Run a process in the background on Linux and keep it after logout

Which method survives what

When you close a terminal or your SSH connection drops, an interactive bash gets SIGHUP and passes it on to its jobs. We tested the methods below on bash 5.2, coreutils 9.7 and util-linux 2.41 by sending SIGHUP to an interactive bash.

None of these survive a reboot, and none restart the job if it crashes.

Background jobs with &, jobs, fg and bg

& at the end of a command runs it in the background of your current shell, while its output still goes to your terminal. If you started a command in the foreground and want your prompt back, press Ctrl-z to suspend it and run bg to resume it in the background.

./backup.sh > backup.log 2>&1 &
jobs -l
fg %1
# Ctrl-z suspends it again, bg resumes it in the background
bg %1
kill %1

jobs -l lists the current shell's jobs with their process IDs, and %1 means job 1. A job suspended with Ctrl-z shows as Stopped, not running, and in our test a stopped job died on SIGHUP.

A plain cmd & does not survive a dropped connection or a closed terminal, because bash sends SIGHUP to its jobs. On a clean exit it survives, because bash's huponexit option is off by default; with shopt -s huponexit in a login shell, such as an SSH session, it dies on exit too.

Keep it running after logout: nohup, disown and setsid

If you have not started the command yet, nohup cmd > log 2>&1 & makes it ignore SIGHUP. setsid cmd > log 2>&1 < /dev/null & starts it in a new session without a controlling terminal, so the hangup never reaches it. For a Python script, add -u or PYTHONUNBUFFERED=1, or the log fills in blocks.

nohup ./backup.sh > backup.log 2>&1 &
setsid ./backup.sh > backup.log 2>&1 < /dev/null &

# Already running as job 1? Keep it alive after logout
disown %1

If the job is already running under cmd &, or you paused it with Ctrl-z and resumed it with bg, disown keeps it alive after you log out: bash removes it from the job table and never sends it SIGHUP. disown -h %1 keeps it in the table but marks it so SIGHUP is not sent. Both survived our test.

disown does not redirect output. In our test, a disowned Python job that kept printing to the terminal died within 2.5 seconds of the terminal closing, from the write error, while the same job with its output redirected kept running. Every detail of nohup's output files is in the nohup guide.

Run it as a systemd user service

systemd-run --user hands the command to your systemd user manager as a transient service. The user manager stops after your last session ends unless lingering is on, so run loginctl enable-linger first if you are logging out.

loginctl enable-linger
systemd-run --user --unit=train --same-dir python3 -u train.py
systemctl --user status train
journalctl --user -u train -f
systemctl --user stop train

systemctl --user status train shows its state and recent output, journalctl --user -u train -f follows the output live, and systemctl --user stop train stops it. The service does not inherit your shell's environment, so pass variables with --setenv=NAME=value.

Or keep a terminal you can come back to

tmux and screen keep the job attached to a terminal you can reattach to later with tmux attach -t NAME or screen -r NAME. If systemd-logind kills user processes at logout (KillUserProcesses=yes, the upstream default that many distributions turn off), start them with systemd-run --scope --user and enable lingering. The tmux detach and attach guide has the full workflow.

Know when it finishes without logging back in

Every method above still makes you log back in to find out whether the job finished.