, ,

Linux Processes Explained: ps, top, and kill (Linux for Beginners, Part 7)

A process is a running program with a number. Learn to find it with ps and top, and stop it with the right signal, using real commands and output.

Linux for Beginners · Part 7 of 24
The short version
A process is a running program with a number attached to it. You list processes with ps, watch them live with top, and stop one by sending it a signal with kill. Everything else in this part is knowing which signal to send, and why a stuck process sometimes ignores you completely.

If you just finished college, this is the part where the command line stops being a place to list files and starts feeling like a control panel. If you already run servers, this is the fast version of a day you will repeat for the rest of your career: something is eating the machine, you find it, and you decide whether it dies politely or by force.

A ticket lands at two in the afternoon. The application server feels slow, pages take ten seconds, and nobody deployed anything today. You ssh in and type one command before you touch anything else.

$ top
top - 14:02:11 up 9 days,  3:47,  2 users,  load average: 7.82, 6.10, 3.44
Tasks: 214 total,   3 running, 210 sleeping,   0 stopped,   1 zombie
%Cpu(s): 91.3 us,  4.2 sy,  0.0 ni,  3.1 id,  0.0 wa
MiB Mem :  15884.0 total,    612.4 free,   9210.8 used,   6060.8 buff/cache

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
  30412 www-data  20   0  812432 402118  10240 R  98.7   2.5   4:11.20 php-fpm
   1104 mysql     20   0 3821004 512880  18220 S   6.3   3.1  88:04.11 mysqld
    884 root      20   0  102340   8912   6120 S   0.3   0.1   2:41.09 sshd

Two numbers answer the ticket. The load average of 7.82 is demand for CPU time over the last minute, and this box has four cores, so demand is running at nearly twice what the hardware can serve. The process with PID 30412, a php-fpm worker, is sitting at 98.7 percent of one core. That single worker is the whole story. Now you need the vocabulary to act on it.

Every running program carries a number

When the kernel starts a program it hands out a process ID, the PID, a plain integer that stays with that program until it exits. Every process also records the PID of whatever started it, the parent PID or PPID. Follow that chain upward from any process and you always arrive at PID 1, the first thing the kernel starts at boot. On almost every current distribution PID 1 is systemd. It is the ancestor of everything else on the machine.

Your own shell is a process too. The special variable $$ holds its PID, so you can look at yourself:

$ echo $$
2417
$ ps -o pid,ppid,comm -p 2417
    PID    PPID COMMAND
   2417    2410 bash

Your bash has PID 2417 and its parent is 2410. That parent is the sshd session that accepted your login, whose parent is the main sshd daemon, whose parent is systemd. The pstree command draws the whole family at once:

$ pstree -p 1 | head
systemd(1)-+-agetty(701)
           |-cron(742)
           |-sshd(884)---sshd(2410)---bash(2417)---pstree(2461)
           |-mysqld(1104)
           `-systemd-journal(410)
systemdPID 1sshdPID 884cronPID 742mysqldPID 1104sshdPID 2410bashPID 2417Follow any PPID upward and you reach systemd at PID 1.

This tree matters for a practical reason. When you kill a parent, its children do not automatically die with it in every case, and when a parent dies first, its children get adopted by PID 1. That adoption is the mechanism that quietly cleans up a lot of messes, and it is also the reason a rare kind of dead process, the zombie, eventually disappears on its own.

Reading ps output without memorizing every flag

The ps command prints a snapshot of processes, one line each, frozen at the moment you run it. The confusion beginners hit is that ps accepts two different flag styles that grew up in two different Unix traditions, and both still work. You will see both in real answers online, so it helps to recognize them.

The two flag styles that trip people up

The BSD style takes no dash, and the most common form is ps aux. The UNIX style takes a dash, and the common form is ps -ef. They show almost the same information in a slightly different order. Pick one and stick with it. I use ps aux when I want the percent CPU and percent memory columns, and ps -ef when I want a clean parent PID column for tracing who started what.

$ ps aux | head -4
USER       PID %CPU %MEM    VSZ   RSS TTY   STAT START   TIME COMMAND
root         1  0.0  0.4 168404 11920 ?     Ss   Jun24  0:31 /sbin/init
www-data 30412 98.7  2.5 812432 402118 ?    R    14:00  4:11 php-fpm: pool www
mysql     1104  0.3  3.1 3821004 512880 ?   Sl   Jun24 88:04 /usr/sbin/mysqld

The column most people ignore is STAT, the process state. It tells you what a process is doing right now, and it is the difference between a process you can stop easily and one that will fight you. Here is what the letters mean.

CodeStateWhat it means
RRunningOn the CPU or waiting in the run queue for a turn.
SSleepingWaiting for an event, and can be woken by a signal. Most processes sit here.
DUninterruptibleWaiting on disk or storage. Ignores signals until the I/O finishes.
TStoppedPaused by a job control signal, waiting to be resumed.
ZZombieFinished, but its parent has not collected the exit status yet.
IIdleAn idle kernel thread, doing nothing at the moment.

A trailing letter often rides along. An s means a session leader, an l means multithreaded, and a + means the process is in the foreground. So Ss is a sleeping session leader, and Sl is a sleeping multithreaded process, which is exactly what a database looks like.

What load average is really telling you

The three numbers in top and uptime are the load average over one, five, and fifteen minutes. Load is roughly the number of processes wanting the CPU or waiting on disk. The single fact that makes it readable is the core count. A load of 4 on a four core machine means fully used with no queue. The same 4 on a single core machine means three processes are always waiting.

$ uptime
 14:05:02 up 9 days,  3:50,  2 users,  load average: 7.82, 6.10, 3.44
$ nproc
4

Four cores, load 7.82 on the last minute. Demand is close to double capacity, and the trend matters as much as the number. The fifteen minute figure of 3.44 is lower than the one minute figure of 7.82, so this spike is recent and getting worse, not something that has been simmering all day. A picture makes the shape obvious.

4 cores7.821 min6.105 min3.4415 minLoad above the core line means processes are waiting for the CPU.

For a live view, top is on every machine, and htop is the friendlier version worth installing. On Debian and Ubuntu that is sudo apt install htop, and on the RHEL family it is sudo dnf install htop. Inside top, press P to sort by CPU, press M to sort by memory, and press q to quit.

Signals, the vocabulary for stopping a process

You do not really kill a process. You send it a signal, a short numbered message the kernel delivers, and the process decides how to respond. The kill command is badly named because most of the time you are asking politely. A handful of signals cover almost everything you will do.

SignalNo.What it asks for
SIGHUP1Terminal closed. Many daemons reuse it to reload their config.
SIGINT2What Ctrl and C send. Interrupt and stop.
SIGKILL9Cannot be caught or ignored. The kernel removes the process.
SIGTERM15The default. Please shut down cleanly. Can be handled.
SIGSTOP19Pause the process. Cannot be caught.
SIGCONT18Resume a paused process.

Plain kill 30412 sends SIGTERM, number 15, the default. You name a different signal by number or name, so kill -9 30412 and kill -KILL 30412 and kill -SIGKILL 30412 are the same command. If you do not know the PID yet, pgrep finds it and pkill signals by name:

$ pgrep -a php-fpm
30412 php-fpm: pool www
30455 php-fpm: pool www
$ pkill -TERM php-fpm      # sends SIGTERM to every match by name

Why kill -9 should not be your first move

Half the advice online reaches straight for kill -9. I disagree, and the reason is in the table. SIGKILL cannot be caught, which sounds like a feature until you think about what catching a signal is for. A well written program catches SIGTERM so it can flush its buffers, finish the current write, close its database connection, and remove its lock file before it exits. SIGKILL gives it none of that. The kernel simply stops it mid step. Send SIGKILL to a database at the wrong instant and you can leave a stale lock file or a half written record that stops the service from starting again.

The order that saves you grief is simple. Send SIGTERM, wait a few seconds, and only escalate to SIGKILL if the process is still there. Here is that sequence against a stuck worker, including the failure you should expect on the first try.

Worked example
$ kill 30412                 # polite request, SIGTERM
$ ps -p 30412 -o pid,stat,comm
    PID STAT COMMAND
  30412 R    php-fpm         # still running, it ignored us
$ sleep 3; kill -9 30412     # give it a moment, then force
$ ps -p 30412 -o pid,stat,comm
    PID STAT COMMAND         # empty output, the process is gone

Two errors show up here often. If you see bash: kill: (30412) - No such process, the process already exited, and there is nothing to worry about. If you see kill: (30412) - Operation not permitted, the process belongs to another user and you need sudo. Neither means kill is broken.

There is one process that ignores even SIGKILL, and it surprises people. A process in state D, uninterruptible sleep, is blocked inside the kernel waiting for storage, often a slow disk or a stalled network mount. Signals do not reach it until the I/O completes or fails. You can send SIGKILL all day and the process will not move. The fix is to solve the storage problem, not to try a bigger hammer.

RunningRSleepingSUninterruptibleDStoppedTZombieZwaitsdisk I/OSIGSTOPexitOnly R and S respond to signals normally. D and Z do not.

Background jobs, and keeping them alive after you log out

Add an ampersand to a command and the shell starts it in the background, handing you the prompt back right away. Press Ctrl and Z and the shell pauses the current foreground job with SIGSTOP. The jobs, fg, and bg commands move work between those states.

$ tar czf backup.tar.gz /var/www &
[1] 6120
$ jobs
[1]+  Running                 tar czf backup.tar.gz /var/www &
$ fg %1        # bring job 1 back to the foreground
tar czf backup.tar.gz /var/www

A background job still belongs to your shell, so when you log out the shell sends SIGHUP and the job usually dies with it. For a long task that must survive the disconnect, start it with nohup, which detaches it from the terminal and sends its output to a file:

$ nohup ./long-import.sh > import.log 2>&1 &
[1] 7042
$ nohup: ignoring input and appending output to 'nohup.out'

For anything you will need to reconnect to later, a terminal multiplexer like tmux or screen is the better tool, because it keeps a live session you can rejoin. That is a topic in its own right and it comes up again when we reach remote work over SSH.

Where this bites in production
A monitoring dashboard shows a process that will not die no matter how many times you send SIGKILL, and the load average keeps climbing. Nine times out of ten the process is in state D, stuck on a storage backend that stopped answering, often an NFS mount that went away. The process is not the problem, the disk is. Run ps aux | awk '$8 ~ /D/' to list every process in that state, then look at what storage they share. Killing harder never works here.
Real interview question
What is a zombie process, and how do you get rid of one? A zombie is a child that has already exited but whose parent has not yet read its exit status with wait. It holds nothing but a slot in the process table, so it uses no CPU or memory. You cannot kill a zombie, because it is already dead. You fix the parent: either the parent recovers and reaps it, or you signal the parent so it exits, at which point PID 1 adopts the zombie and reaps it immediately. A single zombie is harmless. Thousands of them mean a parent with a bug.
Try it yourself
Start a harmless process, find it, stop it, and confirm it is gone. Run sleep 300 &, note the PID it prints, then run pgrep -a sleep to find it again. Stop it with kill $(pgrep -x sleep). The exact verify command: pgrep -x sleep && echo still-here || echo gone. It should print gone.

What people actually ask about processes

Is there a difference between a process and a thread?
A process has its own memory. Threads live inside one process and share that memory, which is why a database shows up as one process with many threads and the l flag in its state. For day to day work you manage processes by PID. Threads are the internal detail.

How do I find which process is using a port or a file?
Use sudo lsof -i :80 for a port and sudo lsof /var/log/app.log for a file. Both print the PID and command holding it, which is how you find the process to signal when a service will not start because the address is already in use.

What is nice and renice?
Niceness sets scheduling priority from -20, the most favored, to 19, the least. A high nice value makes a process yield to others when the CPU is busy. Start a job politely with nice -n 10 ./job.sh, or lower a running job with renice -n 10 -p 6120. Only root can move a process to a negative nice value.

My kill did nothing. What now?
Check the state first with ps -p PID -o stat. If it is D, the process is stuck on storage and no signal will reach it, so look at the disk or mount. If it is Z, it is already dead and you signal its parent instead. If it is R or S and survives SIGTERM, escalate to SIGKILL.

Does killing a process kill its children?
Not always. A signal goes to the process you named. To hit a whole group, signal the process group with a negative PID, as in kill -TERM -6120, or use pkill -P 6120 to target children of a parent.

Processes are where Linux stops being a set of files and starts being a live system you steer. Once you can read top, trace a PID to its parent, and pick the right signal, most of the panic goes out of a slow server. Next in the series we manage the services that start these processes for you, with systemd, so you are not doing this by hand at two in the afternoon. Keep a terminal open and run every command here on a machine you own.

Linux for Beginners · Part 7 of 24
« Previous: Part 6  |  Complete Guide  |  Next: Part 8

References

· ps(1) manual page, process state codes
· signal(7) manual page, standard signals and numbers
· VMware for Beginners: The Complete Guide
· Cloud for Beginners: The Complete Guide

About The Author


Discover more from Journal of Intelligent Infrastructure – By Dr Pranay Jha

Subscribe to get the latest posts sent to your email.

Leave a Reply

Your email address will not be published. Required fields are marked *

Architect’s Toolkit

About the Author

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

Discover more from Journal of Intelligent Infrastructure - By Dr Pranay Jha

Subscribe now to keep reading and get access to the full archive.

Continue reading