System Monitoring

What system monitoring entails:

inspect and manage processes
understand load average
available memory
checking stuff from a shell
disk space, storage available
logging
log size, log rotate
systemd init system
systemd journal

Processes

Controlling Processes

In order to control a process, you must be able to find a process.

Use the ps command to list currently running processes, and pass the aux argument to show everything:

$ ps aux

This shows a lot of information. Filter it out with grep:

$ ps aux | grep httpd

This is also useful because it will display information about the cpu and memory usage.

Now you can see its PID, and use that to kill the process:

$ kill 15800

THis will send a SIGTERM, or Signal 15, to the process. There are 18 different signals you can send, the SIGTERM is the nice polite request that the process stop.

$ man 7 signal

If you want to send a different signal, like 12, or 2,

$ kill -12 15800
$ kill -2 15800

or if you are desparate, send a SIGKILL, Signal 9:

$ kill -9 15800

If you don't know the PID but you do know the process name, you can use the killall command:

$ killall firefox

Process Load

You can see some information about the load average by showing the contents of /proc/loadavg:

$ cat /proc/loadavg

Or use the uptime command:

$ uptime

This displays the uptime and the load average.

The numbers given in /proc/loadavg and by uptime are three numbers less than 1, for example

0.63 0.72 0.71

These represent the system load during a period of 1, 5, and 15 minutes, respectively. The load refers to the number of processes that are waiting on or currently utilizing the CPU during each timeframe.

The number that is reported should be thought of as the number of equivalent CPUs that would be required to handle the work. If you see an average load like

9.23 9.81 8.94

that would be perfectly fine for a beefy machine with 16 cores that could chew through the work in no time. However, if this were a Raspberry Pi or a creaky old desktop, it would be a sign the system is getting hammered.

Memory Load

To check memory, use the free command:

$ free -m

The -m flag makes the output more readable.

used on the first line - corresponds to how much memory is actually being used. This includes disk cache - chunk of memory set aside for data waiting to be written to disk.

this output is also shown in the free command - the number on the far right, labeled "cached".

This memory is not used by processes, so can be contracted as needed.

Actual memory we have free - this number is sown in the second column, second row.

This memory amount is much larger, because it considers the disk cache flexible.

Shell Monitoring

The most obvious monitoring program is top

also install some other popular tools:

$ apt-get insall htop iotop ncdu

Linux/System Monitoring

From charlesreid1

Contents