Understanding memory information on Linux systems

Understanding Linux memory information

Every operating system needs memory to store program code segments and data. This is also true for Linux systems. The problem: there is a lot of information available regarding memory usage and its behavior. Let’s discover how Linux manages its memory and how we can gather memory information. Great for troubleshooting, but also to learn more about the inner workings of our system.

Random access memory

When we talk about memory in this article, we usually mean random access memory (RAM). This is memory which can be used for both showing and storing data. Typically we will find in this type of memory the programs that are running on the system, including the Linux kernel itself. Besides the program code, memory also stores a lot of data. A good example is when you are running a MySQL database server. The program itself is relatively small, the data itself is huge. So we will also have a look on tuning programs and their memory usage, as this is typically a problem with memory-hungry programs.

Determine amount of RAM

The first step is to discover the amount of RAM we have in the system. There are a few ways on how to achieve this, starting from the data stored in dmesg.

dmesg | grep -in mem

The output may look something like this:

This information shows the number of memory available in kilobytes. The first value shows what is currently available, the second value displays the total memory in the system. These values are usually very close. This indicates that most of the memory can be used and is a good thing. The small portion “missing” is used by the initial loading of the kernel. If there is a big gap, then this might be caused by the kernel and how many memory it can allocate. Especially with 32 bits versions of Linux this number is limited.

Details and information about RAM modules

The next step is learning more about the RAM modules itself. You will need the dmidecode utility for this, which is available for most Linux distributions. To gather memory information, tell the dmidecode to only show information for device type 17.

dmidecode --type 17

Depending on your hardware it may be able to extract the specifics of your modules and show detailed information. You may need to run this as root user. Normal users won’t have the right permissions to read all information.

In this output above you can see the details of the first memory module. We see it is a chip of 4 GB and configured at a 1600 MHz speed. This is a great way to determine the memory available in a Linux system, together with detailed output. Unfortunately, the command does not always play well with virtual systems.

No data is displayed on our virtual test system :(

Let’s move on to the next set of utilities and gather details regarding memory usage.

Show available memory on Linux

After the Linux kernel is booted it is time to start programs. The kernel itself is not responsible for the programs. Instead, it delegates this responsibility to a service manager like init or systemd. This process is the first to be started and will be process ID 1. Its duty is to start other services and programs during the lifetime of the system. Each program will consume some amount of memory, depending on the program size and the related data.

See available memory with free command

The first command to obtain available memory information is the perfectly named tool free.

This utility shows two different types of memory: normal memory and swap memory. Swap is a type of memory that you want to avoid needing as much as possible. If it would be used, then it means your normal memory is full. The system will then leverage the swap memory to temporarily store data, at the cost of disk operations. As they are much slower than normal RAM, your system will be impacted. In this screenshot, we see the swap is not used, which is good.

The free utility retrieves this memory information from a file named /proc/meminfo. Let’s have a look at that as well.

Details from /proc/meminfo

The next step to obtain everything available regarding memory is found in the procfs tree, usually mounted under /proc. This file is very extensive, so have a look at it on your system:

cat /proc/meminfo

A partial output listing showing how memory is used

Memory management under Linux is extensive and changed over time to what it is now. This results in a delicate system that optimizes memory usage as much as possible. Let’s get into some of these fields and understand better how Linux does its job.

Cached, SwapCached

The system does a lot of repetition, including reading the same files or data. Everything that goes into memory and is no longer needed, will be kept for a little bit longer. If you then request the same data while it is in memory, you will get it almost instantly. This is what happens when you run a find command on a particular directory the first time, which usually takes a while. Run it again and it will be much quicker.

Active, Inactive

A page cache optimizes access to files. These buffers can be recently used (=active), or not (=inactive).

Active is the total of Active(anon) and Active(file). Similarly, Inactive is the total of Inactive(anon) + Inactive(file).

SwapTotal, SwapFree

These provide insights in the configured swap memory and how much is left. Ideally, the SwapFree value is equal to SwapTotal, meaning no swap is in use at that time. Swapping is disk intensive.

Dirty

The Dirty field refers to data that is stored in memory and still needs to be written to the disk.

You can test this easily by creating a test file and compare the value before and after.

cat /proc/meminfo | grep Dirty && dd if=/dev/zero of=/tmp/testfile.txt bs=1M count=10 && cat /proc/meminfo | grep Dirty

Note: if you repeat this command, you will see the effect of smart memory management. In that case the value before and after will most likely be the same, as some data was cached and directly returned as a finished action.

Slab, SReclaimable, SUnreclaim

The kernel does a lot of repetition during the time it is running. Some objects, like asking for the specific inode of a file may be performed thousand times a day. In such case, it would be wise to store it in a quick reference list, or cache. Slab are the caches for kernel objects, to optimize those activities that happen the most.

The Slab field is the total of SReclaimable and SUnreclaim.

Slab: 32272 kB
SReclaimable: 18144 kB
SUnreclaim: 14128 kB

NFS_Unstable

For systems that use NFS this is a good measurement to see how much data is not committed to the storage yet. For systems without NFS, this value can be ignored and is usually just zero.

More fields

If you compared these fields with your own system, you will discover there are more fields. Depending on your work load, you will have to discover what fields make sense to monitor. What does generally work well during troubleshooting, is comparing similar systems and check for the differences in /proc/meminfo. It may give a good indication where memory is used and what keeps the system busy.

Using vmstat utility

Another nice utility that is often available is vmstat. With -s we can query memory statistics.

We can also query the previous mentioned slabs.

Monitoring memory usage in Linux

If you have a monitoring system in place, then two key attributes from /proc/meminfo should be monitored.

• MemFree
• SwapFree

By monitoring these two values you may discover memory leaks and badly optimized systems. You may also pick up on a misbehaving process now and then.

For environments that have many similar systems with a typical workload (e.g. lots of web servers doing the same), then it would make sense to monitor more of the keys in /proc/meminfo. Storing the data a few times per day may give you the ability to compare systems and find exceptional events.

Conclusion

Linux memory management is an extensive subject and there is a lot to learn. Make sure to understand the basics, like how to obtain memory information, including that of RAM and swap. This is of great help during troubleshooting and know what programs need to optimally do their job.