2009-06-08 13:39:47 Complete memory accounting
John Goyette (UNITED STATES)
Message: 75367
I would like to understand how to get a complete picture of how much RAM is being used by the kernel and user space applications. I have found that 'ps', 'free' and '/proc/meminfo' do not provide a full picture. What other tools and methods to you recommend for capturing this information?
I have built a minimal distribution for the 527 EZ-Kit and booted it with an NFS mounted root file system. I told the kernel via command line that it has 32 Mb of RAM. Searching the dmesg output, I can find what the kernel is reporting as follows:
root:/> dmesg | grep -i memory
Board Memory: 32MB
Kernel Managed Memory: 32MB
Memory map:
Memory available: 29320k/32768k RAM, (100k init code, 1376k kernel code, 640k data, 1024k dma, 304k reserved)
Freeing unused kernel memory: 100k freed
And the 'free' command produces the following:
root:/> free
total used free shared buffers
Mem: 29420 3948 25472 0 0
This tells me that of my 32 Mb of RAM, the kernel is using 3,344k leaving 29,420k available which is reported as "total" in the free command. This, I think, is the easy part. The difficulty comes in determining how the remaining 29,420k is used.
The 'ps' command produces the following output:
root:/> ps
PID USER VSZ STAT COMMAND
1 root 108 S /sbin/init
2 root 0 SW< [kthreadd]
3 root 0 SW< [ksoftirqd/0]
4 root 0 SW< [events/0]
5 root 0 SW< [khelper]
47 root 0 SW< [kblockd/0]
74 root 0 SW [pdflush]
75 root 0 SW [pdflush]
76 root 0 SW< [kswapd0]
77 root 0 SW< [aio/0]
78 root 0 SW< [nfsiod]
128 root 0 SW< [bfin-spi.0]
138 root 0 SW< [hid_compat]
141 root 0 SW< [rpciod/0]
155 root 532 S -/bin/sh
156 root 40 S /sbin/inetd
158 root 424 S /sbin/klogd -n
159 root 428 S /sbin/syslogd -n
167 root 428 R ps
When you add up the running user space applications (not including ps itself) , you end up with 1,532k. However, the free command and /proc/meminfo report that 3,948k is being used. My question is, what is consuming the remaining roughly 2.5 Mb of RAM? Is this being used by the kernel threads indicated in brackets in the ps command? If so, how do I find out how much each of those is using? Is it being used by RAMFS mounts (I have only /var and /tmp mounted as ramfs and tmpfs)? What tools can we use to investigate where that memory is being spent?
For reference, I am also including the contents of /proc/meminfo and the rc file:
root:/> cat /proc/meminfo
MemTotal: 29420 kB
MemFree: 25456 kB
Buffers: 0 kB
Cached: 560 kB
SwapCached: 0 kB
Active: 440 kB
Inactive: 120 kB
Active(anon): 0 kB
Inactive(anon): 0 kB
Active(file): 440 kB
Inactive(file): 120 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 0 kB
Mapped: 0 kB
Slab: 752 kB
SReclaimable: 84 kB
SUnreclaim: 668 kB
PageTables: 0 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 14708 kB
Committed_AS: 0 kB
VmallocTotal: 0 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
root:/> cat /etc/rc
hostname blackfin
mount -t proc proc /proc -o noexec,nosuid,nodev
mount -t sysfs sysfs /sys -o noexec,nosuid,nodev
mkdir -p /dev/pts
mount -t devpts devpts /dev/pts -o noexec,nosuid
mount -t ramfs var /var
mkdir /var/tmp /var/log /var/run /var/lock
mount -t tmpfs tmp /tmp -o nosuid,nodev
Any insight would be appreciated. Thanks.
-John G.
QuoteReplyEditDelete
2009-06-08 16:26:30 Re: Complete memory accounting
Robin Getz (UNITED STATES)
Message: 75370
John:
Anything with a cached based file system is going to be a little hard to tell what is going on - should you include the cache as "free" memory (since it can be dumped and freed for the application - or "used" memory - since it is in use.
/proc/meminfo tells you what things are going on, and let's you make up the decision...
MemTotal is the total memory avalible to userspace and the kernel.
Cached is the file system cache.
Slab is the kernel dynamic memory (kmalloc, and friends) + userspace (since that is how userspace gets it's memory).
Does that help?
QuoteReplyEditDelete
2009-06-08 22:09:40 Re: Complete memory accounting
John Goyette (UNITED STATES)
Message: 75374
Robin,
Thanks, that is helpful. Now I understand /proc/meminfo a little better. So, I can account for another 1.2 Mb between the cache and slab. I suspect that the rest of the RAM is used by my /tmp and /var file systems and the 'cat' program running at the time I view /proc/meminfo. Kind of funny to think of it that way, but if you type cat /proc/meminfo, the MemFree entry must reflect the fact that you have an instance of busybox running that is executing cat, right?
Is there anyway for me to see how much RAM is in use for /tmp and /var? Since those are ramfs, they are not cached, correct? The du command gives the size of the filesystem, but is the kernel using more RAM than that for the filesystem?
Thanks again for your help.
-John G.
QuoteReplyEditDelete
2009-06-08 22:26:04 Re: Complete memory accounting
Robin Getz (UNITED STATES)
Message: 75375
John:
It depends on the backing file system. There are a few - ramfs lives in the page cache - so it should be accounted for in the page cache accounting.
wiki.debian.org/ramfs
ram disks are different. - (see the wiki above). The answer is - it depends.
-Robin