I have certain performance test harnesses that expect to have a lot of free memory. The test harnesses produce a lot of output that facilitate my performance analysis efforts. The output of these harnesses is mostly in the form of text files and pipe-delimited files (to be used by SQL*Loader) and CSV files which I upload to Excel.
Sometimes I execute these performance harnesses for long periods of time in which case I can generate a significant amount of file system content. The problem is that I wind up with little or no free memory.
I Want Free Memory, Not Reclaimable Memory
On Linux (as is the case with every Unix derivation) user memory allocations (e.g., stack/heap growth) can be satisfied by the OS through page reclaims. Page reclaims are simply reuse of memory pages that have clean content such as pages of text files read from the file system. A reclaimable page can be matched to a process request for memory extremely fast, I’ll agree. However, there is still a difference between free memory and reclaimable memory. Whether that difference is no more significant than the output of such tools as free(1) is not the point of this blog entry. In fact, if you want to see the “buffer adjusted” free memory on Linux you can use –o argument to the free(1) command. The problem for me can sometimes be that I don’t want to munge through scripts and programs that expect to start with a lot of free memory and change them to weed out the reclaimable effect.
If you want to effectively clean out physical memory of all the cached file system dead wood, and you have proper permissions, you can write values into /proc/sys/vm/drop_caches and get the results you are looking for.
In the following example I have a 72GB physical memory system that is totally idle. There is a single instance of Oracle Database 11g Release 2 booted with an SGA of roughly 22GB. There are no significantly large processes running either. I’m basically sitting on about 50 GB of cached file system “stuff” that I don’t want cached. As the example shows I’ve echoed the numeral 3 into drop_caches and the subsequent execution of free(1) shows the 50 GB of cached file system junk is now wiped out and shown under the “free” column.
# free
total used free shared buffers cached
Mem: 74027752 73688248 339504 0 528164 62777200
-/+ buffers/cache: 10382884 63644868
Swap: 16779884 954164 15825720
# echo 3 >/proc/sys/vm/drop_caches
#
# free
total used free shared buffers cached
Mem: 74027752 22956444 51071308 0 2144 13320800
-/+ buffers/cache: 9633500 64394252
Swap: 16779884 954164 15825720
#
Of course I understand that this is just a shuffle in memory accounting by the kernel, but for certain scripting purposes this might be helpful for someone searching the web someday.
Kevin Closson's Blog: Platforms, Databases and Storage 
Here’s an interesting outcome. On a 4.8 OEL system with about 4 databases running (non-prod). I did a sync, then echo 3>/proc/sys/vm/drop_caches. Looked at free and I got the expected result. However, all the DB’s then returned this error when connecting to them:
ERROR:
ORA-01034: ORACLE not available
ORA-27102: out of memory
Linux-x86_64 Error: 12: Cannot allocate memory
Additional information: 1
Additional information: 262149
ps -ef | grep pmon showed all the DB’s still running, but it was as if all the memory allocated to them was yanked out. To be safe I rebooted the box and crs moved those DB’s to another node. Any ideas on what happened?
I would suggest to run “sync” BEFORE echo 3>/proc/sys/vm/drop_caches
Sikkandar,
All that does is put more on the list to free. But I agree.
Yep, that helped me out.
Thanks 🙂
It looks like the correct way to do this is using sappiness:
echo 0 > /proc/sys/vm/swappiness
Read this to get a full understanding
http://lwn.net/Articles/83588/
It’s been a while since I wrote that first entry, but I actually did a sync first. Now, since this was a total test environment for my cluster testing I could really do anything I wanted so I played with it extensively. I could reproduce it at will, it was not a 1 time event. My guess is this was a bug or bad behavior in 4.8. We where just about to upgrade to 5.x when I tested this out. When we moved that cluster to 5.x a few days later the problem was gone. I could do the sync and echo without any issues at all.
Of course, I had no real-world need for this functionality. It was purely a curiosity vs. cat scenario. The cat didn’t survive : )