OS Resource Leaks

Anyone who works on an embedded platform or even in PC applications, should probably by now understand full well the implications of a resource leak.  Before we start delving into this matter however it is imperative that we have a full grasp of what types of resources the Operating System (OS) provides to a “user-land” application. For starters let me give you four resources which could possibly be leaked during execution of your program:

• Memory leak – The most celebrated of all the leaks. Simplistically, the failure to inform the OS that you are finished using a memory area thus making the OS reserve that memory area for as long as the CPU is not reset or a “memory map” release is not automatically executed. This applies to malloc(), calloc(), realloc(), opendir(), mmap() and others of the like.

• Thread leak – An OS can only allow a certain number of threads running at the same time for a particular process. Exiting (libc::exit()) unnecessary threads, is a MUST. For they share together with the other threads your process’ precious time quantum. And with special reference to thread-based implementations of server applications (like FTP, HTTP, LDAP, Discovery etc…), you may end up unable to serve further requests should you fail to properly account for your threads.

• Process leak -like thread leaks, unnecessary processes should either be “killed” or “SIGTERM”ed. Don’t tell me you love rogue processes and good-for-nothing-zombies? Failure to properly end a process may result in an OS executing unecessary context switches for processes that dont really matter. Like threads, when your “server” or “daemon” application is serving multiple requests via “fork()”, “vfork()” or “clone()”, you might end up unable to server further requests from your clients if unnecessary processes are left to run idly.

• File Descriptor / Handles Leaks - The operating system also limits the number of files that an application can open simultaneously. So if you have got a leak on this one, you are looking forward to an unnecessary debugging adventure. For starters, some open source libraries, automatically assert() when they fail to take a hold of a valid file descriptor.

Why are leaks dangerous?

A leak in the software realm is not like a leaky pipe that eventually floods up your room if left unfixed over long periods of time. (I left our faucet once fully open overnight and the next day the house was really a mess! The water distribution service bill was almost as messy.) A resource leak in software is pretty much the opposite though. Once there are leaks in certain areas of a process, be it memory or something else, that same process or other newly started programs, will eventually find themselves failing in certain system calls which would have succeeded in an otherwise zero-leak running environment.

Summarizing, “Leaks result to only one thing, the eventual depletion of resources of an otherwise perfectly “enough” system. “Enough” because theoretically, except for some cases, what an OS provides is enough for everything it was designed to do, and that include the facilitation of user-land processes.

What are the consequences of leaks?

The thing I hate most about leaks, is that it has the ability to affect other processes.  And in which case if they do, it is almost virtually impossible to detect. Think of a room full of people where somebody suddenly silently farts. Everyone suffers but when and where it happened nobody really knows. (hahahahah! PEACE!)

They too are extremely difficult to debug and it takes an enormous amount of time to pinpoint their exact location. Fixing them is not really the problem in most cases, but finding out where and when they happen is the most difficult of all.  Enumerating some effects :

1. System fails to allocate memory. (libc :: malloc(), calloc(), etc…)
2. System fails to start a new thread. (POSIX : pthread_create())
3. System fails to start a new process. (libc :: fork(), vfork())
4. System fails to open new files. (libc :: open(), fopen(), dup(), etc…)
5. System fails on system calls. (libc :: socket(), pipe(), etc….)
6. Applications start to run really really slow.
7. Applications suddenly crash due to low level assert().
8. You enter the debugging twilight zone with a ticket to the universal competition for patience. Just kidding.

Probably in simple programs, you dont have to worry too much. But if you are developing an all-too-powerful daemon  which has to exist while the system is online, (or as you morph from intern engineer level to non-assistant level), you have got to be paranoid of leaks. “Resource Leaks = MESS” remember that.

Isn’t my OS the sky-is-the-limit version?

Yes dudes and dudettes! No OS is “mugen” – meaning “infinite” in Japanese.  Let me cite some example just for fun (though completely unrelated and utterly useless) did you know that :

• A server based encrypted data, is most likely be valid for only 5 minutes?
• that the real limit of your system time or the Y2K bug is “2/7/2036″? and the UNIX Y2K bug in 2038?
• The limit of a USB cable is around 3 – 5 meters depending on the speed you use?
• That  a NETBIOS system browser  has to routinely list the domain every 12 minutes?

Now going back to the topic, OS resources are pretty much the same. How they came up with the limits, is I guess an arbitrary science. Limits are most probably based on a careful balance of available memory/actual resources versus the rough average of “extreme usage” and “ordinary usage”. For the moment, I do not question it because, as they are for me “enough”. Plus the fact that so many scientific minds have no major complaints about it, shows that it is well in a way “enough”.

Resources like file descriptors for MOST operating systems have process-wide and system-wide limitations.  Process-wide means that for each process you create, there is a limited amount of file descriptors it can open simultaneously. Or for multi-threading process,  thread count limit is the maximum amount of threads a process can run simultaneously.  System-wide limits however, is the count of all the particular in-use resource, regardless of the parent process.

Let us take for example some older versions of linux which can open up to a maximum of 256 files per process, and roughly 1K system-wide. Therefore, for as long as the 1K system-wide limit is not breached, any process can gain access to 256 files simultaneously at any given time. But should there be 4 other processes, each opening 250 files at the same time, then the 5th process cannot use its full 256 file limit anymore. (Check this out.)

Anyways, if you happen to be running on top of a unix platform, you might want to try “ulimit -a” bash command in your terminal, to see certain limits of your Operating system.

Can these limits be changed?

Yes. However, it is important to note here that changing a per-process limit be it a hard or soft limit might require some special process, like recompiling your kernel. For the most part though, commands like setrlimit(), ulimit() or sysctl() (via libc::system() command) can be called within the program to modify certain soft and hard limits. Note also that setting a particular hard limit to an unreasonable value and then allowing a process to go beyond the hard limit might cause the system to break down eventually. Besides, setting the limit for a particular resource, will never be a solution for a resource leak!

Next Up …  file descriptor leaks …