Too many open files

One puzzling error that happens, albeit rarely, is having too many file handles open at the same time. You can check the limit allowed per process on your system by doing

$ ulimit -a

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 256
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 2784
virtual memory          (kbytes, -v) unlimited

The key value is “open files” which, in my case, is limited to 256. In my application I was running into this. (As a side note, this is not the most common error, and I was not checking for it, so my program was actually doing something much more mysterious, and I had a moment of panic, before I decided this had something with me having too many open files. So, folks, always check your streams for errors properly.)

This limit can be changed using the same ulimit command:

$ ulimit -n 2000
$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 2000
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 2784
virtual memory          (kbytes, -v) unlimited

How on earth did I run into this you ask. Well, for various reasons, my space craft trajectory simulator saves the path of each object in the simulation in a separate file. I ran into this problem when I, you guessed it, ran a simulation with more than 256 objects in it.

PS. In C++ code, you can check for the reason why a file stream failed to open using strerror and the global variable errno.

    std::ifstream infile(path);
    if (infile.fail()) {
        std::cerr << strerror(errno);
    }

I note this because it is not the most obvious thing, and I would rather have something like infile.failstr() or somesuch more OOP like.

PPS. DO NOT do something like

sudo launchctl limit maxfiles ...

and set the number too modestly. I set the number to 2000, little realizing that the system limit was “unlimited” and some programs (looks at Chrome) do seem to use an unlimited number of open files. Executing this command had my system fall over and need a reboot.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.