This is my personal blog. The views expressed on these pages are mine alone and not those of my employer.
Around the turn of the century we started to get a bigger need for high capacity web servers. For example there was the C10k problem paper.
At the time, the kinds of things done to reduce work done per request was pre-forking the web server. This means a request could be handled without an expensive process creation.
Because yes, creating a new process for every request used to be something perfectly normal.
Things did get better. People learned how to create threads, making things more light weight. Then they switched to using poll() / select() , in order to not just spare the process/thread creation, but the whole context switch.
I remember a comment on Kuro5hin from anakata, the creator of both The Pirate Bay and the web server that powered it, along the lines of “I am select() of borg, resistance is futile”, mocking someone for not understanding how to write a scalable web server.
But select() / poll() also doesn’t scale. If you have ten thousand connections, that’s an array of ten thousand integers that need to be sent to the kernel for every single iteration of your request handling loop.
Enter epoll ( kqueue on other operating systems, but I’m focusing on Linux here). Now that’s better. The main loop is now:
set_up_epoll() while True: new, read, write = epoll() epoll_add_connections(new) for con in read: process(con.read()) if con.read_all_we_need: epoll_remove_read_op(con) for con in write: con.write_buffer() if con.buffer_empty: epoll_remove_write_op(con)
All the syscalls are pretty cheap. epoll() only deals in deltas, and it doesn’t have to be re-told the thousands of active connections.
... continue reading