Why This Matters
This piece highlights a persistent performance bottleneck in multithreaded software: standard memory allocation via malloc() often serializes threads, undermining scalability on multi-core systems. This matters broadly, from packet-processing network applications to any high-concurrency software, because inefficient memory management directly limits throughput as hardware parallelism increases. It also traces how allocator designs have evolved—from stack-based and dynamic heap schemes to garbage collection and 'arena'-based approaches like jemalloc—to better address multi-core and NUMA architectures.
Key Takeaways
- Traditional malloc() implementations can become a serialization bottleneck in multithreaded programs, causing performance to degrade as core count increases.
- Techniques like skbuff recycling and PF_RING can reduce memory allocation overhead in network packet processing, improving capture performance by around 10%.
- Memory allocator design has evolved from simple stack/heap models to more sophisticated 'arena'-based systems (e.g., jemalloc) that account for NUMA and multi-core CPU architectures.
Memory Allocation
Problems
Limit Memory Allocation (if not necessary)
Multithreaded programs often do not scale because the heap is a bottleneck.
When multiple threads simultaneously allocate or deallocate memory from the allocator, the allocator will serialize them. Programs making intensive use of the allocator actually slow down as the number of processors increases.
Malloc (libc) is the worst memory allocation API to use.
Programs should avoid, if possible, allocating/deallocating memory too often and in particular whenever a packet is received.
In the Linux kernel there are available kernel/driver patches for recycling skbuff (kernel memory used to store incoming/outgoing packets).
Using PF_RING (into the driver) for copying packets from the NIC to the circular buffer without any memory allocation increases the capture performance (around 10%) and reduces congestion issues.
Design Evolution
... continue reading
GoKawiil