Skip to content

2025

Memory Pool

Virtual memory1 allows programs to use more memory than physically available by mapping logical addresses to physical ones, with the heap being a region within virtual memory for dynamic memory allocation. Functions like malloc in C and new in C++ are used to allocate memory from the heap, with malloc providing raw allocation and new offering both allocation and initialization.

However, frequent memory allocations can lead to performance overhead and fragmentation, which is where memory pools come in. A memory pool pre-allocates a large block of memory and divides it into smaller chunks for reuse, improving efficiency, reducing fragmentation, and enhancing performance in systems with critical memory needs.

Thread Pool

A thread pool is a collection of pre-allocated threads that manage the execution of tasks in a concurrent system, improving efficiency by reusing threads instead of creating new ones for each task. In a producer-consumer model, the producer generates tasks, and the consumer threads in the pool handle their execution. This reduces the overhead of thread creation and destruction and optimizes resource management, making it ideal for high-concurrent environment like web servers or databases. The thread pool controls the number of concurrent threads, ensuring scalable and efficient performance, especially in systems with frequent or I/O-bound tasks.

io_uring

io_uring is a modern Linux I/O interface introduced in Linux 5.1 (2019) to improve upon the limitations of traditional models like epoll, which has been the standard since Linux 2.6 (2003). Unlike epoll, which requires syscalls for every read and write, io_uring uses memory-mapped (mmap) submission and completion queues, allowing user space and the kernel to communicate while minimizing syscalls and avoiding unnecessary memory copies. This reduces context switching, lowers CPU overhead, and enhances efficiency, making it ideal for high-throughput applications.

DPDK

DPDK is a set of libraries designed for high-performance packet processing in user space, bypassing the kernel's networking stack to achieve low-latency and high-throughput networking.

  • in general
  • 5 min read

Coroutine

A coroutine is a programming concept referring to a function or routine that can pause its execution and yield control back to the caller, and later be resumed from where it left off. Unlike traditional subroutines or functions that run from start to finish, coroutines have multiple entry and exit points, enabling cooperative multitasking or asynchronous processing. Coroutines can be implemented using a variety of techiniques, each offering different levels of control, complexity, and performance.

UDP Notes

UDP, or User Datagram Protocol, is a key protocol in the Internet Protocl (IP) suite, functioning at the transport layer. UDP is a connectionless protocol, meaning it does not establish a dedicated connection before data transmission (no handshake). It sends data in discret packets called datagrams, each independent of the others.

Web Server

We can add an application layer or a protocol layer on top of Reactor to build a web server. The Reactor we built in A Million Concurrency handles I/O events at the network layer. While it currently only echos what the client sends, we will extend it by adding support for HTTP and WebSocket protocols.

Epoll Trigger Mode

In Linux, epoll is a high-performance I/O event notification system that allows the application to monitor multiple file descriptors to determine if I/O operations can be performed on any of them.

There are two trigger modes: Level-Triggered (default) and Edge-Triggered.

A Million Concurrency

Building on top of epoll and the Reactor pattern, we will further enhance our TCP server to achieve the capability of handling a million concurrent requests in a non-blocking manner with a single thread.