In this paper, interesting features of the Linux kernel’s network stack are analyzed, which were introduced during the development cycles from Linux v3.7 to Linux v3.16. Special attention is given to the low-latency device polling (introduced in Linux v3.11), the netfilter’s SYNPROXY target (introduced in Linux v3.12) and the new Nftables framework (introduced in Linux v3.13). At the end a trend is presented, which shows the direction in which the Linux network stack is evolving.
A description of how profilers work on modern operating systems.
A frequently occurring idea for IP tunneling applications is to run a protocol like PPP, which encapsulates IP packets in a format suited for a stream transport (like a modem line), over a TCP-based connection. This would be an easy solution for encrypting tunnels […] and also an easy way to compress arbitrary IP traffic, while datagram based compression has hard to overcome efficiency limits.
Unfortunately, it doesn’t work well. Long delays and frequent connection aborts are to be expected. Here is why.
On Linux 3.17, the Linux kernel has gotten an implementation of “relativistic hash tables” that can be resized while lookups proceed concurrently
"Relativistic" refers to the fact that the relative timing of two events (hash table insertions, say) that are not causally related may appear different to independent observers. In other words, one CPU may see two items inserted into the table in one order, while those insertions appear to have happened in the opposite order on another CPU. Despite some interesting performance results, this technique did not find its way into the kernel until the 3.17 merge window opened
There is a nice introduction in Linux Weekly.
In this paper, we explore the performance of traditional virtual machine deployments, and contrast them with the use of Linux containers. We use a suite of workloads that stress the CPU, memory, storage and networking resources.
Our results show that containers result in equal or better performance than VM in almost all cases. Both VMs and containers require tuning to support I/O-intensive applications. We also discuss the implications of our performance results for future cloud architecture.
The results obtained in this papers are far from surprising, but it is good to see it verified by someone like IBM…
An introduction to netlink and details on Tenus, a networking library in golang.
SockMi is a mechanism for the migration of a TCP/IP connection between two Linux systems. Only the migrating peer of the connection needs to reside on a Linux system. The migration is completely transparent for the other peer that can reside on a system running any operating system. SockMi requires neither changes to existing Linux kernel data structures nor new system calls.
Shmoocon 2013 - C10M Defending The Internet At Scale (by theprez98)
Now that we have the C10K concurrent connection problem licked, how do we level up and support 10 million concurrent connections? Impossible you say. Nope, systems right now are delivering 10 million concurrent connections using techniques that are as radical as they may be unfamiliar.