Try to build the Linux kernel on your own (it's easy) and you'll find hundreds of different algorithms for all things networking and TCP. There isn't a global best algorithm or setting. Depending on whether you are a server or a client, whether you are on a connection that offers high banwidth, low latency, both or none, depending on the application (streaming, torrenting, VoIP), some configuration will be better than another.
If I recall correctly, delayed acks and tcp delay are there to provide higher throughput and prevent network congestion —think downloading from an ftp server as fast as possible, without causing trouble to the rest of the channel's tenants.
The solution provided works very well for a specific problem, HAProxy talking to a backend application, but would probably make things worse if you were to use it for your laptop from your shared home connection.
29athrowaway 1274 days ago [-]
Transfer Control Protocol (TCP) provides solutions for many problems affecting data transmission.
The most important aspect, reliability, is addressed via acknowledgments and retransmissions, and shapes the "TCP state machine" (insert diagram here).
Congestion control is another one. One mechanism of congestion control in TCP is the Nagle's algorithm.
Nagle’s Syndrome. This came up in a HN post a few days ago and seemingly every week.
There should be posters made and widely distributed saying if you experience Unexpected 40ms delays in your application read up on the Nagle algorithm and TCP_NODELAY
tleb_ 1273 days ago [-]
The issue isn't Nagle's algorithm but the delayed ACK & Nagle's algorithm combination.
I’m always curious how someone gains this lower level knowledge. It seems there isn’t much substitute to actually implementing a reference yourself or keeping an ear to ground for blogs like this.
Any other good resources to learn low-level (Linux) networking? I would love to know what conntrack actually does rather than just raising the limit in my applications when I see a special log line.
harry8 1273 days ago [-]
The late Richard W. Stevens wrote the classic texts. TCP/IP illustrated and Unix Network Programming.
I have old editions and haven't seen recent ones. There's murmourings that the updated editions are not as good, the writers emplyed to updated them don't have Steven's talents, can't confirm.
Advanced Programming in the Unix Environment by the same author is also worthwhile IMHO.
And absolutely, use the books to write your own webserver (or irc client or ...) and your own TCP/IP stack if you can find the time and motiviation you won't regret it. It's liberating to do it half-assed, doesn't have to be good at all. Improve it as much as you care to.
edit: Julia Evans is inspiring. See how she goes about it. We should something like what she does when things crop up we don't fully understand. Don't have to write them up like she does, but why not? Feynman technique, if you can explain it simply to the person you were before you learned it you understand it.
mr_toad 1273 days ago [-]
> Unix Network Programming.
If you want to learn to send working TCP/IP packets across the internet with just plain hand crafted C code and no networking frameworks or libraries this is a must read.
rramadass 1273 days ago [-]
I highly suggest reading at least one book on Computer Networking and my recommendation is for An Engineering Approach to Computer Networking: ATM Networks, the Internet, and the Telephone Network. You get a detailed overview of all the important network technologies.
Next for TCP/IP; i suggest The TCP/IP Guide: A Comprehensive, Illustrated Internet Protocols Reference followed by the Stevens' books already mentioned by others.
Finally to understand the interaction between HTTP application protocol and the underlying TCP/IP suite i suggest Web Protocols and Practice: Http/1.1, Networking Protocols, Caching, and Traffic Measurement.
Only after you have some idea of the above should you look at the Linux specific implementation for which Understanding Linux Network Internals: Guided Tour to Networking on Linux is a good reference.
I find it hard to get a good single source on Linux networking as the situation tends to change fast enough to make big efforts into documenting it out of date just as they're finished. You can learn a lot by messing around at home though if you have a couple spare machines. These days it might be a better choice to focus on software-defined networking, but so much of SDN is proprietary that it's hard to feel great about learning it.
toast0 1273 days ago [-]
Read the books and the blogs (as time permits), look at tcpdump and wireshark when clients do weird stuff; then try to figure out why, and how you can fix it, when you can only control one side of the connection. Try a bunch of stuff.
Conntrack is pretty simple though; if you've configured your firewall rules for low performance, Xtables keeps track of each of the active connections, and there's a limit. If you hit the limit, it won't track anymore connections and you're stuck. Try to configure stateless firewall rules instead :)
heyoni 1273 days ago [-]
Conntrack is Linux specific. Are you aware of any macOS analogs?
toast0 1273 days ago [-]
MacOS supports pf (search for OpenBSD pf) and ipfw firewalls, both of which can do stateful firewalls.
Be advised, the Mac tcp stack is mostly taken from 2001 FreeBSD, with MPTCP added later, maybe they put in path mtu probing changes from after the initial import, not sure. It was taken before the syncache and syncookies, and definitely doesn't have any of the other major scalability improvements FreeBSD has made over time in the last 20 years. Parts of the firewall code are broken (or were broken in past releases), and there's not really any body of internet knowledge about what parts actually work.
I guess that makes it a good system to play on, because if you try something and it doesn't work, it's a good idea to go spelunking through the code, because there's a good chance it's broken. OTOH, it'll be depressing to see how long it's been broken for, how long it's been fixed upstream, and how likely it is to be fixed in future mac os releases (probably never).
All that said, Apple's TCP stack does better on iOS than Android, because Apple enabled and tweaked path MTU probing, so if you're on a dumb network with MTU set too large and ICMP blocked, iOS send smaller packets pretty quickly and your connections will work, Android ships with probing disabled, so it just sends big packets that get dropped until the connection times out.
Lammy 1273 days ago [-]
> (Linux) networking
Suggestion: don't think of it like this. It's just networking. The thing that worked best for me was just the trial by fire of getting a cheap managed switch (a fanless Dell PowerConnect 2816) and kicking myself offline until I learned to configure it :)
fosk 1274 days ago [-]
Worth nothing that TCP_NODELAY is on by default on NGINX[1].
Tcp is a transport layer protocol on top of ip. So the tcp packet with stuff like sequence and port numbers in its header is encapsulated as the data part of an ip packet, which in turn is send via lower level packets like ethernet frames.
Rendered at 14:56:11 GMT+0000 (Coordinated Universal Time) with Vercel.
If I recall correctly, delayed acks and tcp delay are there to provide higher throughput and prevent network congestion —think downloading from an ftp server as fast as possible, without causing trouble to the rest of the channel's tenants.
The solution provided works very well for a specific problem, HAProxy talking to a backend application, but would probably make things worse if you were to use it for your laptop from your shared home connection.
The most important aspect, reliability, is addressed via acknowledgments and retransmissions, and shapes the "TCP state machine" (insert diagram here).
Congestion control is another one. One mechanism of congestion control in TCP is the Nagle's algorithm.
https://en.wikipedia.org/wiki/Nagle%27s_algorithm
Disabling Nagle's algorithm is usually done via the TCP_NODELAY flag.
If you want to learn more about networking I recommend:
- http://williamstallings.com/DataComm/
- https://beej.us/guide/bgnet/
There should be posters made and widely distributed saying if you experience Unexpected 40ms delays in your application read up on the Nagle algorithm and TCP_NODELAY
https://news.ycombinator.com/item?id=24786196
Any other good resources to learn low-level (Linux) networking? I would love to know what conntrack actually does rather than just raising the limit in my applications when I see a special log line.
I have old editions and haven't seen recent ones. There's murmourings that the updated editions are not as good, the writers emplyed to updated them don't have Steven's talents, can't confirm.
Advanced Programming in the Unix Environment by the same author is also worthwhile IMHO.
And absolutely, use the books to write your own webserver (or irc client or ...) and your own TCP/IP stack if you can find the time and motiviation you won't regret it. It's liberating to do it half-assed, doesn't have to be good at all. Improve it as much as you care to.
edit: Julia Evans is inspiring. See how she goes about it. We should something like what she does when things crop up we don't fully understand. Don't have to write them up like she does, but why not? Feynman technique, if you can explain it simply to the person you were before you learned it you understand it.
If you want to learn to send working TCP/IP packets across the internet with just plain hand crafted C code and no networking frameworks or libraries this is a must read.
Next for TCP/IP; i suggest The TCP/IP Guide: A Comprehensive, Illustrated Internet Protocols Reference followed by the Stevens' books already mentioned by others.
Finally to understand the interaction between HTTP application protocol and the underlying TCP/IP suite i suggest Web Protocols and Practice: Http/1.1, Networking Protocols, Caching, and Traffic Measurement.
Only after you have some idea of the above should you look at the Linux specific implementation for which Understanding Linux Network Internals: Guided Tour to Networking on Linux is a good reference.
I find it hard to get a good single source on Linux networking as the situation tends to change fast enough to make big efforts into documenting it out of date just as they're finished. You can learn a lot by messing around at home though if you have a couple spare machines. These days it might be a better choice to focus on software-defined networking, but so much of SDN is proprietary that it's hard to feel great about learning it.
Conntrack is pretty simple though; if you've configured your firewall rules for low performance, Xtables keeps track of each of the active connections, and there's a limit. If you hit the limit, it won't track anymore connections and you're stuck. Try to configure stateless firewall rules instead :)
Be advised, the Mac tcp stack is mostly taken from 2001 FreeBSD, with MPTCP added later, maybe they put in path mtu probing changes from after the initial import, not sure. It was taken before the syncache and syncookies, and definitely doesn't have any of the other major scalability improvements FreeBSD has made over time in the last 20 years. Parts of the firewall code are broken (or were broken in past releases), and there's not really any body of internet knowledge about what parts actually work.
I guess that makes it a good system to play on, because if you try something and it doesn't work, it's a good idea to go spelunking through the code, because there's a good chance it's broken. OTOH, it'll be depressing to see how long it's been broken for, how long it's been fixed upstream, and how likely it is to be fixed in future mac os releases (probably never).
All that said, Apple's TCP stack does better on iOS than Android, because Apple enabled and tweaked path MTU probing, so if you're on a dumb network with MTU set too large and ICMP blocked, iOS send smaller packets pretty quickly and your connections will work, Android ships with probing disabled, so it just sends big packets that get dropped until the connection times out.
Suggestion: don't think of it like this. It's just networking. The thing that worked best for me was just the trial by fire of getting a cheap managed switch (a fanless Dell PowerConnect 2816) and kicking myself offline until I learned to configure it :)
[1] http://nginx.org/en/docs/http/ngx_http_core_module.html#tcp_...
https://en.wikipedia.org/wiki/File%3AUDP_encapsulation.svg
From: https://en.wikipedia.org/wiki/Internet_protocol_suite?wprov=...