A brief incident of 2005
One day I was interacting with some really junior rookie sys admin guy and he was clueless about IP subnets worked.
I got so frustrated that same evening I sent a routing tutorial to the Linux user group. Since then I have been involved in several networking projects that are of non trivial nature.
How does IP addressing do its job?
First of all what is IP?
IP is short for Internet protocol. We must understand how IP addresses work before we can understand anything about packets and networking.
A typical IP address has 4 parts, say 44.33.22.11.
It has 4 octets.
44
33
22
11
IP addresses come with a subnet mask as well. Usually in /24 or /18 or something like that.
Or it could be in hex like 255.240.0.0.
IP addresses are at the core of the entire Internet packet and all the TCP/IP protocols that make the Internet work and carry your media like youtube videos and podcasts and what not.
An IP address has the ability to uniquely identify your location and it participates in the Internet packet routing topology by identifying its presence.
Actually the BGP routing protocol advertises a BGP prefix for each ISP and that is what tells every other node on Internet to reach you. But suffice it to say that the host address along with the network prefix or subnet tell the world about how many hosts are present in your network and which upstream router can talk to you.
What is routing?
Routing is very closely related to addressing in the sense that packets can be sent from one location to another only by knowing the network and the next hop gateway.
Think of it like this. The network that hosts the target node is away by say 20 hops. it has to cross 20 routers to reach the target network. But how does the source node send the packet? How does it know the route?
Actually it is not necessary. All you need to do is to know if the target host belongs to our network or not. If not, then the packet is sent to the default gateway.
This is the common case with simple networks. In the case of core routers with access to multiple networks(real physical interfaces) then you can send the packets through any of the networks based on the prefix.
So how it works is by going from broader to narrower prefix. In other words you can send to the state and then district and finally the street. Think of it like home addresses on planet earth.
What is IP in TCP/IP?
Now let us talk about the importance of IP and the routing protocols.
Routing protocols establish the topology and construct a graph of connected networks. Common routing protocols are OSPF and BGP.
All this happens in the IP or network layer of the TCP/IP stack. Without a routing table properly populated you cannot forward packets to the destination correctly.
You also need to take into account the network topology changes, outages and new networks coming and going.
Just keep in mind that routing protocols work independent of the forwarding algorithms. They meet in the routing table but otherwise they are quite separate.
What are subnets? What is CIDR?
A subnet is nothing but part of a big network. The common example for the subnets are the RFC1918 private networks, 192.168.0.0/16 or 10.0.0.0/8.
The subnet clearly defines the range of IP addresses that are valid for a particular network. Since Internet works on the principle of address prefixes subnets that are large have shorter prefixes. Every host has a host part and a network part.
CIDR stands for classless Internet Domain Routing which dropped the Class A, B and C networks that was initially theorized in the infancy of the web.
Nowadays all you need to do to participate in the Internet revolution is just catch hold of an ISP and buy an IP address. These protocols that work at the backbone of the Internet have evolved to make things really straight forward for us network engineers.
But still the fact remains that as programmers it is our job to learn how things work in order to troubleshoot as eventually you are going to get into a routing blackhole or asymmetric traffic or latency or packet dropping or IP fragmentation error or firewall blocking our traffic or some such issue.
You could also get into some NAT issue with p2p traffic.
How can network packets get from one location to other?
In addition to seeing how the Internet network of networks work with each other we also saw that packets get from point A to point B using intermediate hops.
Each core router has its own directly connected networks and a default router for those that do not match the prefixes.
If you run a traceroute or tcptraceroute command you can easily trace the hops your packet takes from one place to another.
What are delay tolerant networks?
Ok so the packets are able to easily get from point A to point B.
But what about inter planetary communications? And over links that are unstable and take a long time to communicate? We cannot apply the usual principles of Internet with CDN and high speed low latency traffic there.
Moreover protocols like TCP will fail in that case. Hence UDP which is layer 4 protocol that is mostly similar to IP but much simpler is used.
How can UDP solve problems TCP can’t?
We did not dwell much on TCP but focused only on IP since TCP just adds the concept of flow control, packet sessions and takes care of retransmissions to ensure that your traffic flows from one place to another seamlessly.
But it is very complex and can lead to problems when the assumptions for the mathematics of TCP do not hold good. Hence you need UDP for protocols like NTP, DTN and multicast.
UDP is also very popular with media streams like RTP and friends. If you are doing a voice or video stream live then the delay and latency matters, which is not the case with uploading a file to a Google drive.
So the Internet works as one single unit although there are different protocols doing different things.
Though IP layer takes care of routing , addressing and forwarding there are lot many protocols on top and below that work together for a seamless experience for end users.
Which tools in Linux help you learn networking?
I think the tools tcpdump, netcat, socat and ping will suffice.
hping, p0f and friends will help too.
It wholly depends on your experience with Linux and networking in general.
If you look at a wireshark capture on windows when you are playing a youtube video you will learn a lot about networking. Also when you are sending an email attachment.
A typical Internet use involves a lot of download and some upload but there are situations where gigabytes are uploaded.
You can get a good overview of networking protocols using netflow tools like softflow too. Netflow is a protocol by Cisco later made open to track packet flows. You can quite easily gain insight into a local network’s traffic patterns using just this tool alone.
Answer to last week’s question
How do you validate a json file to see if it parses properly?
You can use jq or json_verify.
json_verify is part of yajl-tools package. We have spoken about jq in one of the previous newsletters. I shall cover it again on a future date since it is a cool tool.
This week’s question
What is the purpose of sessions layer in OSI 7 layer model?
Feedback/questions
If you need me to cover any particular topic of interest let me know and I can gladly oblige. If you wish to simply say Hi that is fine too.