Free lesson · Computer Networking

The Internet A 10000 Foot View

Easy10 min readnetworkingcomputer-networkstcp-ip

The Internet Is Not a Cloud

The word "cloud" is so deeply embedded in the vocabulary of modern software engineering that it is easy to forget what the internet actually, physically is. It is not an abstraction floating in the sky. It is a planet-spanning physical infrastructure of fiber optic cables, copper wires, radio towers, satellite links, data centers, and networking equipment - all owned and operated by thousands of independent organizations that agree to exchange traffic with each other using common protocols.

When you send a request from your laptop in New York to a server in Tokyo, your data does not teleport. It travels as pulses of light through glass fibers under the Atlantic Ocean, across Europe, through the Middle East or Central Asia, under the Pacific, and into a data center in Japan. Alternatively, it might cross the Pacific directly via a submarine cable from the US West Coast. The path depends on routing decisions made by dozens of independent network operators along the way.

Understanding the physical reality of the internet prevents a category of naive assumptions that cause real engineering mistakes - assumptions like "any server can reach any other server instantly," "bandwidth is unlimited," or "the network is always reliable." The internet is fast, capable, and remarkably resilient, but it is a physical system with physical constraints. This topic gives you the 10,000-foot view of that system.

The Internet Is a Network of Networks

The internet is not one big network. It is a cooperative system where thousands of independently operated networks - each owned by different organizations - agree to interconnect and exchange traffic. This architecture is what makes the internet both incredibly resilient and surprisingly fragile.

Internet Service Providers (ISPs)

ISPs are the organizations that provide internet connectivity to end users and businesses. They come in tiers based on the size and scope of their networks:

Tier 1 ISPs own massive backbone networks that span continents. Companies like Lumen (formerly CenturyLink/Level 3), NTT, Telia Carrier, and Cogent operate the internet's long-distance highways. Tier 1 ISPs have a defining characteristic: they can reach every destination on the internet without paying any other network for transit. They achieve this through peering agreements with other Tier 1 networks - mutual agreements to exchange traffic for free because both networks benefit equally.

Tier 2 ISPs are large regional or national networks that buy transit from Tier 1 providers for destinations they cannot reach through their own peering. Companies like Comcast, Vodafone, and Deutsche Telekom are Tier 2 ISPs. They have their own significant network infrastructure but need Tier 1 connectivity for global reach.

Tier 3 ISPs are local providers that buy transit from Tier 2 or Tier 1 networks. Your local cable company or DSL provider is typically a Tier 3 ISP. They serve the "last mile" - the connection between the broader internet and your home or office.

This tiered structure means your data passes through multiple ISP networks on its way to a destination. When you access a website hosted in a distant data center, your request might travel from your Tier 3 ISP to a Tier 2 ISP to a Tier 1 backbone, across the backbone, down to another Tier 2 ISP, and finally to the data center's network - each hop handled by a different organization's equipment.

Internet Exchange Points (IXPs)

ISPs exchange traffic with each other at Internet Exchange Points - physical locations where multiple networks bring their cables together and connect their routers. An IXP is essentially a building (or a floor within a building) full of networking equipment where ISPs plug in and establish peering relationships.

Major IXPs include DE-CIX in Frankfurt (the world's largest by traffic volume, handling over 14 Tbps of peak traffic), AMS-IX in Amsterdam, LINX in London, and Equinix's various exchange facilities worldwide. These IXPs are critical infrastructure - if a major IXP went offline, traffic that normally takes an efficient direct path between two ISPs would have to take longer, less efficient routes, increasing latency and potentially causing congestion.

The existence of IXPs is why the internet works as well as it does. Without them, every ISP would need a separate physical connection to every other ISP - an impossibly complex mesh. IXPs reduce this to a hub model where many networks connect at a shared facility and establish peering through simple cross-connects.

graph TD
    subgraph Backbone["Internet Backbone"]
        T1A["Tier 1 ISP<br/>(Lumen)"]
        T1B["Tier 1 ISP<br/>(NTT)"]
        IXP["Internet<br/>Exchange Point"]
        T1A --- IXP
        T1B --- IXP
    end

    subgraph Regional["Regional Networks"]
        T2A["Tier 2 ISP<br/>(Comcast)"]
        T2B["Tier 2 ISP<br/>(Vodafone)"]
    end

    subgraph LastMile["Last Mile"]
        T3A["Tier 3 ISP<br/>(Local Cable)"]
        HOME["Your Home<br/>Router"]
    end

    subgraph DataCenter["Cloud / Data Center"]
        CDN["CDN Edge<br/>(Cloudflare)"]
        AWS["Cloud Provider<br/>(AWS)"]
    end

    T2A --- T1A
    T2B --- T1B
    T3A --- T2A
    HOME --- T3A
    CDN --- IXP
    AWS --- T1B

    style IXP fill:#4f46e5,stroke:#4338ca,color:#fff
    style T1A fill:#7c3aed,stroke:#6d28d9,color:#fff
    style T1B fill:#7c3aed,stroke:#6d28d9,color:#fff
    style T2A fill:#8b5cf6,stroke:#7c3aed,color:#fff
    style T2B fill:#8b5cf6,stroke:#7c3aed,color:#fff
    style T3A fill:#a78bfa,stroke:#8b5cf6,color:#fff
    style CDN fill:#4f46e5,stroke:#4338ca,color:#fff
    style AWS fill:#4f46e5,stroke:#4338ca,color:#fff

This diagram shows the hierarchical structure of the internet. At the top, two Tier 1 ISPs (Lumen and NTT) connect through an Internet Exchange Point. Below them, Tier 2 ISPs (Comcast and Vodafone) connect to the Tier 1 providers. A Tier 3 local cable ISP connects to Comcast, and your home router connects to the Tier 3 ISP. On the other side, a CDN edge node connects directly to the Internet Exchange Point, and a cloud provider connects to a Tier 1 ISP. Data from your home traverses multiple networks to reach its destination.

The Physical Infrastructure

Submarine Cables

Approximately 95% of intercontinental data travels through submarine fiber optic cables on the ocean floor. These cables are about the diameter of a garden hose, yet they carry the vast majority of global internet traffic. There are currently over 500 active submarine cable systems worldwide, with a total length exceeding 1.4 million kilometers - enough to circle the Earth over 35 times.

Submarine cables are laid by specialized ships that unspool cable from enormous drums as they cross the ocean. In shallow coastal waters, cables are buried in the seabed using water-jet plows to protect them from fishing trawlers and anchors (the two most common causes of cable damage). In deep ocean, cables simply rest on the seabed, where they are largely safe from human activity.

Major tech companies including Google, Meta, Microsoft, and Amazon now invest billions in private submarine cables to ensure they have sufficient bandwidth between their global data center regions. Google's Dunant cable, for example, connects the US East Coast to France with a capacity of 250 Tbps - enough to carry the entire global internet traffic of 2015 through a single cable.

Data Centers

Data centers are the warehouses of the internet. They house the servers, storage systems, and networking equipment that run the websites, applications, and services you use daily. A large cloud data center can contain hundreds of thousands of servers consuming hundreds of megawatts of electricity.

Data centers are strategically located based on several factors: proximity to users (lower latency), proximity to cheap electricity (lower operating cost), fiber connectivity (access to multiple ISP networks), geographic stability (low risk of natural disasters), and political and legal considerations (data sovereignty laws).

The major cloud providers operate data centers in dozens of regions worldwide. AWS has over 30 geographic regions. Azure and GCP have similar global footprints. This geographic distribution is what enables low-latency access for users worldwide - your request is typically served by a data center within a few hundred kilometers of you rather than one on the other side of the planet.

Content Delivery Networks (CDNs)

CDNs place copies of popular content at edge nodes distributed around the world, close to end users. When you load a website that uses a CDN (and most large websites do), static assets like images, CSS files, JavaScript files, and even cached API responses are served from an edge node near you rather than from the origin server that might be thousands of kilometers away.

Companies like Cloudflare, Akamai, Fastly, and Amazon CloudFront operate CDN networks with thousands of edge nodes in hundreds of cities worldwide. These edge nodes are often physically located inside ISP facilities or Internet Exchange Points - as close to the end user as possible. This placement minimizes the number of network hops between the user and the content, reducing latency to single-digit milliseconds for cached content.

CDNs are one of the most important networking concepts for software engineers because they fundamentally change the performance characteristics of web applications. Without a CDN, every user request travels to the origin server, which might be 10,000 km away. With a CDN, most requests are served from an edge node 100 km away. The difference in user-perceived latency is dramatic.

How Data Gets from Here to There

When you access a website, your data does not travel in a straight line. It follows a path determined by routing - the process by which routers at each hop decide which direction to forward your data. The path is dynamic and can change from second to second based on network conditions.

Routing in Brief

Every router on the internet maintains a routing table - a map that says "to reach network X, send the data to router Y." Routers exchange information about which networks they can reach using routing protocols (BGP - Border Gateway Protocol - is the primary one used between ISP networks). When a router receives a packet, it reads the destination IP address, consults its routing table, and forwards the packet to the next hop.

The path your data takes is not necessarily the shortest geographic path. It is the path that the routing protocols determine is optimal based on factors like network capacity, current load, peering agreements, and business relationships between ISPs. Your data from New York to London might go through Chicago and across the Atlantic via a submarine cable landing in Cornwall - a geographic detour that turns out to be the fastest available path because of the network topology.

Redundancy and Resilience

The internet was designed to survive partial failures. If a submarine cable is cut, traffic reroutes through other cables. If a router fails, neighboring routers detect the failure and update their routing tables within seconds. If an entire data center goes offline, traffic shifts to other data centers through DNS or anycast routing.

This resilience is not perfect. Major cable cuts have caused significant internet disruptions in regions served by few cables. BGP misconfigurations (where a network accidentally advertises incorrect routing information) have caused global outages. Individual ISP failures can isolate entire countries or regions.

But the overall architecture is remarkably robust. The internet has survived natural disasters, undersea earthquakes, and even deliberate attacks without a global outage. This resilience comes from redundancy at every layer - multiple cables, multiple paths, multiple DNS servers, multiple data centers - combined with protocols designed to detect and route around failures automatically.

graph LR
    subgraph Physical["Physical Internet Infrastructure"]
        direction LR
        USER["User Device"]
        CELL["Cell Tower /<br/>Home Router"]
        ISP["ISP<br/>Network"]
        IXP2["Internet<br/>Exchange"]
        CABLE["Submarine<br/>Cable"]
        RISP["Remote<br/>ISP"]
        DC["Data<br/>Center"]
    end

    USER --> CELL --> ISP --> IXP2 --> CABLE --> RISP --> DC

    style USER fill:#c4b5fd,stroke:#a78bfa,color:#000
    style CELL fill:#a78bfa,stroke:#8b5cf6,color:#fff
    style ISP fill:#8b5cf6,stroke:#7c3aed,color:#fff
    style IXP2 fill:#7c3aed,stroke:#6d28d9,color:#fff
    style CABLE fill:#4f46e5,stroke:#4338ca,color:#fff
    style RISP fill:#7c3aed,stroke:#6d28d9,color:#fff
    style DC fill:#4f46e5,stroke:#4338ca,color:#fff

This diagram shows the physical path that data takes from a user to a remote data center. Starting from the user's device, data goes to a cell tower or home router, then to the ISP network, through an Internet Exchange Point, across a submarine cable, to a remote ISP on the other side, and finally to the destination data center. Each step represents physical infrastructure with real cables, routers, and switching equipment.

Who Runs the Internet?

A common question with a surprising answer: nobody runs the internet. There is no single organization, government, or company in charge. Instead, the internet is governed by a loose collection of organizations that manage different aspects:

IANA (Internet Assigned Numbers Authority) and ICANN (Internet Corporation for Assigned Names and Numbers) manage the global allocation of IP addresses and the DNS root zone. They delegate IP address blocks to regional registries (ARIN for North America, RIPE NCC for Europe, APNIC for Asia-Pacific, AFRINIC for Africa, LACNIC for Latin America), which in turn allocate blocks to ISPs and organizations.

The IETF (Internet Engineering Task Force) develops and maintains the protocols that make the internet work - TCP, IP, HTTP, DNS, TLS, and hundreds of others. The IETF publishes its specifications as RFCs (Requests for Comments), which are the authoritative technical documents for internet protocols.

Individual ISPs and network operators make their own decisions about how to route traffic, which networks to peer with, and how much capacity to build. No central authority tells Comcast how to route your traffic or requires AT&T to peer with a particular network.

This decentralized governance model is both the internet's greatest strength (no single point of failure, no single entity can shut it down) and its greatest weakness (no single entity can fix it, coordinate upgrades, or enforce quality standards). The internet works because enough organizations choose to cooperate - not because anyone requires them to.

Interview: The physical architecture of the internet comes up in System Design interviews more than you might expect:

"Why is a CDN necessary?" - Because the physical distance between users and servers introduces latency that no software optimization can eliminate. Light in fiber travels at about 200,000 km/s, so a round trip from New York to Tokyo (about 11,000 km of cable path) takes at minimum ~110 ms - plus processing time at every router along the way.

"How would you design a globally distributed system?" - Understanding that the internet is a network of networks with variable latency between regions helps you make concrete decisions about data replication, consistency trade-offs, and where to place services.

"What happens when an AWS region goes down?" - Understanding the physical reality (data centers, power, networking) helps you design multi-region architectures that survive regional failures.

What Comes Next

Now that you understand what the internet physically is - cables, routers, data centers, ISPs cooperating to exchange traffic - the next topic walks through what happens during one of the most common internet interactions: typing a URL into your browser and pressing Enter. This end-to-end walkthrough ties together every layer of the network stack and serves as a roadmap for the rest of the course.

Exercise: Starter: Visit a submarine cable map (search for "submarine cable map" - TeleGeography maintains an excellent interactive one). Find the cables that connect your continent to other continents. How many cables connect North America to Europe? How many connect to Asia? Are there any single points of failure where a cable cut could isolate a region?

Intermediate: Run traceroute (or tracert on Windows) to a server on a different continent - try traceroute bbc.co.uk from the US, or traceroute cnn.com from Europe. Examine the hostnames of the routers along the path. Can you identify when your data crosses from one ISP to another? Can you identify the approximate geographic location of each hop from the hostname?

Advanced: Research BGP (Border Gateway Protocol) hijacking incidents. In 2018, traffic destined for Amazon's Route 53 DNS service was rerouted through a Russian ISP for two hours. In 2008, Pakistan Telecom accidentally took YouTube offline worldwide by announcing incorrect BGP routes. How do BGP hijacks work at a high level, and what mechanisms (RPKI, BGPsec) are being deployed to prevent them? Why has adoption been slow?

Back to Computer Networking