In my recent update to the Internet Map, I expanded the BGP layer on map.kmcd.dev to show all attributed IP address space globally instead of just the largest hubs.
On the map, blue dots are cities with public Internet Exchange Points (IXPs) and purple dots are cities that only announce BGP IP space. Once I put all of the smaller cities on the globe, I noticed a lot of big purple dots sitting right on top of major fiber lines with millions of IPs and no local IXP.
Blue IXP cities and purple BGP-only cities on map.kmcd.dev.
Even if you have gigabit fiber to your apartment, living in a city without an IXP affects how your traffic is routed.
Trombone Routing
If you and your neighbor across the street are on two different ISPs, your fiber cables probably run through the same conduit under the road. If those two ISPs don’t have a place to connect in your city, your packets have to leave town to reach your neighbor, go to another city where both ISPs peer, and come back. This is called trombone routing:
Without a local exchange, traffic between two Odense networks takes a detour to Copenhagen and back.
Take two users in Odense whose ISPs only peer in Copenhagen. For their packets to cross the street, they have to leave the island of Funen, cross over to Zealand, hit an exchange in Copenhagen, and turn right back around.
With a local IXP in Odense, both ISPs can exchange local routes directly in town:
With a local IXP in Odense, the traffic stays in town.
Same Fiber, Different Paths
You can see this in latency measurements from RIPE Atlas and M-Lab if you compare cities in the same region that use the same ISPs:
| Cities | With IXP | Without IXP | Detour |
|---|---|---|---|
| Copenhagen / Odense | 1.5–3 ms | 8.5–13 ms | Copenhagen |
| Columbus / Cincinnati | 2.5–4.5 ms | 18–26 ms | Chicago / Ashburn |
| Calgary / Saskatoon | 2.5–5 ms | 22–34 ms | Seattle / Winnipeg |
| Stuttgart / Karlsruhe | 1.8–3 ms | 11–15 ms | Frankfurt |
| Lyon / Clermont-Ferrand | 2–3.5 ms | 14–19 ms | Paris |
I live in Copenhagen, where the major Danish ISPs peer with each other at DIX and Netnod. In Odense (165 km west of Copenhagen), those same ISPs carry their regional traffic back to Copenhagen to peer, so packets between two ISPs in Odense cross the Great Belt bridge to Copenhagen and back. You can see the same thing in Ohio, where Columbus has a local exchange (Ohio IX) and Cincinnati routes cross-ISP traffic through Chicago or Ashburn. And in Calgary, local traffic used to go through Seattle, Chicago, or Toronto at ~30 ms until the city set up YYCIX and brought local pings under 5 ms.
Empty IXPs and Private Cables
The blue and purple dots on the map don’t show everything, since PeeringDB only lists public exchanges.
Some cities have a blue IXP dot on the map, but the big residential cable and phone companies in town don’t connect to it. Large providers often require a network to meet minimum traffic levels and connect with them in several major cities before they will peer for free, especially if that smaller network would otherwise have to pay them for transit. Often the local exchange is only used by a university and a few small regional ISPs, and the big national providers still route all of their home customers through Chicago, Ashburn, or Frankfurt.
Other cities only show up in purple on the map because the big networks there connect privately. When an ISP and a CDN have routers in the same building, they can plug a fiber cable directly between them using a Private Network Interconnect (PNI). Since PNIs are private agreements and cables between two companies, they aren’t in PeeringDB and don’t show up on my map.
Check Your City on the Map
The cable layer on the map shows where the physical fiber goes, and the IXP and BGP layers show where networks actually connect to each other. You can check your own city on map.kmcd.dev to see whether it shows up in blue or purple.