Internet Topology

COS 461: Computer Networks Jennifer Rexford

Goals of Todays Lecture

Internets two-tiered topology
Autonomous Systems, and connections between them Routers, and the links between them

AS-level topology
Autonomous System (AS) numbers Business relationships between ASes

Router-level topology
Points of Presence (PoPs) Backbone and enterprise network topologies

Inferring network topologies

By measuring paths from many vantage points
2

Internet Routing Architecture

Divided into Autonomous Systems
Distinct regions of administrative control Routers/links managed by a single institution Service provider, company, university,

Hierarchy of Autonomous Systems

Large, tier-1 provider with a nationwide backbone Medium-sized regional provider with smaller backbone Small network run by a single company or university

Interaction between Autonomous Systems

Internal topology is not shared between ASes but, neighboring ASes interact to coordinate routing
3

Autonomous System Numbers

AS Numbers are 16 bit values.

Currently just over 20,000 in use.

Level 3: 1 MIT: 3 Harvard: 11 Yale: 29 Princeton: 88 AT&T: 7018, 6341, 5074, UUNET: 701, 702, 284, 12199, Sprint: 1239, 1240, 6211, 6242,
4

AS Topology
Node: Autonomous System Edge: Two ASes that connect to each other

4 3 5 2

1
5

What is an Edge, Really?

Edge in the AS graph
At least one connection between two ASes Some destinations reached from one AS via the other

d AS 1

d AS 1

Exchange Point
AS 2 AS 2 AS 3
6

Interdomain Paths
Path: 6, 5, 4, 3, 2, 1
4

3
5

2 7 6

Web server Client

7

Business Relationships
Neighboring ASes have business contracts
How much traffic to carry Which destinations to reach How much money to pay

Common business relationships

Customer-provider
E.g., Princeton is a customer of AT&T E.g., MIT is a customer of Level 3

Peer-peer
E.g., Princeton is a peer of Patriot Media E.g., AT&T is a peer of Sprint
8

Customer-Provider Relationship
Customer needs to be reachable from everyone
Provider tells all neighbors how to reach the customer

Customer does not want to provide transit service

Customer does not let its providers route through it Traffic to the customer
advertisements provider traffic customer

Traffic from the customer d

provider

customer
9

Peer-Peer Relationship
Peers exchange traffic between customers
AS exports only customer routes to a peer AS exports a peers routes only to its customers Often the relationship is settlement-free (i.e., no $$$)

Traffic to/from the peer and its customers

advertisements peer traffic peer

10

Princeton Example
Internet: customer of AT&T and USLEC Research universities/labs: customer of Internet2 Local residences: peer with Patriot Media Local non-profits: provider for several non-profits AT&T USLEC Internet2

peer

Patriot
11

AS Structure: Tier-1 Providers

Tier-1 provider
Has no upstream provider of its own Typically has a national or international backbone UUNET, Sprint, AT&T, Level 3,

Top of the Internet hierarchy of 12-20 ASes

Full peer-peer connections between tier-1 providers

12

Efficient Early-Exit Routing

Customer B

Diverse peering locations

Both costs, and middle

Provider B

Comparable capacity at all peering points

Can handle even load

Consistent routes
multiple peering points Same destinations advertised at all points Same AS path length for a destination at all points

Early-exit routing

Provider A Customer A
13

AS Structure: Other ASes

Tier-2 providers
Provide transit service to downstream customers but, need at least one provider of their own Typically have national or regional scope E.g., Minnesota Regional Network Includes a few thousand of the ASes

Stub ASes
Do not provide transit service to others Connect to one or more upstream providers Includes vast majority (e.g., 85-90%) of the ASes

14

Characteristics of the AS Graph

AS graph structure
High variability in node degree (power law) A few very highly-connected ASes Many ASes have only a few connections

All ASes have 1 or more neighbors

CCDF

0.1
0.01 0.001

Very few have degree >= 100

10

100

1000

AS degree

15

Characteristics of AS Paths
AS path may be longer than shortest AS path Router path may be longer than shortest path 2 AS hops, 8 router hops

s 3 AS hops, 7 router hops

16

Intra-AS Topology
Node: router Edge: link

17

Hub-and-Spoke Topology
Single hub node
Common in enterprise networks Main location and satellite sites Simple design and trivial routing

Problems
Single point of failure Bandwidth limitations High delay between sites Costs to backhaul to hub
18

Princeton Example
Hub-and-spoke
Four hub routers and many spokes

Hub routers
Outside world (e.g., AT&T, USLEC, ) Dorms Academic and administrative buildings Servers

19

Simple Alternatives to Hub-and-Spoke

Dual hub-and-spoke
Higher reliability Higher cost Good building block

Levels of hierarchy
Reduce backhaul cost Aggregate the bandwidth Shorter site-to-site delay

20

Backbone Networks
Backbone networks
Multiple Points-of-Presence (PoPs) Lots of communication between PoPs Accommodate traffic demands and limit delay

21

Abilene Internet2 Backbone

22

Points-of-Presence (PoPs)
Inter-PoP links
Long distances High bandwidth Inter-PoP Intra-PoP

Intra-PoP links
Short cables between racks or floors Aggregated bandwidth

Links to other networks

Wide range of media and bandwidth

Other networks

23

Where to Locate Nodes and Links

Placing Points-of-Presence (PoPs)
Large population of potential customers Other providers or exchange points Cost and availability of real-estate Mostly in major metropolitan areas

Placing links between PoPs

Already fiber in the ground Needed to limit propagation delay Needed to handle the traffic load
24

Customer Connecting to a Provider

Provider Provider 2 access links

1 access link

Provider

Provider

2 access routers

2 access PoPs
25

Multi-Homing: Two or More Providers

Motivations for multi-homing
Extra reliability, survive single ISP failure Financial leverage through competition Better performance by selecting better path Gaming the 95th-percentile billing model

Provider 1

Provider 2

26

Shared Risks
Co-location facilities (co-lo hotels)
Places ISPs meet to connect to each other and co-locate their routers, and share space & power E.g., 32 Avenue of the Americas in NYC

Shared links
Fiber is sometimes leased by one institution to another Multiple fibers run through the same conduits and run through the same tunnels, bridges, etc.

Difficult to identify and accounts for these risks

Not visible in network-layer measurements E.g., traceroute does not tell you links in the same ditch
27

Learning the Internet Topology

Internet does not have any central management
No public record of the AS-level topology No public record of the intra-AS topologies

Some public topologies are available

Maps on public Web sites E.g., Abilene Internet2 backbone

Otherwise, you have to infer the topology

Measure many paths from many vantage points Extract the nodes and edges from the paths Infer the relationships between neighboring ASes
28

Inferring an Intra-AS Topology

Run traceroute from many vantage points
Learn the paths running through an AS Extract the hops within the AS of interest
1 169.229.62.1 2 169.229.59.225 3 128.32.255.169 4 128.32.0.249 5 128.32.0.66 7 209.247.9.170 8 66.185.138.33 9 66.185.142.97 10 66.185.136.17 11 64.236.16.52 inr-daedalus-0.CS.Berkeley.EDU soda-cr-1-1-soda-br-6-2 vlan242.inr-202-doecev.Berkeley.EDU gigE6-0-0.inr-666-doecev.Berkeley.EDU qsv-juniper--ucb-gw.calren2.net pos8-0.hsa2.Atlanta2.Level3.net pop2-atm-P0-2.atdn.net Pop1-atl-P3-0.atdn.net pop1-atl-P4-0.atdn.net www4.cnn.com
29

6 209.247.159.109 POS1-0.hsipaccess1.SanJose1.Level3.net

AOL

Challenges of Intra-AS Mapping

Firewalls at the network edge
Cannot typically map inside another stub AS because the probe packets will be blocked by firewall So, typically used only to study service providers

Identifying the hops within a particular AS

Relies on addressing and DNS naming conventions Difficult to identify the boundaries between ASes

Seeing enough of the edges

Need to measure from a large number of vantage points And, hope that the topology and routing doesnt change
30

Inferring the AS-Level Topology

Collect AS paths from many vantage points
Learn a large number of AS paths Extract the nodes and the edges from the path

Example: AS path 1 7018 88 implies

Nodes: 1, 7018, and 88 Edges: (1, 7018) and (7018, 88)

Ways to collect AS paths from many places

Mapping traceroute data to the AS level Measurements of the interdomain routing protocol

31

Map Traceroute Hops to ASes

Traceroute output: (hop number, IP)
1 169.229.62.1 AS25 Berkeley 2 169.229.59.225 AS25

3 128.32.255.169 AS25
4 128.32.0.249 5 128.32.0.66 AS25 AS11423 Calren

6 209.247.159.109 AS3356

7 *
8 64.159.1.46 9 209.247.9.170 10 66.185.138.33

AS3356
AS3356 AS3356 AS1668

Level3

11 *
12 66.185.136.17 13 64.236.16.52

AS1668
AS1668

AOL

AS5662 CNN

32

Challenges of Inter-AS Mapping

Mapping traceroute hops to ASes is hard
Need an accurate registry of IP address ownership Whois data are notoriously out of date

Collecting diverse interdomain data is hard

Public repositories like RouteViews and RIPE-RIS Covers hundreds to thousands of vantage points Especially hard to see peer-peer edges

AT&T

Sprint ???

d1

Harvard

Harvard B-school

d2

33

Inferring AS Relationships
Key idea
The business relationships determine the routing policies The routing policies determine the paths that are chosen So, look at the chosen paths and infer the policies

Example: AS path 1 7018 88 implies

AS 7018 allows AS 1 to reach AS 88 AT&T allows Level 3 to reach Princeton Each triple tells something about transit service

Collect and analyze AS path data

Identify which ASes can transit through the other and which other ASes they are able to reach this way
34

Paths You Should Never See (Invalid)

Customer-provider Peer-peer

two peer edges

transit through a customer

35

Challenges of Relationship Inference

Incomplete measurement data
Hard to get a complete view of the AS graph Especially hard to see peer-peer edges low in hierarchy

Real relationships are sometime more complex

Peer is one part of the world, customer in another Other kinds of relationships (e.g., backup and sibling) Special relationships for certain destination prefixes

Still, inference work has proven very useful

Qualitative view of Internet topology and relationships
36

Conclusions
Two-tiered Internet topology
AS-level topology Intra-AS topology

Inferring network topologies

By measuring paths from many vantage points Intradomain and interdomain routing

Next class

37