You are on page 1of 126

NET403

AWS Direct Connect: Deep Dive

Justin Davies
Solutions Architect
AWS/Solutions Architecture

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What’s going on here?
policy-options

policy-statement TO-AWS

term tag-aws

from

route-filter 0.0.0.0/0 exact;

then

community add TAG-TO-AWS;

accept;

community TAG-TO-AWS-HIGH-PREF members 7224:7300;

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
Level set—review

New features and functionality

Route manipulation and traffic engineering

How is AWS Direct Connect billed?

How to manage hybrid DNS scenarios over


AWS Direct Connect

Architectural best practices and resiliency


© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
On-premises

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
On-premises
Amazon Virtual Private Cloud (Amazon VPC)

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
On-premises

Availability Zone

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
On-premises

Availability Zone

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
On-premises

Subnet

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
On-premises

Subnet

Virtual private
gateway

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
On-premises

Subnet

Virtual Private
Gateway

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
Direct
Connect On-premises

Subnet

Virtual Private
Gateway

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
Direct Connect On-premises

Amazon

Customer
backbone

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
Direct Connect On-premises

Amazon

Customer
backbone

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
Direct Connect
Amazon
Private

Public


© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Level set—Review
Direct Connect
Amazon
Private

Public


© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Direct Connect specifications
Direct Connect

1G, 10G,

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Direct Connect specifications
Direct Connect

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Direct Connect specifications
Direct Connect

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Direct Connect specifications
Direct Connect

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Direct Connect specifications
Direct Connect

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Private VIF
Virtual Private
Gateway
Private

Physical connection
VLAN ID
VIF name & owner
On-prem ASN
*AWS ASN

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Private VIF
Virtual Private
Gateway
Private

Physical connection
VLAN ID
VIF name & owner 50 VIFs
On-prem ASN
*AWS ASN

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Public VIF Physical connection
VLAN ID
VIF name & owner
On-prem ASN
Public peer IPs (v4)

Public
Public VIF


© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Public VIF Physical connection
VLAN ID
VIF name & owner
On-prem ASN
Public peer IPs (v4)

Public
Public VIF


© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
”Home” region
us-east-1 us-west-2

https://aws.amazon.com/directconnect/features/

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Do I need to have a BGP session for every VPC?

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Can I connect to VPCs outside of my “home” region?

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Can I connect to VPCs outside of my “home” region?

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Can I reduce my BGP peers and simplify connectivity?

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
So what is a Direct Connect Gateway?

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
You specify: Direct Connect
“name”

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect Gateway

Attached
1

10

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect Gateway

Account 1

Account 1

Account 2
Direct Connect

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
So how does this scale?

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect Gateway—Scaling
Attach 10

Account 1

Account 1

Account 1
Direct Connect

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect Gateway—Scaling
Attach 10

Account 1

Account 1

Account 2
Direct Connect

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect Gateway—Scaling
Attach 10

Account 1

Account 1

Account 2
Direct Connect

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect Gateway—Scaling

Account 1

Account 1

Account 2
Direct Connect

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How do routes work?

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How do routes work?

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Before Logical Redundancy Direct Connect

Direct Connect Customer


Device

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Before Logical Redundancy Direct Connect

Direct Connect Customer

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Before Logical Redundancy Direct Connect

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Logical Redundancy (NEW) Direct Connect

Direct Connect Customer

Direct Connect

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Logical Redundancy (NEW) Direct Connect

Customer

Direct Connect

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How does this change my
physical redundancy?

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Logical & Physical Redundancy Direct Connect

Direct Connect Customer

Direct Connect

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Is logical redundancy available?

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Redundant BGP Sessions

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VPC & Direct Connect route selection

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route selection
East - DC

172.16.0.0/16
East
65001, 65001, 65001
us-east-1

172.16.0.0/16
65001, 65001 West - DC

West

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route selection
East - DC

172.16.0.0/16
East
65001, 65001, 65001
us-east-1

172.16.0.0/16
65001, 65001 West - DC

*Preferred route
leaving AWS
West

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route selection
East - DC

172.16.0.0/16
East
65001, 65001, 65001
us-east-1

172.16.0.0/16
65001, 65001 West - DC

*Preferred route
leaving AWS 172.16.0.0/16
65001 West

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route selection
East - DC

172.16.0.0/16
East
65001, 65001, 65001
us-east-1

172.16.0.0/16
65001, 65001 West - DC

*Preferred route
leaving AWS 172.16.0.0/24
65001 West
*Longest prefix match
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
BGP communities & local—preference

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Public VIF communities—Controls your prefix scope

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Public VIF communities—Controls AWS prefix scope

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Private VIF communities: AWS egress local-pref

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route selection
East - DC

172.16.0.0/16
East
65001, 65001, 65001
us-east-1

172.16.0.0/16
65001, 65001 West - DC

*Preferred route
leaving AWS 172.16.0.0/16
65001 West

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route selection
East - DC
172.16.0.0/16
7224:7100 (low) East
us-east-1 65001, 65001, 65001

172.16.0.0/16
7224:7100 (Low) West - DC
65001, 65001

*Preferred route
leaving AWS 172.16.0.0/16
7224:7300 (high) West
65001
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Applying communities to prefixes Juniper example

policy-options

policy-statement TO-AWS

term tag-aws

from

route-filter 0.0.0.0/0 exact;

then

community add TAG-TO-AWS;

accept;

community TAG-TO-AWS-HIGH-PREF members 7224:7300;

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Applying communities to prefixes Cisco example

ip bgp-community new-format

ip prefix-list TAG-TO-AWS permit 0.0.0.0/0 le 32

route-map TO-AWS permit 10

match ip address prefix-list TAG-TO-AWS

set community 7224:7300

router bgp 65400

address-family ipv4

neighbor 169.254.221.5 send-community

neighbor 169.254.221.5 route-map TO-AWS out

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
I manage the network.
I’m not sure what all these VPCs are really doing.
How does billing work?

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect Billing 1G = $0.30/port hour
10G = $2.25/port hour
*All locations except Japan

Data-Transfer-OUT
Source: United States
VPC, S3, DDB …

Destination:
Switch, SUPERNAP
Las Vegas

$0.0200/GB Out

https://aws.amazon.com/directconnect/pricing/

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect Billing 1G = $0.30/port hour
10G = $2.25/port hour
*All locations except Japan

Data-Transfer-OUT
Source: Ireland
(eu-west-1)
VPC, S3, DDB …

Destination:
Switch, SUPERNAP
Las Vegas

$0.0282/GB Out
https://aws.amazon.com/directconnect/pricing/

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect: Port cost

https://aws.amazon.com/directconnect/pricing/
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect: Data-transfer-out cost

https://aws.amazon.com/directconnect/pricing/
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What if I have multiple accounts?

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect Billing
Organization (master payer account)

Account 2

$ Account 3

Account 4 Account 1

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Direct Connect Billing
Source account

Account 2

$ Account 3

Account 4 Account 1

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
I manage DNS servers on-premises today.
How can I resolve resources between
my VPC resources and on-premises?

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
192.168.1.0/24 (myvpc.com)

192.168.1.10
one.myvpc.com 10.0.0.0/16 (mydc.com)

192.168.1.11
two.myvpc.com

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
1. Host one: Where is ”two.myvpc.com”
192.168.1.0/24 (myvpc.com)

192.168.1.10 1
one.myvpc.com
192.168.1.2 mydc.com

192.168.1.11
two.myvpc.com

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
1. Host one: Where is ”two.myvpc.com”
192.168.1.0/24 (myvpc.com) 2. Amazon Route 53: Oh, that’s 192.168.1.11

192.168.1.10 1
one.myvpc.com 2
192.168.1.2 mydc.com

192.168.1.11
two.myvpc.com

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
1. Host one: Where is ”two.myvpc.com”
192.168.1.0/24 (myvpc.com) 2. Amazon Route 53: Oh, that’s 192.168.1.11

192.168.1.10 1
one.myvpc.com 2
192.168.1.2 mydc.com

192.168.1.11
two.myvpc.com

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
1. Client: Where is ”two.myvpc.com”
192.168.1.0/24 (myvpc.com)

192.168.1.10
one.myvpc.com
192.168.1.2 mydc.com

192.168.1.11
two.myvpc.com
1

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
1. Client: Where is ”two.myvpc.com”
192.168.1.0/24 (myvpc.com) 2. On-prem DNS: Conditional forward?

192.168.1.10
2
one.myvpc.com
192.168.1.2 mydc.com

192.168.1.11
two.myvpc.com
1

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
1. Client: Where is ”two.myvpc.com”
192.168.1.0/24 (myvpc.com) 2. On-prem DNS: Conditional forward?
3. I don’t know, can’t reach 192.168.1.2

192.168.1.10
2
one.myvpc.com
192.168.1.2
X mydc.com

192.168.1.11
two.myvpc.com
1

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
1. Client: Where is ”two.myvpc.com”
192.168.1.0/24 (myvpc.com)

192.168.1.10
one.myvpc.com
192.168.1.2 mydc.com

192.168.1.11
two.myvpc.com Unbound
1

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
1. Client: Where is ”two.myvpc.com”
192.168.1.0/24 (myvpc.com) 2. On-prem DNS: Forward to Unbound

192.168.1.10
one.myvpc.com
192.168.1.2 mydc.com
2

192.168.1.11
two.myvpc.com Unbound
1

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
1. Client: Where is ”two.myvpc.com”
192.168.1.0/24 (myvpc.com) 2. On-prem DNS: Forward to Unbound
3. Unbound forward to Route 53

192.168.1.10
one.myvpc.com
3
192.168.1.2 mydc.com
2

192.168.1.11
two.myvpc.com Unbound
1

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid hosted zones
1. Client: Where is ”two.myvpc.com”
192.168.1.0/24 (myvpc.com) 2. On-prem DNS: Forward to Unbound
3. Unbound forward to Route 53
4. Reply to requester
192.168.1.10
one.myvpc.com
3
192.168.1.2 mydc.com
2

192.168.1.11
two.myvpc.com Unbound
4 1

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Route 53 Resolver

Primary Secondary Tertiary

Availability Availability Availability


Zone 1 Zone 2 Zone 3
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route 53 Resolver
1. Client: Where is ”two.myvpc.com”
192.168.1.0/24 (myvpc.com)

192.168.1.10
one.myvpc.com
192.168.1.xyz mydc.com

192.168.1.11
two.myvpc.com 1

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route 53 Resolver
1. Client: Where is ”two.myvpc.com”
192.168.1.0/24 (myvpc.com) 2. On-prem DNS: Forward to AWS resolver

192.168.1.10
one.myvpc.com 2
192.168.1.xyz mydc.com

192.168.1.11
two.myvpc.com 1

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route 53 Resolver
1. Client: Where is ”two.myvpc.com”
192.168.1.0/24 (myvpc.com) 2. On-prem DNS: Forward to AWS resolver
3. Reply to requester

192.168.1.10
one.myvpc.com 2
192.168.1.xyz mydc.com
3

192.168.1.11
two.myvpc.com 1

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route 53 Resolver 1. Host one: Where is ”client.mydc.com”

192.168.1.0/24 (myvpc.com)

192.168.1.10 1
one.myvpc.com
192.168.1.xyz mydc.com

192.168.1.11
two.myvpc.com

10.0.0.7
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route 53 Resolver 1. Host one: Where is ”client.mydc.com”
2. Route 53: Forward *.mydc.com to on-prem DNS
192.168.1.0/24 (myvpc.com)

192.168.1.10 1
one.myvpc.com
2
192.168.1.xyz mydc.com

192.168.1.11
two.myvpc.com

10.0.0.7
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route 53 Resolver 1. Host one: Where is ”client.mydc.com”
2. Route 53: Forward *.mydc.com to on-prem DNS
192.168.1.0/24 (myvpc.com) 3. On-prem DNS: Oh, that’s 10.0.0.7

192.168.1.10 1
one.myvpc.com
2
192.168.1.xyz mydc.com
3

192.168.1.11
two.myvpc.com

10.0.0.7
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Route 53 Resolver 1. Host one: Where is ”client.mydc.com”
2. Route 53: Forward *.mydc.com to on-prem DNS
192.168.1.0/24 (myvpc.com) 3. On-prem DNS: Oh, that’s 10.0.0.7
4. Reply to requester

192.168.1.10 1
one.myvpc.com
2
4 192.168.1.xyz mydc.com
3

192.168.1.11
two.myvpc.com

10.0.0.7
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
“Everything fails all the time.”

Werner Vogels
VP & CTO, AWS
Start with the application

Availability Zone 1

Availability Zone 2

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Start with the application

us-east-1

us-west-2

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Start with the application

us-east-1

us-west-2

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Consider the ingress and egress points

Availability Zone 1

Availability Zone 2

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Consider the ingress and egress points

Availability Zone 1

Availability Zone 2

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Consider the ingress and egress points
Direct Connect On-premises

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Know your traffic profile
Direct Connect On-premises

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Know your dependencies

Everything API

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Understand impact

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Understand impact

Guilty until proven innocent


Test it! Test it often!

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Understand impact

Guilty until proven innocent


Test it! Test it often!

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Understand impact

Guilty until proven innocent


Test it! Test it often!

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you!
Justin Davies
@mrjustind

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

You might also like