Centralizing AWS Network Connectivity with Transit Gateway

TL;DR: AWS Transit Gateway (TGW) acts as a central hub and spoke model for connecting Amazon Virtual Private Clouds (VPCs) and on-premises networks. This article details its architecture, implementation, routing, security, monitoring, and cost considerations for experienced engineers.

$1

AWS Transit Gateway is a network transit hub that you can use to interconnect your virtual private clouds (VPCs) and your on-premises networks. It acts as a cloud router for all traffic that flows between these networks. This centralizes your network topology, simplifying management and scalability compared to traditional VPC peering or complex VPN mesh configurations.

$1

Transit Gateway offers a managed, scalable solution for inter-network connectivity. Its core components are:

| Component | Description | Benefits |

| :----------------- | :------------------------------------------------------------------- | :------------------------------------------------------------------------- |

| Transit Gateway | The central routing and forwarding hub for all connected networks. | Simplifies connectivity management, reduces operational overhead. |

| Transit Gateway Attachments | Connects network resources (VPCs, VPNs, Direct Connect gateways) to the Transit Gateway. | Provides flexible integration points for various network types. |

| Transit Gateway Route Tables | Controls how traffic is routed between attachments. | Enables granular routing policies, network segmentation, and isolation. |

| Transit Gateway Peering | Allows connecting Transit Gateways across different AWS Regions. | Extends your network reach globally, enabling cross-region connectivity. |

| Network Manager | A network management service that provides visibility and control. | Centralized visibility and operational management of your TGW deployment. |

``mermaid

graph LR

TGW[Transit Gateway]

subgraph VPCs

VPC1[VPC A (10.1.0.0/16)]

VPC2[VPC B (10.2.0.0/16)]

VPC3[VPC C (10.3.0.0/16)]

end

subgraph On-Premises

OnPrem[On-Premises Network (192.168.0.0/16)]

VPN[Site-to-Site VPN]

DX[AWS Direct Connect]

end

TGW -- Attachment --> VPC1

TGW -- Attachment --> VPC2

TGW -- Attachment --> VPC3

TGW -- Attachment --> VPN

TGW -- Attachment --> DX

VPC1 -- Traffic --> TGW

VPC2 -- Traffic --> TGW

VPC3 -- Traffic --> TGW

VPN -- Traffic --> TGW

DX -- Traffic --> TGW

`

$1

$1

When deploying Transit Gateway, consider default route table association and propagation settings. Enabling defaultRouteTableAssociation and defaultRouteTablePropagation can simplify initial setups, but disabling them offers more control for complex routing scenarios.

`typescript

import * as aws from "@pulumi/aws";

const mainTgw = new aws.ec2.TransitGateway("main-tgw", {

// Recommended for production: Set to 'disable' for explicit route table management.

autoAcceptSharedAttachments: "enable",

// 'enable' for simpler setups, 'disable' for explicit route table management.

defaultRouteTableAssociation: "enable",

// 'enable' for automatic propagation, 'disable' for static route definition.

defaultRouteTablePropagation: "enable",

// Set to 'enable' if you require DNS resolution through TGW.

dnsSupport: "enable",

// 'enable' for multiple paths to VPN endpoints.

vpnEcmpSupport: "enable",

// Use a private ASN for BGP with on-premises.

amazonSideAsn: 64512,

tags: {

Name: "main-transit-gateway",

Environment: "production",

},

});

`

$1

Attaching a VPC to Transit Gateway involves creating a TransitGatewayVpcAttachment. You must specify the VPC ID and one or more subnet IDs within that VPC. The subnets determine the Availability Zones in which the TGW attachment will reside, providing high availability.

Crucially, for the VPC to communicate with other networks via TGW, you must add a route in the VPC's route table pointing to the Transit Gateway.

`typescript

import * as aws from "@pulumi/aws";

// Assume 'mainTgw' is the created Transit Gateway resource.

// Assume 'vpcA' is the VPC resource and 'vpcASubnets' are its subnet IDs.

const vpcAAttachment = new aws.ec2.TransitGatewayVpcAttachment("vpcA-attachment", {

subnetIds: vpcASubnets, // e.g., ["subnet-xxxxxxxxxxxxxxxxx", "subnet-yyyyyyyyyyyyyyyyy"]

transitGatewayId: mainTgw.id,

vpcId: vpcA.id,

opts: {

dependsOn: [mainTgw, vpcA], // Ensure TGW and VPC exist first

},

tags: {

Name: "vpcA-tgw-attachment",

VpcName: "vpcA",

},

});

// Route traffic destined for other networks (e.g., 10.0.0.0/8) to the Transit Gateway.

// This is often a '0.0.0.0/0' route if TGW is the default internet gateway for the VPC.

// Adjust destinationCidrBlock based on your network plan.

const routeTableForVpcA = aws.ec2.getRouteTable({

vpcId: vpcA.id,

// Filter for the main route table or a specific one.

filters: [{

name: "association.main",

values: ["true"],

}],

});

const tgwRouteForVpcA = new aws.ec2.Route("tgw-route-vpcA", {

routeTableId: routeTableForVpcA.then(rt => rt.id),

destinationCidrBlock: "0.0.0.0/0", // Example: route all non-local traffic via TGW

transitGatewayId: mainTgw.id,

opts: {

dependsOn: [vpcAAttachment], // Ensure attachment is created before route

},

});

`

$1

On-premises connectivity is established via Site-to-Site VPN connections or AWS Direct Connect. Both are configured as TransitGatewayVpnAttachment or TransitGatewayDirectConnectGatewayAttachment respectively. These attachments require BGP configuration (ASN) and route exchange.

$1

Transit Gateway route tables are fundamental for controlling traffic flow and implementing network segmentation.

`mermaid

graph TB

TGW[Transit Gateway]

subgraph Route Tables

RT_PROD[TGW RT: Production]

RT_DEV[TGW RT: Development]

RT_SHARED[TGW RT: Shared Services]

end

TGW --> RT_PROD

TGW --> RT_DEV

TGW --> RT_SHARED

RT_PROD -- Propagates --> VPC_PROD[VPC Production]

RT_DEV -- Propagates --> VPC_DEV[VPC Development]

RT_SHARED -- Propagates --> VPC_SHARED[VPC Shared Services]

VPC_PROD -- Attachment --> TGW

VPC_DEV -- Attachment --> TGW

VPC_SHARED -- Attachment --> TGW

RT_PROD -- Static Route --> VPC_DEV

RT_DEV -- Static Route --> VPC_PROD

RT_SHARED -- Static Route --> VPC_PROD

RT_SHARED -- Static Route --> VPC_DEV

RT_PROD -- Static Route --> ONPREM[On-Premises]

`

$1

* Propagation: Automatically adds routes learned from connected VPCs or VPNs. Use with caution to avoid unwanted route leaks.

* Static Routes: Manually configured routes for precise control. Essential for directing traffic to specific destinations or implementing segmentation.

* Route Table Association: Links an attachment to a specific route table. A VPC can only be associated with one route table.

* Route Table Propagation: Controls which learned routes are advertised to the associated route table.

Example: Isolating Development VPCs from Production

1. Create separate route tables for Production and Development.

2. Associate the Production VPC attachment with the Production route table.

3. Associate the Development VPC attachment with the Development route table.

4. In the Production route table, do not propagate routes from the Development VPC. Add a static route for the Development VPC CIDR pointing to the Development VPC attachment.

5. In the Development route table, do not propagate routes from the Production VPC. Add a static route for the Production VPC CIDR pointing to the Production VPC attachment.

6. Ensure Security Groups and Network ACLs further enforce desired traffic flows.

$1

Transit Gateway itself does not inspect traffic. Security remains a responsibility of the attached VPCs and on-premises firewalls. Security Groups within VPCs and Network ACLs (NACLs) should be configured to permit/deny traffic between attached networks as needed. For traffic inspection, consider deploying a Network Virtual Appliance (NVA) in a dedicated "Inspection VPC" attached to Transit Gateway, with routes pointing to it for inter-VPC/on-prem traffic.

$1

Effective monitoring is key to maintaining a healthy Transit Gateway network.

| Metric | Description | Target/Action |

| :---------------- | :----------------------------------------------- | :------------------------------------------------------------------------------ |

| BytesIn/Out | Total data processed by the Transit Gateway. | Monitor for unexpected spikes or sustained high utilization, indicative of traffic patterns or misconfigurations. |

| PacketsIn/Out | Total packets processed. | Correlate with BytesIn/Out to identify potential packet-per-byte ratios. |

| PacketDropCount | Packets dropped by Transit Gateway. | Aim for near zero. High values may indicate upstream congestion or routing issues. |

| PacketErrorCount| Packets with errors. | Aim for near zero. High values suggest network link issues or faulty hardware on connected networks. |

$1

* No Connectivity:

* Verify Route Tables: Ensure correct associations and propagation settings. Check if static routes are accurately defined and pointing to the correct attachment.

* Security Groups/NACLs: Confirm that Security Groups in the source and destination VPCs, and NACLs, permit the traffic.

* Attachment Status: Check the status of all Transit Gateway attachments in the AWS console.

* Intermittent Connectivity/Performance Degradation:

* Bandwidth Limits: Review BytesIn/Out` metrics for attachments. Individual VPC attachments can become a bottleneck.

* Overlapping CIDRs: Ensure no overlapping CIDR blocks between directly connected networks via TGW.

* MTU Issues: While TGW generally handles MTU well, inconsistent MTU settings in connected networks can cause fragmentation and performance problems.

* BGP State (VPN/DX): For hybrid connectivity, check BGP peer status and advertised routes.

$1

Transit Gateway costs are primarily based on:

* Transit Gateway per hour: A flat hourly fee for each running Transit Gateway.

* Data Processing: A per-GB charge for data processed by the Transit Gateway.

* Attachment Hours: An hourly fee for each active attachment.

* Data Transfer: Standard AWS data transfer charges apply between Availability Zones and Regions.

$1

* Route Optimization: Design route tables to minimize unnecessary traffic crossing the Transit Gateway. For example, if VPC A and VPC B are in the same region and only need to communicate, and their traffic doesn't need to exit the region, avoid routing them through a cross-region TGW peering attachment or an on-premises VPN if direct VPC peering would suffice (though this scales poorly).

* Attachment Consolidation: Consolidate VPCs into fewer, larger regions if possible, reducing the number of attachments.

* Traffic Engineering: Understand traffic patterns. If most traffic stays within a region, leverage regional Transit Gateways and only use cross-region peering for essential inter-region communication.

* Dedicated Connections: For high-volume, consistent hybrid traffic, AWS Direct Connect is often more cost-effective than VPNs over the long term, despite higher initial setup costs.

$1

* Pros:

* Simplified Network Architecture: Central hub-and-spoke model reduces complexity compared to full mesh VPC peering.

* Scalability: Designed to handle large-scale network interconnections.

* Centralized Management: A single point for managing connectivity and routing policies.

* Network Segmentation: Powerful route table features enable strong isolation.

* Hybrid Connectivity: Seamlessly integrates VPCs with on-premises networks.

* Cons:

* Cost: Can be more expensive than VPC peering for very small deployments due to per-hour charges for the TGW and attachments. Data processing costs can accumulate significantly with high traffic volumes.

* Single Point of Failure (without HA): While TGW itself is a managed, highly available service, a misconfiguration or single TGW instance in a complex multi-region setup can disrupt connectivity if not architected for redundancy.

* Lack of Direct Traffic Inspection: TGW does not inspect traffic; security must be managed at the attachment level (VPC Security Groups/NACLs) or via NVAs.

Limited Granularity within VPCs: Routing is at the VPC attachment level. Fine-grained routing *within a VPC still relies on its native route tables.

$1

* Very Small Deployments: For a single VPC or two VPCs with minimal interaction, direct VPC peering might be simpler and more cost-effective.

* Need for Global Transit Without Region Separation: If you require a truly global, flat network where all subnets can talk to all others without explicit routing policies, and cost is not a primary concern, a different approach might be considered, though TGW with aggressive propagation can approximate this.

* Strictly On-Premises to Single VPC: If you only need to connect an on-premises network to a single VPC, a direct AWS Site-to-Site VPN or Direct Connect to that VPC might suffice and avoid TGW overhead.

* Requirements for Deep Packet Inspection at the Hub: If your requirement is to inspect all inter-network traffic at a central point without deploying NVAs, TGW alone will not meet this need.

$1

| Decision Point | Considerations | Recommendation Status (Yes/No/N/A) | Notes |

| :-------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

| Number of VPCs to connect? | More than 2-3 VPCs, especially across different AWS accounts or regions, strongly favors TGW. | | |

| Need to connect on-premises networks? | TGW is the primary AWS service for hybrid connectivity scaling. | | |

| Requirement for network segmentation/isolation? | TGW route tables offer robust control for isolating environments (e.g., prod vs. dev). | | |

| Scalability requirements? | TGW is designed for scale, handling thousands of VPCs and routes. | | |

| Cost sensitivity for small deployments? | For 2-3 VPCs with minimal interaction, VPC peering might be cheaper than TGW hourly/data processing fees. | | |

| Need for centralized routing policy management? | TGW offers a single pane of glass for routing policies. | | |

| Cross-region connectivity requirements? | TGW peering is the standard AWS solution for inter-region VPC connectivity. | | |

| Desire to simplify network topology? | TGW reduces the complexity of managing a mesh of VPC peerings. | | |

| Need for centralized security inspection (NVA)? | TGW can facilitate this by directing traffic to an NVA VPC, but TGW itself doesn't inspect. | | |

| Existing complex network architecture? | TGW can simplify migration from complex meshes or point-to-point VPNs. | | |

$1

AWS Transit Gateway provides a robust, scalable, and manageable solution for centralizing your AWS network connectivity. By understanding its architecture, mastering route table management for segmentation, and implementing diligent monitoring and cost optimization strategies, you can build a highly effective network foundation that supports your organization's evolving needs. Carefully weigh the trade-offs against simpler solutions like VPC peering for smaller deployments to ensure you are using the most appropriate tool for the task.

$1

1. [AWS Transit Gateway Documentation](https://docs.aws.amazon.com/vpc/latest/tgw/what-is-transit-gateway.html)

2. [Transit Gateway Design Patterns](https://docs.aws.amazon.com/vpc/latest/tgw/tgw-design-patterns.html)

3. [AWS Network Manager Documentation](https://docs.aws.amazon.com/network-manager/latest/tgw/what-is-network-manager.html)

4. [AWS Transit Gateway Pricing](https://aws.amazon.com/transit-gateway/pricing/)