Architecture Overview
Aviatrix multi-cloud overlay across AWS, Azure, and GCP — 3 CSPs, 3 regions each, centralized NGCN Account
App Fabric - East/West Traffic
Same-region and cross-region E/W flows with Palo Alto firewall inspection in NGCN Account
| Component | Location | Role |
|---|---|---|
| App Spoke GW (Source) | Workload Account | Originates traffic, attached to E/W Aviatrix Transit GW |
| E/W Aviatrix Transit GW | NGCN Account | Aggregates spoke traffic; sends to LB for FW inspection |
| CSP Load Balancer | NGCN Account | AWS: GWLB · Azure: ILB · GCP: ILB — sits between TGW and FW |
| Palo Alto FW (1-arm) | NGCN Account | Inspects and policy-enforces, returns via same LB |
| App Spoke GW (Dest) | Workload Account | Receives inspected traffic from E/W Aviatrix Transit GW |
- App A Spoke GW (East)
- → E/W Aviatrix Transit GW (East, NGCN)
- → LB → FW (East) → LB (inspection only in source)
- → E/W Aviatrix TGW (East)
- → HE Aviatrix Transit GW (East)
- → HE Aviatrix Transit GW (Central)
- → E/W Aviatrix Transit GW (Central, NGCN)
- → App B Spoke GW (Central)
Hybrid Edge — Cloud to On-Premises
Edge Spoke GWs in Equinix and Cologix colos bridge cloud to on-prem via BGPoLAN
● NGCN Team Owns
- Edge Spoke GWs (in colos)
- All Aviatrix cloud infrastructure
- E/W Aviatrix Transit GWs + HE Aviatrix Transit GWs
- MTT Gateways
- App Fabric Controllers + Hybrid Edge Controller
- CSP circuit-terminating accounts (cloud side)
● On-Prem Team Owns
- Cages, routers, switches in colo
- Physical circuits (Direct Connect, ExpressRoute, Interconnect)
- Circuit-terminating cloud accounts
- Colo switches (BGPoLAN peer side)
- Everything from colo switches toward on-prem
| Parameter | Equinix (AWS preferred) | Cologix (Azure/GCP preferred) |
|---|---|---|
| ESGs per CSP per colo | 4 | 4 |
| Total ESGs per colo | 12 (4 × 3 CSPs) | 12 (4 × 3 CSPs) |
| Total ESGs across both | 24 total | |
| WAN interface | 25 Gbps (underlay) | 25 Gbps (underlay) |
| LAN interface | 25 Gbps (overlay/BGPoLAN) | 25 Gbps (overlay/BGPoLAN) |
| Circuits per CSP per colo | 2 × 100 Gbps | 2 × 100 Gbps |
| BGPoLAN | Between ESG LAN interface and colo switches | |
| Routes accepted via BGPoLAN | RFC 1918 only (from colo switches) | |
Cross-CSP Traffic — MTT Gateways
Multi-Tenant Transit gateways in Central regions connect across CSPs via Aviatrix Transit GW Attachments — managed by Hybrid Edge Controller
| # | Hop | Component | Notes |
|---|---|---|---|
| 1 | Source App | App in AWS (any region) | Originates from AWS workload |
| 2 | E/W Transit | E/W Aviatrix TGW (AWS region, NGCN) | Routes toward HE TGW |
| 3 | HE Transit | HE Aviatrix TGW (AWS Central) | Inter-region hop if needed; connects to MTT |
| 4 | MTT GW (AWS) | MTT GW — AWS Central | Connects to Azure MTT GW via TGW Attachment |
| 5 | ↔ TGW Attachment | Cross-CSP Aviatrix TGW Attachment | Hybrid Edge Controller manages both sides |
| 6 | MTT GW (Azure) | MTT GW — Azure Central | Receives traffic from AWS MTT GW |
| 7 | HE Transit | HE Aviatrix TGW (Azure Central) | Distributes into Azure regions |
| 8 | E/W Transit | E/W Aviatrix TGW (Azure target region, NGCN) | Routes to destination spoke |
| 9 | Destination App | App in Azure | Receives cross-CSP traffic |
CSP Underlay Connectivity
Native CSP circuits connecting colos to cloud — managed by on-prem team's accounts
| CSP | Cloud Router/GW | Aggregation | Physical Layer | Preferred Colo |
|---|---|---|---|---|
| AWS | VPG (Virtual Private Gateway) | DXGW (Direct Connect Gateway) | Direct Connect circuit | Equinix |
| Azure | ER Gateway | ExpressRoute Circuit | ExpressRoute Direct | Cologix |
| GCP | Cloud Router | VLAN Attachment | Interconnect Direct | Cologix |
Traffic Flow Animations
Animated packet walks through the architecture — press Play to watch each flow
Trace a Path
Select source & destinationE/W Same-Region Flow
App A → E/W Aviatrix Transit GW → LB → FW → LB → E/W Aviatrix Transit GW → App B (same region, same CSP)
Cross-Region Flow (Same CSP)
App A (East) → E/W Aviatrix TGW → FW (source only) → E/W Aviatrix TGW → HE TGW (East) → HE TGW (Central) → E/W Aviatrix TGW → App B (Central)
Hybrid Edge Flow (Cloud to On-Prem)
E/W Aviatrix TGW → HE Aviatrix TGW → Edge Spoke GW → BGPoLAN ↔ Colo Switches → On-Prem
Cross-CSP Flow (AWS → Azure)
App A (AWS East) → E/W TGW → FW → MTT GW (AWS) → MTT GW (Azure) → E/W TGW (Azure) → App B
L2 Operational Scenarios
Hop-by-hop with ops verification notesSelect a scenario above to view the operational walkthrough with verification notes per hop.
Management Fabric
Firewall management architecture — how controllers connect to and manage Palo Alto firewall infrastructure
● NGCN Team Owns
- Aviatrix management plane connectivity
- Panorama management access paths
- Management VPC/VNet infrastructure
- FireNet integration configuration
● Firewall Team Owns
- Palo Alto Panorama servers
- Firewall policy management
- Security rule lifecycle
- Log collection and SIEM integration
Alerts & Monitoring
Grafana to ServiceNow alerting pipeline — 300+ alerts across the multi-cloud environment
| Alert Name | Condition | Threshold | Severity | Status |
|---|---|---|---|---|
| Controller CPU Used % | CPU utilization exceeds threshold | Minor: >70% · Major: >85% · Critical: >95% | Minor | Active |
| Gateway Status | Gateway Down / Keep Alive Fail | Status ≠ Up | Critical | Active |
| PPOS Limit Exceed Drop (%) | Packet processing drop rate exceeds limit | >1% | Major | Active |
| Tunnel Status | IPSec/GRE tunnel state change | Status ≠ Up | Critical | Active |
| BGP Session Status | BGP neighbor session down | State ≠ Established | Critical | Active |
| Circuit Utilization | Direct Connect / ExpressRoute / Interconnect bandwidth | >80% sustained 5min | Warning | Active |
| ESG WAN/LAN Interface | Edge Spoke GW interface down | Status = Down | Critical | Active |
| Firewall Health Check | Palo Alto FW instance health | Health ≠ Healthy | Critical | Active |
| Memory Utilization | Gateway/Controller memory usage | >85% | Major | Active |
Main Aviatrix monitoring dashboard
Alert rules management page
ServiceNow instance for ticket creation
Webhook endpoint for automated ticket creation from Grafana alerts
Aviatrix CoPilot for gateway and topology management
On-call escalation endpoint (optional)
Environment Certification
Test your knowledge across Operations, Engineering, Architecture, and Management domains — 80% required to pass
Quick Reference
Acronyms, components, and key concepts
| Acronym | Definition |
|---|---|
| AFC | App Fabric Controller. 1 per region per CSP (9 total). Manages E/W Aviatrix Transit GWs and App Spoke GWs. |
| BGPoLAN | BGP over LAN. Overlay protocol used on ESG LAN interfaces to exchange routes with colo switches. |
| CSP | Cloud Service Provider (AWS, Azure, GCP). |
| DC | Direct Connect. AWS's dedicated circuit service. |
| DXGW | Direct Connect Gateway. AWS construct that connects VPG to Direct Connect circuits. |
| E/W | East/West. Traffic flowing between workloads within or across regions of the same CSP. |
| ER | ExpressRoute. Azure's dedicated circuit service. |
| ESG | Edge Spoke Gateway. Physical Aviatrix appliance in colocation facilities. 25G WAN + 25G LAN. |
| GWLB | Gateway Load Balancer. AWS-specific LB used in the 1-arm firewall design. |
| HE | Hybrid Edge. The connectivity path from cloud to on-premises via colocations. |
| IC | Interconnect. GCP's dedicated circuit service (Interconnect Direct). |
| ILB | Internal Load Balancer. Used by Azure and GCP in the 1-arm firewall design. |
| MTT | Multi-Tenant Transit. Gateways that enable cross-CSP routing via Aviatrix TGW Attachments in Central regions. |
| NGCN | The centralized network account/subscription/project in each CSP. Hosts E/W Transit GWs, HE Transit GWs, and firewall infrastructure. |
| TGW | Transit Gateway. Aviatrix construct that aggregates and routes traffic between spokes. |
| VPG | Virtual Private Gateway. AWS construct in the underlay path between HE Aviatrix TGW and DXGW. |
- 9 App Fabric Controllers — 1 per region per CSP. Manage E/W TGWs + App Spoke GWs.
- 1 Hybrid Edge Controller — Manages all HE TGWs, MTT GWs, and Edge Spoke GWs.
Hybrid Edge — Underlay & BGPoLAN
Operational verification of underlay paths and BGPoLAN neighbor status via CoPilot and CSP consoles
1 Concept Overview
The Hybrid Edge path connects cloud workloads to on-premises networks via physical colocation facilities. The full chain is:
E/W Aviatrix Transit GW → HE Aviatrix Transit GW → CSP underlay → Edge Spoke GW (physical, in colo) → BGPoLAN → colo switches → on-prem
Edge Spoke GWs (ESGs) are physical appliances — they are not cloud-based VMs. They sit in Equinix (Chicago) and Cologix (Minneapolis) colocation facilities.
Three CSP Underlay Paths:
2 Where to Click — CoPilot Navigation
View individual Edge Spoke GW status, including WAN/LAN interface state (Up/Down), BGP neighbor count, and route table.
View routing table of the HE Aviatrix Transit GW. Verify on-prem prefixes (10.x.x.x/16) are learned, and cloud prefixes (6.x.x.x/8) are present.
Verify prefix propagation across the fabric. Ensure on-prem routes are present in the expected cloud regions.
Check BGP neighbor status between ESG LAN interfaces and colo switches. Expect Established state.
3 Where to Click — CSP Underlay (High Level)
AWS:
- Console → VPC → Virtual Private Gateways — verify VPG is attached and active
- Console → Direct Connect → Connections — verify connection state is available
Azure:
- Portal → ExpressRoute circuits — verify circuit provisioning state and peering status
- Portal → Virtual network gateways — verify ER Gateway is active and routes are learned
GCP:
- Console → Hybrid Connectivity → Interconnect — verify attachment operational status
- Console → Hybrid Connectivity → Cloud Routers — check BGP session status
4 What to Look For
6.x.x.x/8), on-prem routes (10.x.x.x/16)Overlay Routes — CoPilot Gateway Views
Verify route presence and correctness via CoPilot Diagnostics: Gateway Routes and Cloud Routes
1 Concept Overview
All workload routing in this architecture is via the Aviatrix overlay fabric. There is no native TGW or VWAN used for routing — all routes are managed through Aviatrix transit gateways and spoke gateways.
Gateway Routes shows the per-gateway routing table: what each individual E/W Aviatrix Transit GW, HE Aviatrix Transit GW, or Edge Spoke GW knows about reachable prefixes.
Cloud Routes shows the aggregate view: all routes visible within a given VPC/VNet/VPC project, including which gateway provides the next hop.
6.0.0.0/8 (6.x.x.x) internally. On-prem uses RFC1918 ranges: 10.10.0.0/16, 10.20.0.0/16, 10.30.0.0/16.
2 Where to Click — CoPilot Gateway Routes
Opens the per-gateway routing table view.
- Select gateway type: E/W Aviatrix Transit GW, HE Aviatrix Transit GW, or Edge Spoke GW
- Filter by prefix (e.g.,
10.10.0.0/16) to narrow results - Key columns: Destination, Next Hop, Gateway, Metric/Preference, AS-Path, Status
3 Where to Click — Cloud Routes
Aggregate route view across the fabric.
- Filter by VPC/VNet/VPC to see all routes in a given network
- Verify that on-prem prefixes (
10.x.x.x/16) are present in cloud gateways - Verify that cloud prefixes (
6.x.x.x/8) are propagated toward on-prem direction
4 What to Look For
6.x.x.x/8 routes present in HE Aviatrix Transit GW and ESG route tables10.x.x.x/16 routes learned from on-prem via BGPoLAN on ESGs| Destination | Next Hop | Gateway | Metric | AS-Path | Status |
|---|---|---|---|---|---|
| 10.10.0.0/16 | 192.168.1.1 | edge-aws-equinix-1 | 200 | 65000 65000 | Active |
| 10.10.0.0/16 | 192.168.2.1 | edge-aws-cologix-1 | 200 | 65000 65000 65000 65000 65000 | Backup |
| 6.10.0.0/16 | 10.100.1.1 | ew-aws-east-1 | 100 | — | Active |
| 10.20.0.0/16 | 192.168.3.1 | edge-aws-equinix-2 | 200 | 65000 65000 | Active |
Traffic Steering — AS-Path & Colos
Primary/secondary colo selection via BGP AS-path prepend — no PBR, exclusively BGP path selection
1 Concept Overview
Each CSP has a primary and secondary colocation for on-prem connectivity. Traffic steering is achieved exclusively via BGP AS-path prepend — NOT policy-based routing (PBR).
- Active colo: Shorter AS-path — 2x prepend (e.g., AS65000 AS65000)
- Non-active colo: Longer AS-path — 5x prepend (e.g., AS65000 AS65000 AS65000 AS65000 AS65000)
BGP will always prefer the path with the shorter AS-path. If the primary colo withdraws its route, traffic automatically fails over to the secondary colo.
2 Colo Preference Table
| CSP | Primary Colo | Primary Prepend | Secondary Colo | Secondary Prepend |
|---|---|---|---|---|
| AWS | Equinix (Chicago) | 2x | Cologix (Minneapolis) | 5x |
| Azure | Cologix (Minneapolis) | 2x | Equinix (Chicago) | 5x |
| GCP | Cologix (Minneapolis) | 2x | Equinix (Chicago) | 5x |
3 AS-Path Teaching Widget
4 Troubleshooting: Wrong Colo Active
If traffic is taking a non-preferred colo path, follow this workflow:
Open CoPilot›Diagnostics›Gateway Routes for the relevant HE Aviatrix Transit GW
Filter by the on-prem prefix (e.g., 10.10.0.0/16)
Compare AS-path lengths from both colos. The active route should have the shorter AS-path (2x prepend). If both show 5x, the primary colo’s ESG may be down.
If the non-preferred colo has the shorter path, check if the preferred colo’s ESG is down or if the route has been withdrawn. Navigate to CoPilot›Cloud Fabric›Hybrid Edge›Edge Gateways and verify the ESG’s WAN/LAN status.