Site-to-site VPN over the internet works for low-bandwidth, latency-tolerant connectivity between on-premises and cloud. For workloads that cannot tolerate internet variability, for data transfers that saturate a VPN tunnel, or for compliance frameworks that require private connectivity, VPN is the wrong tool. FastConnect is the right one.
OCI FastConnect is a dedicated private circuit between your on-premises network and OCI. Traffic does not traverse the public internet at any point. You get consistent bandwidth, predictable latency, and a private path that satisfies most regulatory requirements for cloud connectivity. This post covers how FastConnect is architected, how to provision a virtual circuit with Terraform, how to configure BGP on both ends, and how to build a redundant setup with IPSec VPN as an automatic failover path.
FastConnect Architecture
FastConnect works through a physical cross-connect at a colocation facility where Oracle has a presence. You or your network provider brings a circuit to that facility and connects it to Oracle’s edge router. Oracle provisions a virtual circuit on top of that physical connection. The virtual circuit connects to a Dynamic Routing Gateway in your OCI VCN, which advertises routes to your on-premises network over BGP.
On-Premises Network
|
Provider Edge Router (your carrier)
|
Colocation Facility (Oracle FastConnect location)
|
Oracle Edge Router
|
FastConnect Virtual Circuit
|
Dynamic Routing Gateway (DRG)
|
OCI VCNTwo connection models are available. The first uses an Oracle network partner: a carrier or colocation provider that has an existing agreement with Oracle and can provision the circuit on your behalf. This is faster to set up and requires less physical infrastructure work on your end. The second model is a direct cross-connect where your organization physically colocates equipment at an Oracle FastConnect location and cables directly to Oracle’s router. This gives you more control but requires your own equipment in the colocation facility.
Step 1: IAM Policy
resource "oci_identity_policy" "fastconnect_policy" {
compartment_id = var.compartment_id
name = "fastconnect-management-policy"
description = "Permissions to manage FastConnect and DRG resources"
statements = [
"Allow group ${var.network_admin_group} to manage virtual-circuit in compartment id ${var.compartment_id}",
"Allow group ${var.network_admin_group} to manage drg in compartment id ${var.compartment_id}",
"Allow group ${var.network_admin_group} to manage drg-attachment in compartment id ${var.compartment_id}",
"Allow group ${var.network_admin_group} to manage cross-connect in compartment id ${var.compartment_id}",
"Allow group ${var.network_admin_group} to manage cross-connect-group in compartment id ${var.compartment_id}"
]
}Step 2: Dynamic Routing Gateway
resource "oci_core_drg" "production_drg" {
compartment_id = var.compartment_id
display_name = "production-drg"
defined_tags = {
"Operations.Environment" = "production"
"Operations.ManagedBy" = "terraform"
}
}
resource "oci_core_drg_attachment" "vcn_attachment" {
drg_id = oci_core_drg.production_drg.id
display_name = "production-vcn-attachment"
network_details {
id = var.vcn_id
type = "VCN"
}
}
# DRG route table for VCN traffic
resource "oci_core_drg_route_table" "vcn_route_table" {
drg_id = oci_core_drg.production_drg.id
display_name = "vcn-route-table"
is_ecmp_enabled = true
}
# Import route distribution - accept routes from FastConnect
resource "oci_core_drg_route_distribution" "fc_import" {
drg_id = oci_core_drg.production_drg.id
display_name = "fastconnect-import-distribution"
distribution_type = "IMPORT"
}
resource "oci_core_drg_route_distribution_statement" "fc_import_statement" {
drg_route_distribution_id = oci_core_drg_route_distribution.fc_import.id
action = "ACCEPT"
priority = 1
match_criteria {
match_type = "DRG_ATTACHMENT_TYPE"
attachment_type = "VIRTUAL_CIRCUIT"
}
}
output "drg_id" {
value = oci_core_drg.production_drg.id
}Step 3: Cross-Connect Group and Cross-Connect
For a direct cross-connect model, you provision the physical port on Oracle’s edge router. For a partner model, Oracle’s partner handles this step on your behalf and provides you with a service key to reference in the virtual circuit.
# Cross-connect group enables LAG (Link Aggregation Group) for redundancy
resource "oci_core_cross_connect_group" "production_ccg" {
compartment_id = var.compartment_id
display_name = "production-cross-connect-group"
defined_tags = {
"Operations.Environment" = "production"
"Operations.ManagedBy" = "terraform"
}
}
# Primary cross-connect - first physical port
resource "oci_core_cross_connect" "primary_cc" {
compartment_id = var.compartment_id
display_name = "primary-cross-connect"
cross_connect_group_id = oci_core_cross_connect_group.production_ccg.id
location_name = var.fastconnect_location
port_speed_shape_name = "10 Gbps"
is_active = true
}
# Secondary cross-connect - second physical port for LAG redundancy
resource "oci_core_cross_connect" "secondary_cc" {
compartment_id = var.compartment_id
display_name = "secondary-cross-connect"
cross_connect_group_id = oci_core_cross_connect_group.production_ccg.id
location_name = var.fastconnect_location
port_speed_shape_name = "10 Gbps"
is_active = true
}
# Query available FastConnect locations
data "oci_core_cross_connect_locations" "fc_locations" {
compartment_id = var.compartment_id
}Step 4: Virtual Circuit with BGP Configuration
resource "oci_core_virtual_circuit" "production_vc" {
compartment_id = var.compartment_id
display_name = "production-virtual-circuit"
type = "PRIVATE"
bandwidth_shape_name = "10 Gbps"
# Attach to DRG for VCN routing
gateway_id = oci_core_drg.production_drg.id
# Use cross-connect group for physical connectivity
cross_connect_mappings {
cross_connect_or_cross_connect_group_id = oci_core_cross_connect_group.production_ccg.id
# VLAN for traffic tagging on the physical circuit
vlan = 100
# BGP peering addresses - /30 subnet
# Oracle side uses the first usable IP, customer side uses the second
oracle_bgp_peering_ip = "169.254.0.1/30"
customer_bgp_peering_ip = "169.254.0.2/30"
}
# Customer BGP ASN
customer_asn = var.customer_asn
# BGP MD5 authentication key - store in OCI Vault, reference here
bgp_md5_auth_key = var.bgp_auth_key
# Customer on-premises routes to be reachable from OCI
public_prefixes {
cidr_block = var.onprem_cidr_block
}
defined_tags = {
"Operations.Environment" = "production"
"Operations.ManagedBy" = "terraform"
}
}
output "virtual_circuit_state" {
value = oci_core_virtual_circuit.production_vc.bgp_session_state
}
output "oracle_bgp_asn" {
value = oci_core_virtual_circuit.production_vc.oracle_bgp_asn
description = "Oracle BGP ASN to configure on your edge router"
}Oracle uses BGP ASN 31898 for FastConnect sessions. Your edge router needs this configured as the remote ASN for the BGP peer. The link-local addresses in the 169.254.0.0/16 range are used for the BGP peering session itself. They are not routable outside the circuit. The actual network prefixes you want accessible from OCI are advertised via BGP from your on-premises router.
Step 5: On-Premises Router BGP Configuration
The following is a reference configuration for a Cisco IOS-XE edge router. Adapt it to your specific router platform.
! Interface configuration for FastConnect circuit
interface GigabitEthernet0/0/0.100
encapsulation dot1Q 100
ip address 169.254.0.2 255.255.255.252
no shutdown
!
! BGP configuration
router bgp 65000
bgp router-id 10.0.0.1
bgp log-neighbor-changes
!
neighbor 169.254.0.1 remote-as 31898
neighbor 169.254.0.1 description OCI-FastConnect-Primary
neighbor 169.254.0.1 password YOUR-BGP-MD5-KEY
neighbor 169.254.0.1 soft-reconfiguration inbound
neighbor 169.254.0.1 route-map OCI-IN in
neighbor 169.254.0.1 route-map OCI-OUT out
!
address-family ipv4
neighbor 169.254.0.1 activate
! Advertise your on-premises network to OCI
network 10.100.0.0 mask 255.255.0.0
exit-address-family
!
! Route maps for BGP policy control
ip prefix-list OCI-PREFIXES seq 10 permit 10.0.0.0/16 le 24
!
route-map OCI-IN permit 10
match ip address prefix-list OCI-PREFIXES
!
route-map OCI-OUT permit 10
match ip address prefix-list LOCAL-NETWORKSStep 6: VCN Route Table for On-Premises Traffic
# Add a route in the VCN private subnet route table
# pointing on-premises traffic toward the DRG
resource "oci_core_route_table" "private_subnet_rt" {
compartment_id = var.compartment_id
vcn_id = var.vcn_id
display_name = "private-subnet-route-table"
route_rules {
destination = var.onprem_cidr_block
destination_type = "CIDR_BLOCK"
network_entity_id = oci_core_drg.production_drg.id
description = "Route to on-premises via FastConnect DRG"
}
# Default route for internet-bound traffic via NAT Gateway
route_rules {
destination = "0.0.0.0/0"
destination_type = "CIDR_BLOCK"
network_entity_id = var.nat_gateway_id
}
}
# Security list allowing traffic from on-premises
resource "oci_core_security_list" "allow_onprem" {
compartment_id = var.compartment_id
vcn_id = var.vcn_id
display_name = "allow-onprem-traffic"
ingress_security_rules {
protocol = "6"
source = var.onprem_cidr_block
source_type = "CIDR_BLOCK"
stateless = false
tcp_options {
min = 1521
max = 1522
}
description = "Oracle DB from on-premises via FastConnect"
}
ingress_security_rules {
protocol = "6"
source = var.onprem_cidr_block
source_type = "CIDR_BLOCK"
stateless = false
tcp_options {
min = 443
max = 443
}
description = "HTTPS from on-premises via FastConnect"
}
}Step 7: IPSec VPN as Automatic Failover
A single FastConnect circuit is a single point of failure. The physical port, the cross-connect cable, and the colocation facility all introduce risk. For production environments, configure an IPSec VPN as an automatic failover path that activates if the FastConnect BGP session drops.
resource "oci_core_cpe" "onprem_router" {
compartment_id = var.compartment_id
display_name = "onprem-cpe-device"
ip_address = var.onprem_public_ip
cpe_device_shape_id = data.oci_core_cpe_device_shapes.cisco.cpe_device_shapes[0].cpe_device_shape_id
}
resource "oci_core_ipsec" "failover_vpn" {
compartment_id = var.compartment_id
display_name = "fastconnect-failover-vpn"
cpe_id = oci_core_cpe.onprem_router.id
drg_id = oci_core_drg.production_drg.id
static_routes = [var.onprem_cidr_block]
# CPE config hint helps Oracle generate accurate device-specific configuration
cpe_local_identifier = var.onprem_public_ip
cpe_local_identifier_type = "IP_ADDRESS"
defined_tags = {
"Operations.Environment" = "production"
"Operations.ManagedBy" = "terraform"
}
}
resource "oci_core_ipsec_connection_tunnel_management" "tunnel_1" {
ipsec_id = oci_core_ipsec.failover_vpn.id
tunnel_id = data.oci_core_ipsec_connection_tunnels.vpn_tunnels.ip_sec_connection_tunnels[0].id
routing = "BGP"
bgp_session_info {
customer_interface_ip = "169.254.1.2/30"
oracle_interface_ip = "169.254.1.1/30"
customer_bgp_asn = var.customer_asn
}
# Lower BGP local preference on VPN so FastConnect is always preferred
# Configure this on your router: set local-preference 50 for VPN routes
# vs local-preference 100 for FastConnect routes
ike_version = "V2"
phase_one_details {
auth_algorithm = "SHA2_256"
encryption_algo = "AES_256_CBC"
dh_group = "GROUP20"
lifetime = 28800
}
phase_two_details {
auth_algorithm = "HMAC_SHA2_256_128"
encryption_algo = "AES_256_GCM"
lifetime = 3600
pfs_dh_group = "GROUP20"
}
}The failover mechanism works through BGP route preference. Your on-premises router advertises the same prefixes over both FastConnect and the VPN tunnel but sets a higher BGP local preference on the FastConnect routes. When FastConnect is healthy, all traffic uses the dedicated circuit. When the FastConnect BGP session drops, the router stops receiving the higher-preference routes and automatically uses the VPN path. No manual intervention is required.
Step 8: Monitoring the Circuit
resource "oci_monitoring_alarm" "fastconnect_bgp_down" {
compartment_id = var.compartment_id
display_name = "fastconnect-bgp-session-down"
is_enabled = true
metric_compartment_id = var.compartment_id
namespace = "oci_fastconnect"
query = "BgpSessionState[5m]{virtualCircuitId = '${oci_core_virtual_circuit.production_vc.id}'}.mean() 9000000000"
severity = "WARNING"
pending_duration = "PT10M"
destinations = [var.ops_notification_topic_id]
body = "FastConnect circuit utilization is above 90% of 10 Gbps capacity. Plan bandwidth upgrade to avoid saturation."
}Check circuit state and BGP session status from the CLI:
# Get virtual circuit lifecycle and BGP state
oci network virtual-circuit get \
--virtual-circuit-id ${VC_OCID} \
--query 'data.{state:"lifecycle-state", bgp:"bgp-session-state", bandwidth:"bandwidth-shape-name"}' \
--output table
# List advertised BGP prefixes from on-premises
oci network virtual-circuit get \
--virtual-circuit-id ${VC_OCID} \
--query 'data."public-prefixes"'
# List cross-connect status
oci network cross-connect list \
--compartment-id ${COMPARTMENT_ID} \
--query 'data[*].{name:"display-name", state:"lifecycle-state", port:"port-speed-shape-name", location:"location-name"}' \
--output tableOperational Notes
FastConnect provisioning is not instant. Physical cross-connect setup at a colocation facility takes days to weeks depending on the provider and location. Plan your timeline accordingly. The Terraform resources for the virtual circuit will be created quickly, but the circuit will remain in a PROVISIONING state until the physical layer is completed.
Use LAG with two physical cross-connects rather than a single cross-connect. A single port failure takes down the entire circuit. With LAG, one port failure reduces bandwidth but maintains connectivity. The cross-connect group resource in Terraform enables this automatically when you attach two cross-connects to the same group.
BGP MD5 authentication is not optional in production. Without it, anyone who can inject packets into the peering session can potentially disrupt your BGP session. Store the MD5 key in OCI Vault and reference it from Terraform using a data source rather than hardcoding it in your configuration.
Test the failover path before you need it. Bring the FastConnect BGP session down intentionally during a maintenance window, verify that traffic fails over to the IPSec VPN within your acceptable time window, and confirm that applications running in OCI can still reach on-premises resources through the VPN path. Document the failover time so you have a baseline when something changes.
Regards,
Osama
#OCI #OracleCloud #FastConnect #HybridCloud #Terraform #Networking #IaC #OracleCloudInfrastructure #CloudArchitecture #PlatformEngineering #DevOps #CloudNetworking #TechBlog #Oracle #BGP #PrivateConnectivity #NetworkEngineering #CloudSecurity #DRG #HybridIT
Technology Geek 
Leave a comment