BGP Vortex: Update Message Floods Can Create Internet Instabilities

BGP Vortex: Update Message Floods Can Create Internet Instabilities

Speakers: Felix Stöger

Conference: USENIX Security

YouTube: https://www.youtube.com/watch?v=dd6L1mdQLmk

Overview

The Border Gateway Protocol (BGP) forms the foundational routing fabric of the internet, orchestrating how data traverses autonomous systems (ASes) globally. For decades, the stability and convergence of BGP have been a cornerstone assumption, particularly underpinned by the seminal work of Gao and Rexford in 2001. This research, presented at USENIX Security 2025 by Felix Stöger and his co-authors, critically re-evaluates these long-held assumptions, revealing a profound vulnerability that can induce persistent and widespread internet instabilities.

The talk introduces the BGP vortex, a novel and potent attack that shatters the premise of BGP convergence. By exploiting specific BGP community attributes intended for traffic engineering, an adversary can trigger an endless cycle of route oscillations among a small group of ASes with just three well-formed BGP route updates. This localized instability then propagates across a significant portion of the global internet, leading to massive floods of BGP update messages, delayed route convergence, and prolonged data plane outages. The implications are severe, challenging fundamental beliefs about internet resilience and posing a significant threat to global network stability.

Background

▶ Watch: Introduction: BGP Vortex and its internet instability threat (0:00)

BGP operates as a distributed, hop-by-hop route dissemination protocol, where individual ASes make routing decisions based on local incentives, often economic. The question of whether such a decentralized system could reliably converge to a stable state was addressed by Gao and Rexford in 2001. Their work established that BGP systems converge under a critical assumption: ASes prefer routes learned from their customers over those from peers or providers. This preference hierarchy—customer > peer > provider—is rooted in the economic model of the internet, where customers pay providers for transit, peers exchange traffic without billing, and providers are paid by customers. An AS benefits most from customer traffic and incurs costs for provider traffic, naturally leading to this preference order.

Specifically, routes received from customers are typically re-announced to all other customers, peers, and providers because they represent revenue. Routes from peers are only re-announced to customers to avoid incurring transit costs through a provider for free peering traffic. Similarly, routes from providers are also only re-announced to customers to recoup costs. This economic logic, deeply embedded in BGP’s decision-making process, has historically been considered robust and essential for global routing stability.

However, the modern internet has evolved, introducing sophisticated traffic engineering capabilities that can subtly undermine these foundational assumptions. One such capability is the use of BGP communities, defined in RFC 1997. These are optional transitive attributes that can be attached to route advertisements, allowing for enhanced control over routing policies. The talk highlights two specific types of communities that are leveraged in the BGP vortex attack:

1. Local preference lowering community: This allows a customer to instruct its provider to reduce the local preference of a specific route. This can cause the provider to deprioritize a direct route from that customer, potentially making it less preferred than a route received from a peer or even another provider.
2. Do not forward route to specific AS community: This custom, but widely used, community allows a customer to tag a route, instructing the provider not to redistribute that specific advertisement to a designated AS.

These communities, while seemingly benign and designed for operational flexibility, introduce a mechanism to override the traditional customer > peer > provider preference hierarchy. By selectively manipulating local preference and route propagation, they create a pathway to violate the very conditions that Gao and Rexford identified as necessary for BGP convergence, thus opening the door to the BGP vortex.

Key Findings

▶ Watch: Foundational BGP convergence assumption (Gao & Rexford) (3:05)

The research unveils several critical findings that challenge the long-standing assumptions about BGP stability:

• Existence of BGP Vortices: The core discovery is the existence of BGP vortices, which are specific sub-topologies of the internet capable of inducing persistent, indefinite route instabilities. These vortices trap a set of ASes in continuous route oscillations.
• Minimal Trigger: A BGP vortex can be triggered by an adversary sending as few as three well-formed BGP route updates. Crucially, these updates are legitimate and BGP-conformant, requiring no hijacking, malformed packets, or other illicit activities, making the attack difficult to detect based on malfeasance.
• Widespread Susceptibility: The analysis of the Kaido topology, augmented with data on BGP community support, reveals a pervasive vulnerability. Out of the 30 largest ASes, 21 were found to support the necessary BGP communities. Of these, 20 are contained in at least one peering triangle that forms a BGP vortex. In total, 340 viable BGP vortices were identified among these 20 ASes alone.
• Massive Internet Coverage: Due to the central role of these large ASes, the customer cone of the identified BGP vortices covers an enormous portion of the internet—over 96% of all ASes are within the customer cone of at least one BGP vortex. This means almost the entire internet could potentially be affected by updates emanating from these oscillations.
• High Update Rates: A single BGP vortex can generate a significant number of route updates. In the median, each oscillation of a BGP vortex propagates 801 BGP route updates into its customer cone. With oscillation periodicities ranging from 6.6 to 40 periods per second, this translates to a staggering 5,000 to 32,000 BGP route updates per second being received by an AS in the median customer cone if multiple vortices are triggered.
• Impact on Convergence and Data Plane: Experiments demonstrated that BGP vortices significantly delay route convergence and cause prolonged data plane outages. Route convergence can be delayed by almost 40 seconds in worst-case scenarios, even with a small number of fluctuating prefixes. Data plane outages were observed to last over 30 seconds, directly impacting network reachability and service availability.
• Vulnerability of Major Implementations: Widely used BGP router implementations, including Cisco IOS XR 9000, FRR (Free Range Routing), and BIRD, were found to be susceptible to BGP vortices, with varying fluctuation speeds.

These findings collectively demonstrate that the long-standing assumption of BGP stability, contingent on the economic preference model, no longer holds true in the presence of modern traffic engineering capabilities.

Technical Deep Dive

▶ Watch: BGP business relationships and route preference logic (4:40)

The BGP vortex attack exploits a specific sub-topology and two BGP community attributes to induce infinite route oscillations. The core topology required for a BGP vortex consists of three benign peering ASes (let's call them AS2, AS3, AS4) and one malicious AS (AS1) that is a customer to all three. All three benign peers must support both the local preference lowering community and the do not forward route to specific AS community. The adversary's only capability is to control a BGP speaker in AS1 and send legitimate, well-formed BGP updates.

The attack unfolds in two phases:

1. Seeding the malicious advertisement: The adversary (AS1) initiates the attack by announcing a specific IP prefix P to its three providers (AS2, AS3, AS4). Each announcement is carefully crafted with BGP communities:

• To AS2: P is tagged with the local preference lowering community and a "do not forward to AS4" community.
• To AS3: P is tagged with the local preference lowering community and a "do not forward to AS2" community.
• To AS4: P is tagged with the local preference lowering community and a "do not forward to AS3" community.

This setup induces a specific directionality or cyclicity (e.g., clockwise) in how the route will be preferred and propagated among the peering ASes. The local preference lowering community ensures that the direct route from AS1 will be less preferred than a route received via a peer. The "do not forward" communities prevent ASes from propagating the route back to specific peers, completing the cycle.

1. Inducing Infinite Oscillation: Once the initial advertisements are seeded, the three peering ASes (AS2, AS3, AS4) begin to oscillate the availability of the route P by themselves. Let's trace a simplified version of the oscillation, assuming an initial clockwise direction:

• Initial State: Each AS (AS2, AS3, AS4) receives route P directly from AS1. Due to the local preference lowering community, this direct route is deprioritized compared to a potential route from a peer.
• AS2's Action: AS2, having received P from AS1, can only re-announce it to AS3 (due to "do not forward to AS4").
• AS3's Decision: AS3 now has two options for P: its direct route from AS1 (deprioritized) and the route from AS2 (a peer). Because the direct route from AS1 has been tagged with the local preference lowering community, AS3 will prefer the route received from its peer, AS2. AS3 updates its routing table to use the path via AS2. However, since this new best route was received from a peer, AS3 will not redistribute it to other peers (specifically AS4, as per standard BGP peering policy: peer routes are not sent to other peers). AS3 also withdraws its direct route to AS1 (since it's no longer the best path).
• AS4's Action: AS4, not having received a new route from AS3, still has its direct (deprioritized) route from AS1. It will then announce its route P to AS2 (due to "do not forward to AS3").
• AS2's Decision: AS2 now has its current best route (from AS1 via AS3, if it had one, or its own deprioritized direct route) and a new route from AS4 (a peer). Again, due to the local preference lowering community, AS2 will prefer the route via AS4. AS2 updates its routing table, invalidating and withdrawing its old direct route to AS1 (or its route via AS3). Since this new route is peer-received, AS2 will not re-advertise it to AS3.
• AS3's Dilemma: AS3 now loses its most preferred route (the one via AS2) because AS2 withdrew its route to AS1. With no other preferred peer route, AS3 falls back to its less preferred direct route from AS1. This direct route, having come from a customer (AS1), can be re-advertised to peers. So, AS3 re-advertises its direct route P to AS4.
• AS4's Decision: AS4 now receives a new route P from AS3. It will compare this with its current best route (the one via AS2, if it had switched, or its own deprioritized direct route). This new route from AS3 is now preferred, causing AS4 to withdraw its previously announced route to AS2.
• The Cycle Continues: AS2, having lost its preferred route from AS4, falls back to its direct route from AS1, which it then re-advertises to AS3. This completes the cycle, and the process repeats indefinitely, with each AS continuously oscillating between different paths for prefix P, causing a cascade of route updates and withdrawals.

This cyclical manipulation of BGP preferences and propagation rules, enabled by specific BGP communities, effectively traps the ASes in a state of perpetual instability, generating a constant flood of update messages that ripple outwards into the broader internet.

Demo / Proof of Concept

▶ Watch: BGP communities enabling traffic engineering attacks (6:15)

To validate the theoretical underpinnings of the BGP vortex and quantify its impact, the researchers conducted practical experiments using real-world BGP router implementations.

Firstly, they confirmed the susceptibility of widely used BGP software. By constructing BGP vortices with combinations of Cisco IOS XR 9000 (a commercial virtual router), FRR (Free Range Routing), and BIRD (two popular open-source implementations), they found that all three are vulnerable to the BGP vortex attack. The oscillation speeds varied, with BIRD exhibiting the fastest fluctuations and FRR the slowest. This demonstrates that the vulnerability is not specific to a niche implementation but affects the core technologies underpinning the internet.

Secondly, two key experiments were performed to measure the impact:

1. Route Convergence Delay Experiment:

• Setup: A BGP vortex was instantiated using BIRD routers (chosen for their faster fluctuation speed). This vortex was connected to an upstream AS running FRR and a bystander AS also running FRR.
• Methodology: The experiment measured the time it took for the upstream AS to announce a route and for the bystander AS to receive and install this new route in its routing table. This process was observed with varying numbers of prefixes fluctuating inside the BGP vortex.
• Results: The BGP vortex significantly delayed data plane convergence. Even in a relatively small example topology consisting of just six ASes, the convergence time was extended by almost 40 seconds in the worst case. This is a dramatic increase compared to the near-instantaneous convergence observed when the BGP vortex was inactive, illustrating how the constant churn within the vortex hinders stable route propagation.

1. Data Plane Outage Experiment:

• Setup: This experiment extended the previous topology by adding a downstream AS. This downstream AS was configured with a primary path through the bystander AS and the BGP vortex, and a backup path directly to the upstream AS.
• Methodology: The goal was to determine how long a data plane outage would persist if traffic was sent through the more preferred (primary) route, and a network outage occurred within the BGP vortex. The time taken for this outage information to propagate and cause the downstream AS to switch to its backup route was measured.
• Results: Corroborating the control plane findings, data plane outages were observed to last over 30 seconds. The duration of the outage showed a distinct upward trend with an increasing number of routes oscillating within the BGP vortex. This demonstrates that the BGP vortex not only disrupts routing tables but directly impacts the availability of network services for end-users.

These experiments provide concrete evidence that BGP vortices are not just a theoretical construct but a practical threat capable of causing significant disruptions to internet stability and performance.

Defensive Implications

▶ Watch: Visualizing local preference lowering community impact (7:10)

Addressing the BGP vortex requires a multi-faceted approach, ranging from immediate policy adjustments to long-term architectural considerations. The researchers propose both partial mitigations and a concrete solution to tackle the root cause.

Partial and Temporary Mitigations: The paper discusses temporary measures to alleviate the immediate impact of a BGP vortex, such as rate-limiting BGP updates or more aggressive damping strategies. However, these are acknowledged as symptomatic treatments that do not resolve the underlying vulnerability.

Alternative Internet Architectures: The talk briefly touches upon alternative internet architectures, such as SCION (Scalability, Control, and Isolation On Next-generation networks), which are path-aware and designed with different fundamental assumptions, making them inherently more resilient to such routing instabilities. While these represent promising long-term solutions, their widespread adoption is a monumental undertaking.

Concrete Mitigation: Enforcing Gao-Rexford Conditions: The most direct and actionable mitigation proposed targets the core insight exploited by the BGP vortex: the ability of BGP communities to violate the fundamental Gao-Rexford routing assumptions. Specifically, the local preference lowering community allows customers to reduce the preference of their routes to a level below that of peer routes, directly inverting the crucial customer > peer preference hierarchy.

The proposed concrete mitigation is for network operators to pay very close attention to and disallow BGP communities that violate these fundamental Gao-Rexford conditions. In practice, this means:

• Strictly enforcing preference hierarchies: Providers should configure their BGP speakers to ensure that routes received from customers, even if tagged with a local preference lowering community, never fall below the preference level assigned to routes received from peers.
• Auditing BGP community policies: Operators need to audit their existing BGP community implementations and policies to identify any configurations that could inadvertently allow a customer route to be treated with lower preference than a peer-received route.

The rationale is that while customers should be able to reduce the preference of their routes for traffic engineering, this reduction must not cross the critical threshold that makes a customer route less desirable than a peer route. As long as the reduced preference for a customer route remains strictly greater than that of a peer route, the BGP convergence model holds. It is only when this boundary is crossed that the conditions for a BGP vortex emerge. By preventing this inversion, operators can effectively neuter the primary mechanism by which BGP vortices are formed, restoring the stability assumptions that BGP relies upon.

Key Takeaways

• BGP Convergence Assumptions are Broken: The long-standing assumption of BGP stability, based on economically rational preference hierarchies (customer > peer > provider), is no longer universally valid due to modern traffic engineering capabilities.
• BGP Vortices Induce Infinite Instability: A new attack, the BGP vortex, can trap three benign peering ASes in indefinite route oscillations using just three well-formed BGP updates from a malicious customer.
• Widespread Vulnerability and Impact: Over 96% of the internet's ASes are within the customer cone of identified BGP vortices, involving 20 out of the 30 largest ASes. This can lead to 5,000 to 32,000 BGP updates per second in affected regions.
• Significant Service Disruption: BGP vortices cause severe network instability, leading to route convergence delays of up to 40 seconds and data plane outages lasting over 30 seconds.
• Major Implementations are Susceptible: Widely used BGP router software, including Cisco IOS XR 9000, FRR, and BIRD, are all vulnerable to this attack.
• Mitigation Requires Policy Enforcement: The most effective immediate mitigation is for operators to disallow BGP communities that enable customers to lower the local preference of their routes below that of peer routes, thereby restoring the fundamental Gao-Rexford convergence conditions.

About the Speaker(s)

This research was presented by Felix Stöger at the USENIX Security Conference 2025. It is a joint project developed in collaboration with his co-authors, Henry Burchley, Jako Jiuliari, Jodi Subarinetto, and Adrien Perrick. While specific affiliations were not detailed in the presentation, the nature of the work, involving deep analysis of BGP behavior, internet topology, and router implementations, suggests a background in network security research, likely within an academic or institutional setting. Their collective expertise has led to a critical re-evaluation of fundamental internet routing stability.