Circuit Breakers and Client Diversity Within the Chainlink Network
Chainlink Price Feeds are purpose-built to provide DeFi applications with the maximum amount of price oracle security, reliability, and data quality. These properties are generated by focusing on several key design features:
- Decentralization at both the oracle node and data source levels to prevent any single points of failure
- Selection of secure node operators and premium data sources to ensure every individual component is highly reliable
- Provable security with on-chain performance metrics that users can independently monitor
- Crypto-economic incentives to ensure price feed updates occur even during extreme blockchain network congestion and unexpected hard forks.
These strong guarantees have led to the widespread adoption of Chainlink Price Feeds, which now secure billions of dollars in user funds across DeFi. As the amount of value being secured by Chainlink grows, we continue to explore additional ways in which developers can further secure their smart contracts. While Chainlink Price Feeds are designed to guarantee reliability even during the recent flash loan attacks, there are additional layers of redundancy in the form of circuit breakers and Chainlink client diversity that generate hyper-reliability for Chainlink Price Feeds, protecting user funds even amidst an unexpected black swan event.
The Multiple Layers of Redundancy in Chainlink Price Feeds
To ensure the highest level of reliability and tamper-resistance, Chainlink Price Feeds feature three layers of aggregation to filter out outlier data and hedge against black swan events.
First, Chainlink Price Feeds source data from professional data aggregators (e.g. BraveNewCoin), which fetch market data from all trading environments (both off-chain and on-chain exchanges) to create a single, volume-adjusted market-wide average price. Second, each Chainlink oracle node pulls data from multiple data aggregators and takes the median value, ensuring each node’s individual response is resistant to API downtime and doesn’t reflect merely a single source of data. Third, the final price consumed by a smart contract represents the median value from numerous independent, security-reviewed node operator responses, preventing any single node from being a point of failure.
These multiple layers of aggregation already provide an extreme level of reliability. However, the Chainlink Network also enables two additional layers of redundancy to further ensure smart contracts receive a consistent stream of high-quality data. This includes:
- Historical circuit breakers – an additional price point to compare the latest price feed update against, where a flag is raised if the difference between the two is beyond a predefined percentage threshold. This optional additional layer of security provides a backstop to prevent potentially outlier data from being consumed by contracts and can be based on the last Chainlink Price Feed update (historical) or a separate user-operated price feed.
- Diversity of clients – Chainlink oracle networks always operate multiple versions of the Chainlink oracle client software, providing automatic failover to previous client versions should any unexpected event occur in the current version. While it hasn’t happened before, client diversity provides protection against any unexpected bug in the node software.
By building in multiple layers of redundancies and optional safety nets, the Chainlink Network provides users with strong guarantees that their smart contracts will always receive high quality and extremely secure data in a reliable manner. To showcase these key pieces of infrastructure used for redundancy, we’ll explore good and bad circuit breaker designs and the benefits of client diversity.
Understanding Traditional and Chainlink Circuit Breakers
A circuit breaker is an automated device in nearly every occupied building designed to protect an electrical circuit from damage caused by large fluctuations in current. Its basic function is to detect and prevent large changes to the power grid’s output from adversely affecting any electronics connected to the circuit. Circuit breakers protect your own home every day but are also used to secure mission-critical equipment like data warehouse servers, hospital ventilators, life support systems, and other uptime sensitive infrastructure. In essence, circuit breakers are the last line of defense before a dangerous amount of current reaches an electronic device.
A typical electrical circuit breaker in someone’s home
The concept of circuit breakers can also be seen throughout traditional financial markets to trigger a temporary halt in trading when there is a large surge in downward price movement. The circuit breaker trips when the price moves more than a defined percentage within a given amount of time. For example, the S&P 500 Index has a circuit breaker that triggers a 15-minute delay in trading when there is a 7% decrease against the prior day’s closing price. It is designed to protect against panic selling, cascading liquidations, and unnatural deviations from the wider market. Similar to the electrical breakers in your home, these financial market circuit breakers are the last line of defense against undesired damages.
Historical Chainlink Circuit Breaker
Historical Chainlink circuit breakers are an optional layer of redundancy for Chainlink Price Feeds that are triggered if and when the latest update from a Price Feed deviates beyond a predefined threshold from the previous round or an aggregation of multiple past rounds (e.g. average of the past 5 price updates). If there is an abnormal deviation detected, a flag is raised which can trigger application-specific logic for dApps, such as a temporary pause in trading or preventing new collateralized loans from being opened.
In addition to being conceptually simple, Chainlink’s Historical Circuit Breaker design provides an easy to integrate and gas-efficient layer of redundancy. The threshold used within the circuit breaker (e.g. 10%) is defined on a per-price feed basis, providing a high degree of flexibility. For example, a historical circuit breaker for a stablecoin/FX price feed may have a tighter deviation than a price feed for highly volatile assets, allowing for customized adjustments specific to each project and use case.
User-Operated Chainlink Circuit Breaker
Chainlink can also support the creation of custom circuit breakers where a user chooses to launch a Chainlink oracle node in order to create a user-operated circuit breaker network that can be used together with Chainlink’s existing Price Reference Feeds. This allows a project to retain some control over the data being consumed by their smart contracts by independently running the circuit breaker responsible for securing the protocol against any black swan event.
Please note: Chainlink circuit breakers are not a multi-oracle solution for mixing data, as this introduces a large number of unintended attack vectors, particularly around dilution of the inherent high levels of security and data quality offered by Chainlink Price Feeds. When you mix vinegar and wine, you don’t get better wine. Instead, Chainlink Price Feeds operate as the primary oracle solution supplying applications with high-quality aggregated price data with full market coverage, while the optional circuit breaker simply raises a flag and triggers a subsequent action in response to detecting a large deviation.
CAUTION: Avoid DEX-Based Circuit Breakers
While the integration of a secure and reliable circuit breaker can serve as a protective backstop during a worst-case scenario, a poorly designed circuit breaker can actually introduce several new and unintended risks. In particular, using a Decentralized Exchange (DEX)-based price feed as a circuit breaker can easily lead to false positives being triggered during times of market volatility, which is when the circuit breaker is most needed.
The reason being is that the application using the DEX-based circuit breaker exposes itself to either data manipulation attacks or stale price data. In the event that the circuit breaker represents a DEX spot price, it becomes extremely vulnerable to sandwich attacks, where a large trade temporarily skews the price reported by the DEX in comparison to the wider market. This attack vector has been increasingly exposed recently using flash loans, thus we absolutely do not recommend a DEX spot price as a circuit breaker.
If the circuit breaker reflects a DEX Time Weighted Average Price (TWAP), the user can achieve resistance to sandwich attacks but at the direct expense of price accuracy. That’s because TWAP represents the average price over an extended period of time, leading to a dynamic where price oracle security is inversely correlated with price accuracy. Thus, a TWAP circuit breaker will often report stale prices that lag behind the wider market during high volatility situations, introducing serious risks to user funds.
For example, if a decentralized money market was using a TWAP-based circuit breaker during a rapid 50% market crash, the TWAP circuit breaker could easily become out of sync with the wider market for an extended period of time. As a result, the circuit breaker would generate a false positive, causing liquidations and/or user deposits to be disabled. When the TWAP circuit breaker later catches up the market wide price and allows liquidations to occur again, such liquidations would be unprofitable as the collateral would have dropped too much in value compared to the debt that needs to be paid back. The lack of incentives could result in a major undercollateralization of the protocol, creating an insolvency situation and a direct loss of user funds.
Because of the risk of stale pricing data, we recommend developers avoid TWAP-based circuit breakers and instead use a Historical or user-operated Chainlink Circuit Breaker, as they provide more accurate data and can scale security without sacrificing accuracy (through the addition of more oracle nodes and data sources).
Diversity of Chainlink Software Clients
Further adding to the hyper-reliability of the Chainlink Network, Chainlink oracle node operators are always running multiple different client versions of the Chainlink Core software. This ensures that if one version were to fail, nodes can seamlessly fall back to a separate client with an entirely different codebase, allowing nodes to continue fetching and delivering external data on-chain for smart contracts. The Chainlink nodes powering Price Reference Data Feeds use this system design by operating Flux Monitor as the primary oracle client and RunLog in the background as a backup oracle client.
With the ability for nodes to independently switch between oracle clients, the probability of a single bug taking down the Chainlink Network is mitigated. This approach of client diversity increases network liveness through additional redundancies and is a well established and proven strategy deployed by multiple mission-critical networks such as Ethereum ,which consists of a multiple of clients, including Geth and Open Ethereum (formerly known as Parity).
With the launch of Chainlink’s Off-Chain Reporting functionality, node operators will have an additional third oracle software client at their disposal, offering even more client diversity. While there hasn’t been a situation where Chainlink nodes needed to switch to a backup oracle client, oracle nodes do have the capability, which serves as a powerful primitive to mitigate any type of black swan event should one client implementation experience any issues.
Through the use of multiple built-in layers of Price Feed aggregation and several carefully constructed optional backstops in the form of circuit breakers and client diversity, the Chainlink Network provides a robust defense-in-depth approach that ensures users have the most secure and reliable oracle solution in the market. This enables users to receive all the benefits of Chainlink Price Feeds around oracle security, reliability, and data quality, while also having the tools to mitigate unexpected situations, allowing DeFi to fully benefit from Chainlink’s large and growing network effects.
Start Building with Chainlink Today
If you are interested in integrating a highly reliable Chainlink oracle network for your decentralized application, reach out to us here and we can assist in the integration process.
If you are a DeFi project and would like to launch a new Chainlink Price Feed or integrate an existing one, visit the developer documentation or schedule a call to discuss it more in-depth. You can easily integrate one or many Chainlink oracle networks live on mainnnet and testnets today, adding more security and capabilities to your smart contracts.