Research conducted and written by Ari Juels, Lorenz Breidenbach, and Florian Tramèr of Chainlink Labs
As Ethereum’s popularity continues to increase thanks to tokenization, then DEXes, and now DeFi, we’re seeing unprecedented, consistently high gas prices for users of the network. While this is a mark of Ethereum’s tremendous success, it unfortunately means higher transaction costs and longer mining latencies for users. As transactions languish in the mempool, another undesirable side effect occurs: Transactions become more vulnerable to front-running.
Even when gas prices are low, as users’ transactions pass through the mempool, savvy traders and bots get a sneak peek at them before they are mined. Sophisticated players of this kind can profit from this form of advance intelligence by strategically generating their own transactions. By then paying a higher gas price than the transactions they exploit, they can ensure that their own transactions are processed first.
This problem isn’t hypothetical. A recent study by Daian et al. (including authors of this blog) documented millions of dollars of such activity per year in just a subset of DeFi contracts.
Frontrunning gains today are almost certainly higher than the Daian et al. paper indicates. And given that most trading activity happens in centralized exchanges, where frontrunning by operators and potentially others is possible, what that study found may be just the tip of the tip of the tip of the iceberg. Wall Streeters have known for many years how lucrative the ability to front-run trades can be, and firms regularly sell their clients’ orders to high-frequency traders. The example below illustrates the issue as it can affect smart contracts today.
Ordering problems in DeFi and beyond
DeFi contract operators want fairness and transparency for their customers. They often include countermeasures to front-running, such as gas caps. But these are not foolproof. Additionally, they provide no protection against adversarial miners, which of course can order transactions as desired. Front-running of oracle reports and other transaction types pose a similar threat.
Daian et al. coined the term Miner-Extractable Value (MEV) to denote opportunities to extract money from a smart contract system through adversarial transaction ordering. The term MEV is a little misleading: Some (or most) MEV can be extracted by ordinary users, not just miners. Miners have unlimited power to order transactions in a block, so MEV just represents an upper bound on the value that other classes of adversaries can extract. MEV arises because of a dirty little open secret in most blockchains: They’re temporarily centralized, in the sense that a single miner gets to decide unilaterally how to order transactions in each mined block.
Recently, proposals have arisen to create services that systematize MEV exploitation, sometimes referred to as MEV auctions. While there are legitimate arguments for and against MEV auctions, we believe that MEV auctions are likely to be detrimental to the community. A compelling argument to this effect is given here.
New transaction-processing approaches like rollups, an excellent idea with many benefits for scalability and transaction privacy, unfortunately don’t help with the MEV problem either. A rollup places the power to order transactions in the hands of whoever rolls it up. It basically swaps MEV for REV: Rollup-Extractable Value.
Introducing The Fair Sequencing Service (FSS)
As we think through the next phase of the Chainlink roadmap, we’ve been developing an idea in Chainlink Labs that can help make transaction ordering fair in decentralized (and even centralized) systems. It can help resolve the ordering problems in rollups and DeFi more generally. In fact, it takes aim simultaneously at the problems of both high gas costs and unfair transaction ordering. It’s called the Fair Sequencing Service (FSS).
FSS provides benefits to any contract written to use it. Contracts can benefit from FSS without any changes to the layer-1 blockchain on which they run.
In a nutshell, the idea behind FSS is to have an oracle network order the transactions sent to a particular contract SC, including both user transactions and oracle reports. Oracle nodes ingest transactions and then reach consensus on their ordering, rather than allowing a single leader to dictate it. Oracle nodes then forward the transactions to the contract SC. They sequence these transactions by attaching nonce or sequence numbers to them or sending them in batches.
Like MEV auctions, FSS decouples the ability to extract MEV from the ability to mine blocks, but unlike MEV auctions, doesn’t inherently incentivize front-running. In general, FSS removes the temporary centralization of mining and decentralizes the process of transaction ordering.
Given the problems with gas-based ordering, though, what criteria should oracle nodes use to order users’ transactions? A natural option is time of arrival, i.e., sequencing transactions on a first-come-first-serve basis. But what precisely does that mean?
The question of how transactions should be ordered isn’t trivial. In fact, it’s deep and challenging.
It turns out that decades of research on Byzantine Fault Tolerance, i.e., consensus for permissioned systems, has overlooked the issue of order-fairness. Most BFT systems aim to achieve liveness and consistency, meaning that transactions need to be processed in a timely way and that all nodes need to have the same view of their ordering. But these properties say nothing about how transactions close in time should be ordered. Even if all nodes receive T1 before T2, the usual requirements of BFT systems deem it acceptable for T2 to be ordered before T1.
Recent work by Kelkar et al. (which one of us co-authored) is the first to set forth a formal notion of the property of order-fairness in consensus. While the details are complicated, the general idea is that if a large fraction of nodes receive T1 before T2, then T1 shouldn’t be scheduled after T2. That work also proposes a family of protocols called Aequitas to achieve this property. Those protocols incur undesirable performance overhead, but we believe that coming advances will soon yield more practical variants.
FSS is a framework for implementing ordering policies, of which Aequitas is one example. It can alternatively support simpler approaches, such as straightforward encryption of transactions, which can then be decrypted in a threshold manner by oracle nodes after ordering. It will also support various policies for inserting oracle reports into a stream of transactions. (It can even support MEV auctions, if desired.)
In short, the goal of FSS is to help DeFi developers ensure that their instruments treat users fairly, according to their own criteria.
How do transactions reach FSS?
Users can submit their transactions directly to the oracle network. To make fair ordering work by giving multiple nodes a perspective on transaction order, a user would submit her transaction simultaneously to multiple nodes.
We’re exploring a variant in which oracle nodes monitor the mempool of a target blockchain and pluck transactions from it on behalf of a relying contract SCON. Oracle nodes typically use web services as data sources. But they can also treat the mempool as a data source and generate reports that correspond to user transactions. We depict the idea in the figure below.
Such use of the mempool is a powerful idea with two big benefits:
- Legacy compatibility: Users can transmit transactions directly to the blockchain. There’s no need to provide them with special-purpose software for interacting with the oracle network.
- Lower gas costs: If getting a transaction into the mempool suffices to ensure receipt, then users don’t need to worry about mining speed: They can send transactions with low gas prices. When these transactions are eventually mined, they provide an auditable record showing that transactions weren’t censored. The oracle network can re-transmit sequenced user transactions to a relying contract with a high gas price, ensuring timely processing. But by using batching, rollups, or any of a number of other techniques, the network can keep per-transaction gas costs low.
Peer-to-peer networks are complicated, and an adversary with many peers and a fast network connection might be able to front-run users. But if oracle nodes have peers distributed throughout the network and actively detect flooding and other deceptive practices by adversaries, they can significantly raise the bar for front-running attempts.
Additionally, some of the same techniques that have been proposed to combat front-running directly in smart contracts (e.g., commit-reveal schemes or verifiable delay functions) could also be applied by the oracle nodes. The crucial advantages here is that the oracle nodes would implement these defenses off-chain, and to prevent front-running on much smaller time scales (on the order of the network delay, rather than the block interval).
The Fair Sequencing Service also enables new, simpler approaches to combat front-running at the network-level. For example, users can threshold-encrypt transactions sent to oracle nodes, so that the transaction content is revealed only after the transaction ordering has been agreed upon. Encrypting transactions prior to ordering them is a simple and powerful way of defeating front-running in DEXes and other DeFi systems based on transaction content, but cannot prevent more subtle forms of front-running based on metadata alone. (E.g., simply knowing that some user has issued a transaction could cause other users to front-run it, or cause colluding orderers to prioritize it.) Defending against this more powerful form of front-running is possible in Ethereum using ideas like Submarine Sends, and it is an interesting open question whether similar defense mechanisms could be deployed at the network level.
Even systems that seem to achieve fairness in isolation, like batch auctions, don’t compose well with other systems. Fairness in a broad sense is and will always be a tricky thing to achieve. Our goal with FSS is to give DeFi developers the tools they need to make their systems fairer according to principled policies of their choice, while keeping a UX with a familiar user experience. We’ll have more to say about FSS in our upcoming whitepaper. Our hope, though, is that fair ordering of transactions can help DeFi achieve its promise of a more egalitarian financial system—ideally one much fairer and more transparent than Wall Street's.
Thanks to Andrew Miller, Dan Moroz, Fan Zhang, and Sergey Nazarov for comments on a draft of this blog post.
1. Literally. The portion of an iceberg above water is about 10%. DEX volume is around 0.1% of total trading volume across all exchange types.
2. The Latin for “fairness.”
3. For users concerned about censorship, a slower direct channel to the contract can operate in parallel with oracle-based submission.
4. Users can replace transactions by resending them with higher gas costs. How overwritten transactions get handled by the mempool is a matter of policy. For example, the oracle network might treat only the first transmitted transaction as valid and require users not to overwrite transactions if they want full auditability.
5. A nice related approach has been proposed by Khalil et al. for centralized exchanges.
6. If a batch-auction system sells assets that are also sold in a continuous-time market, then participants in the latter can perform auction sniping—deciding at the last moment whether to participate in a batch based on up-to-date market information.