The best price is not necessarily the best adjusted price. Just like buying a plane ticket, there may be a lot of follow-up expenses and costs.
Further reading:
“
Original Title: “Introduction | Comprehensive Analysis of DEX Aggregator Performance”
Author:
Translation & Proofreading: Min Min & A Jian
Too long to watch
- In 70% of cases, the adjusted price provided by 0x API is better than 1inch, Dex Ag, Paraswap and Uniswap. The so-called adjusted price is the price after the transaction fee has been paid.
- 0x API 1.0 is the market leader in gas efficiency. 0x API provides a highly optimized custom architecture. Therefore, the cost for users to access Uniswap liquidity through this API is as low as or even lower than that of direct access to Uniswap.
- The aggregator promises to allow users to get the best quotation for the entire ecosystem, to eliminate the psychological barriers of users when choosing a trading platform. However, not all aggregators are the same: they differ to varying degrees in terms of pricing, quote accuracy, gas usage, revert rate, and response time.
- The best price is not necessarily the best adjusted price. Just like buying a ticket, there may be a bunch of follow-up expenses/costs. Although the base fare may be very cheap, when you include all additional fees (baggage fees, priority boarding fees, designated seat fees, etc.), the actual cost may be very high.
- 0x API and Matcha (0x API is used in the background) are two products launched by 0x Labs, which allow DeFi developers and traders to easily buy and sell assets at the best price among all DEX liquidity sources.
The battle for DEX liquidity and the rise of aggregators
A large number of new sources of liquidity have emerged in the past few weeks. There are more than $4 billion in funding distributed across 15 sources, and new sources will appear every week. As a result, DEX liquidity has become more dispersed than ever, making it more difficult for DeFi users to find the best price for a particular transaction.
This promoted the rise of aggregators. Over the past 12 months, the aggregator has made tremendous development, and its transaction volume accounted for about 9.6% of the total transaction volume in the third quarter. However, people misunderstand aggregators. For example, although quotes are usually regarded as the only indicator for comparing the performance of aggregators, people often encounter situations where aggregators quote $100 but settle at $90. The quoted price is not the actual settlement price. This begs the question: what is the correct index to measure the performance of the aggregator?
This article aims to reveal the current state of the aggregator industry and how to objectively measure and compare the performance of aggregators.
Adjusted prices-compare the gold indicators of aggregators
We believe that to build a high-performance professional-grade aggregator, the following principles must be followed:
1. Accuracy : The aggregator should have accuracy, that is, the settlement price should be equal to the quoted price. Slight differences are acceptable because there are often market fluctuations between the time the transaction is submitted and the time the transaction is confirmed on the chain. However, all other conditions remain the same, the more accurate the quotation, the better. What you see is what you get.
2. Low cost : The more operations performed on the chain, the more gas usage, which in turn leads to higher gas costs. The goal of the aggregator should be to reduce costs as much as possible to ensure that users get the best transaction price. The aggregator should adapt to the current gas price market and adjust its path accordingly. Every penny spent on gas should bring more value to users.
3. Low rollback rate : In order to protect users, the aggregator must control the rollback rate of transactions of any size within a low range. If the transaction fails, the user will feel very frustrated because the gas fee has already been paid.
Finally, we summarize the above principles into a gold indicator: adjusted price .
Although we analyzed the above indicators separately, this article focuses on adjusted prices. The adjusted price refers to the price after the user has paid the transaction cost (gas fee and handling fee) when the transaction is completed on the chain . The calculation formula is as follows:
Adjusted price = USD value of sold assets / (USD value of purchased assets-USD value of transaction costs)
I believe that this metric is more suitable for comparing aggregators, because this is the price at which users ultimately affect user revenue. Let’s look at an example:
A user wants to spend 100 DAI to buy USDC. The price of aggregator 1 is 100 USDC, and the transaction cost is 5 USD. According to the above formula, we can get the adjusted price = 100/(100-5) = 1.05 USD.
Look at another aggregator. For the same 100 DAI, the price of aggregator 2 is 104 USDC, but the transaction cost is 10 USD. In this case, the adjusted price = 100/(104 – 10) = 1.06 USD.
In the above example, aggregator 1 has found a cheaper price. Although the price of aggregator 2 is higher, we can see the impact of transaction costs on the adjusted price. All in all, it is obviously slightly more expensive to buy 4 USDC for an extra $5.
We simulated 59,975 transactions and calculated adjusted prices on different exchanges, and came to the following conclusions:
_0x API, 1inch, Dex Ag, Paraswap and Uniswap adjusted price win rate. The adjusted price provided by 0x API is better than other aggregates in 72% of cases _
On average, 0x API outperforms other exchanges in 70% of cases. In other words, the probability of a trader using the 0x API aggregator (such as Matcha) to get the best price of all DEX is 70% .
As far as the adjusted price is concerned, we found that as the transaction volume increases, the exchange’s winning rate will change. In terms of small transactions, Uniswap performs better because its transaction costs are lower than most aggregators. As the transaction volume increases, the aggregator will perform its functions and begin to split transactions into different sources. For transactions of any size, the 0x API performs well. As the transaction volume increased, aggregators began to show their competitiveness. For example, in terms of large transactions, 1inch is more competitive than 0x API because it has more sources.
Methodology
The data shared in this section comes from a simulation tool built by our 0x Labs. In the face of different token pairs, transaction amounts and confirmation delays, we can continue to quote and execute transactions through this tool. If you know the technology, you will understand as soon as I say it: this simulation is to execute eth_call
calls on the Geth node, and use state overrides to fund and track transactions, including indicators such as the number of transactions and gas usage .
The exchanges we track include 0x API, 1inch, Dex Ag, Paraswap and Uniswap. In this study, we selected the 20 most common trading pairs on these DEXs, and took 11,995 transactions per exchange on average through the public API as a sample. The transaction volume ranges between 100 USD and 1 million USD, and the settlement delay is between 0 and 180 seconds (simulating the waiting time in the trading pool). The simulation lasted about 78 hours, and the gas price fluctuated between 47 and 194 gwei, which ensures that our research reflects market conditions.
In the graphs in this report, the green bars and Matcha both represent 0x API. Only the exchange whose adjusted price exceeds 0.005% of other exchanges will be considered the final winner. If the gap is within 0.005%, it is considered a tie. “A tie including Metcha” refers to a tie between 0x API and one or more exchanges. “A tie without Metcha” refers to a tie between two or more exchanges other than 0x API. The colors used in all charts are uniform.
Accuracy, gas usage, rollback rate and response time
In this study, we analyzed three different prices. Therefore, we give the following definitions:
Quotation: The initial advertised price for a specific transaction-the list price that users see in the UI.
Actual price: The actual settlement price after the delay. It usually takes 0 to 180 seconds from transaction submission to confirmation on the chain.
Adjusted price: When a transaction is completed, the price after the user has paid the transaction cost (gas fee and handling fee). In other words, it is the actual price minus the cost.
Let’s take a look at the quotation and actual price first.
0x API, 1inch, Dex Ag, Paraswap and Uniswap quotes and actual price win rate. Many aggregators seem to be optimized for quotes, but the best quote does not necessarily mean the best adjusted price
In the figure above, we can see that Uniswap’s overall performance is not outstanding. The above chart is in sharp contrast with the adjusted price win rate chart. In other words, although Uniswap cannot provide the best price, it can sometimes provide the best adjusted price for transactions under $5,000 because of low transaction fees. As mentioned above, as the transaction volume increases, Uniswap’s winning rate will be greatly reduced, because the aggregator can split the transaction into different sources to find the best price.
From these charts, we can also see that 0x API’s winning rate on quotations and actual prices is much lower than its winning rate on adjusted prices-the difference is an average of 19%. The actual price win rates are 57% and 63% respectively. The average win rate of 0x API after price adjustment is about 72%. The reason why 0x API has such a low winning rate in terms of quotation and actual price is that we have made a clear trade-off when designing the smart contract routing logic, and considered the gas cost when calculating the transaction path of the order. By considering the cost of gas in the smart order routing of the 0x API, we can ensure that the price advantage exceeds the cost of obtaining liquidity from different sources.
Unfortunately, many aggregators seem to be optimized only for winning in the bidding competition, because it can attract more users for them. However, as shown in the aggregator 2 example above, aggregators that do not consider transaction costs can provide better quotes because they are willing to pay more, but this is actually not the most favorable solution for users. Therefore, the best price does not necessarily bring the best adjusted price .
An apt analogy is booking a flight ticket. Have you booked a ticket for a certain airline because its flights seem to be cheaper than those of other companies, only to find out that you need to pay extra for seat selection, and the baggage charge is a little more expensive? Although the basic fare may be cheaper, but with the miscellaneous fees, it pays more. The same goes for some aggregators: it seems that the quotes are higher than other aggregators, but once the transaction is completed, the actual money received is lower than expected.
But why does this happen?
Take 1inch as an example, because it has the highest winning rate in the quoted and actual prices. From the quotation to the actual price and then to the adjusted price, the winning rate of 1inch declined stepwise, from 22% to 19% to 14%. This means that 1inch seems to have a 20% chance of reaching the best deal. However, once the transaction is completed, the cost is deducted, which is often not the case .
Generally speaking, the reason for the difference between the quotation and the actual price is that the quotation is inaccurate or the market price changes during the period from the quotation to the on-chain settlement, which causes the quotation to become invalid. The difference between the actual price and the adjusted price is the additional cost of using the aggregator.
Let’s take a look at the accuracy of all exchanges. What is the probability that the exchange will settle according to the quoted price?
Accuracy is important because it can build user trust and reflect whether the aggregator’s offer matches the price actually paid by the end user. If there is enough time and samples, the difference will tend to zero.
As shown in the above figure, 0x API and Uniswap always have a positive deviation (that is, greater than zero), which means that the price fluctuations between the transaction quotation and the on-chain settlement are beneficial to users. This is the so-called positive slippage. Once this happens, 0x API will always transmit the positive deviation to the user .
What is interesting is that the deviation of 1inch is always less than zero, which means that no matter how the market price fluctuates, users will not get a more favorable price. A deviation of less than zero also indicates that the aggregator is using outdated information quotes, or the response speed is not fast enough. These problems may lead to a higher probability of transaction rollbacks. If a transaction rollback occurs, although there is no token sale, users still need to pay transaction fees. A detailed explanation will be given later.
Now, let’s talk about cost, which is measured by gas usage on Ethereum. In the figure below, we can see that there is a big difference between the gas usage of these exchanges.
As far as the aggregator is concerned, it corresponds to the old saying “less is more”. If the aggregator can bring more value to the user, then the cost is no harm, but as we have shown in the adjusted price win rate chart before, most aggregators cannot do this. Optimizing the code to reduce gas usage increases the likelihood that the aggregator will split the transaction into more sources to get more value.
Uniswap is one of the exchanges with the lowest gas usage, so this is a goal that all aggregators should pursue, especially for small transactions. It is impossible for a transaction worth $100 to be split across multiple exchanges, even if the gas price is only 1 gwei.
Generally, the cost of 0x API is higher than Uniswap. However, as shown in the adjusted price win rate chart, although 0x API is slightly higher in gas usage, it has a 70% probability that users will get the best adjusted price. The cost of all other aggregators is at least twice that of 0x API, that is, it brings double the cost to users, but it creates almost no additional value . 1inch and Paraswap are the exchanges with the highest gas usage, which can also be seen from their low winning rate after adjustment.
To facilitate an objective comparison, the following is the average cost of all simulated transactions worth 500 to 1000 USD:
For the past 8 months, reducing gas usage has been the focus of our team. Our initial goal is to reduce transaction costs by 50%. However, in 0x API 1.0, we successfully reduced the average transaction cost by about 76%. For a $50,000 transaction, the gas usage of 0x API 1.0 is about 460,000 gwei, and the gas usage of 0x API 0.0 is about 1.5 million gwei. These optimizations have turned the 0x API into the most efficient aggregator, regardless of transaction size.
In fact, the gas price was much higher than 100 gwei last month. We want to show you the adjusted price win rate of 0x API in this case. We conducted the second simulation and manually increased the gas price. The result found that when the gas price is high, the 0x API still outperforms other competitors-when the gas price is higher than 400 gwei, provide The probability of the optimal adjusted price is 80%.
Now, back to the gas usage of 0x API and Uniswap. Normally, 0x API is higher than Uniswap in gas usage. However, in 70% of cases, 0x API offers a higher adjusted price than Uniswap . How does the 0x API do this? Our 0x Labs team has built a highly optimized contract that allows 0x API to obtain 100% liquidity from Uniswap in a way that is more gas-efficient than the default Uniswap contract . This is especially beneficial for small transactions (less than $5,000) routed to a single source.
Take the following two transactions as an example:
On the left is an ETH<>UNI transaction routed directly to Uniswap. It is an ETH<>UNI transaction that is 100% routed to Uniswap via 0x API. As shown in the “transaction gas usage” field in the figure, the transaction cost of Uniswap is about 114,000 gwei, and the transaction cost of 0x API is about 110,000 gwei, which is 3.7% lower than the former. Certain on-chain states and conditions may change the result (for example, if the user needs to set a new limit), but generally speaking, the cost of routing transactions to Uniswap through the 0x API will only be lower than that of direct transactions on Uniswap.
So far, we have discussed accuracy and gas usage, and now it’s time to talk about transaction rollbacks. When users trade on these exchanges, how likely is the transaction to fail?
The aggregator should keep the rollback rate as low as possible, because after the transaction fails, paying for gas is a poor experience for users. Common reasons for transaction rollbacks include: market fluctuations during the period from quotation to on-chain settlement, and inaccurate quotations (especially when quotations are provided using outdated information).
Our simulation tool always executes transactions under perfect conditions, that is, sufficient funds, unlimited gas and no runaway transactions (that is, the message does not leak into the dark forest). We ruled out transaction rollbacks caused by these reasons and user behavior in real life. Therefore, for all exchanges, the simulated rollback rate is lower than the actual rollback rate. Nevertheless, the simulated rollback rate can well reflect the performance of the exchange.
In the image above, the delay of 0 to 10 seconds is the most interesting. A delay of less than 10 seconds means that the transaction is executed immediately, or the transaction is packaged into the next block just after it is submitted. Therefore, during this delay period, a high rollback rate means inaccurate quotes.
Generally speaking, we found that Paraswap has the highest rollback rate compared to other exchanges, followed by Dex Ag and 1inch. In a simulated environment where human behavior does not cause a transaction rollback, one of the possible reasons for a rollback is that the quotes are out of date (as shown in the high-latency period) or inaccurate (as shown in the low-latency period). If you remember, Dex Ag and Paraswap have the lowest accuracy.
Then look at the impact of rollback rates on end users. If a user makes 100 transactions worth 500 to 1000 USD on each exchange, the average loss caused by a failed transaction is:
0x API — $6.89
1inch — $62.52
Dex Ag — $3.47
Paraswap — $119.22
Uniswap — $0.21
In the real world, we see Uniswap’s daily rollback rate as high as 30%. However, in our simulation environment, the highest rollback rate in the case of a long delay is around 1.3%. The difference between the actual rollback rate and the simulated rollback rate means that there is a lot of room for improvement in the user experience of all DEXs, but this is another topic.
The result of the simulation experiment is that the simulation rollback rate of our 0x API is the lowest under different delay conditions. The analog rollback rate of 0x API ranges from 0.6% (shorter latency) to 1.8% (longer latency). In the real world, we found that the average weekly rollback rate of applications integrated with 0x API such as Matcha ranged from 3% to 4%.
Now, there is another important indicator that we haven’t mentioned: response time. End users of applications such as Matcha may not have a deep impression of this indicator. However, when you build a product on a third-party infrastructure, you must ensure that the product is as reliable and fast as possible . Faster aggregators can allow users to complete transactions faster, reduce slippage and rollback rates, and improve user experience.
Since our 0x API takes samples from more than 13 sources (including systems that provide liquidity by professional market makers), keeping the response time low is a technical challenge. However, our goal is to make 0x API the most professional liquidity aggregation API. Currently, the response time of 0x API is about 1.5 seconds, which we are proud of.
Among all the exchanges involved in this study, 1inch has the longest response time, which is 2.7 times higher than that of 0x API. The response time of 1inch is 4 seconds, which is about 1/3 of the time required to dig a new block, which may cause time-sensitive transactions to roll back, for example, during periods of high volatility when market conditions are constantly changing.
Given all the above information, how to choose the best aggregator? We can calculate the average gains or losses of transfers under various trading scenarios to get the expected value of gains from one exchange to another. This is the expected benefit of users.
For users who switch from other aggregators to applications that integrate 0x APIs such as Matcha, the average expected benefit of transactions less than $10,000 is:
1inch — $7.17
Dex Ag — $10.36
Paraswap — $14.89
Uniswap —$17.14
In general, if a user who trades on 1inch turns to an application that integrates 0x API such as Matcha, an average of USD 7.17 per transaction is expected. If they switch from Uniswap to 0x API, they are expected to receive an average of $17.14 per transaction.
to sum up
First of all, I am glad you guys can see here!
Our goal is to share our gains in building 0x APIs and to clarify key differences between aggregators that are not obvious. As we mentioned above, we want to make this industry more transparent and cause discussions on how to objectively measure and compare the performance of different aggregators, so that DeFi developers and traders can make informed decisions .
By making our data public, we hope to advance the DEX aggregator industry and continue to build the best applications for DeFi developers and users.
If you are a trader, you might as well come to Matcha to experience it!
If you are a DeFi developer, you can read our 0x API 1.0 announcement and 0x API documentation. If you have integrated 0x API 0.0 into your project, the migration process to 1.0 is very simple. Just in case, we wrote this guide.
Many thanks to Ben, Jacob, Kroeger, Lawrence, and Michael for their contributions and feedback on this article.