72 total views
The well-known white hat Sam Sun worked with Tina Zhen, Xinghuo Mining Pool and others overnight to help the DeFi stablecoin protocol Lien avoid vulnerabilities and protect US$9.6 million in funds.
Written by: Sam Sun, Alex Wade, Scott Bigelow, Tina Zhen, Shaoping
On September 15, 2020, an action team worked together overnight to rescue $9.6 million from a fragile smart contract. The following tells their story.
Sam Sun aka samczsun Readme
Let me summarize the story of reviewing some smart contracts.
Of course I didn’t expect anything interesting. In the past few weeks, I have seen countless yield farming clone projects that have exactly the same propaganda model: put your tokens in our hands and you may become the next crypto millionaire. Most cloning projects simply fork the well-audited code with some adjustments, which can sometimes lead to disastrous results.
But in all the hustle and bustle, some code I have never seen before. The protagonist in this article, this smart contract holds more than 25,000 Ether, which was worth more than 9.6 million US dollars at the time. For those who desperately find errors logically, this will be a rewarding hunt.
I quickly took a look at the code to transfer ETH out and found two matches. One of them transferred ETH to a hard-coded token address, so it can be ignored. The second is the incineration function that transfers ETH to the sender. After tracking the usage of this function, I found that anyone can mint tokens for themselves for free, and then burn them in exchange for all the Ether in the smart contract without any effort. My heart beat faster. Suddenly, the problem became serious.
My smart contract is a Lien Finance agreement. Unfortunately, their team is anonymous! The only instant messaging support platform is Telegram, and I am not sure whether the administrator of the Telegram channel is a developer or just some early supporter. I never want to accidentally disclose this loophole to the wrong person.
After browsing their website for a while, I found that they had cooperated with ConsenSys Diligence and CertiK to accept their code audit. This seems like a very good way. ConsenSys Diligence and CertiK must have interacted with developers during the audit. I quickly chatted privately with John Mardlin (aka maurelian), a security engineer at ConsenSys Diligence, on Telegram.
Initiating a chat and waiting for a reply is too painful. Unfortunately, time passed and Maurelian did not respond. He seems to have fallen asleep. In despair, I sent a message to the ETHSecurity Telegram channel, asking ConsenSys if anyone is not asleep, please contact me as soon as possible.
A few minutes later, I received a reply from someone who had worked with me several times in the past-Alex Wade, another security engineer from ConsenSys Diligence.
Alex Wade Readme
My head just got on the pillow and I heard my roommate knock on the door. “Sam asks anyone who knows anyone in ConsenSys Diligence on the ETHSec Telegram channel.”
In the conversation between Sam and Alex, it might be a long night tonight, which is indeed the case.
As soon as I heard it was Sam, I knew it wouldn’t be a good thing. I found a channel to communicate with Lien and an email address that I established a few months ago. Their team is an anonymous team, it is better to have these than nothing at all.
I was still half asleep and half awake. Sam didn’t want to go into details in the chat and asked for a conversation with Zoom. While regretting why I had to get up, I assessed the seriousness of the situation:
Five minutes later, I knew I had to drink coffee to refresh myself and start working.
Sam and I reviewed the code together. Sam had prepared a sample test at the time, which could confirm the problem on his machine. Our dialogue search shifted to discussing feasible options:
- We save this money by ourselves.
- Contact the Lien team, let them disclose their identities, and urge users to divest.
Both are not very satisfying choices. The first move is extremely dangerous. If you read the article on the Ethereum DeFi Dark Forest ( Chain Wen Chinese version ) discussed by Dan Robinson and Paradigm research partner Georgios Konstantopoulos, the probability of our transaction being intercepted is extremely high. The second option is almost as dangerous, because a public statement will cause the outside world to pay attention to the issue and give attackers a window of opportunity. What we need is the third option.
Recalling some of the content in the article on the Dark Forest of Ethereum DeFi, Sam contacted Scott Bigelow, Vice President of Engineering at Amberdata: “If you really get into this predicament, I suggest you go to Scott Bigelow, Vice President of Engineering at Amberdata. He has been studying this. Security researchers on one subject have a prototype implementation strategy that can better achieve the goal of transcending the sky.”
Scott Bigelow’s narrative
After I participated in the fund rescue operation in the article on the Ethereum DeFi Dark Forest , and finally lost to the Dark Forest Sniper, I was very eager to have a rematch . I spent some time monitoring the rush to intercept Hu, and designed a simple system that seemed to be able to fool the average rush runner. At least the $200 I personally invested in the test was successfully saved. When Sam contacted me late at night and said sincerely “mind you take up about an hour or so of your time”, I was ready to try! I have imagined that picture: How do I make a few technical adjustments, it takes a few hours, and then successfully save the user’s thousands of dollars in assets, wear that sense of accomplishment, and sleep a good night.
When Sam shared this smart contract with me, these plans fell apart in an instant: about 25,000 ETH, worth $9.6 million. Although I really want to do this kind of rematch, my few lines of broken code are not designed for assets of 9.6 million US dollars.
For the past few months, I have been trying to establish relationships with the miners, just for this purpose: to conduct such white hat rescue deals. If there is a point in time when you need the help of miners to write a transaction in their mining, so as not to be stolen by the runners, it is now. Fortunately, Tina Zhen and I have joined forces in the past few months to find this collaborative relationship. At the time, there was little hope for the success of this rescue operation, but it was worth a try: let Tina participate in this rescue operation and cooperate with a mining pool to dig a private transaction.
Tina Zhen’s self-report
I just evacuated from my home because of the California forest fire codenamed “Lynx”. I was drinking an unknown beach drink and listening to the sound of waves from the dim Pacific Ocean. Sam’s Telegram message brought me back to a darker reality: “Funds are at risk and may be intercepted.” In the past few weeks, I have been collaborating with Sam and Scott on a research project on “Miner Extractable Value” MEV, and I guessed it before they sent the request: a direct channel that can protect white hat transactions in Ethereum Memory pool mempool (pending, unconfirmed transaction set) in this “dark forest” is protected from looting by “snatchers”.
Since this is a risky move that requires exposing our strategy to the miners, we decided to first try to obtain green light approval from the anonymous Lien team. When Alex tried to contact through ConsenSys internal channels, we also tried to contact CertiK.
I realized that Certik’s auditors in the United States will take four hours to get up, but time can’t be delayed. I don’t know much about CertiK. I just know that they have audited several Asian projects, so I tried to contact the CertiK China team. I left a voice message in the two WeChat groups “DeFi the World” and “Yellow Hats”. I received four messages within 30 minutes in my private chat, confirming that the ID in my WeChat friend is the real Zhaozhong Ni, CTO of CertiK. I was pulled into a WeChat group composed of five CertiK team members, and at the time I still could not disclose related projects or vulnerabilities. In order to minimize exposure and potential liability, we can only invite one Certik member to join our white hat operation. After the final confirmation by official email, Georgios Delkos, head of engineering at CertiK, joined our conversation.
With the help of Georgios, Alex was able to quickly contact the Lien team and verify his identity. We let them understand the current grim situation as quickly as possible, win their consent, and directly cooperate with a mining pool to rescue this fragile fund. After deliberation, the Lien team agreed that it would be too dangerous to save funds directly or issue a warning statement, and agreed to advance this plan.
Now we need to find a mining pool with adequate infrastructure and willing to cooperate with us as soon as possible. Which mining pool should we look for? Which contact person of this mining pool can quickly make a technical decision to help us execute this action against the clock?
We thought of SparkPool, and I know that they have been building a public infrastructure called Taichi Network, which can easily give us the support we need. I decided to chat privately with Shaoping Zhang, the co-founder of Spark Mine. He had previously helped me investigate mempool security incidents.
Half an hour later, Shaoping replied: “You mean I opened a whitelist for the transaction? Sorry, we can’t.” Well, lost in translation, the Chinese “white hat” whitehat and “whitelist” whitelist look a little bit Like.
“Now there are 10 million US dollars at stake. Samczsun and I are online,” I tried again to communicate the situation without revealing any specific details.
“You two are saving the world? Do you need help from the mining pool?” I was a little surprised and relieved, Shaoping teased and expressed willingness to help. After confirming by official email, Shaoping entered our marathon-like Zoom conference, and was given technical support by a large number of Spark Mine pool developers.
After lunch, I was about to take a nap, and then I received a WeChat message from Tina: “Have Spark Mine help with a white hat transaction?” I misread it at first, but it was a whitelist for the transaction. No one has contacted us for white hat transactions before, and we are not familiar with the specific circumstances of “white hat transactions”. After Tina explained in more detail, I realized that they need a private transaction service. For example, a white hat hacker wants to send a transaction to save a DeFi smart contract, but to prevent others from being intercepted by others, they need the mining pool to broadcast In the case of a transaction, write the transaction to the block.
We have been building a private transaction feature on Taichi Network. This feature is still under development and has not been tested. I told our development team the request of the white hat hacker and emphasized the urgency: Our private transaction function needs to enter the production stage smoothly within a few hours. Our developers said they would try their best to complete, and then quickly put into work. We completed the development of the private transaction function within two hours, and spent some time fixing the bug.
After we completed the internal test, we sent the hitehat.taichi.network endpoint to Scott Bigelow to complete the white hat task.
Scott Bigelow’s narrative
Spark Mine Pool worked overtime and launched a new white hat API. Sam and I also completed script programming to generate four consecutive signature transactions. Processing these transactions in turn will not take out 25,000 ETH itself, but will transfer 30,000 SBT+LBT tokens (generated by “error”) to the Lien team, who can submit them to the final transaction and convert these tokens into ETH.
By transferring SBT+LBT tokens that can be minted indefinitely to the Lien team instead of ETH, we used more transactions as a blinding method to confuse the attacks of general coercors (in case of reorganization), which can make 9.6 million US dollars in revenue. It’s not possible to get into my pocket for a moment.
After we generated four signature transactions, Sam and I spent a lot of time using various multi-party transaction simulation tools to verify their chain actions. The four transaction programs with a total data volume of less than 1.5 KB are a killer to save 9.6 million US dollars of assets, ensuring that no one will be foresight before the Spark Pool captures these transactions.
I tested the white hat endpoint of Spark Pool with a meaningless transaction, and its execution process was unbiased: the transaction would not be seen in mempool, and then suddenly appeared in a block of Spark Pool! It’s like watching the water vapor directly turn into ice, and the nasty liquefaction process is completely invisible!
We adjusted the transaction generation script and submitted the transaction directly to the new endpoint of the Spark Pool. Now is the time to act. I hesitated for a while, but this is definitely the best effort we can do. We may lose this $9.6 million, but we won’t regret it: I clicked “Run” in IntelliJ. I’m not sure why, but I expected that the whole process will take some time, just like the node will understand the severity of the situation and spend some time in it. But in fact it does not; transactions are sent in milliseconds.
Everyone in the Zoom call started to refresh Etherscan frantically. I wonder if the Etherscan team has seen this 3-minute traffic peak. Since only Spark Pool has transactions, and only a portion of Spark Pool’s hash rate is dedicated to this purpose, all we can do is wait anxiously in a cold sweat. Every block mined by every other miner scared us. In the Zoom conference, someone will utter the name of the miner who dug the block with nervous laughter. About 15 blocks will be generated before our transaction is written into the block, but we feel like a year, like a few hours, but in the end, we completed a perfect transaction: successively dug out, without rollback.
We watched with relief as more and more blocks were superimposed on our blocks, and worries about block reorganization quickly disappeared. The Lien team now has enough SBT + LBT tokens to liquidate the funds of the entire system, and Sam is responsible for coordinating the final stage of the rescue.
Sam Sun Readme
Now that we have successfully transferred the tokens to the Lien team, we have not found any signs of forced censorship, attempts or other signs. We quickly told them the good news in private chat. They confirm that they have received the tokens and immediately send a transaction to withdraw most of the ETH locked in the smart contract. A few seconds later, this pending transaction appeared on Etherscan.
While watching the progress bar go round, I took this opportunity to reflect on this action. At the beginning, I looked at the smart contract, which eventually evolved into a “Save Private Ryan” operation that attracted the collaboration of experts from all over the world. Without Alex and Georgios, we would not be able to contact Lien developers. Without Scott, our rescue operation may have long been troubled. Without Tina, we would not be able to contact CertiK or Spark Mine. Without Spark Mine, we are destined to repeat the tragedy that Dan wrote in the article a few weeks ago.
However, late on Tuesday night, our seemingly impossible team worked together for the same goal. After unremitting efforts, we finally ensured the return of $9.6 million to the original owner. Our hard work over the past seven hours finally converged into a pending transaction and a progress bar for this circle.
When the progress bar finally showed a green tick, the tension of the Zoom conference finally disappeared, and everyone breathed a sigh of relief.
We finally managed to get out of the dark forest.
This article records the success of many people’s hard work. Special thanks to Alex Wade, Scott Bigelow, Tina Zhen, Georgios Delkos, SparkPool who rescued from distress, and Alex Obadia and Dan Robinson who reviewed this article and gave feedback.
If you are interested in the technical details behind this action, please click here to learn more. If for various reasons, you haven’t read the article “Dark Forest” of Ethereum DeFi , you should read it.