Decentralized finance can add value to the network and to people’s lives… as long as our excitement doesn’t result in us dancing right off a cliff.
This was originally a talk I gave at Dystopia Labs’ DeFi Discussions event.
At the time of this talk (early May 2020), the dForce and Hegic exploits had both happened in the previous 14 days. It was also before the advent of governance tokens, IDO’s, and yield farming. But this is less about specific events and more building a system that will benefit people, not harm them.
DeFi has grown remarkably fast and, as always, growth brings hacks, attacks, exploits, incidents, non-incidents, and generally unintended consequences. Between January 2020 and May 2020, it seemed like all the Bad Things™ were with DeFi products. Or was I just hyper-focused on the events in the DeFi space? Let’s take a look...
One of the most interesting things is that publicly disclosed/reported exchange hacks are way down compared to last year. By this time in 2019, there were seven massive exchange hacks: Cryptopia, Bitrue, Coinmama, Coinbin, DragonEx, Bithumb, and then Binance (May 4). The fact that there are fewer hacks on exchanges is interesting, but it’s also interesting that the sizes of these hacks have been relatively small. This is probably the first time in 5+ years that the number of exchange hacks has been this low.
Since then, BlockFi experienced a data breach after an employee was sim swapped. No customer funds or passwords were taken, but customer info was.
After giving this talk, some commented that it may be that less exchanges are disclosing the hacks, not that the actual number has decreased. This is somewhat supported by the CryptoCore report, although even that group’s activity has dwindled in the first half of 2020. Potential reasons for the drop in actual hacks or reported hacks could be due to more assets being separated in hot and cold storage, setting aside money as an “insurance” fund, increased profitability/maturity in general, macro conditions like COVID-19, or bearish market conditions attracting less attention.
Another interesting takeaway from looking at the collection of Bad Things™ is that the wallet exploits and robberies, and the sizes of those robberies have been way up.
- We kicked off the year with IOTA’s Trinity wallet, which was compromised via Moonpay’s infrastructure in a targeted attack. Rumors have it that $2M was stolen, and their post-mortem is an excellent read.
- Shortly thereafter, a Chinese investor got SIM-swapped and $30M worth of BTC and BCH was stolen. This is likely the largest recorded loss due to a SIM swap. (Protect yourself from swaps, please.)
- There has been an uptick in malicious chrome extensions targeting Ledger, MEW, MetaMask, and many more.
- Most recently, we saw this huge EOS wallet exit scam. The perpetrators may have gotten away with $52M worth of EOS. I don’t follow the EOS ecosystem closely, so this number may be wrong—I hope it’s wrong—but regardless, there was evidently a big exit scam by a wallet creator. That hurts.
- I also forgot to include various ZecWallet issues, which rivals DeFi when it comes to vulnerabilities and security drama.
And finally, let’s look at our own ecosystem. This is DeFi right now. This is a pretty long list. Granted, I’ve included the near misses, but I think that we should pay attention them because we cannot assume that people like Sam Sun and 1inch.exchange will save the day again and again and again.
😈 Actual exploits where money was lost:
- Feb 15: bZx exploited via flash loans for $350,000.
- Feb 17: bzx exploited again for $630,000. Or this podcast that covers both.
- Apr 18: imBTC pool on Uniswap attacked & drained for $220,000 (ERC-777).
- Apr 19: LendFMe / dForce hack for $25M (ERC-777).
- Apr 24: Hegic typo / bug locks $30,000+ in contract day after launch.
😬 The “near misses” where a disclosure and resolution occurred:
- Jan 20: 1inchexchange finds & discloses exploit in Fulcrum/bZx.
- Feb 18: Sam Sun responsibly discloses Authereum exploit.
- Feb 24: Sam Sun AND Mudit Gupta responsibly disclose two Nexus Mutual exploits.
- Mar 24: Sam Sun responsibly discloses Hegic Exploit.
😱 The “near misses” where if it hadn’t been caught, it would have led to significant losses:
- Jan 25: Sam Sun responsibly discloses exploit in Curve contracts.
- Apr 19: Curve sUSD pool contract shut down after discovery of exploit.
- April 25, 2020: Etheroll.
Then, in the middle of all of this, we had ⚰ Black Thursday.
This was a catastrophic day in the traditional markets that quickly impacted the crypto world. All things considered, we’ve recovered pretty well from cascading effects that threatened the DAI peg, skyrocketed gas costs, liquidated positions, and limited the use of oracles.
- Multicoin’s The Day Crypto Market Structure Broke
- MakerDAO Timeline & Analysis
- More MakerDAO Analysis
- DeFi Pulse’s Report
- Aave’s Analysis
- DeFi Saver: Black Thursday
- Argent: Gas Issues
- Chainlink: Oracle Issues
- NexusMutual Claims
- Liquidation numbers for Maker, Compound, dydx, etc.
- Consensys Q1 2020 Report by Alethio
- I hate Twitter threads, but if you want to converse, here’s where we are currently at. One. Two. Three. (They are all the same thread but different sub-threads, scroll up and read from the top down.)
- More CEX / wider analysis that isn’t DeFi focused can be found in this Twitter thread.
Black Thursday is a good reminder that risks come in all forms. A single platform can be drained or global economic conditions combined with increased composability can lead to cataclysmic events. While hopefully we won’t see another Black Thursday again anytime soon, the cascading effects are something to learn from and attempt to protect against, especially as the interconnected DeFi legos become even more tightly coupled. It may not take something so large to cause problems.
Lastly, it’s worth noting that we haven’t had too many exit scams (in DeFi or Ethereum) where a product gains the trust of everyone and then runs away with the bags. That said, there have been a lot of Ponzi schemes and obvious scams which have managed to make off with millions, or are ongoing *cough* Hex *cough*. We will like see the admin functionality of contracts be abused at some point. We will likely see exit scams. That should scare us. And, yes, please, everyone knock on wood.
This is how much money is “locked” in the specific DeFi protocols as of April 29, 2020. Since we denominate everything in USD, that number can go up really quickly when the ETH price goes up. Right before this talk, we saw a pump where the price was $172 when I went to bed and over $200 when I woke up.
If this market swings upward as it did in 2017, we will see these types of pumps day after day. If you’re developing a DeFi smart contract, you could go from having $1M to $100M too quickly for you to process and adapt to the new circumstances. This is one of the things that scares me the most. 2017 was exciting but also terrifying when I saw usage take off with my product. For the entirety of 2017 and 2018 I was two or three steps behind and constsntly putting out fires. And I wasn’t holding anyone’s money, literally or via a smart contract.
Even if no new users start using your product or deposit money into your smart contract, the value jumps when the market jumps.
All things considered, we have been super lucky that many of the catastrophic exploits have been responsibly disclosed. The number of tricks we’ve pulled in the 13th hour has been remarkable.
We have been walking around acting like we’re building a better tomorrow, saving the world, and making boatloads of money. In reality, we’re locking up crypto, burning crypto, throwing crypto away, flushing it down the toilet, and everything in between.
Only one or two million dollars have been lost in the last 4 months from these DeFi protocols. But the reason I use the word ‘only’ is that you can see eight of these were responsibly disclosed or discovered by internal members of a team. For example, in a remarkable turn of events, 1inch and others were able to recover most of the funds stolen in the $25M DForce hack. that doesn’t happen. Hackers don’t give back money. Again, we cannot rely on Sam Sun, 1inch, or a cavalry of white hats returning $25M every time. We need to be better at securing our products and constraints and systems so $25M isn’t taken in the first place.
If we continue down the path we’re on, we’re going to get our asses handed to us and it’s going to hurt the prospects of DeFi and this ecosystem. And I really like DeFi. The reason I talk about DeFi so much is that I want it to succeed. Unfortunately, the way we’re approaching it right now — the things that we’re doing, the things we’re not doing, the attitudes we have — does not end in rainbows and unicorns. We want a path where we’re benefitting, we’re making money, and we’re transforming the financial systems that control our lives. We need to level up on a number of fronts if we want to get there.
Hegic and Trail of Bits
In May there was a ton of discussion about how the Hegic exploit could have been prevented and the audit that Trail of Bits did.
A lot of people called attention to the fact that Trail of Bits “audited” Hegic’s smart contracts, but Trail of Bits has clarified that it was a very short review rather than a comprehensive audit. I could do a whole talk on just Hegic, but there have been some interesting suggestions… One is that auditors should stake a whole boatload of money and if a contract gets hacked, then that money is used to pay people back. Obviously, there’s the drama around software engineers being licensed. Some have suggested that before users can put their money into any DeFi projects, they need to pass a multiple-choice test to prove that they fully understand and accept the risks that accompany their actions.
I’m not a fan of any of these. If we zoom out and look at everything that’s happening in relation to the world we live in, we realize there’s a bigger issue than calling a review an audit.
Let’s look at how audits have happened throughout history. Most traditional corporations (you know, huge offices with servers or even the multi-national conglomerates with people in their warehouses running around with hard hats and using forklifts) all do audits of various types. These audits serve a very different purpose than a TrueCrypt audit. TrueCrypt and projects like it are open-source and have an ever-changing group of contributors who are sometimes anonymous.
For example, in a traditional corporate setting, you may have frequent, ongoing internal audits conducted by people outside the company (especially if you have regulatory hurdles) to strengthen security. Because if it’s not secure, money is lost, employees lose jobs, the CEO gets thrown out, etc.
Over in the OG, cypherpunk, open-source world, it’s very different. There have been instances where a security audit was crowdfunded for a project in order to ensure that the developers were writing good, secure code but also to determine they weren’t actually back-dooring everything for the NSA.
Those are two totally different ends of the spectrum, and those of us in our little DeFi bubble are in the middle somewhere.
When you live in this space it’s easy to forget just how fleeting the infrastructures are, but it’s insane. You have tokens that are distributed today but can be sold tomorrow for 2X (or .5X) without vesting schedules. You don’t have people investing in your company so much as they are investing in your tokens — these are kids on the internet who probably have a bit of ADHD and jump from token to token, always wanting the newest thing.
We want to decentralize everything and sometimes we can go a bit overboard. For example, some projects will keep kill switches but if something bad happens to the smart contract then we’ll just shift the blame around a bit.
I think we’re gonna have a reckoning with the whole anonymous cypherpunk culture. We have Satoshi, and Satoshi did good, so some people assume that a semi-centralized project that’s going to take all your money that has an anonymous founder is also good. We have to remember that being anonymous is actually an untrustworthy signal, not a trustworthy one. Satoshi was an anomaly. We need to be more skeptical.
DeFi builders are different. I call them builders to capture the full range of people creating DeFi products, not just the CEOs.
DeFi builders have new organizational structures; they’re based in all countries or no specific countries, or they’re a DAO, or any number of things. They’re very experimental and disruptive of tradition. Their “equity” is usually a token and it MAYBE has a lockup but maybe doesn’t, and it can be sold tomorrow. They’re building these systems with cryptocurrency, using a new organizational structure that’s not recognized by any government.
The people building products have a very high appetite for risk. If you look at the spectrum of builders in this space, you’ll see the ones who you consider more or less risky, but if you zoom out… we’re all over here on the super high-risk side. When you zoom in very closely it’s easy to get the wrong idea that someone is being safe. Everyone is engaging in really risky behavior and it affects the choices that we make, how we see and build things, and how we talk about them.
And of course, we have this whole shiller/token holder aspect — the closest equivalent to being a shareholder. They can influence you and the price of your stock (or in this case, your token). Unfortunately, tokens are not the same as investing equity, as people are flipping these tokens constantly. They want that token/project/whatever to launch and they want to gain from it.
We have this tendency to treat DeFi projects a little bit like traditional companies in certain ways — YOU get the audit, YOU prove you’re safe, YOU pay the bills — but then also treat them like super decentralized magical beasts in other ways. This creates a lot of conflict.
We have created this crazy influential community of individuals who invest in these tokens and take huge risks for these very-short-term gains. They will pump it. They will build bots for it. They will arbitrage it. They will shill, shill, shill it. And if something goes wrong, those same individuals will fight tooth and nail to defend that project, pushing to have protocols turned back on even if they were hacked two days earlier. The reason these “investors” do this is that they already lost X amount and if that protocol doesn’t get turned on ASAP they’re gonna lose more. So these token holders, the people that have money in these protocols, they’re incentivized to push the builders of these protocols to turn it back on now.
So where do we go from here?
We need to provide a safeguard against the worst offenders, the worst tokens, and the worst shillers— those with no regard for safety and no awareness of something bad that could happen. But we also need to reward the good behavior, and that’s missing right now. If MakerDAO takes two years to even get to testnet, nobody is like ‘hey guys, thank you so much for doing your diligence and taking the time necessary to build this complex system that is going to hold billions of dollars.”
Instead, people are like, “Duuude it took you TWO YEARS?! and you’re only on TESTNET? Why aren’t you on mainnet yet?”
That’s gotta go. We need to be better at rewarding good behavior. Even when someone slips up, if they show that they are taking it seriously and learning from the errors, and take action on their missteps, they should be rewarded.
If we keep pushing out the bad and pulling in the good, everything will be stronger.
A security audit only means that a project has enough money to pay for an audit. It does not mean that a project is secure. I could talk all day about audits and what they are and what they’re not. I merely want to emphasize that a security audit is like a third-party set of eyes that comes in and takes a look at your code. It is one tiny aspect of a good secure culture and a good secure company.
If this one sliver of effort is all you do for the security of your product, you’re not secure. And (it shouldn’t need to be said but here we are) if your audit comes back with a whole bunch of errors, you need to address them in order to actually be more secure.
Auditors have been catching some heat lately because their audit reports don’t come out and say things frankly. Their professional politeness requires you to read between the lines but as this space matures, I hope we’ll have more journalists and researchers who can translate these complex, technical documents into human language. But for now, we have to remember that an audit is a good thing that people have to do, but it’s just one step. It does not guarantee security and you should not treat it as a stamp of approval.
Okay, and then I want everyone to try to be more skeptical. I mean when you look at a new project or go on DefiPulse and there’s a name of a protocol that you’ve never heard before… instead of being excited and immediately throwing money into it, stop and think:
“These people want my money. This page is designed to manipulate me into giving them my money. Is that a good decision? What things do I need to know to be able to make that decision?”
We need to be skeptical and that is going to take some time. It’s an attitude shift that we need to instill in each other and ourselves, as well as the newcomers entering the space, because they’re the ones that are most at risk for this type of stuff.
Ask questions. It’s one of the most powerful things that anyone in this community can do, whether you’re a designer, artist, best security engineer, or a basketball player! It doesn’t matter if your technical.
Don’t feel like it’s stupid. It’s not stupid. Even if it is stupid, someone else had the same question, so you can be stupid together. 😉 But seriously, there is such a lack of information in this space that I can sincerely say there are no stupid questions. There truly aren’t.
Here are some examples:
- Do they have an audit?
- When was the audit performed?
- Who did the audit? How long were the auditors on it?
- When did it go live on testnet? How much volume did it handle? Has the code changed?
- Where is the documentation? Is it any good?
- How do they communicate with people on social media? Are they transparent? Do they lash out if you question their security culture?
- Where are their tests? How often are they updated?
- Is their code even open-source?
- Is their code verified on Etherscan?
- Do they have a security email?
- Do they have a bug bounty program?
- What do they talk about most on social media or in talks? Security or the next big thing?
- Do they have a security engineer on staff? Do they have five? Do they have 500?
- Do they have any engineers? Are these engineers real people? So many ICOs just had fake team pages full of fake people — even celebrities!
Consensys Diligence just wrote an entire post about questions you should be asking!
- What special actions can administrators take?
- Which oracles does your system depend on?
- What exchanges does your system depend on?
- Is the source code of your contracts publicly available?
- What is the range of bounty payments?
- Do you have a written plan outlining how to handle a security incident?
- Which scenarios does your plan take into consideration?
- Was any portion of your system excluded from the scope of the audit?
…and SO MANY MORE. Definitely read it.
Social pressure is a powerful motivator and probably the only motivator that can counteract the financial incentives that we talked about earlier.
Internally, we have to be skeptical. But externally, we have to show that skepticism. You have to show these projects hunting for money what matters to you—that you care about security and safety and diligence. Make them dedicate resources to be aware of the Bad Things that can potentially happen.
It’s easier to not get an audit or not create a threat model. It’s easier to throw Google Analytics on your site. And it’s even easier if nobody cares about these things and there is no reward for doing them. The only reward is intangible. It’s the default state of “not being hacked.” But the community has the power to reward people for the good choices they make.
The blockchain is actually really amazing at incentives. It’s sort of the whole purpose of the blockchain. It was built to create this remarkable incentive structure so anyone from anywhere in the world can be on the same team. Which means that we’re all on the same team. That means everyone reading this, whether you’re building something, or a CEO. or just watching the drama unfold on Twitter, you are part of this ecosystem. Therefore, it is your responsibility to help ensure that there is more good than bad and more success than failure.
Reward the good, kick out the bad, and we’ll all be better off.
I’m Taylor Monahan. I’m the founder and CEO of MyCrypto. We are always on Twitter. Check out beta.mycrypto.com and please tell us like what you like about it or what can be improved because we’ve been working so hard on this for you. Thank you so much.
More Resources and Stuff
Security / smart contract audits for products mentioned above
- bZx (ZK Labs, Sept 2018)
- Uniswap Audit (Consensys Dilligence, Dec 2018)
- Authereum (Quantstamp, Feb 2020)
- Nexus Mutual: Solidified, April 2019 & G0 Group, June 2020
- Hegic (ToB Rapid Review, March 2020)
- Curve (ToB Rapid Review, March 2020)
- Etheroll (Piper Merriam, April 2017)
- Compound Audits: Tether Integration (OZ), Gas Optimization (OZ), Governance (ToB), Governance (OZ), Open Oracle (OZ), MCD/DSR Integration (OZ), Market Risk Assessment (Gauntlet), Timelock (OZ), Compound Upgradable Tokens (OZ), Compound Refactor Fee/cToken (OZ), Compound v2 Patch (OZ), Compound v2 & Summary (OZ), Compound v2 — August, Compound v2 — April (ToB) 👀👀👀
- Aave Audits: ToB — Oct 2019 and OZ — Jan 15, 2020
- ERC-777 / Uniswap: Analysis by OpenZeppelin, July 2019