The Bug That Lost $440 Million In 45 Minutes
In 2012, one server missed a software update, and Knight Capital lost $440 million in 45 minutes.
The script
This company lost almost ten million dollars a minute... for forty-five minutes straight.
In twenty twelve, Knight Capital, a huge stock trader, updated its software on eight servers.
Someone missed ONE.
On that server, a reused setting woke up old code that had been dead for years.
When the market opened, it bought high and sold low. Millions of orders.
Their fix? They rolled back the seven good servers... which made it worse.
Forty-five minutes later, four hundred and forty million dollars were gone.
Now Dropstone reads my deploy script before I run it. Because one missed server is the reason...
The facts, checked
- 1 August 2012. New code was deployed to 8 servers; a technician missed one.
- A repurposed flag activated old, unused "Power Peg" code on that server.
- Engineers uninstalled the new code from the 7 correct servers, which spread the problem (SEC order, 2013).
- About $440 million lost in roughly 45 minutes; Knight needed a rescue and was acquired.
