Designing an Order Execution Engine: Why Traditional Scaling Laws Break Down ?
A few weeks ago, I had an amazing opportunity to interview at a top proprietary trading firm(can't disclose the name ). Even though I didn’t land the job, the technical evaluation pushed my understanding of backend systems to its absolute limit. They asked me a deceptively simple question: 'How do you design an order execution engine that handles a 100-million-order rush on a single stock ticker without cross-server latency spikes?' Here is exactly what I told them and I will try to serve you my understanding in a story form so it can make sense easily :).
Imagine it is 10:00 AM. A hot stock like Apple (AAPL) opens for trading.
10 Million People open their mobile apps and smash the BUY button.
1 Million People smash the SELL button.
How does a computer system match these people up instantly without crashing, getting confused, or losing money?
If you ask a regular web developer, they might say: "Just add more servers! If one server gets full, turn on Server 2, Server 3, and Server 4!"
But that is a trap! If a buyer is on Server 1 and a seller is on Server 4, how do they talk to each other? For Server 1's memory to talk to Server 4's memory over the internet takes time (latency). In high-speed trading, even a tiny delay of a millisecond means failure.
So, how do the richest trading companies in the world actually solve this? They use a 3-Step System.
Step 1: The "Dumb Mailmen" (The Ingress Servers)
When 10 million people click BUY, they don't actually talk to the trading machine. They talk to a front row of basic servers. Think of these servers as Dumb Mailmen.
They do not keep any records of who bought what. They only do the boring, basic checks:
"Is this user logged in?"
"Do they have enough money in their bank account to buy this stock?"
Because this job is simple, you can spin up 10, 20, or 100 of these servers. If a billion people click buy, you just add more mailmen. This handles the heavy traffic noise.
Step 2: The Ticket Dispenser (The Sequencer)
Once the mailmen check that the users have money, they take all those millions of orders and send them to a single machine called the Sequencer.
Think of the Sequencer like the little machine at a bank that gives you a token number (Token #1, Token #2, Token #3).
Even if 10 million people press the button at the exact same microsecond, the Sequencer forces them to stand in a single, straight line. It slaps a strict number on every order so the system knows exactly who arrived first.
Step 3: The King Core (The Matching Engine)
Now, that single straight line of orders enters Server 3.
This is a special, super-powerful server. Inside this server, there is one single CPU core handling the actual order book for Apple stock.
This is the golden rule of trading systems: The actual matching book for a single stock lives on exactly ONE thread on ONE server. It is never split up.
Because this single CPU core doesn't have to talk to other servers, doesn't have to check bank balances, and doesn't use a slow database, it is blindingly fast. It uses a super-fast memory structure called a Doubly Linked List or similar DS directly inside its RAM.
It takes only 200 nanoseconds to process an order. How fast is that?
- In just 1 second, this single core can process 5,000,000 orders!
So, even if 10 million people hit the system all at once, this single core clears the entire crowd in just 2 seconds without sweating .
What Happens to the Rest of the Orders?
As we said, 10 million people wanted to buy, but only 1 million wanted to sell. The King Core matches 1 million buyers with 1 million sellers instantly.
Now, 9 million buyers are left waiting. What happens to them?
1. Does the RAM run out of space?
No. An order is just a tiny piece of text data (User ID, Price, Quantity). It takes up almost zero space. Even if 1 billion orders are waiting in a line, they only take up about 64 Gigabytes of memory. A high-end trading server easily has 512 Gigabytes of RAM. It fits like a drop of water in a bucket.
2. Does the price change?
Yes! Because there are 9 million buyers waiting and 0 sellers left at that low price, the price of the stock shoots up. Sellers start demanding more money, and the single matching core starts matching trades at higher and higher prices.
3. What happens at 7:00 PM when the market closes?
Do the remaining orders get lost? No!
Day Orders: If a user just placed a normal order for the day, the server automatically deletes it from RAM at closing time, cancels it, and gives the user their money back.
Good 'Til Canceled (GTC) Orders: If a user said "keep my order active until I cancel it manually," the server takes that order out of the super-fast RAM at night and saves it safely onto a hard disk database. The next morning, before the market opens, the server reads the hard disk and loads those orders back into the fast RAM so they are ready to go again.
To sum up, I hadn't prepared for this specific system design scenario, I panicked and also I had to pivot on the spot and answer purely based on my engineering intuition which I expand in this article which might be wrong approach :( .
Because my headspace was compromised, I ended up messing up the subsequent mathematics, statistics, and machine learning segments, including a round where I had to implement a minimal predictive analysis algorithm completely from scratch without using any external libraries.
However, I learned a lot from this , I will implement this execution engine and share the live code link soon :)
