Backtesting vs paper trading vs live trading: what each stage actually proves
These three stages get treated as interchangeable validation steps — "I tested it" — when they actually prove different, non-overlapping things. Skipping straight from backtest to live trading is one of the most common ways a strategy that looked good ends up losing real money.
Backtesting: does this idea have any merit at all, on data it's already seen
A backtest runs a strategy against historical price data and reports what would have happened. It's fast, cheap, and lets you test years of history in seconds — and it's also the easiest of the three to fool yourself with. The core risk is curve-fitting: with enough parameter tweaking, almost any strategy can be made to look good on a specific historical window, without that meaning it captured anything real about how markets behave.
What a backtest actually proves: whether the basic mechanics work (does the strategy do what you think it does, computed correctly) and whether the idea clears a bare-minimum bar — beating a trivial baseline like buy-and-hold, after realistic fees and slippage are included, not just as raw price movement. What it doesn't prove: that the strategy has real, repeatable edge going forward, since it was tested on the exact data it could have been (even unintentionally) tuned against.
Paper trading: does it hold up on data it hasn't seen yet
Paper trading runs the strategy against live market data in real time, with fake money. This is the first honest out-of-sample test — the strategy has no way to have been tuned against days that haven't happened yet. It also surfaces problems a backtest can't: how the strategy behaves with real-time data quirks, what happens when a data feed hiccups, whether the logic that seemed fine in a vectorized backtest script actually runs correctly as a live, scheduled process.
What paper trading actually proves: whether the backtested edge (if any) persists on fresh data, and whether the operational plumbing works. What it doesn't prove: that the strategy will behave identically with real capital — fake money doesn't flinch at a big drawdown, and paper fills don't reflect real slippage under real market stress the way live orders do.
Live trading: does it hold up when real money and real psychology are involved
This is the only stage that tests the thing that actually matters in the end — real capital, real fills, real fees, and (for a human running it) real emotional pressure during a drawdown. It's also the only stage where mistakes cost actual money, which is exactly why it should be the last stage, not the first real test.
Why skipping stages is the common failure mode
Going straight from backtest to live skips the one step that tests for curve-fitting without any capital at risk. Going straight from paper to live with a large position skips the step of confirming small-scale live behavior matches paper behavior before scaling up. The stages are cheap to expensive, in that order, specifically so problems get caught before they're expensive — treating any of them as optional defeats the reason they exist as separate steps at all.
What this looks like as actual code, not just a concept
A real three-stage setup is usually three separate small pieces of infrastructure, not one script with a flag:
run_backtest.py # stage 1 — vectorized, runs against 2 years of history in seconds
paper_trader.py # stage 2 — scheduled (e.g. daily via cron), tracks fake P&L on live data
# stage 3: live trading — a broker connection + real capital, added only after 1 and 2 hold up
Each stage is meaningfully more expensive to run wrong than the last: a bad backtest costs nothing but time, a bad paper-trading setup costs nothing but a wasted validation window, and a bad live setup costs real money. That cost ordering is the actual argument for not skipping stages, not just process for its own sake.
Sources: standard quantitative trading practice regarding out-of-sample validation and curve-fitting risk; this project's own methodology follows exactly this three-stage sequence, with paper trading currently running on a daily cron schedule and no live capital committed yet.