Trading · 71 / 86 · 2 min
Backtests
A backtest shows how a rule would have done on old prices. Old prices do not include every fee, every moment you would have hesitated, or the future.
Old prices do not include every fee, every hesitation, or the future.
A worked case
A rule looks perfect on five years of prices. The test used the close, ignored fees, and never had a moment where the person refused a signal. Live, the first month pays the fees the test skipped.
The pretty history
Rules get tuned until the past looks good. That tuning is called fit. It is also how a useless rule learns the past by heart.
What is missing
Fees, spreads, failed fills, and the days the data was wrong. A curve that never loses is a warning, not a comfort.
A backtest is a lab note. It is not a balance.
The perfect curve ended yesterday
A backtest shows how a rule would have done on old prices. It is easy to make beautiful, and the beauty is usually the fees you left out, the trades you would have skipped, and the fact that you designed the rule after seeing the prices. Academic finance has a long literature on this, under the name overfitting. Crypto Twitter has a long catalogue of screenshots. They are the same disease at different prices.
The test ends today. The money you would risk starts tomorrow, in a market that knows the old trick and charges you the spread the spreadsheet set to zero. A rule that doubles in the past and dies in a month of fees has not been unlucky. It has been measured properly for the first time.
If someone sells you a curve, ask for the fees, the slippage, and the date they froze the rules. If the rules were free to change until the curve looked good, you are buying a story about the past.
Read it yourself
These links are the record. They are not a recommendation, and they are not instructions. A news story or a court paper can still be wrong about a detail. The check does not change because a famous name is in the story.
Apply the check
Open a question. The line is about this topic. It is not a verdict that anything is safe.
The person who sold the perfect curve.
The test, which used a past it was allowed to see.
The next month, which the test did not contain.
Check yourself
If a backtest never loses, should you trust it more?
No. That is a sign it was fitted to the past or ignored costs.
After this you can ask what fees were included, and be suspicious of a curve that never loses.