Testing an idea
How to turn an observation into a statement that can be wrong, and how many trades it takes to trust the result. Usually far more than it seems.
5 lessons · 37 min
Who this course is for
For anyone who has a trading idea and no certainty that it works. After this course you can state an idea so that it can be refuted, you know the sample size your edge requires, you recognise four ways of fooling yourself during testing, and you understand the signs that an idea should be declared dead.
- 1A testable idea: one that can turn out to be wrong8 minMost trading rules are phrased so that they fit any outcome. Such a rule cannot be tested — and therefore cannot be relied on.
- 2How many trades before you can trust the result7 minOver twenty trades a random system shows anywhere between 28% and 72% wins. Proving an edge of five percentage points takes a thousand trades — and that changes which ideas are worth looking for at all.
- 3Four ways to fool yourself while testing7 minA test more often confirms an idea not because it works but because the tester quietly helped it. All four ways look like careful work.
- 4Forward testing: a check on a market you have never seen8 minA good result on history is necessary but not sufficient. Three things appear only going forward, and one of them costs more than the fee.
- 5When to declare an idea dead7 minA loss by itself proves nothing: a run of five losers happens almost always. Death conditions are set in advance — otherwise the decision is made by mood.