1 / 5 · 8 min
A testable idea: one that can turn out to be wrong
Most trading rules are phrased so that they fit any outcome. Such a rule cannot be tested — and therefore cannot be relied on.
The distinction that decides everything
Take your idea and ask: what result would show that it is WRONG? If there is no answer, there is nothing to test. Price always returns to gaps is irrefutable: it did not return today, so it will in a month; it did not return in a month, so it was the wrong gap. Such a phrasing is compatible with any future and therefore says nothing about the future. The testable version sounds different: in 70% of cases price touches the gap within five days. Now it is clear what to count and at what result the statement must be discarded.
The same observation in two forms
| Untestable | Testable |
|---|---|
| A level is stronger the more times it is touched | After the third touch a level breaks more often than after the first |
| Volume confirms the move | A breakout on above-median volume holds more often than one without |
| Trade with the trend | Entering with the daily candle's direction gives higher expectancy than against |
| The market grabs liquidity before a move | After the day's low is taken, price rises within 4 hours in 60% of cases |
The difference is not in strictness of language but in the presence of a number and a deadline. A testable statement has three parts: what happens, in what share of cases, and over what time. Without any one of the three you get a phrase to which data cannot be applied — and an argument about it is endless, settled by authority rather than by measurement.
Four parts of a working formulation
First, the ENTRY CONDITION, described so that two people would mark it identically on a chart. A strong level will not do; the high of the last twenty candles will. Second, the ACTION: what exactly we do, at what price, with what order. Third, the OUTCOMES: where we take profit and where we admit the mistake, both prices known before entry. Fourth, the DEADLINE: how long the idea has to come true, after which we exit on time. Only with all four does an idea become countable.
What the transformation looks like
Before: I buy from a strong support when I can see the buyer defending the level. After: I take a coin from the top fifty by turnover; the condition is that price touched the low of the last twenty hourly candles and returned above it within two hours; entry at market on the hourly close; stop 0.7 ATR below the low; target 1.5 ATR; if neither happens within 12 hours I exit on time. The first phrase cannot be tested, the second can, and that is the whole difference between an opinion and a hypothesis.
Leaving the word usually in the rule
There is usually a pullback after such a move looks cautious and therefore honest. In fact it makes the rule untestable: any exception is explained by this was not that case. Caution is expressed with a number, not a word: not usually but in 62 cases out of 100. If you do not know that number, write down that you do not know it and make obtaining it the first step. The word usually in a trading rule almost always means I have not counted.
An idea and its explanation are different things
It helps to separate the statement about WHAT happens from the story about WHY. The first is checked against data, the second is not, and mixing them is expensive. After the low is taken price goes up can be counted. Because the market maker collected liquidity cannot: a chart shows only price and volume, intentions are not written there. An explanation helps you remember the rule and invent similar ones, but you must lean on the measurement. When an idea stops working, the first thing to discard is the explanation, not the data.
Testing an idea on the same stretch where you spotted it
An idea always comes from an observation: you noticed a repeating pattern on a chart. Testing it on that same stretch of history is meaningless — it works there by definition, that is where you took it from. This is the most common and least visible testing mistake: the result looks brilliant and then falls apart on the next stretch. The correct order is to see it on one stretch, formulate it, and test on another you have not looked at. How to split history honestly is the subject of the next lesson.
The service has bar replay: the chart advances candle by candle with the future genuinely hidden. It is the only way to mark signals honestly — without knowing what came next. Open it on an instrument you know and mark ten entries by your rule, writing the decision down BEFORE advancing. Half of all rules do not survive this procedure, and it is better to find that out in replay.
The chart and bar replayRewrite your rule in four parts
Take the rule you trade most often and write it in four parts: entry condition, action, two outcomes, deadline. Then give the formulation to another trader and ask them to mark the last five occurrences on a chart. If their marks match yours, the rule is described unambiguously. If not, the disagreement shows exactly which part is phrased in words instead of numbers, and that is the part to rewrite first.
How does price respects the level differ from a testable statement?
By having no number and no deadline. Respects fits a bounce, a one-hour pause and a slowdown before a breakout — that is, any behaviour of price near a line, including opposite ones. A testable version names the observable event and its share: for instance, after the first touch price moves at least 1 ATR away from the level in 55% of cases within six hours. Now it is clear what to count, and there exists a result at which the statement must be declared wrong.
Do you need an explanation of why an idea works?
It is useful but not required, and it does not replace testing. An explanation helps you invent similar ideas and hints at where the rule may break — if the cause disappears, so does the effect. But an idea cannot be confirmed by an explanation: a convincing story can be composed for any set of data, including a random one. The order is measurement first, explanation second, never the reverse. And if the explanation is beautiful while the measurement showed nothing, it is the explanation that gets discarded.