Is an expert advisor backtest reliable?
-
- 0
- 1564
- 0
There are often expert advisers whose backtests are efficient, but once activated on a trading account, we notice that the performance just isn’t there. What are the causes? What solutions can be found to remedy this?
Backtests are generally distorted if the expert advisor works with technical indicators.
Effectively, the backtest only takes into account the level or direction of the indicator once the candlestick being formed has closed. However, under real trading conditions, some positions are opened (or not), or some stops loss /take profit orders are executed (or not), even before the candlestick being formed is closed.
It’s clearer with an example:
Let's take the example of the expert advisor Super Trend; this one opens a position in the direction of the Super Trend indicator with a stop loss on the active position and a stop on the opening of a new reverse position at the same level at X pips above or below the Super Trend.
The problem usually comes from the wicks! (not included in backtests).
Solution:
Ideally, backtests should include the impact of wicks.
- or the candlesticks close; the backtest would then take into account the highest/lowest points formed in the previous candlestick, the impact of these wicks on the indicator(s) used, and the final impact on the active expert advisor.
- or backtests take up the global and complete development of each candlestick formed, in rapid succession. But that would take longer.
Depending on the trading platform on which the backtest is performed, the spread is or isn’t taken into account.
If the spread is not taken into account, the backtest is completely distorted. Effectively, the tester only uses the Bid price, the Ask price, or the average price, to calculate performance. However, within a few pips accuracy, the tester may miss a stop loss that would have actually been triggered, or a position that would have actually been opened. In the case of a missed stop loss, this is to the advantage of the backtest, because in reality the trading account would have stopped the gain or loss position (depending on the level of the stop loss that could be a "follower"), and moreover, the profit subsequently recorded by the backtest would not actually have been recorded.
If the spread is taken into account, it can generally only be fixed (for the backtest). Two new problems that further distort the backtest:
- the width of spreads are not taken into account; this means that with certain economic news releases, when volatility rises and spreads widen, some stop losses have not been triggered by the backtest whilst in real life they are.
- if we include a fixed average backtest spread that best reflects the real variable trading conditions, mini spreads inevitably appear; depending on the EA, and its aggressiveness, its small spreads can completely distort performance.
Solution:
The backtest should take into account the actual spread recorded at any time during the formation of each candlestick. It sounds very complicated.
A simple accumulation of causes 1 and 2.
Beware of scams! Backtests generally do not reflect the actual performance of expert advisers.
Pay attention to the way the EA works. Check if the backtest signals are identical to those that would have been detected in real life. See if the correct spread has been applied to the displayed backtests (because it is easy to display a double winning backtest simply by setting the spread to zero).
Only expert adviser backtests that only work on a value/price basis (and therefore without indicators) give a better indication of performance in real trading conditions.
IMPORTANT:
Depending on the trading platform used, an EA or an automatic trading system’s backtests are more or less realistic. On ProRealTime the backtest results are very realistic (not like on other trading platforms).
Cause 1: calculation at the candlestick’s close
Backtests are generally distorted if the expert advisor works with technical indicators.
Effectively, the backtest only takes into account the level or direction of the indicator once the candlestick being formed has closed. However, under real trading conditions, some positions are opened (or not), or some stops loss /take profit orders are executed (or not), even before the candlestick being formed is closed.
It’s clearer with an example:
Let's take the example of the expert advisor Super Trend; this one opens a position in the direction of the Super Trend indicator with a stop loss on the active position and a stop on the opening of a new reverse position at the same level at X pips above or below the Super Trend.
The problem usually comes from the wicks! (not included in backtests).
Solution:
Ideally, backtests should include the impact of wicks.
- or the candlesticks close; the backtest would then take into account the highest/lowest points formed in the previous candlestick, the impact of these wicks on the indicator(s) used, and the final impact on the active expert advisor.
- or backtests take up the global and complete development of each candlestick formed, in rapid succession. But that would take longer.
Cause 2: the spread
Depending on the trading platform on which the backtest is performed, the spread is or isn’t taken into account.
If the spread is not taken into account, the backtest is completely distorted. Effectively, the tester only uses the Bid price, the Ask price, or the average price, to calculate performance. However, within a few pips accuracy, the tester may miss a stop loss that would have actually been triggered, or a position that would have actually been opened. In the case of a missed stop loss, this is to the advantage of the backtest, because in reality the trading account would have stopped the gain or loss position (depending on the level of the stop loss that could be a "follower"), and moreover, the profit subsequently recorded by the backtest would not actually have been recorded.
If the spread is taken into account, it can generally only be fixed (for the backtest). Two new problems that further distort the backtest:
- the width of spreads are not taken into account; this means that with certain economic news releases, when volatility rises and spreads widen, some stop losses have not been triggered by the backtest whilst in real life they are.
- if we include a fixed average backtest spread that best reflects the real variable trading conditions, mini spreads inevitably appear; depending on the EA, and its aggressiveness, its small spreads can completely distort performance.
Solution:
The backtest should take into account the actual spread recorded at any time during the formation of each candlestick. It sounds very complicated.
Cause 3
A simple accumulation of causes 1 and 2.
Conclusion
Beware of scams! Backtests generally do not reflect the actual performance of expert advisers.
Pay attention to the way the EA works. Check if the backtest signals are identical to those that would have been detected in real life. See if the correct spread has been applied to the displayed backtests (because it is easy to display a double winning backtest simply by setting the spread to zero).
Only expert adviser backtests that only work on a value/price basis (and therefore without indicators) give a better indication of performance in real trading conditions.
IMPORTANT:
Depending on the trading platform used, an EA or an automatic trading system’s backtests are more or less realistic. On ProRealTime the backtest results are very realistic (not like on other trading platforms).
