Basic Points

• Backtesting is a way to test a strategy against historical data. This can give an idea (but not certainty) of how it will perform in the future. 
• One way to approach backtesting is to test a strategy and then to test variations of each parameter. For example, one might try differently sized moving averages, or experiment with different lengths of stop losses (exit point for a loss) or profit targets (exit points for a profit). 
• However, if changing parameters too aggressively in an attempt at optimization, then this can easily become overfitting, in which case historical data can greatly outperform future data. 
• Backtesting is much different than Monte Carlo (forward testing) which will analyze thousands of random outcomes and then derive statistics from that. Being a simulation based on random data, this is not a certainty either. 

Advanced: Pitfalls of Backtesting

As a general practice, it comes with a couple of main problems, namely path-dependency and overfitting. Path-dependency is when the overall directional drift of a security or index is a main driver for the results of a test. This can be mitigated against by using large samples, testing conditions in multiple regimes, such as high/low IV conditions, and also bullish/flat/bearish markets. 

Overfitting, being another pitfall of backtesting, happens when strongly profitable results are generated from a strategy using historical data, but that strategy falls short or is unprofitable on newer data. These instances of overfitting are often caused by testing too many factors at once, or changing factors or parameters until a test does well; the hazard is overconfidence in a trading strategy. Anti-bias techniques can be important and arguably necessary to establish the data as having sufficient integrity for it to help build a strategy or thesis.