Historical Trading Simulations

Some dos and don'ts when running simulations

I very recently started alpha testing the simulation platform that I have been consulting on for the last 2.5 years and thought I’d mention a few things that I view as important when setting up, running and interpreting historical simulations in the trading world.

Setting up the simulation: Objectives

The first point where traders go off the rails with simulations is how they set them up. In my opinion, you start completely away from the simulation with the objectives of the new strategy. Whatever you come up with you have to be able to flawlessly execute the trading strategy or it is of zero use to you. If you have a risk tolerance levels at a very low level, you are not going to want to test highly leveraged, high risk level strategies. If you want more action, you’ll not want to mess with strategies that are likely to plod along with very few trades. You have to know where you are trying to end up, before embarking on the journey through simulation land.

Buy/Sell Engine:

Next, now that you know what you are designing this strategy to do, you need to develop the logic. If you want to capture some longer term trends and make only occasional trades, the you should set up your simulation with some long term trend following models. If you are looking to trade more frequently and want to trade in the noise band of prices, then you will go shorter-term trend following perhaps. If you want to catch snap backs against the current trend, the mean reversion indicators will be a logical choice. Finally, if you want to day trade, you’ll need to keep things very short term as each day you’re going to run out of time.

Position Sizing:

Now that you’ve decided on your Buy/Sell Engine or indicators that you want to examine, you’ll need to decide how much risk of loss and volatility you are willing to take on. This will vary with each of the thousands of traders that will read this “Thoughts” article. If you are looking for more risk, then ratchet up the position sizing algorithm, so that you are heavily invested, maybe on margin, maybe with futures or options and expect a wild simulation. If you are retired like me, and are not looking to win the US Trading Championship, then dial the exposures back. My risk as a percent of equity and volatility as a percent of equity formulas in my position sizing book are an easy way to do that. I’m running initial risk as a percent of equity at 0.5% and volatility as a percent of equity of 0.2%, fairly low levels.

Total Strategy Risk:

I also at this point ask myself the question, “How much am I willing to lose if every position moves against me to my stop loss points?” It’s a very important question and one which most people never ask themselves until they are in a drawdown and not sleeping well at night. For me in my futures trading for instance, I use -15% for that strategy. More than that would be uncomfortable for me and I don’t see why I can’t design what I’m doing to fit my situation. I am no longer managing other peoples’ money and don’t have to consider anyone else in setting up a new strategy.

Running the simulation:

I have actually had traders that I know personally get through all the above checkpoints and then set up a run so that it steps through every possible parameter from 1 to 100 incrementing each period by 0.5 for example. If you have numerous parameters, the number of runs will become exponential and you may be waiting a very long time for the simulations to finish running. You are wasting computer time. You will end up with a best case that will be very compelling, but will essential be a force-fit of the specific history you fed the simulation. I won’t call it garbage, but I believe you are wasting your time over-optimizing history to prepare yourself for the future. Simply pick a few parameters and increment in jumps to levels that you will be able to handle in the future. If you want very long-term, maybe 50, 75, 100, 150 and 200 days should give you a feel for it. If looking at shorter term variables, maybe try 3, 5, 7 and 9 days. Don’t obsess over getting the perfect best answer on the simulation, because the future is going to be different anyway.

Interpreting the simulation:

I look at simulations different from what I see others doing. First, I remind myself over and over again, this is just a simulation not the future real world. Next, I’m a return to risk junkie. I don’t mind making more return, as long as I’m not taking on more risk above some level that I’m comfortable with. I find many traders seek the higher returns and look at some of the risk numbers casually thinking they can stand the heat for those returns and when reality sets in during their future trading, they really cannot stand the heat.

I start from risk, looking at average depth and time spent in maximum drawdowns and average drawdowns. I mentally double both the time and depth, because, as the old trader axiom states, “Your worst drawdown is ahead of you.” I’m also looking at the number of trades that I have to do to run the strategy years into the future. The more trades a strategy has, the more work I have to do to monitor, pay commissions, deal with possible trading errors, etc. With less trades, the work load is easier.

I then identify periods in the simulation where markets were going up, down and sideways. Knowing those dates, I dig into the details of the simulation results and try to understand how the math or logic I used in the simulation reacted to up, down and sideways periods. Is it what I would have expected it to be? If I’m looking at a mean-reversion strategy and it lost money during a sideways period, when I would have thought it would be producing lots of profits, then I’m going to dig in deeper and try to understand why that was happening. If a market has a long run up or down and I’m simulating a long-term trend following approach, I would expect large profits in those periods. If I checked the same strategy for results in a sideways period, I would expect whipsaws and drawdowns. Do I see those in the simulation?

Lastly, I check how this strategy fits into the total portfolio. Is this logic giving me the same things as some other strategy that I’m already running every day? That’s seems like adding more work for no extra benefit. My friend Laurens Bensdorp of the Trading Mastery School talks about “filling the potholes.” The colored area in the equity chart below show me times that I want this new strategy to perform.

Sure, you might want to juice up returns when your current strategies are doing well. You end up with more ending equity that way. But, if you add more deep or lengthy potholes, you are not doing your psyche any favors.

Notice that returns were almost the last thing I looked at in the simulations. It’s dangerous to lock your mind onto some Compound Average Growth Rate (CAGR) and start planning your retirement. I simply look at the pothole periods, drill down into the simulation and see how the new strategy’s simulation did during those periods. If they will help fill in the potholes, adding the strategy to the portfolio mix should help stabilize the the total portfolio going forward, helping you in your efforts to “enjoy the ride”.