Broad back testing methodology

At sharpely, we provide “on-the-fly” back-tested results so that investors can quickly analyse the historical performance of a metric, screen or basket.

In this article, we will discuss a broad backtesting methodology that we use including the all-important bias avoidance.

Historical walk forward back test

The backtest we use is technically referred to as the “historical walk forward” backtest in financial literature. This is the most standard technique used for backtesting.

In this, we simulate the “portfolio” performance “walking forward” from a start date. At each rebalancing period, past data is used to decide the allocation for the next period and the portfolio is rebalanced accordingly till we reach the end of our back-testing period.

Prices used in backtesting

By default, all prices used in the backtest are adjusted for splits, bonuses and rights issue. Prices are not adjusted for dividends or do not assume dividend reinvestment.

Therefore, when comparing the back-tested performance of a screen or a basket with a benchmark (like Nifty 50 TRI), do keep in mind that your screen backtest does not include dividends while the benchmark generally does. To that extent, back-tested performance will be understated.

The reason for not including dividends is that we deliberately want to keep the back-tested result a bit understated because there are many other pitfalls of blindly using a backtest as an investment criterion.

On a side note, the performance of individual instruments (stocks, ETFs and mutual funds) is shown after adjusting for dividends as well.

Handling rebalancing

A common mistake in most backtests is that “portfolio decision” and “transactions” are done on the same date. This induces a look-ahead bias in the back test as we will discuss in the next article.

In order to avoid this, we use the following methodology:

• On the day of the rebalancing, a new portfolio is selected which includes instruments and weights.
• For stocks and ETFs, all transactions (buying and selling) are done at prices that are average of the next day’s open, high, low and close (OHLC).
• For Mutual Funds, the next day’s closing NAVs are considered.

Since fractional shares are not allowed, rebalancing is done to minimize the leftover cash and still be as close as possible to the desired weights. Any leftover cash is assumed to earn no interest but is available for use on the next rebalancing date. For Mutual Funds, since the fractional unit is allowed, no cash is left over.

Note that there is no correct approach since all approaches are based on some or the other assumption. However, using the same day’s price for a transaction is definitely not correct when your portfolio decisions (and allocations) are based on closing prices/NAVs for that day.

Accounting for transaction costs and taxes

At present, the back-tested results do not take into account transaction costs and hence investors should be cognizant of the fact. Further, all results are shown pre-tax and do not account for tax incidence during rebalancing. We are in the process of including transaction costs and taxes in our backtest.

Stock scores in backtesting

On sharpely, all the scores are calculated daily. So what you see on the stock detail pages is the latest data calculated after EOD prices are available.

However, in backtesting the treatment is a bit different as we have to optimize for storage to enhance backtest speed. So in backtesting, only 2 scores - Price Momentum and Value Score are “stored” on a daily basis while all other scores are stored on a monthly basis.

For example, if you backtest a screen that uses quality score and value score, and you have a rebalancing due on 9th April 2023, the value score used will be as of 8th April 2023 while the quality score used will be as of 31st March 2023.

Note that since value and momentum are most impacted by price, it was a conscious call to store them daily so that the backtest always picks the latest data. On the other hand, other scores are not dependent on price (or if dependent, only indirectly), we have stored them at monthly frequency without compromising too much “information”.

Do also note that in paper trading, the screen will always pick the latest scores based on the latest price.