Equity Fundamentals: Part 3

Adding an equity backtester to Serenity

Kyle Downey

Silvrback blog image

I have been thinking a lot about different models for backtesting and strategy development. While I would like to think it's possible to develop one universal backtester, I believe that different time horizons require materially different programming interfaces. In particular, tick-by-tick strategies are better expressed in terms of a reactive programming paradigm, while close-on-close strategies for a portfolio of assets are better suited for an imperative programming style. One of the goals of Serenity is to provide a single platform for both styles of programming, so with the addition of equities support via Sharadar's U.S. equity research database in Part 1 and Part 2 of this mini-series, we are going to wrap up with backtesting a basic equity strategy using the new Serenity Strategy API, leveraging the equity research database we built already.

Strategy paradigms

Strategies that depend upon 5 minute signals and trade constantly look very different from multi-week signals for strategies that hold positions for a relatively longer time. This principle is asset class-agnostic: you can have Bitcoin or FX strategies that depend upon real-time price signals or you could invest in a portfolio of cryptocurrency, and similarly in the equities space you can aim to write a real-time strategy or invest a portfolio of stocks.

That said, in general cryptocurrencies and FX have much less in the way of fundamental price drivers, and so your strategies will look different. Personally I am fond of a quantamental model for equities, and more "follow the price signal" for Bitcoin. But whatever your preferences, Serenity should offer an API that serves both lenses for trading.

As noted in the introduction, in my opinion the FRP (functional reactive programming) model is best for the real-time strategy case, and an imperative programming model that revolves around the portfolio of assets you are investing in is better for longer-term signals. In Serenity today these two models correspond to the code modules serenity.algo and serenity.strategy -- but for today we're going to focus on the strategy package only. (For those interested in the algo package, I highly recommend looking at strategies/bbands1.py, a simple Bollinger Bands-based strategy.)

The Strategy API

The strategy API is fundamentally an IoC (Inversion of Control) type model: there is a container, which could be a backtester or a live strategy engine, and a strategy class, and the container injects various services into the strategy class. This is very important, because it allows the strategy code to be entirely agnostic about whether it's running in research or live mode, so you can use the exact same business logic for testing and for trading.

Long-term strategies implement InvestmentStrategy, which is a subclass of the simpler Strategy abstract base class. In terms of the lifecycle, it proceeds in three phases:

initialization, corresponding to Strategy#init(StrategyContext)
policy preparation, which we'll cover in more detail next
rebalancing, corresponding to InvestmentStrategy#rebalance(RebalanceContext)

Policy preparation deserves a bit of explanation. Policies are objects that determine how the strategy behaves and interacts with the container. Policies include:

the initial portfolio, i.e. typically the cash we start with when investing
RebalanceSchedule, which tells the container when to rebalance the strategy
TradableUniverse, which tells the container what instruments the strategy wants to trade
DividendPolicy, which tells the container how to invest dividends (where applicable)

So if you have a strategy that trades takes $10K, trades mid-cap equities once a month and reinvests dividends in the portfolio, that corresponds to one set of policies, while a strategy that takes $1M, trades a bespoke selection of stocks quarterly based on their fundamentals and and collects dividends as cash corresponds to a completely different policy set. And the strategy is in control because it creates all these policy objects in code:

class InvestmentStrategy(Strategy):
    @abstractmethod
    def get_initial_portfolio(self) -> Portfolio:
        pass

    @abstractmethod
    def get_tradable_universe(self, base_universe: TradableUniverse):
         pass   

    @abstractmethod
    def get_rebalance_schedule(self, scheduler: NetworkScheduler, universe: TradableUniverse,
                               msp: MarketScheduleProvider) -> RebalanceSchedule:
        pass

    @abstractmethod
    def get_dividend_policy(self, trading_ctx: TradingContext, pricing_ctx: PricingContext) -> DividendPolicy:
        pass

Once you have decided on your strategy's policies, all you need to do is write a rebalance() method, which is the essential trading logic of the strategy. Every time the schedule indicates you want a rebalance the container is guaranteed to call rebalance(RebalanceContext) passing you appropriate objects for rebalancing.

This is all pretty abstract, so let's look at a real stock trading strategy: buy and hold AAPL for 10 years, rebalancing at inception only, and reinvesting dividends in more AAPL stock -- i.e. a total return strategy. This is an important strategy to have it our toolkit because we often want to benchmark our investments vs. buying and holding the market: a good return means nothing if all you were doing was riding a rising market and could have obtained the same result more cheaply and at lower risk by simply buying a broad-based ETF like SPY!

Buy-and-hold, step-by-step

In all the posts about writing investment strategies we're going to break the problem down into initialization, policy declaration and rebalancing logic, so we'll do the same here.

First we'll initialize:

class BuyAndHold(InvestmentStrategy):
    def __init__(self):
        self.ctx = None
        self.ticker_id = None
        self.ccy = None
        self.default_account = None

    def init(self, ctx: StrategyContext):
        self.ctx = ctx
        self.ticker_id = self.ctx.get_configuration()['universe']['ticker_id']
        self.ccy = self.ctx.get_configuration()['portfolio']['ccy']
        self.default_account = self.ctx.get_configuration()['portfolio']['default_account']

Here we're making reference to a TOMLformat configuration file, where each [section] corresponds to a top-level key in the dictionary and variables underneath correspond to a second-level dictionary key:

[strategy]
module="stock_buy_and_hold"
class="BuyAndHold"

[universe]
ticker_id=33

[portfolio]
ccy="USD"
default_account="main"

[portfolio.main]
cash_balance=10000

Next we create several policies. Starting investment is easily done with the Portfolio API -- we just deposit cash into the trading account:

    def get_initial_portfolio(self) -> Portfolio:
        portfolio = Portfolio([self.default_account], self.ccy)
         portfolio.get_account(self.default_account).get_cash_balance().
        deposit(Money(self.initial_investment, self.ccy))
        return portfolio

Next we need a tradable universe:

    def get_tradable_universe(self, base_universe: TradableUniverse):
        tradable = base_universe.lookup(self.ticker_id)
        return FixedTradableUniverse([tradable])

This is the simplest possible case: given a base universe (the database of all tickers from Sharadar in this case) we select a single ticker and create a fixed universe out of it. This is a very important optimization. By declaring which stocks we want as well as the time range of the backtest we can easily pre-load data in bulk, accelerating our backtest.

Next we want a rebalancing schedule:

    def get_rebalance_schedule(self, scheduler: NetworkScheduler, universe: TradableUniverse,
                               msp: MarketScheduleProvider) -> RebalanceSchedule:
        return OneShotRebalanceOnCloseSchedule(scheduler, universe, msp)

This is a special case but ultimately the simplest possible schedule: we rebalance once, at inception, on the first non-holiday date after our start time.

Finally we declare our dividend investment policy, using a built-in policy that corresponds to DRIP, i.e. reinvesting the dividends in the stock itself:

    def get_dividend_policy(self, trading_ctx: TradingContext, 
                             pricing_ctx: PricingContext) -> DividendPolicy:
        return ReinvestTradableDividendPolicy(trading_ctx, pricing_ctx)

Now that we have established our trading policies, we can look at trading logic. In this case all we need to do is buy stock, so this gives us a chance to introduce some other API's provided by the RebalanceContext.

First we ask for the ticker to trade, calling back into the TradableUniverse we created as a policy and passing it the unique ticker ID from the Sharadar equity database (in my system that corresponds to ticker_id=33 for Apple):

    def rebalance(self, rebalance_ctx: RebalanceContext):
        # get the instrument to trade
        tradable = rebalance_ctx.get_tradable_universe().lookup(self.ticker_id)

Next we apply some slightly tricky logic to determine how much stock we can afford:

        # compute how many shares we can afford to buy
        initial_cash = rebalance_ctx.get_portfolio().get_account(self.default_account).get_cash_balance().get_balance()
        px = rebalance_ctx.get_pricing_ctx().price(tradable, rebalance_ctx.get_rebalance_time().date(),
                                                   PriceField.CLOSE)
        qty = Decimal(int(initial_cash / px.amount))
        cost = rebalance_ctx.get_trading_ctx().get_trading_cost_per_qty(Side.BUY, tradable)
        est_total_cost = qty * cost
        remaining_cash = initial_cash - qty * px
        while remaining_cash < est_total_cost:
            qty -= 1
            est_total_cost = qty * cost
            remaining_cash = initial_cash - qty * px

This introduces another part of Strategy API, the TradingContext. You should think of the trading context as an abstraction of the connection to the exchange. As shown you can ask it for expected trading cost per quantity, but you can also use it to execute transactions. And that's exactly what we do as the final step, buying stock and storing the position in our trading account:

        # execute the buy
        tx = rebalance_ctx.get_trading_ctx().buy(tradable, qty)
        rebalance_ctx.get_portfolio().get_account(self.default_account).apply(tx)

This particular abstraction lets us fine-tune backtest and strategy behavior yet further. Though not yet exposed as an API or configuration parameter, internally the backtester plugs in a specific TradingCostCalculator to assist it in determining transaction costs. We also can fine-tune the details of how we simulate exectation, e.g. if you want to dig into the code take a look at MarketOnCloseTradingSimulator in serenity.strategy.historical -- this again is hard-coded in the current version, but in theory you can write your own TradingContext that makes different execution assumptions.

The complete code for this strategy is under strategies/stock_buy_and_hold.py in the Serenity codebase, and the configuration for backtesting is in strategies/stock_buy_and_hold.cfg.

Running a backtest

So we've written our first strategy. What do the 10 year returns look like? To answer this question we need to backtest our strategy. Run:

venv/bin/python src/serenity/strategy/backtester.py \
    --strategy_dir=strategies/ \
    --config_path=strategies/stock_buy_and_hold.cfg \
    --start_time="2011-01-01T00:00:00" 
    --end_time="2020-01-01T00:00:00"

What you'll see in the log is a single rebalance execution (it buys 849 shares) followed by "Performing daily bookkeeping" logs for every market close on NYSE, with periodic logs showing dividend reinvestment, and finally this simple output:

Account: [main]
    cash: USD 33.50
    AAPL: 976 shares (USD 71,648.16)

One thing to note: in my testing of the Shardar database I noticed some discrepancies vs. the dividend history published by Apple, which I suspect has something to do with the total return not quite matching my expectations. However it's still possible there is a bug lurking, as I'd be surprised if Sharadar had Apple dividends wrong, so caveat hackor.

Next steps

The backtester is very primitive, and is missing a strong risk & analytics package. Thankfully Quantopian's pyfolio can support other backtesters so long as they can export returns, transactions and positions as simple Pandas dataframes, so we can get all of those nice charts & analytics for free by modifying Serenity to export its portfolio object in that format. This is the goal for the 0.10.x release of Serenity in February 2021, and I'll blog about it when it's ready.

Once we have some better strategies backtested, the longer-range goal is to integrate with the E*Trade Python API for test and then live trading of equity strategies. Using a different platform from E*Trade? Consider contributing! We'd welcome other exchange API's both for equities and for cryptocurrencies.