Improving Serenity's production infrastructure
Serenity's production server is a Linux box sitting next to my desk which runs Ubuntu's microk8s Kubernetes distribution. It runs 24x7 collecting tick data from several cryptocurrency exchanges and once a day uploads the tick data to Azure Blob Storage. This presents a problem: this highly valuable, impossible-to-reproduce data is not . . .
Integrating Azure identity services with Serenity
Serenity's tick-by-tick backtester relies on a tick database called Behemoth, which in turn sits on top of Azure blob storage. Serenity's crypto market data feed handlers run 24x7, but every day at midnight they dually-publish the tick data to local RAID disk and Azure blob storage. The only problem: in the current . . .
Adding an equity backtester to Serenity
I have been thinking a lot about different models for backtesting and strategy development. While I would like to think it's possible to develop one universal backtester, I believe that different time horizons require materially different programming interfaces. In particular, tick-by-tick strategies are better expressed in . . .
Building a daily Sharadar batch pipeline with Luigi
In Part 1 we looked at using TimescaleDB and SQLAlchemy to build a relational database model of the Sharadar equity dataset with a Python object model sitting on top. The initial cut of this project ran on my desktop and broke up each of the dataset loads into a simple script that I could run in PyCharm. In this next blog post we'll dive . . .
Building an equity database with Sharadar & sqlalchemy
After experimenting with Polygon.io I decided to try out Sharadar, which is delivered via the Quandl API.It had a very good reputation as a survivorship-bias free dataset and Quandl offered a $89/month package deal for a decade of Sharadar fundamentals, equity prices, holdings data, corporate actions and more. The Sharadar Core US Equities . . .
Bringing Serenity to the cloud with AKS
Up until now all of the Kubernetes work I've done has been local on my Ubuntu server, charger. But there is only so much compute & memory I can wring from that server, so in this blog post we will look at Azure Kubernetes Service (AKS) and use Azure to deploy both a private container registry and a Kubernetes cluster, with the intent . . .
Scaling backtest capacity with Dask and Kubernetes
Two months ago I upgraded the motherboard in my home Linux PC to accommodate an Intel i9 CPU, specifically the Intel Core i9-10900 Comet Lake 10-Core CPU running at 2.8 GHz paired with the Cooler Master MasterLiquid ML240R CPU liquid cooler. This box runs production market data capture as well as a test trading strategy that I am working on, . . .