Kyle Downey

Serenity DevOps #1 - Motivation

Improving Serenity's production infrastructure

Serenity's production server is a Linux box sitting next to my desk which runs Ubuntu's microk8s Kubernetes distribution. It runs 24x7 collecting tick data from several cryptocurrency exchanges and once a day uploads the tick data to Azure Blob Storage. This presents a problem: this highly valuable, impossible-to-reproduce data is not . . .

Read More

June 16, 2021

Securing Behemoth

Integrating Azure identity services with Serenity

Silvrback blog image

Serenity's tick-by-tick backtester relies on a tick database called Behemoth, which in turn sits on top of Azure blob storage. Serenity's crypto market data feed handlers run 24x7, but every day at midnight they dually-publish the tick data to local RAID disk and Azure blob storage. The only problem: in the current . . .

Read More

January 08, 2021

Equity Fundamentals: Part 3

Adding an equity backtester to Serenity

Silvrback blog image

I have been thinking a lot about different models for backtesting and strategy development. While I would like to think it's possible to develop one universal backtester, I believe that different time horizons require materially different programming interfaces. In particular, tick-by-tick strategies are better expressed in . . .

Read More

January 03, 2021

Equity Fundamentals: Part 2

Building a daily Sharadar batch pipeline with Luigi

In Part 1 we looked at using TimescaleDB and SQLAlchemy to build a relational database model of the Sharadar equity dataset with a Python object model sitting on top. The initial cut of this project ran on my desktop and broke up each of the dataset loads into a simple script that I could run in PyCharm. In this next blog post we'll dive . . .

Read More

December 10, 2020

Equity Fundamentals: Part 1

Building an equity database with Sharadar & sqlalchemy

After experimenting with I decided to try out Sharadar, which is delivered via the Quandl API.It had a very good reputation as a survivorship-bias free dataset and Quandl offered a $89/month package deal for a decade of Sharadar fundamentals, equity prices, holdings data, corporate actions and more. The Sharadar Core US Equities . . .

Read More

December 05, 2020

Up, Up and Away

Bringing Serenity to the cloud with AKS

Up until now all of the Kubernetes work I've done has been local on my Ubuntu server, charger. But there is only so much compute & memory I can wring from that server, so in this blog post we will look at Azure Kubernetes Service (AKS) and use Azure to deploy both a private container registry and a Kubernetes cluster, with the intent . . .

Read More

November 06, 2020

Distributed Backtesting

Scaling backtest capacity with Dask and Kubernetes

Two months ago I upgraded the motherboard in my home Linux PC to accommodate an Intel i9 CPU, specifically the Intel Core i9-10900 Comet Lake 10-Core CPU running at 2.8 GHz paired with the Cooler Master MasterLiquid ML240R CPU liquid cooler. This box runs production market data capture as well as a test trading strategy that I am working on, . . .

Read More

October 22, 2020