Some of the projects I’ve worked on, and my areas of interest and expertise:

Hadoop

I was one of the first committers on Apache Hadoop, and worked on it and many other distributed systems projects in the Hadoop ecosystem for the best part of a decade while working at Cloudera. I wrote four editions of the bestselling book Hadoop: The Definitive Guide, published by O’Reilly.

Genomics

I was the primary author of Spark support in GATK, working with members of the Broad Institute GATK team. I also created Disq for reading and writing bioinformatics sequencing formats from Spark.

I ported the single cell preprocessing pipeline in Scanpy so it could run in parallel using Dask, and on GPUs using RAPIDS.

SciPy

I have made contributions to the dimensionality reduction software UMAP and the related project pynndescent (for calculating approximate nearest neighbours).

Open Data

In early March 2020 I began collating the disparate sources of UK COVID-19 data, by writing web crawlers to integrate the data for COVID-19 tests, confirmed cases, and deaths into a set of CSV files. My work has been used by many different individuals and organisations, including John Burn-Murdoch’s visualizations for the Financial Times.

In 2019 I produced the data analyses and visualizations of Welsh school funding data, for the Level the Playing Field campaign for fair funding for schools in Wales.

Visualization

In 2020 I started a blog about data visualization with the goal of creating one interesting visualization per week - with no constraints on dataset, visualization type, or technology.

Over the years I’ve created many geometric visualizations in my spare time.

Diabetes tech

In 2018 I was diagnosed with Type 1 Diabetes, and since then I have written various pieces of software to help manage the condition.

Games

I’m interested in board games and puzzles, and how to get computers to play them. Examples include: Mastermind and SET®.