Matt Hagy

I’m a software engineer and have previously been a data scientist, manager, and academic. Here’s an assortment of various personal projects and blog posts.

matthew.hagy@gmail.com | matthagy.substack.com | github.com/matthagy

---

DB From Zero

A series of small projects implementing various database components and benchmarking their performance. Here are a few highlighted projects:

• Disk-Based Storage Engine for Read-Only Key/Value Data
• Log-Structured Merge-Tree for Persistent Reads and Writes
• Write-Ahead Logging for Durable Key/Value Persistence

Substack analysis and experiments

• An interactive map of Substack comment hierarchical categories (methods exposition)
• Using OpenAI APIs to categorize and label substack comments
• Developing a Custom Substack Front-end
• Newsletter Stats - Analysis of some of the newsletters that I subscribe to

Software engineering

• Initial impressions of Scala from a Java and Python data engineer
• Don’t give me another domain specific language. Give me a library
• Introducing scalaps: Scala-inspired data structures for Python

Data science, analytics, and statistical modeling

• Our Exciting Journey as Data Scientists Onwards to Higher Levels of Abstraction
• The limits of SQL in data science, analytics, and engineering
• An introduction to Spark GraphFrame with examples analyzing the Wikipedia link graph
• A review of GCP for data science and engineering

Computational statistical chemistry

• Investigating the long-time behavior of the two-dimensional Ising model by leveraging the Java Virtual Machine (JVM)
• Developing simple chemical physics simulations that run in the browser

Small machine learning research projects

• Analyzing Recurrent Neural Networks (RNNs) Using Polymer Dynamics Theory
• Python to Scala transpiler using neural machine translation (NMT)

Open source software

• scalaps - Scala-inspired data structures for Python
• MockRDD - A Python module for testing PySpark code
• PyLjFluid - Classical simulations of Lennard-Jones fluids using Python
• rtchemstats - Python library to compute statistical descriptions of chemical systems in real-time without the need to write trajectories to disk
• substack_client - Library for accessing Substack content using the REST API

In-browsers chemistry simulations

A few simple simulations developed in Scala.js that run within your web browser to demonstrate various simulations and analysis methods.

• 2D Ising Model Simulation
• Random Walk in a Periodic Box
• Simulation of Soft Spheres in a Periodic Box
• Simulation of Polymer Chains

Learning and leveraging Scala

A small series that aims to teach Scala to non-engineers with a focus on quickly learning the basics to start getting things done with this powerful programming language.

1. Quickly learning the basics of Scala through Structure and Interpretation of Computer Programs examples
2. A brief tour of lists in Scala and algorithmically processing them in SICP exercises
3. Interactively exploring Reddit posts using basic Scala in your browser