[Matt Hagy]

I'm a software engineer and have previously been a data scientist, manager, and academic. Here's an assortment of various personal projects and blog posts.

matthew.hagy@gmail.com | matthagy.substack.com | github.com/matthagy

---

Substack analysis and experiments w/ generative AI

• AI Categorization of Slow Boring Articles
• AI Categorization and Summarization of Substack Comments using GPT-4o: Slow Boring | The Free Press | Richard Hanania | Matt Hagy Comments
• An interactive map of Substack comment hierarchical categories (methods exposition)
• Developing a Custom Substack Front-end
• Newsletter Stats - Analysis of some of the newsletters that I subscribe to

DB From Zero

A series of small projects implementing various database components and benchmarking their performance. Here are a few highlighted projects:

• Disk-Based Storage Engine for Read-Only Key/Value Data
• Log-Structured Merge-Tree for Persistent Reads and Writes
• Write-Ahead Logging for Durable Key/Value Persistence

Software engineering

• Initial impressions of Scala from a Java and Python data engineer
• Don't give me another domain specific language. Give me a library
• Introducing scalaps: Scala-inspired data structures for Python

Data science, analytics, and statistical modeling

• Our Exciting Journey as Data Scientists Onwards to Higher Levels of Abstraction
• The limits of SQL in data science, analytics, and engineering
• An introduction to Spark GraphFrame with examples analyzing the Wikipedia link graph
• A review of GCP for data science and engineering

Computational statistical chemistry

• Investigating the long-time behavior of the two-dimensional Ising model by leveraging the Java Virtual Machine (JVM)
• Developing simple chemical physics simulations that run in the browser

Small machine learning research projects

• Analyzing Recurrent Neural Networks (RNNs) Using Polymer Dynamics Theory
• Python to Scala transpiler using neural machine translation (NMT)

Open source software

• scalaps - Scala-inspired data structures for Python
• MockRDD - A Python module for testing PySpark code
• PyLjFluid - Classical simulations of Lennard-Jones fluids using Python
• rtchemstats - Python library to compute statistical descriptions of chemical systems in real-time without the need to write trajectories to disk
• substack_client - Library for accessing Substack content using the REST API

In-browsers chemistry simulations

A few simple simulations developed in Scala.js that run within your web browser to demonstrate various simulations and analysis methods.

• 2D Ising Model Simulation
• Random Walk in a Periodic Box
• Simulation of Soft Spheres in a Periodic Box
• Simulation of Polymer Chains

See also background academic work in Characterizing the Molecular-Level Dynamics of Polystyrene/Toluene Solutions: Development of Methods for Dynamical Simulation of Heterogeneous Coarse-Grained Models (defense slides PDF) and resulting thesis Dynamical simulation of structured colloidal particles

Learning and leveraging Scala

A small series that aims to teach Scala to non-engineers with a focus on quickly learning the basics to start getting things done with this powerful programming language.

1. Quickly learning the basics of Scala through Structure and Interpretation of Computer Programs examples
2. A brief tour of lists in Scala and algorithmically processing them in SICP exercises
3. Interactively exploring Reddit posts using basic Scala in your browser