systems-mcp: generate systems models via LLM

llm (12), systems (6)

Back in 2018, I wrote lethain/systems as a domain-specific language for writing runnable systems models, and introduced it with this blog post modeling a hiring funnel. While it’s far from a perfect system, I’ve gotten a lot of value out of it over the last seven years, because it allows me to maintain systems models in version control.

As I’ve been playing with writing Model Context Protocol (MCP) servers, one I’ve been thinking about frequently is one to help writing systems syntax, and I finally put that together in the lethain/systems-mcp repository.

More detailed installation and usage instructions are in the GitHub repository, so I’ll just share a couple of screenshots and comments here. Starting with the load_systems_documentation tool which loads a copy of lethain/systems/README.md and a file with example systems into the context window.

The image contains text and code snippets related to creating a systems model for user acquisition in a social network. It includes a brief description of systems modeling tools and a specification for modeling user flow from initial discovery to active contribution within the network.

The biggest challenge of properly writing DSLs with an LLM is providing enough in-context learning (ICL) examples, and I think the idea of providing tools that are specifically designed to provide that context is a very interesting idea. Eventually I imagine there will be generalized tools for this, e.g. a search index of the best ICL examples for a wide variety of DSLs. Until then, my guess is that this sort of tool is particularly valuable.

The second tool is run_systems_model which passes the DSL (and an optional parameter for number of rounds) to the tool and then returns the result.

The image shows an interactive chart and data selection interface for a social network user acquisition model, allowing the user to select and display various metrics such as user types and funnel stages. The chart visualizes data over simulation rounds, indicating trends among different user segments like AwareUsers, CasualUsers, PowerUsers, and EngagedUsers.

I experimented with interface design here, initially trying to return a rendered chart of the results, but ultimately even multi-modal models are just much better at working with text than with images. This meant that I had the best results returning JSON of the results and then having the LLM build a tool for interacting with the results.

Altogether, a fun little experiment, and another confirmation in my mind that the most interesting part of designing MCPs today is deciding where to introduce and eliminate complexity from the LLM. Introduce too little and the tool lacks power; eliminate too little and the combination rarely works.