Some delightful developer experiences in 2019.

apis (1), product (6)

I once worked at a company that built most of their functionality on top of Facebook’s advertising APIs. GraphQL was not publically a thing at that point, but the API design was more or less equivalent to GraphQL. Properties would appear and disappear without warning, and reacting to changes required frequent fire drills.

One conclusion might be that they didn’t care much about the experience of integrating with their APIs, but I’m pretty sure another guess is closer to the truth: it’s extremely difficult to support integrations into complex and evolving systems, and Facebook Ads very much met those criteria.

This past year has in many ways been the debut party for GraphqQL. While many of its ideas represent a generational improvement from HATEOAS, they’re not particularly new.

So what is new?

I wanted to know what developer experiences had been the most rewarding or for folks over the past year, and tweeted for folks more interesting developer experiences. From those responses, plus some browsing through Product Hunt and my own memory, here are some reflections on delightful developer experiences in 2019.

Code

An increasingly common paradigm is allowing folks to write code that runs directly on the platform. This provides a powerful, dynamic interface, and surprisingly reduces complexity in many cases.

Platforms can abstract away the complexity of deployment, managing operating systems, etc, and let their users focus exclusively on the control logic. All particulars, no glue. This is especially prominent for scenarios where decision-making is relatively stateless, although stateful examples are becoming less rare.

Some good examples of code as interface are:

  • “Serverless computing” continues to be a dominant theme, with a feature race emerging among the major cloud providers across Alibaba Cloud Function Compute, AWS Lambda, Microsoft Functions, and Google Cloud Functions. Language support continues to expand, with Java, Python, NodeJS being common place, and differentiation in the long tail. For example, Alibaba supports PHP and AWS supports Go.

    What’s maybe more interesting is seeing more niche products enter the serverless ecosystem, either trying to fill in usability gaps or specializing to support narrower usecases:

    Zeit Now promises selective build and deployment of only modified code paths (similar to what you might self-roll with Bazel).

    Firebase has specialized on supporting mobile development, providing a complete platform for mobile applications (mostly by specializing or extending existing Google Cloud functionality).

    Twilio has traditionally relied on their HTTP API, but in 2017 introduced Twilio Functions, which allow users to react to Twilio’s events running code on Twilio’s servers rather than their own.

  • WebAssembly on Cloudflare Workers allows folks to write WebAssembly, sometimes abbreviated as Wasm, and execute it on Cloudflare’s edge compute infrastructure.

    Wasm is a mediocre product but an exceptional platform layer: I think long-term folks won’t want to write Wasm directly, and will instead write more familiar languages and compile down to Wasm. That approach is being explored in Fastly’s Terrarium, which I write about a bit more later on.

  • AWS Lambda@Edge is in similar vein to Cloudflare Workers or Fastly’s Terrarium, but they’ve reached it from a different direction. For Fastly, the move into Wasm is increased flexibility compared to their existing VCL based configuration. However, Lambda@Edge only supports Node.js, which is much less featureful than AWS Lambda which [supports Java, Node.js, C# and Python.

    In some ways it’s unfair to compare AWS Lambda with Lambda@Edge, as it’s pretty clear the implementations are distinct and the similarity is mostly a marketing concern, but either way it’s interesting to see how physical and efficiency constraints create product limitations and specialization in “code as configuration” offerings.

  • Chrome Extensions are an interesting, different slant on integration through code, and an another aspect of what “edge computing” might mean in the future. Here you write JavaScript applications which users of the Chrome browser can install from the Chrome web store, running on a cloud of internet browers.

Writing code is such a powerful platform interface because it is a fairly unique combination of extreme expressivity while retaining extreme control at the platform tier. However, it’s worth pointing out that these offerings are largely Infrastructure-as-a-Service, which can assume a significant degree of technical expertise on behalf of their users. Will we see “code as integration” mechanism expand further beyond IaaS?

Containers

Containers such as Docker are an interesting choice of interface. They allow extraordinary customization and configuration, while often choosing to restrict or eliminate the statefulness offered by running a complete virtual machine. However, container security remains a pressing concern in a multi-tenant environment, and containers often provide an inferior interface between platform and user, forcing users to address operating system upgrades and platforms to appease multi-gigabyte images.

That said, I found one particularly interesting example of using containers as an integration point that I think is digging into, which is Azure IoT Edge.

Azure IoT Edge allows you to create and specify Docker containers, that process data on your physical internet-of-things devices, offloading some processing from the cloud to your cloud of physical devices. Containers represent an even higher abstraction than code, as you can control an entire virtual machine underneath. Containers represent some potential security risk in a shared computation environment, but that doesn’t apply in the case of running on your IoT devices, so this usecase is a clever increase in flexibility without many downsides.

I also generally think this is an interesting product because it introduces the idea that “edge” is a broad concept, with both physical devices and CDNs representing different facets of edge computation.

All of that said, I do believe containers represent an effective internal interface within companies as they provide an interesting composable interface in terms of the complexity and flexibility tradeoff, but perhaps not working well externally.

Domain Specific Languages

Using domain specific languages to specify integration behavior is a bit of cross between configuration-driven integrations and code-driven integrations. Some of these examples use DSLs as an artifact of their initial implementation (similar to AWS Redshift using the Postgres protocol because the early versions were implement using Postgres), and others use them to improve correctness.

  • Github Actions allow you to write commands in a simple DSL to configure your build, test and deploy workflows:

      action "Deploy to Production" {
    needs = "Provision Database"
    uses = "actions/gcloud"
    runs = "gcloud deploy"
  }

    This is a nice approach, because it describes how the actions relate in their DSL, but they don’t try to describe the actions themselves. Instead the actions are described as scripts or shell commands. I could see Github Actions replacing large swathes of continuous integration tooling, and it’s done so in a very flexible way that will allow for users to innovate on top of their platform, not trying to define all the ways that someone might want to integrate.

  • Fastly’s use of Varnish Configuration Language, sometimes abbreviated as VCL, is another interesting example of a domain specific language. It’s a pretty fascinating choice, because VCL is an extremely powerful language that is intentionally constrained to what is possible to do in a high performance web server, but also a rather dense syntax.

      sub pipe_if_local {
    if (client.ip ~ local) {
      return (pipe);
    }
  }

    In this specific case, I think VCL is probably too complex a DSL for wide-spread adoption, and I suspect that Fastly agrees based on their experimentation with Terrarium, which is compiling Rust, C and TypeScript to WebAsembly and running it at the edge.

  • Terraform is pretty fascinating as a DSL written by HashiCorp to represent integrations with third party cloud providers like Alibaba, AWS, Azure or GCP.

      resource "aws_elb" "frontend" {
    name = "frontend-load-balancer"
    listener {
      instance_port     = 8000
      instance_protocol = "http"
      lb_port           = 80
      lb_protocol       = "http"
    }
    instances = ["${aws_instance.app.*.id}"]
  }

    The entire idea of your product being a DSL on top of other products is pretty fascinating, and a powerful testament to how important effective interfaces can be. In this case, I think Terraform’s biggest value propositions are in abstraction (sort of theoretically decoupling from vendor specific configuration and making it easy to support multi-cloud, although in practice this is a bit tenuous) and verifiabiility (much better tools to validate correctness than in e.g. YaML or JSON).

IDEs and development environments

Several examples were of powerful IDEs and development environments. Some of these are very focused specific tools, others are development platforms, and others fall somewhere inbetween.

  • Glitch which will run your entire frontend and backend Javascript application, providing an entire online IDE, development and deployment environment. Some of their existing examples use Airtable’s API, Slack’s API and Google Sheets’ API. This is a powerful showcase of just how good Javascript sandboxing has gotten from security and performance isolation perspectives.

  • VSCode is a free, open source IDE from Microsoft that has been getting widespread adoption as a light, configurable and powerful IDE. Beyond being free, it does a lot of interesting things well: debuggers, smart completion, Git integration, extensions, etc.

    Potentially the thing is does best, though, is the Language Server Protocol, which abstracts language support from IDE particulars, allowing one LSP integration to support a wide range of IDEs. This is a very powerful approach to encourage adoption of VSCode, but also will lower the entry barriers for future IDEs as well. These sorts of platforms allow tools to compete on quality, and make it feasible to support hyper-specialized tools for particular workflows. Very excited to see more products that are viral vectors for platforms that can be used by the rest of us.

  • Google Cloud Shell is a pretty powerful idea, allowing folks to interact with and control their Google Cloud environment with a dedicated VM for each individual. This is pretty crafty, as it allows Google to upgrade the clients automatically on their images (reducing backwards compatability overhead), and also provides better security and auditability primitives.

    From a user perspective, I also love that it prevents the proliferation of the much-dreaded “shared management server” anti-pattern, where folks do a bunch of critical work off a single, shared server that is forever running out of disk space, getting its CPU pegged on a bad script, or causing an outage when it goes down.

  • Hyper is a HTML/CSS based terminal application, that allows heavy customization while using familar web-technologies. This opens the door to some neat ideas around embedding terminals into tools for powerful IDEs, without having to (directly) rely on native technologies to do so.

  • Merlin is an editor service for the OCaml programming language that supports autocomplete and such. By default it supports Emacs and Vim, but has also been integrated with other editors such as… VSCode. I think this example is interesting in three different ways. First, it’s a reminder of how we can draw inspiration from many places. Second, it’s a testament to how well design Emacs and Vim are to remain heavily used and actively extended so many years after their first development. An inspiration! Finally, it’s a testament to what a great idea the Language Server Protocol is, because the future it’ll enable will allow tools like this to integrate cleanly with any editor.

  • Pharo is a Smalltalk implementation which combines the operating system and IDE into a single bundle. Pharo expects far more from us than most tools–to give up our entire operating system–and I think that’s part of what makes it so interesting. Many of the barriers we create for tools are social constructs, we have the technology to do much more than we typically do in our tools.

Progressive configuration

In practice, very few developer-centric tools provide only a single vehicle for integration, but instead provide a range of integration capabilities from simple to rather complex.

Picture of Google Cloud Build’s configuration interface.

  • Clubhouse is another standard, modern example. Default configuration done through their UI, and more powerful customization done with a REST API and webhooks.

Programming without code

I won’t belabor this too much, but products like Airtable and If This Then That still hold a very special place in my heart. They are these spiritual successors to HyperCard, providing fairly simple tools that “non-programmers” can compose into deeply powerful programs.

These products slot into a void between standard GUI-driven integration and programmatic integration. The difference between these and GUI-driven integration is they aren’t particularly opinionated. They aren’t designed to solve a specific thing, but rather a tool that can solve many things by providing some composable, open-ended building blocks.

Libraries and tools

This is kind of a grab bag categories of tools that folks called out as offering particularly good developer excperiences, which maybe didn’t fit perfectly into other groups.

  • Flutter is a tool for building native apps on iOS and Android from a single codebase. Similar in some aspects to React Native but seemingly to better results.
  • pytest is a Python test runner that simplifies test running and in particular test writing, relative to the standard library’s Unittest.
  • Laravel and Laravel Spark are powerful PHP frameworks for developing web applications.
  • Gatsbyjs bundle together the full modern frontend stack into something with all the power and a more graduated learning curve.
  • elm is a language for writing reliable webapps, with powerful type inference in the vein of Haskell.
  • React Hooks are a simpler way to deal with state changes in React.
  • Sourcegraph is a powerful code search tool, that makes it easy to search across large code repositories.
  • OASGraph is a tool to transpate OpenAPI Specifications into GraphQL APIs, which lowers the barrier to start running an GraphQL API.

Things that weren’t mentioned

Given the endless hype, I thought it was interesting to just briefly mention a few things that that no one brought up:

  • Blockhain in general or Ethereum in particular as examples of great developer experiences.

    Depending on usecase, it seems like either the wallets are still difficult to use securely, or the transaction costs remain prohibitively high or volatile. Together, it seems like the developer experience for blockchain is still lacking.

  • Chat bots didn’t come up either. despite getting a lot of excitement. I think this might be in large part due to US and Europe focus of the folks I know on Twitter, as it seems like chat bots are getting a great deal of use in other markets.

  • React wasn’t brought up as much as I expected, although React Hooks were mentioned. Perhaps React is so commonplace at this point that folks don’t even think to mention it, much as no one mentioned Python, Ruby or Node.js.

Altogether, this was quite an interesting way to spend some time! I was unfamiliar with many of these tools before today, and it was a good survey to help refresh my thinking about where developer tooling and great developer experiences are headed.

Very hopeful to hear folks suggestions for other impressive and inspiring developer experiences they’ve had recently!