Goals and baselines.

metrics (6), management (208)

There is a moment in every company’s growth when top-level planning shifts from discussing specific projects to talking about goals. This happens recursively across each scope of leadership, as areas of accountability become too broad or complex for their leaders to consistently understand every project’s details.

This can be a very empowering moment because goals decouple the “what” from the “how”, but it can also be a confusing transition for everyone involved: writing clear goals takes a bit of practice.

Defining goals

Bad goals are indistinguishable from numbers. “Our p50 build time will be below 2 seconds” or “We’ll finish 8 large projects.” You’ll know a goal is just a number when you read it and aren’t sure if it’s ambitious or whether it matters.

Good goals are a composition of four specific kinds of numbers:

  1. A target states where you want to reach,
  2. A baseline identifies where you are today,
  3. A trend describes the current velocity,
  4. A timeframe sets bounds for the change.

Putting these all together, and a well structured goal takes the form of: “In Q3, we will reduce time to render our frontpage from 600ms (p95) to 300ms (p95). In Q2, render time increased from 500ms to 600ms.”

The two tests of an effective goal are whether someone who doesn’t know much about an area can get a feel for a goal’s degree of difficulty, and whether afterwards they can evaluate if it was successfully achieved. If you define all four aspects, typically will fulfill both criteria.

Investments and baselines

There are two particularly interesting kinds of goals: investments and baselines. Investments describe a future state that you want to reach, and baselines describe aspects of the present you want to preserve.

Imagine you wanted to speed up your data pipeline, your goal might be “Core batch jobs should finish within 3 hours (p95) by the end of Q3. They currently take 6 hours (p95), and over the course of Q2 they got 2 hours slower.” This is a well structured goal, but it’s also incomplete because you could likely reach that goal tomorrow by doubling the size of your cluster, which is probably not a desirable outcome.

The best way to avoid such unintended outcomes is to pair your investment goals with baseline metrics, sometimes referred to as countervailing metrics. For the data pipeline example, a few of the baseline metrics might be:

  • Efficiency of running core batch jobs should not exceed current price of $0.05 per GB.
  • Core batch jobs should not increase alert load on teams operating or using the pipeline, currently alerting twice per week.

Baseline metrics are useful for narrowing the solution space you explore in order to accomplish your investment goals. They are also useful for identifying when you should pause pursuing your goals and instead invest into platform quality. For example, if you were making excellent progress towards launching a new feature, but site stability has regressed below your baselines, this framework provides a structure to trigger rebalancing your priorities.

Although your baselines will often be about preserving a current property, you can also decide to accept some degradation before you want to trigger reprioritization. Perhaps you’re okay with costs increasing by 10% as long as your investment goals are accomplished. This kind of upfront clarity around tradeoffs is can be quite powerful.

Plans and contracts

The most common way to use goals is during a planning process. By agreeing on the mix of investment and baseline goals for each team, you’re able to agree on clear expectations with a team while still giving them full ownership of how they’ll satisfy the constraints. I’ve found that you should specify as few investment goals as possible, maybe three, and those should be the focus of planning discussions.

You’ll probably want to more baseline goals than investment goals, but it’s easiest to separate them out to avoid bogging down the conversation. Ideally, baselines are carried over across planning periods, and such that they frame the investment goals but don’t require too much active discussion during any given planning cycle.

Potentially one exception is when you’re using a baseline as a contract with a second party, probably specifying an SLO, at which point you’ll probably want to discuss it more explicitly than other baselines: missing an SLO will probably require immediate reprioritization, whereas missing most other baselines can generally be addressed more methodically.

From OKRs onward, there are dozens of different approaches to setting metrics, but I’ve found this format to be a useful, lightweight structure to start from. If folks have found other approaches easier or more useful, I’d love to hear from you!