Thesis on engineering onboarding products.

As I’ve started doing a bit more angel investing, I’ve ended up chatting with a handful of folks working on the engineering boarding space, and here is my collection of thoughts on the space.

Engineering onboarding is the process by which a recently hired engineer becomes a proficient contributor to your company. As companies scale, they tend to devote an increasingly large amount of energy into this space. Pulling a few data points from my own experience: Uber sent every new engineer through a week of “engucation” to onboard them, Stripe had new engineers form short-term teams that spent two-to-four weeks building and shipping a project together. Most other companies do something similar, although always tweaked to their culture and practices (e.g. Facebook’s bootcamp model).

Engineering onboarding is an interesting problem for a few reasons:

• Quality of onboarding greatly impacts whether you benefit from rapid hiring, and many companies believe that engineering is their constraint on product delivery. (A belief that I often disagree with for the reasons Shreyas describes.)
• Engineering headcount is a large item in most companies budgets, often the highest departmental headcount cost, with overall headcount as one of the top two to three costs. This means even a marginal improvement has significant value to would-be buyers
• Very few companies have a team or individuals responsible for engineering onboarding, although increasing are seeing some “Engineering Operations” roles with a focus on this problem. Historically this problem was either unowned or owned by a “developer productivity” team whose expertise is primarily in tooling rather than training
• Customer experience teams have clear efficiency metrics to evaluate onboarding quality against, but engineering teams firmly disagree whether it’s possible to measure engineering impact or engineering efficiency. A number of recent companies attempt to create measurable engineering efficiency scores, but I think they’re focused on measuring efficiency rather than effective behavior (Slack is a great book on why efficient and effective are meaningfully different). The industry has started to standardize on the DORA metrics (e.g. those covered in Accelerate), but those are input metrics rather than output metrics (e.g. you can do well on them and still have bad efficiency) so they’re not particularly helpful for evaluating onboarding quality, e.g. they are advantageous to but not guarantees of effective onboarding
• The Pandemic-drven rapid switch to remote work over course of 2020 and 2021 has put even more pressure on onboarding as many previous strategies (e.g. “shoulder tapping”) aren’t working as well
• Many companies are finding it relatively easy to attain high company valuations at early stages, providing the capital to start rapidly growing their engineering team at earlier phases than in the past

Despite the significant potential value in this space and trends supporting investment, there are relatively few tools doing well in this space. Some of the core challenges are: \

• The would-be buyer is typically the head of engineering who is not directly impacted by bad onboarding, because executive onboarding is always very custom and they’re able to quickly get help from anyone. This leads to limited intuitive connection to the problem
• The would-be user, e.g. newly hired engineers at the company, have little influence at the company to drive tool adoption, and typically direct that influence towards other problems that they understand better (day to day practices like code review, tool selection, etc)
• Even if buyer agrees with the problem statement, they’ve rarely experienced effective onboarding and may not believe there’s much opportunity to impact this problem, even if it’s going poorly
• Even if buyer theoretically believes the problem is solvable, the lack of agreed upon measurement for engineering impact (and the bespokeness of engineering impact measures that companies adopt in practice) makes it difficult to show value even if your offering is quite good
• Many approaches require an internal operator to use effectively, which leads many companies to prioritize hiring the “engineering operations” role first, and then rely on those hires to select tools (if any). However, the folks hired to solve the problem are rarely aware of these tools and consequently they don’t advocate for their adoption
• This sort of tooling potentially requires integration with important data (e.g. HRIS data) which brings security, legal and compliance issues into play for both the buyer and the seller

Reflecting on those challenges, the most valuable questions I’d encourage folks working in this space to answer are:

• Who is the buyer, and what budget are they pulling from?
• What is your security and compliance strategy?
• What is your go-to-market strategy to convince would-be buyers that it is possible to improve engineering onboarding?
• What is your go-to-market strategy to create conviction that your product can improve engineering onboarding?
• How do you avoid getting sequenced behind hiring “engineering operations” role? Conversely, how do you lean into getting sequenced behind hiring someone into the engineering operations role?
• How does your product show value when no one agrees on measuring engineering efficiency? (If you use a synthetic metric to evaluate efficiency, how do you avoid measuring something harmful and how do you ensure it’s widely applicable? If you use surveys, how do you go beyond NPS surveys that inevitably show a one-time improvement but are difficult to tie to impact?)
• If you rely on consulting-heavy approaches to solve any of the above problems, how will you scale revenue at a reasonable margin?
• If you don’t rely on a consulting-heavy approach or intend to pivot out of the consulting-heavy approach quickly, how will you manage quality without humans in the loop?

If you have convincing answers to those problems, then I think most of the other problems are probably pretty solvable in comparison (e.g. driving up margin once you’re there seems pretty straightforward: attach more higher margin offerings once you’re in the door).

I’ve probably done just enough angel investing that I should really figure out a more structured template for thinking about investment verticals, and hopefully will figure out a template over the next bit.