Irrational Exuberance

Notes on how to use LLMs in your product.

Mon, 08 Apr 2024 09:00:00 -0700

Pretty much every company I know is looking for a way to benefit from Large Language Models. Even if their executives don’t see much applicability, their investors likely do, so they’re staring at the blank page nervously trying to come up with an idea. It’s straightforward to make an argument for LLMs improving internal efficiency somehow, but it’s much harder to describe a believable way that LLMs will make your product more useful to your customers.

I’ve been working fairly directly on meaningful applicability of LLMs to existing products for the last year, and wanted to type up some semi-disorganized notes. These notes are in no particular order, with an intended audience of industry folks building products.

Rebuild your mental model

Many folks in the industry are still building their mental model for LLMs, which leads to many reasoning errors about what LLMs can do and how we should use them. Two unhelpful mental models I see many folks have regarding LLMs are:

LLMs are magic: anything that a human can do, an LLM can probably do roughly as well and vastly faster
LLMs are the same as reinforcement learning: current issues with hallucinations and accuracy are caused by small datasets. Accuracy problems will be solved with larger training sets, and we can rely on confidence scores to reduce the impact of inaccuracies

These are both wrong in different but important ways. To avoid falling into those mental model’s fallacies, I’d instead suggest these pillars for a useful mental model around LLMs:

LLMs can predict reasonable responses to any prompt – an LLM will confidently provide a response to any textual prompt you write, and will increasingly provide a response to text plus other forms of media like image or video
You cannot know whether a given response is accurate – LLMs generate unexpected results, called hallucinations, and you cannot concretely know when they are wrong. There are no confidence scores generated that help you reason about a specific answer from an LLM
You can estimate accuracy for a model and a given set of prompts using evals – You can use evals – running an LLM against a known set of prompts, recording the responses, and evaluating those responses – to evaluate the likelihood that an LLM will perform well in a given scenario
You can generally increase accuracy by using a larger model, but it’ll cost more and have higher latency – for example, GPT 4 is a larger model than GPT 3.5, and generally provides higher quality responses. However, it’s meaningfully more expensive (~20x more expensive), and meaningfully slower (2-5x slower). However, the quality, cost and latency are improving at every price point. You should expect the year-over-year performance at a given cost, latency or quality point to meaningfully improve over the next five years (e.g. you should expect to get GPT 4 quality at the price and latency of GPT 3.5 in 12-24 months)
Models generally get more accurate as the corpus it’s built from grows in size – the accuracy of reinforcement learning tends to grow predictability as the dataset grows. That remains generally true for LLMs, but is less predictable. Small models generally underperform large models. Large models generally outperform small models with higher quality data. Supplementing large general models with specific data is called “fine-tuning” and it’s currently ambiguous when fine-tuning a smaller model will outperform using a larger model. All you can really do is run evals based on the available models and fine-tuning datasets for your specific usecase
Even the fastest LLMs are not that fast – even a fast LLM might take 10+ seconds to provide a reasonably sized response. If you need to perform multiple iterations to refine the initial response, or to use a larger model, it might take a minute or two to complete. These will get faster, but they aren’t fast today
Even the most expensive LLMs are not that expensive for B2B usage. Even the cheapest LLM is not that cheap for Consumer usage – because pricing is driven by usage volume, this is a technology that’s very easy to justify for B2B businesses with smaller, paying usage. Conversely, it’s very challenging to figure out how you’re going to pay for significant LLM usage in a Consumer business without the risk of significantly shrinking your margin

These aren’t perfect, but hopefully they provide a good foundation for reasoning about what will or won’t work when it comes to applying LLMs to your product. With this foundation in place, now it’s time to dig into some more specific subtopics.

Revamp workflows

The workflows in most modern software are not designed to maximize benefit from LLMs. This is hardly surprising–they were built before LLMs became common–but it does require some rethinking about workflow design.

To illustrate this point, let’s think of software for a mortgage provider:

User creates an account
Product asks user to fill in a bunch of data to understand the sort of mortgage user wants and user’s eligibility for such a mortgage
Product asks user to provide paperwork to support the data user just provided, perhaps some recent paychecks, bank account balances, and so on
Internal team validates the user’s data against the user’s paperwork

In that workflow, LLMs can still provide significant value to the business, as you could increase efficiency of validating the paperwork matching with the user supplied information, but the user themselves won’t see much benefit other than perhaps faster validation of their application.

However, you can adjust the workflows to make them more valuable:

User creates an account
Product asks user to provide paperwork
Product uses LLM to extract values from paperwork
User validates the extracted data is correct, providing some adjustments
Internal team reviews the user’s adjustments, along with any high risk issues raised by a rule engine of some sort

The technical complexity of these two products is functionally equivalent, but the user experience is radically different. The internal team experience is improved as well. My belief is that many existing products will find they can only significantly benefit their user experience from LLMs by rethinking their workflows.

Retrieval Augmented Generation (RAG)

Models have a maximum “token window” of text that they’ll consider in a given prompt. The maximum size of token windows are expanding rapidly, but larger token windows are slower to evaluate and cost more to evaluate, so even the expanding token windows don’t solve the entire problem.

One solution to navigate large datasets within a fixed token window is Retrieval Augmented Generation (RAG). To come up with a concrete example, you might want to create a dating app that matches individuals based on their free-form answer to the question, “What is your relationship with books, tv shows, movies and music, and how has it changed over time?” No token window is large enough to include every user’s response from the dating app’s database into the LLM prompt, but you could find twenty plausible matching users by filtering on location, and then include those twenty users’ free-form answers, and match amongst them.

This makes a lot of sense, and the two phase combination of an unsophisticated algorithm to get plausible components of a response along with an LLM to filter through and package the plausible responses into an actual response works pretty well.

Where I see folks get into trouble is trying to treat RAG as a solution to a search problem, as opposed to recognizing that RAG requires useful search as part of its implementation. An effective approach to RAG depends on a high-quality retrieval and filtering mechanism to work well at a non-trivial scale. For example, with a high-level view of RAG, some folks might think they can replace their search technology (e.g. Elasticsearch) with RAG, but that’s only true if your dataset is very small and you can tolerate much higher response latencies.

The challenge, from my perspective, is that most corner-cutting solutions look like they’re working on small datasets while letting you pretend that things like search relevance don’t matter, while in reality relevance significantly impacts quality of responses when you move beyond prototyping (whether they’re literally search relevance or are better tuned SQL queries to retrieve more appropriate rows). This creates a false expectation of how the prototype will translate into a production capability, with all the predictable consequences: underestimating timelines, poor production behavior/performance, etc.

Rate of innovation

Model performance, essentially the quality of response for a given budget in either dollars or milliseconds, is going to continue to improve, but it’s not going to continue improving at this rate absent significant technology breakthroughs in the creation or processing of LLMs. I’d expect those breakthroughs to happen, but to happen less frequently after the first several years, and slow from there. It’s hard to determine where we are in that cycle because there’s still an extraordinary amount of capital flowing into this space.

In addition to technical breakthroughs, the other aspect driving innovation is building increasingly large models. It’s unclear if today’s limiting factor for model size is availability of Nvidia GPUs, larger datasets to train models upon that are plausibly legal, capital to train new models, or financial models suggesting that the discounted future cashflow from training larger models doesn’t meet a reasonable payback period. My assumption is that all of these have or will be the limiting constraint on LLM innovation over time, and various competitors will be best suited to make progress depending on which constraint is most relevant. (Lots of fascinating albeit fringe scenarios to contemplate here, e.g. imagine a scenario where the US government disbands copyright laws to allow training on larger datasets because it fears losing the LLM training race to countries that don’t respect US copyright laws.)

It’s safe to assume model performance will continue to improve. It’s likely true that performance will significantly improve over the next several years. I find it relatively unlikely to assume that we’ll see a Moore’s Law scenario where LLMs continue to radically improve for several decades, but lots of things could easily prove me wrong. For example, at some point nuclear fusion is going to become mainstream and radically change how we think about energy utilization in ways that will truly rewrite the world’s structure, and LLM training costs could be one part of that.

Human-in-the-Loop (HITL)

Because you cannot rely on LLMs to provide correct responses, and you cannot generate a confidence score for any given response, you have to either accept potential inaccuracies (which makes sense in many cases, humans are wrong sometimes too) or keep a Human-in-the-Loop (HITL) to validate the response.

As discussed in the workflow section, many companies already have humans performing validation work who can now move into supervision of LLM responses rather than generating the responses themselves. In other scenarios, it’s possible to adjust your product’s workflows to rely on external users to serve as the HITL instead. I suspect most products will depend on both techniques along with heuristics to determine when internal review is necessary.

Hallucinations and legal liability

As mentioned before, LLMs often generate confidently wrong responses. HITL is the design principle to prevent acting on confidently wrong responses. This is because it shifts responsibility (specifically, legal liability) away from the LLM itself and to the specific human. For example, if you use Github Copilot to generate some code that causes a security breach, you are responsible for that security breach, not Github Copilot. Every large-scale adoption of LLMs today is being done in a mode where it shifts responsibility for the responses to a participating human.

Many early-stage entrepreneurs are dreaming of a world with a very different loop where LLMs are relied upon without a HITL, but I think that will only be true for scenarios where it’s possible to shift legal liability (e.g. Github Copilot example) or there’s no legal liability to begin with (e.g. generating a funny poem based on their profile picture).

“Zero to one” versus “One to N”

There’s a strong desire for a world where LLMs replace software engineers, or where software engineers move into a supervisory role rather than writing software. For example, an entrepreneur wants to build a copy of Reddit, and uses an LLM to implement that implementation. There’s enough evidence that you can assume it’s possible today to go from zero to one on a new product idea in a few weeks with an LLM and some debugging skills.

However, most entrepreneurs lack a deep intuition on operating and evolving software with a meaningful number of users. Some examples:

Keeping users engaged after changing the UI requires active, deliberate work
Ensuring user data is secure and meets various privacy compliance obligations
Providing controls to meet SOC2 and providing auditable evidence of maintaining those controls
Migrating a database schema with customer data in it to support a new set of columns
Ratcheting down query patterns to a specific set of allowed patterns that perform effectively at higher scale

All of these are straightforward, basic components of scaling a product (e.g. going from “one to N”) that an LLM is simply not going to perform effectively at, and where I am skeptical that we’ll ever see a particularly reliable LLM-based replacement for skilled, human intelligence. It will be interesting to watch, though, as we see how far folks try to push the boundaries of what LLM-based automation can do to delay the onset of projects needing to hire expertise.

Copyright law

Copyright implications are very unclear today, and will remain unclear for the foreseeable future. All work done today using LLMs has to account for divergent legal outcomes. My best guess is that we will see an era of legal balkanization regarding whether LLM generated content is copyright-able, and longer-term that LLMs will be viewed the same as any other basic technical component, e.g. running a spell checker doesn’t revoke your copyright on the spell checked document. You can make all sorts of good arguments why this perspective isn’t fair to copyright holders whose data was trained on, but long-term I just don’t think any other interpretation is workable.

Data Processing Agreements

One small but fascinating reality of working with LLMs today is that many customers are sensitive to the LLM providers (OpenAI, Anthropic, etc) because these providers are relatively new companies building relatively new things with little legal precedent to derisk them. This means adding them to your Data Processing Agreement (DPA) can create some friction. The most obvious way around that friction is relying on LLM functionality served via your existing cloud vendor (AWS, Azure, GCP, etc).

Provider availability

I used to think this was very important, but my sense is that LLM hosting is already essentially equivalent to other cloud services (e.g. you can get Anthropic via AWS or OpenAI via Azure), and that very few companies will benefit from spending too much time worrying about LLM availability. I do think that getting direct access to LLMs via cloud providers–companies that are well-versed at scalability–is likely the winning pick here as well.

There’s lots of folks out there who have spent more time thinking deeply about LLMs than I have–e.g. go read some Simon Willison–but hopefully the notes here are useful. Curious to discuss if folks disagree with any of these perspectives.

Ex-technology companies.

Fri, 22 Mar 2024 04:00:00 -0700

One of the most interesting questions I got after joining Calm in 2020 was whether Calm was a technology company. Most interestingly, this question wasn’t coming from friends or random strangers on the internet, it was coming from the engineers working there! In an attempt to answer those questions, I wrote up some notes, which summarize two perspectives on “being a technology company.”

The first perspective is Ben Thompson’s “Software has zero marginal costs.” You’re a technology company if adding your next user doesn’t create more costs to support that user. Yes, it’s not really zero, e.g. Stripe has some additional human overhead for managing fraud for each incremental user it adds, but it’s a sufficiently low coefficient that it’s effectively zero. This is the investor perspective, and matters predominantly to companies because it will change how their valuation is calculated, which in turn plays a significant role in investor, founder, and employee compensation.

If a company is a technology company in a “good” vertical, then the valuation might be 7-10x revenue. If it’s not a technology company, the valuation might be 2-5x revenue. The rationale behind this difference is that a technology company should be able to push its gross margin to 70+% as it matures, which will drive significantly higher cash flow, and most valuations are anchored in discounted future cash flow. This also means that if you’re perceived as a technology company one year, and then not perceived as a technology company a few years later, your company’s valuation plummets.

The second perspective on being a technology company is captured by Camille Fournier, “A company where engineering has a seat at the table for strategic discussions.” This is the employee perspective regarding how it feels to work within a company. If engineering has a meaningful influence in how the company makes decisions, then doing engineering work at that company will generally be a rewarding experience. If they don’t have much influence, then they’ll generally be treated as a cost center.

The recent trend that I want to apply these definitions to is that I see a number of companies losing their “technology company status.” These fallen technology companies are creating a new striation of company, whose employees and investors think of themselves as being in a technology company, but where the company itself is no longer able to effectively provide that experience to employees, or valuation to investors.

Companies are falling out of technology company status for a few reasons:

Their zero marginal costs were always aspirational constructs to attract investors. They can no longer find investors who believe in their dream, and they are not able to reach those zero marginal costs with their remaining cash reserves
They no longer believe they can change the business’ outcomes through R&D efforts, and as a result they shouldn’t include engineering as a major stakeholder in business decisions. I recently chatted with a data science leader who described their company reaching this state. They couldn’t show any business impact from the past two years of their product releases, so the finance team identified a surefire way for R&D to make a business impact: laying off much of the R&D team.

This is more extreme than the typical example of companies which have “overhired.” Those companies believe they have impactful R&D work to do, it’s simply that they have more capacity than high quality projects. These companies cannot identify non-maintenance R&D work that will make their business more valuable

The experience of working in this new striation of ex-technology company shares some of the ideas developed in Jean Yang’s Building Observability for 99% Developers: many of the latest ideas and trends are no longer appropriate or perhaps even affordable. However, compared to the 99% developer experience, this striation of ex-technology companies is in an even harder spot: they grew up, and established their R&D cost structures, believing they were technology companies. Now they have poor financial fundamentals and have downsized the R&D teams previously intended to build their way out of that predicament.

Organizations that spun up dedicated in-house build and deployment infrastructure have fixed engineering costs to maintain that infrastructure, and the math that convinced executives earlier–some sort of argument about the multiplicative effect on the impact of other engineers–doesn’t make as much sense anymore. But often you can’t just stop maintaining those investments, because that would require slowing down to decommission the existing systems, and ex-technology companies have little capacity for maintenance. Instead they’re focused on survival or roleplaying the motions of rapid product development despite actually spending the large majority of their time on maintenance.

This was the exact challenge we encountered after Digg’s layoffs: our architecture and processes had been designed for a team of ~50 engineers and now a team of ~10 had to operate it. Our service architecture of seven services made a great deal of sense for seven engineering teams, but was a lot less convenient for one engineering team with little redundancy in skillset.

If you’re trying to determine whether you’re in this ex-technology striation, the question I’d encourage you to ask yourself is whether R&D at your company can significantly change your company’s financial profile over the next three years. If the answer is yes, then I don’t think you’re a member. Even if you’re in a dark moment–and many people are in 2024–as long as you see a path for R&D to change your company’s financial fundamentals, stay hopeful.

On the other hand, if you simply don’t see a path to changing the underlying financials, then you probably have joined the new striation of ex-technology companies. This might be because your business never made sense as a technology company to begin with. Or it might be because your R&D operating structure is simply designed for a larger team than you’ll ever have again, and the work to solve that problem is uninvestable in your new circumstances.

The beautiful thing about our industry is that it’s a dynamic, living thing. We’re in a challenging pocket, but the good times are never too far around the corner either.

Leadership requires taking some risk.

Sun, 17 Mar 2024 05:00:00 -0700

At a recent offsite with Carta’s Navigators, we landed on an interesting topic: leadership roles sometimes mean that making progress on a professional initiative requires taking some personal risk.

This lesson was hammered into me a decade ago during my time at Uber, where I kicked off the Uber SRE group and architectured Uber’s self-service service provisioning strategy that defined Uber’s approach to software development (which spawned a thousand thought pieces, not all complimentary). I did both without top-down approval, and made damn sure they worked out. It wasn’t that I was an anarchist, or that I hated all authority, rather that I could have never gotten explicit approval for either approach. It was the sort of culture where occasionally you were told not to do something, but you were only rarely given explicit permission to do anything. My choice was to either point fingers about why I was stuck, or take on some personal risk to make forward progress.

I love making progress, and hate being stuck, so for me it was an easy decision: take ownership and move forward. There’s a reasonable argument to be made that I was a bit reckless here, but I believe that a surprising number of environments distribute leadership decisions exactly this way. Indeed, if you want a bottom-up decision-making environment, but feel that taking on personal risk is an unreasonable demand, then maybe you actually don’t want a bottom-up decision-making environment.

In these environments, decisions aren’t solely the purview of executives. As a staff engineer or a line manager, you’ll be celebrated if you succeed, lampooned if you fail, and you’ll have to own the risk directly rather than shifting the risk to senior leadership by getting their approval. In this mode of operation, senior leadership doesn’t provide direction on navigating demands, rather they provide demands to be satisfied (sometimes described as “context” because that’s a nicer sounding word than “demands”), and outsource solving the constraints to the team.

If you want to make choices, expect that you’re going to be accountable for outcomes.

When should you take risks?

This isn’t a recommendation to always take on personal risks. Sometimes that isn’t just ill-advised, it will put you in active conflict with your company. For example, if your company has a clearly articulated engineering strategy, then explicitly violating that strategy is unlikely to succeed. Further, the risk is magnified, because you’re not just filling in blank space, you’re undermining the organization’s explicit decisions. This is true even when the strategy isn’t explicitly documented, but is nonetheless widely recognized.

You should generally only take bottom-up decision-making risk in two scenarios:

It’s a blank space without an articulated or practiced strategy (e.g. rolling out a caching strategy at a company without any consistent approach to caching).

Creating the SRE organization at Uber fell into this bucket, as there simply wasn’t an existing point of view on whether to have such an organization
It’s an existential issue, where respecting the strategy is secondary to survival (e.g. solving a compliance problem that your company has irrationally decided to deprioritize).

Our switch to self-service service provisioning at Uber was in this bucket, as part of that strategy was deliberately slowing down support for manual provisioning while we built the new solution, and no one would have approved a slow down

If there’s a way to make progress without taking on personal risk, that’s your first option. Get approval from the decision-making body. Find an executive sponsor for the initiative. It’s only when you run out of “approved paths forward” that you should consider taking on the risk yourself. Depending on your company, you may find there are abundant opportunities for approval, or none at all.

Owning the risk

For a long time, I considered it an enduring irony that executives are rarely held accountable for their decisions. This seems unfair, but it’s also true that the typical executive holds a basket of risks, and some of them are going to come due even if they do an excellent job of managing the overall basket. When you take on a risk as a non-executive, your situation is a bit different. You probably own exactly one significant risk, and similarly to the pressure to ensure your “staff project” succeeds, every time you take on a personal risk, you need to ensure it’s a success.

When attempts to own risk fail, it usually comes down to two issues:

a lack of interest in user needs, generally because you’re anchored on the adoption of a particular approach or technology (e.g. we must use a serverless architecture)
It’s unclear if the approach is working for so long that it gets canceled from above before showing impact (e.g. you’re nine months into building a vastly superior service framework, but nothing important is able to migrate to it)

There are a handful of techniques to reduce risk:

Engineering strategy techniques: are useful even if no one will approve your strategy, because they force you to think through the constraints and challenges before jumping into the solution
Modeling techniques: like systems thinking or Wardley mapping (explained in Simon Wardley’s original book or The Value Flywheel Effect) will help you build conviction that both the problem is real and your solution is viable
Skunkwork prototyping: don’t take on the risk until you’ve validated your approach is viable
Effective migrations: iterate rapidly across usage cohorts to understand the full breadth of requirements before driving adoption numbers to ensure you don’t stall out in late stages
Validate across your network: derisk your approach by reaching out to peers at similar companies who’ve already solved the problem and understanding why your proposed approach did or did not work well for them
Engage an executive sponsor: convince an executive to care enough about the risk you’re taking on that they’re going to absorb it themselves. This usually requires a strong pre-existing relationship with the executive that you’ve built by listening to them and taking on problems that they’re trying to solve

If none of those are directly applicable, then at a minimum ensure that you’re anchored in the data about your users and have done the work to understand their needs.

Obfuscated capacity

As hinted at earlier, sometimes bottom-up leadership requires obfuscating the work being done, because it addresses an implied system problem rather than directly solving their current problem. Sometimes your approach will even make things worse short-term, which is an idea I touch on in the Trunk and Branches Model for Scaling Infrastructure Organizations. In that case, we had so many incoming requests that servicing them effectively would have consumed our entire bandwidth, and we created time to invest into systems work by degrading our response to short-term requests.

Overwhelmed teams generally turn to executive leadership to prioritize their incoming asks, but overwhelmed teams in a bottom-up decision-making environment will generally find that approach doesn’t work very well. Executives have become comfortable articulating demands, and will restate their demands, but are often not particularly good at solving for underlying constraints. The bottom-up team itself has to take the risk to solve their own constraints.

In most cases, that means that these teams develop some mechanism for hiding internal work that needs to be done but doesn’t directly solve an executive demand. They’ll all describe this somewhat differently, whether it’s “engineering-allocated story points”, mildly inflating the sizing of every project, preventing on-call engineers from being tasked with product work, a platform team that’s not included in roadmapping, or just a sufficiently messy planning process that an engineer or two’s efforts can be quietly redirected.

Long-term, teams retain the right to obfuscate capacity by delivering something useful with the capacity they previously obfuscated. If not, long-term that capacity is “detected” and recaptured by the encompassing organization. Most organizations are glad to ignore the details of your team’s allocation for a quarter, but very few will ignore your output for an entire year. If you obfuscate capacity without solving something meaningful with it, you’ll find that trust takes quite a long time to rebuild.

Leadership requires some risks

Taking direct, personal risk is a prerequisite to taking ownershsip of interesting problems that matter to your company. A risk-free existence isn’t a leadership role, regardless of whatever your title might be. Indeed, an uncomfortable belief of mine is that leadership is predicated on risk. The upside is that almost all meaningful personal and career growth is hidden behind the risk-taking door. There’s a lot of interesting lessons to learn out there, and while you can learn a lot from others, some of them you have to learn yourself.

Friction isn't velocity.

Fri, 15 Mar 2024 05:00:00 -0700

When you’re driving a car down a road, you might get a bit stuffy and decide to roll your windows down. The air will flow in, the wind will get louder, and the sensation of moving will intensify. Your engine will start working a bit harder–and louder–to maintain the same speed. Every sensation will tell you that you’re moving faster, but lowering the window has increased your car’s air resistance, and you’re actually going slower. Or at minimum you’re using more fuel to maintain the same speed.

There’s nothing that you didn’t already know in the first paragraph, but it remains the most common category of reasoning error that I see stressed executives make. If you’re not sure how to make progress, then emotionally it feels a lot better to substitute motion for lack of progress, but in practice you’re worse off.

Grounding this in a few examples:

Many companies realize that their monolithic codebase is slowing them down. It’s easy to decide to migrate from your monolith to services to “solve” this problem, but without a clear service architecture, most attempts take a long time without improving on the underlying issues. That’s because an effective service migration requires the same skill to operate an effective monolith: good technical design.

However, the microservice migration itself provides a reassuring sensation of progress, delaying for a year or two the realization that you’re in roughly the same place that you started in.
When your engineering organization doesn’t seem to be shipping enough software, an easy solution is to rollout a new development process. For example, you might say that an ineffective team needs to start following the scrum development technique.

In rare case that the team has never considered any approach to organize their work, then this might well help. In most cases, this will just paper over whatever problem is causing the slow down, creating an appearance of progress that’ll quickly fade away.
It’s common for new executives to rollout their preferrenced knowledge base, e.g. Notion or Confluence or whatnot, operating from the belief that the tool itself is the fundamental driver of an effective knowledge base.

This will create months of work to move to a new knowledge base, but generally does not improve the underlying knowledge being managed. Poorly managed knowledge bases are always related to incentives and culture, not checkbox ready feature lists like “effective search.”

The pattern here is generally an intuition-driven decision driven by a senior leader, unclear criteria for success, an orientation towards motion as an effective proxy for progress, and being too busy to reflect on whether prior actions accomplished their intended goals. This recipe passes as leadership, and does share some of the characteristics from leading from conviction, but is always an inferior tactic to another available option.

If you see someone following this tactic, it’s a genuine kindness to point it out to them. If they’re not interested in that feedback, you’ve learned something important: they’re more focused on the performance act of leadership than in the impact of their work.

To provide one caveat, in cases where you’re wholly stuck, then minimizing friction doesn’t matter so much. In that case, Travis Kalanick’s classic quote is appropriate, “Fear is the disease. Hustle is the antidote.” Frenetic motion is worse than thoughtful pursuit, but some frenzy is preferable to going quietly into that good night.

More (self-)publishing thoughts.

Sat, 24 Feb 2024 05:00:00 -0700

I recently got an email asking about self-publishing books, and wanted to summarize my thinking there. Recapping my relevant experience, I’ve written three books:

An Elegant Puzzle was published in 2019 as a manuscript by Stripe Press (e.g. I wrote it and then it was released as is), which has sold about 100,000 copies (96k through the end of 2023, and selling about 4k copies a quarter over past two years),
Staff Engineer which I self-published in 2021, which has sold about 70,000 copies (also selling roughly 4k copies a quarter over the past two years)
The Engineering Executive’s Primer which was published by O’Reilly earlier this month. It’s too early to have sales numbers at this point

Putting those in context, my sense is that these are “very good” numbers, but not “breakout” numbers. For example, my best guess is that a breakout technology book like Accelerate or The Manager’s Path has sold something closer to 300-500k copies.

I’ve also written about publishing a few times:

Self-publishing Staff Engineer (2021) – this remains a comprehensive summary of my self-publishing process
Thoughts on writing and publishing Primer (2023) – my process writing with O’Reilly and how it contrasted with self-publishing
What I learned writing [An Elegant Puzzle] (2019) – I wrote this shortly after finishing writing Puzzle, and rereading this five years later, I’m most surprised at how little I knew about writing books at this point. It’s also a poorly formatted post, but whatever, who knows what I was doing back then

Building on that, the general elements I’d encourage someone to think through if they’re deciding whether to self-publishing:

There’s a learning curve to publishing book, and I’ve learned a lot from every book I’ve written. Both working with publishers and self-publishing accelerate your learning curve. To maximize learning, I’d recommend doing a mix of both. If your goal is to only write a single book, I’d recommend working with a publisher already has gone through the learning curve and can guide you on navigate it as well
Publishers might not take your book, which means sometimes you can’t publish a given book with a publisher. I’d generally argue that means you should work on your own distribution before trying to publish the book. Having your own distribution is critical to getting a publisher to take your book, and also critical to being able to self-publish successfully. If you can’t find a publisher willing to take your book, I think there’s a lot of risk in self-publishing it (not because self-publishing is inherently risky, but because publishers filter for the sorts of criteria that derisk self-publishing), and you should reflect on that
Pricing control is lost when you work with a publisher. Stripe Press prices to maximize distribution, selling a hard cover at roughly $20. O’Reilly prices to maximize profit, selling a paperback at roughly $40. Neither of these is right or wrong, but your goals may or may not align with your publisher’s pricing strategy. When self-publishing, there’s no potential for misaligning with the publisher’s pricing strategy. Of course, pricing strategy also impacts your compensation a great deal, e.g. I probably make twice as much from each copy of Staff Engineer sold as I do from a copy of The Engineering Executive’s Primer, despite the fact that Staff Engineer costs half as much.
Pring quality is highly variable across publishing solutions. In particular, Kindle Direct Publishing–which is the dominant on-demand printing solution for self-published books–has highly variable print quality. In general, on-demand print quality is variable because there are 10,000s of small batch print runs. Even when print quality is high 99% of the time, it still means shipping some badly printed books. Anecdotally, my sense is that quality is highly dependent on the specific region where your book is printed, so you might never get a badly printed copy, but many of your readers in another region might frequently receive low quality print. This has been the largest “hidden tax” of self-publishing for me.

If you work with a publisher, they handle this, and their large volume print runs are generally error free because they are infrequent and represent a major investment for both the publisher and printer
Creative control may be significantly lower working with a publisher on many dimensions. This ranges from creating your book’s cover to decisions about how content and topics are treated. Similar to pricing strategy, you can largely derisk this issue upfront by understanding what a given publisher wants in these regards, but you can get into a lot of trouble if you don’t align early
Editorial support is highly variable across publishers and editors within publishers I’ve adored every publisher and editor I’ve worked with, but I think that’s largely due to good luck (asking around about a given editor goes a long way here)
Other sorts of support is highly variable, but working with a publisher you don’t have to find the folks, and generally you’re going to run into fewer operational issues because you’re working with folks who publish books frequently
Release timing and control is very low when you work with a publisher. When you self-publish, particularly with a print on-demand solution, you have immense control here
Payment nuances are someone else’s problem if you work with a publisher. If you’re an individual author who is taking full revenue (and costs), this is trivial. However, if you want to split revenue from a book, this is going to be fairly annoying as a self-publisher
International rights management is pretty painstaking as a self-published author, although if you’re lucky you can find an agency to work with like Nordlyset who take on most of the burden for this. You can do this yourself (and I did for one language, just to understand the process), but you won’t have a good sense of the quality of those international publishers, how to do the negotiations, and so on. Not all publishers will handle this for you either, for example I work with Nordlyset for both my Stripe Press and self-published books, but O’Reilly handles this for me

In sum, I don’t think there’s any right decision on whether or not to self-publish, it’s all very context dependent. The only thing I’d push back on is the sense that there’s only one obviously right decision, that statement is resoundingly untrue from my experience.

Digital release of Engineering Executive's Primer.

Wed, 07 Feb 2024 01:00:00 -0600

Quick update on The Engineering Executive’s Primer. The book went to print yesterday, and physical copies will be available in March. Also, as of this moment, you can purchase the digital edition on Amazon, and read the full digital release on O’Reilly. (You can preorder physical copies on Amazon as well.)

Thesis on value accumulation in AI.

Wed, 31 Jan 2024 14:00:00 -0600

Recently, I’ve thinking about where I want to focus my angel investing in 2024, and decided to document my thinking about value accumulation in artificial intelligence because it explains the shape of my interest–or lack thereof–in investing in artificial intelligence tooling. I’ll describe my understanding of the current state, how I think it’ll evolve over the next 1-3 years, and then end with how that shapes what I’m investing in.

My view on the the state of play today:

There are three fundamental components: Infrastructure (cloud providers, NVIDIA, etc), Modeling & Core (OpenAI, Anthropic, etc), and AI-enhanced products (Github Copilot, etc)
Today there’s significant value being captured in the Modeling & Core layer, and many new companies attempting to compete in that tier. Valuations in this tier are extremely rich at this point
Infrastructure hasn’t captured too much value, except for NVIDIA who arguably should be split into their own bucket of “hardware” instead of lumped in with cloud providers. Cloud vendors have the scale of physical resources to participate in AI, but generally don’t yet have strong offerings. However, these companies have a structural advantage in preexisting legal contracts with companies to govern API and data usage, along with the economy of scale to rapidly grow these businesses once they find product-market fit in the AI segment
AI-enhanced product has relatively few sophisticated entries today. There’s a lot of handwaving and loud statements, but very few companies that have proven out their AI-enhanced products are meaningfully better than preexisting alternatives. I think this is a matter of time rather than exceptionally difficult, so we will see more value accumulate here.

However, I think this is a transitory state. Where I see things moving over the next several years (and generally I think the transition here will be faster than slower):

I believe Infrastructure will eat an increasingly large amount of the Modeling & Core tier. Even today, the cloud providers of the Infrastructure have significant ownership and control in the leading Modeling & Core tier. This will make it harder to perceive the shift, but I think it’s already happening and will accelerate
Because I believe AI-enhanced product will successfully capture value thoughtfully using AI, the interesting question is what sorts of products will capture the majority. Ultimately I think the question is whether it’s harder to get the necessary data to power AI (fast-moving incumbents capture majority of value) or whether learning to integrate and integrating products with genuinely useful AI-capabilities is harder (new challengers capture majority of value)

There’s no interesting way to invest in the Infrastructure tier in 2024 (the main players are all public at this point), and I think the Modeling & Core tier is shrinking (and largely over-valued by interest from folks with a different thesis on value accumulation), which means that the interesting place to angel invest in 2024 is, in my opinion, in products that are well-suited to adopt AI-capabilities. That’s a broad category–we’re still learning where these techniques are powerful–but I think it’s particularly any company that works heavily with documents, and any company where it’s product is capable of keeping a human in the loop (e.g. LLMs are cheap, fast and imperfectly accurate, but in a system where someone uses it to draft replies and review them by a human, you’d be fine).

Not angel-investing, but if you wanted to make a career bet, I think the interesting career bet is finding an established company with significant existing data and product workflows that could be enhanced by recent AI advances.

High-Context Triad.

Wed, 24 Jan 2024 15:00:00 -0600

The past couple weeks I’ve been working on three semi-related articles that I think of as the “High Context Triad.” Those are Layers of context, Navigating ambiguity, and Tradeoffs are multi-dimensional. One of my background projects, probably happening in 2025 or 2026 after I’ve finished my nascent project on engineering strategy, is publishing a second edition of Staff Engineer, and I intended these three articles as supplements.

I’ve really enjoyed writing these pieces, because the first on context layers is really necessary to establish the vocabulary to even talk about the other two effectively. I’ve been trying to write about navigating ambiguity for four or five years now, but really struggled to do so until I was able to write “Layers of context.” Once I wrote about context layers, then the piece on navigating ambiguity fells together in an hour or two, following years of staring at a blank page. Similarly, I struggled to write “Layers of context” until I was in a specific set of discussions with an engineer that framed the specific concept clearly enough in my head that I could write it down, which is a good articulation of why I believe so deeply in the unique opportunity of Writers who operate.

In addition to pulling in this triad, Navigators is another likely supplement, perhaps following Where do Staff-plus engineers fit into the org? Altogether, I think that Staff Engineer is holding up well, but there’s a lot of interesting thinking happening in the space–especially Tanya Reilly’s The Staff Engineer’s Path–and a light revision will be worthwhile. Eventually. Until then, I hope folks interested in this topic get something out of this High-Context Triad.

Useful tradeoffs are multi-dimensional.

Wed, 24 Jan 2024 14:00:00 -0600

In some pockets of the industry, an axiom of software development is that deploying software quickly is at odds with thoroughly testing that software. One reason that teams believe this is because a fully automated deployment process implies that there’s no opportunity for manual quality assurance. In other pockets of the industry, the axiom is quite different: you can get both fast deployment and manual quality assurance by using feature flags to decouple deployment (shipping the code) and release (enabling new functionality).

The deeper I get into my career, the more I believe that example holds within it a generalizable pattern for making useful tradeoffs:

Two-dimensional tradeoffs always disappoint someone
You can usually make a tradeoff that doesn’t disappoint anyone by introducing a new dimension

In the “quick vs safe deployment” tradeoff, the additional dimension is decoupling feature activation (“release”) from shipping the code necessary to enable that feature (“deployment”). Introducing that dimension makes it possible for engineers to get fast, predictable deployments and for quality assurance to get the chance to review before enabling the feature for users.

While most people have already intuited these rules to some extent, I think that stating them explicitly is a lightly transformative experience, and I’ll dig into applying these rules a bit.

Examples

Before talking about the mechanisms of identifying dimensions to improve tradeoffs, let’s briefly walk through a few more examples of where adding a dimension makes for cleaner tradeoffs:

Project budgets – During annual planning, many companies struggle with intense debates about whether they invest into international expansion in new markets or do they instead prioritize their existing markets. By adding the dimension of fixed budgets, they can get varying degrees of both rather than debating existentially about doing one or the other
Diversified portfolio – For a long time, investors felt stuck either making safe investments that underperformed the stock market or making risky bets that might overperform the stock market but also might go to zero. Burt Malkiel’s A Random Walk Down Wall Street introduced the dimension of diversification, such that you could both get stock market-like performance and lower risk
Data-informed restrictions – You’ll often hear debates between Product and Security teams about the tradeoff between safety for your users and usability of your product. However, by taking a data informed approach you can often get both. For example, instead of debating about removing permissions from users, start by removing all permissions that each given user currently doesn’t use. By including real-world usage as a dimension of the tradeoffs, you can usually identify a tradeoff that improves security without reducing usability
Feature flags – As discussed in the introduction, many engineers believe we must have slow-and-safe deployment or fast-and-risky deployment, but decoupling deploy and release via feature flags allows us to get fast-and-safe deployments

Beyond this small handful of examples, I suspect you can identify quite a few more tradeoffs from your work history where an additional dimension turned a messy disagreement into an obvious path forward. When you work with someone who’s particularly good at this, the entire idea of tradeoffs starts to melt away to be replaced by thoughtful solutions.

How to add dimensions

Once you start thinking about tradeoffs this way, you’ll notice people who already take this approach to improving tradeoff decisions. The challenge is that most people do this intuitively rather than following a specific set of steps, which makes it difficult for them to explain it. Frankly, I have this challenge as well. Over time I’ve gotten better at doing it, but it was only very recently that I found the right vocabulary to describe it.

Here’s by best attempt to reverse engineering this practice into steps:

Go into each tradeoff discussion believing that there’s an additional dimension you can add that will greatly reduce the current tension in decision-making. Socialize this belief with others so they understand where you’re coming from, this can be as simple as a statement like, “I wonder if there’s a dimension we can add to this tradeoff to make it easier.”
Get very specific on all stakeholder requirements. The missing dimension is usually only evident in the details, so you need to force people to be precise about their needs. If you have stakeholders who cannot be precise about their needs, then you should spend time working with them to get more clarity.

Yes, it is their problem that they can’t articulate their needs, but it’s also your problem now too.
Seeing dimensions is the same as seeing layers of context. You’ll either need to expand your awareness of additional context layers or pull together a working team who have broad knowledge. This doesn’t need to be the decision making stakeholders, just folks who understand the relevant teams, technologies, and product.
Test new dimensions for usefulness. At the simplest, as your working group “How might we simplify untangling this tradeoffs with this additional dimension?” The key is to explore many dimensions quickly, try them on for usefulness, and then move on to another. Don’t go deep into any given dimension until it shows some promise.
See around corners by asking those who’ve solved similar tradeoffs before. I feel like a broken record, but it really does work to just ask people who’ve solved this specific problem before. Once again, this is why it’s so valuable to develop a network of peers. They can probably just tell you what the missing dimension is!
Ultimately, you should only add a dimension to a tradeoff if it provides significantly better outcomes for the stakeholders involved. Once you start thinking about this idea, there’s a temptation to add dimensions everywhere, but avoid additional dimensions that make things decisions harder to explain without greatly improving your options.

This process won’t work every time, because some working groups simply won’t know enough about the missing dimension to suggest it. This is why you shouldn’t get discouraged if you can’t find the missing dimension in any given tradeoff, and also why it’s useful to reconsider hard tradeoffs every couple of years. Just because you didn’t know about the missing dimension last time doesn’t mean you are unaware of it now.

Late-career abilities

Sometimes people will talk about engineers becoming senior in five to seven years, and then being wholly competent at the job they do. This is true in one sense–you can be a very good engineer with five years of experience–but also misses on the many abilities that are only beginning to take root at that point. Adding dimensions to tradeoffs is a good example of the latter category: there are very few folks with the necessary context layers and the breadth of experience to get good at identifying the missing dimension to make difficult tradeoffs easier. There’s always more to learn.

Navigating ambiguity.

Fri, 19 Jan 2024 05:00:00 -0600

Perceiving the layers of context in problems will unlock another stage of career progression as a Staff-plus engineer, but there’s at least one essential skill to develop afterwards: navigating ambiguity. In my experience, navigating deeply ambiguous problems is the rarest skill in engineers, and doing it well is a rarity. It’s sufficiently rare that many executives can’t do it well either, although I do believe that all long-term successful executives find at least one toolkit for these kinds of problems.

Before going further, let’s get a bit less abstract by identifying a few examples of the kinds of problems I’m referring with the label deeply ambiguous:

At Stripe, we knew that data locality laws were almost certainly coming, but we didn’t know when or what shape that regulation would come in. One scenario was that many countries would require domestic transactions (e.g. transactions where the buyer and seller reside in the same jurisdiction) to be stored in a domestic datacenter, which India did indeed require. Another scenario was that all transactions would have to store a replica in jurisdictions that had a buyer or seller present. There were many such scenarios, and which seemed most likely changed as various political parties won and lost elections in various regions When explaining this problem to new colleagues, my starting explanation became, “The only thing we can know is that the requirements will change every six months for the lifespan of the internet.”

If the requirements were ambiguous, so were our tools for solving it. Many solutions involved degrading the reliability or usability of our user-facing functionality for impacted zones. Evaluating those solutions required Product, Engineering, and Sales alignment. Other solutions reduced the user-impact but reduced our operating margin, which required alignment from Product, Engineering and Finance. Even implementing this work had a significant opportunity cost relative to other work, which was also difficult to get agreement on.

This was a deeply ambiguous problem.
At Calm, we would eventually acquire Ripple Health Group to enter the healthcare space, but beforehand we made a series of efforts to enter the space ourselves. None of our Product, Engineering or Legal teams were from a healthcare background, and we ran into an unexpected source of ambiguity: HIPAA.

It quickly became clear that we couldn’t make forward progress on a number of product and engineering decisions without agreeing on our interpretation of HIPAA. Some interpretations implied significant engineering costs, and others implied almost no engineering costs at all, but some potential legal risk. Teams were glad to highlight concerns, but no one had conviction on how to solve the whole set of concerns.

This, too, was a deeply ambiguous problem.

These examples highlight why perceiving layers of context are a prerequisite to effectively navigating deeply ambiguous problems: they almost always happen across cross-functional boundaries, and never happen in the seams where your organization has built experience solving the particular problem. They are atypical exceptions, that involve a lot of folks, and where the decision has meaningful consequences.

It’s also true that what’s a deeply ambiguous problem for one company isn’t necessarily that ambiguous for another company. For example, Amazon has solved data locality in a comprehensive way, so it’s certainly not a deeply ambiguous problem for them at this point, but I suspect it was until they figured it out. What falls into this category at any given company changes over time too: data locality isn’t deeply ambiguous for Stripe anymore, either.

It would be disingenuous to claim that there’s a universal process to navigating ambiguous problems–they’re ambiguous for a reason!–but there is a general approach that I’ve found effective for most of these problems that I’ve encountered: map out the state of play, develop options for discussion, and drive a decision.

First, map out the state of play:

Talk to involved teams to understand what the problems at hand are, and rough sketches of what the solution might look like. Particularly focus on the points of confusion or disagreement whose existence are at the root of this problem being deeply ambiguous, e.g. data locality is important, for sure, but wouldn’t it be better to delay solving it until we have clearer requirements?
Debug the gaps in cross-functional awareness and partnership that are making this difficult to resolve. You’re not looking to assign blame, simply to diagnosis the areas where you’ll need to dig in to resolve perceived tradeoffs, e.g. as Product, we can’t be responsible for interpreting HIPAA, we need Legal to tell us exactly how to implement this
Identify who the key stakeholders are, and also the potential executive sponsors for this work, e.g. the General Counsel, Chief Technology Officer, and Chief Product Officer

Next, develop potential options to solve the state of play:

Cluster the potential approaches and develop a clear vocabulary for the clusters and general properties of each approach. For the data locality situation, the clusters might be something like: (1) highly-available and eventually consistent, (2) strongly consistent and single-region, and (3) strongly consistent with backup region
Develop the core tradeoffs to be made in the various approaches. It helps to be very specific, because getting agreement across stakeholders who don’t understand the implications will usually backfire on you.

For example, if we allow transactions to be processed in any region, and then forwarded them for storage in their home region, then it means that balances in the home region will sometimes be stale. This is because a transaction may be completed in another region and not yet forwarded to the home, causing the sum of all calculations to be stale. Are you willing to temporarily expose stale balances? If you’re not comfortable with that, you have to be comfortable failing to complete transactions if the home region is unavailable. What’s the right tradeoff for our users?
Talk to folks solving similar problems at other companies. It’s one thing for you to say that you want to run a wholly isolated instance of your product per region, but it’s something else entirely to say that Amazon used to do so. As you gather more of this data, you can benchmark against how similar companies approached this issue. (It’s true that companies rely too heavily on social proof to make decisions, but it’s also true that there are few things more comforting for leadership making a decision they don’t understand particularly well than knowing what another successful company did to solve it.)

Finally, drive a decision:

Determine who has the authority to enforce the decision. The right answer is almost always one or more executives. The wrong answer is the person who cares about solving the problem the most (which might well be you, at this point)
Document a decision making process and ensure stakeholders are aware of that process. No matter how reasonable the process is, some stakeholders may push back on the process, and you should spend time working to build buy-in on the process with those skeptics. Eventually, you will lean on the authorities to hold folks to the process, but they’ll only do that if you’ve already mostly gotten folks aligned
Follow that process to its end. Slow down as necessary to bring people along, but do promptly escalate on anyone who withholds their consent from the process
Align on the criteria to reopen this decision. One way that solutions to ambiguous problems die is that the debates are immediately reopened for litigation after the decision is made, and you should strive to prevent that. Generally a reasonable proposal is “material, new information or six months from now”

This formula often works, but sometimes you’ll follow it diligently and still find yourself unable to make forward progress. Let’s dig into the two most common challenges: trying to solve the problem too early, and not having an executive sponsor.

Is the time ripe?

Something that I’ve learned the hard way is that there’s a time and place for solving any given ambiguous problem. The strongest indicator that the time isn’t now is if you drive a decision to conclusion with the relevant stakeholders, but find the decision simply won’t stay made. It’s easy to feel like you’re failing at that moment, but usually it’s not personal. Instead, it probably means that the company values the optionality of various potential solutions more than it values the specific consequences those solutions imply.

Going back to the data locality challenge for Stripe, I found it quite difficult to make forward progress–even after getting stakeholders bought in–and it was only when the risk of legal penalties for non-compliance became clear that the organization was able to truly accept the necessary consequences to solve the problem at hand. Until the legal consequences became clear, the very clear opportunity cost, product tradeoffs, and margin impact weren’t worth tolerating. Once the legal consequences were better understood, it became obvious which trade offs were tolerable.

Here are some questions to ask yourself if you’re debugging whether your approach is flawed or it’s simply the wrong time to solve a given problem:

Does new information keep getting introduced throughout the process, despite your attempts to uncover all the relevant information? If so, you’re probably trying to solve the problem too early
Are there clear camps advocating for various approaches? If so, it’s probably not a timing issue. Rather it sounds like there’s more stakeholder management to do, starting with aligning with your executive sponsor
Do your meetings attempting to agree on a decision keep ending with requests for additional information? This may or may not indicate being early. Many leaders hide from difficult decisions by requesting more information, so this might be either a timing issue or a leadership issue. Don’t assume requests for additional information mean it’s too early

If you’re still unclear, then escalate to your leadership chain for direction. That will make it clear whether they value you solving this problem at this moment. If they don’t, then slow down and wait for the circumstances to unfold before returning to push on the problem again.

Maybe you’re reading my advice to escalate up the leadership chain and thinking to yourself, “Yeah, I wish I had someone to escalate this to!” To be explicit: that’s a bad sign! If you can’t find an executive committed to solving an ambiguous problem, then you’re unlikely to solve it. Yes, there are some exceptions when it comes to very small companies or when you yourself are a quasi-executive with significant organizational resources, but those are exceptions.

My general advice is that you should only take on deeply ambiguous problems when an executive tells you to, and ensure that the executive is committed to being your sponsor. If not, the timing might be good, but you’re still extremely likely to fail. Flag the issue to your management chain, and then let them decide when they’re willing to provide the necessary support to make progress.

Don’t get stuck

My biggest advice on solving deeply ambiguous problems is pretty simple: don’t overreact when you get stuck. If you can’t make progress, then escalate. If escalating doesn’t clarify the path forward, then slow down until circumstances evolve. Failing to solve an ambiguous problem is often the only reasonable outcome. The only true failure is if feeling stuck leads you to push so hard that you alienate those working on the problem with you.

Layers of context.

Mon, 15 Jan 2024 05:00:00 -0600

Recently I was chatting with a Staff-plus engineer who was struggling to influence his peers. Each time he suggested an approach, his team agreed with him, but his peers in the organization disagreed and pushed back. He wanted advice on why his peers kept undermining his approach. After our chat, I followed up by talking with his peers about some recent disagreements, and they kept highlighting missing context from the engineer’s proposals. As I spoke with more peers, the engineer’s problem became clearer: the engineer struggled to reason about a problem across its multiple layers of context.

All interesting problems operate across a number of context layers. For a concrete example, let’s think about a problem I’ve run into twice: what are the layers of context for evaluating a team that wants to introduce a new programming language like Erlang or Elixir to your company’s technology stack? I encountered this first at Yahoo! when my team lead introduced Erlang to the great dismay of the Security and tooling teams. I also experienced it later in my career when dealing with a team at Uber that wanted to implement their service in Elixir.

Some of the layers of context are:

Project’s engineering team
- The problem to be solved involves coordinating work across a number of servers
- Erlang and Elixir have a number of useful tools for implementing distributed systems
- The team solving the problem has an experienced Erlang engineer, and the rest of the team is very excited to learn the language
Developer Experience and Infrastructure teams
- There’s a fixed amount of budget to support the entire engineering organization
- Each additional programming language reduces the investment into the more frequently used programming languages across the organization. This makes the organization view the Infrastructure organization as less efficient each time it supports a new programming language, because on average it is less efficient
- The team is telling Infrastructure that they’ll take responsibility for all atypical work created by introducing Erlang. However, the Infrastructure team has heard this promise before, and frequently ends up owning tools in new languages after those teams are reorganized. At this point, they believe that any project in a new programming language will become their problem, no matter how vigorously the team says that it won’t
Engineering leadership
- Wants to invest innovation budget into problems that matter to their users, not into introducing new technologies that are generally equivalent to their existing tools
- Is managing a highly constrained financial budget, and is trying to maximize budget spend on product engineering without impacting stability and productivity. Introducing new languages is counter to that goal
- Wants a standardized hiring and training process focused on the smallest possible number of programming languages
- Has been burned by teams trying to introduce new programming languages and ending up blocked by lack of Infrastructure support for the language

Seeing this specific problem twice in my career was enlightening, because the first time it seemed like a great idea to introduce a new programming language. The second time, my context stack had expanded, and I pushed back on the decision firmly. In my current role as an executive, introducing another programming language is a non-starter as it would violate our engineering strategy.

A mid-level engineer on the project team is expected to miss some parts of the infrastructure perspectives. A mid-level engineer on the infrastructure team is expected to miss some parts of the product engineering perspectives. Succeeding as a Staff-plus engineer requires perceiving and reasoning across those context layers: seeing both product and infrastructure perspectives, and also understanding (or knowing to ask about) the leadership perspective.

How to see across layers

In any given role, you’ll be missing critical context to expand your understanding of the layers around you. In the best case, your peers and manager will take the time to explain the context in those layers, but often they won’t. For example, it took me a long time to understand how the company’s financial plan connected with our planning process, in part because no one ever explained it to me. Folks are generally so deep in their own layer of context that they fail to recognize how unintuitive it might be to others.

If you want to develop your sense for additional layers around you, here are some of the techniques I’ve found most effective for developing that context yourself:

Operate from a place of curiosity rather than conviction. When folks say things that don’t make sense to you, it’s almost always because they’re operating at a layer whose context you’re missing. When you’re befuddled by someone’s perspective, instead of trying to convince them they’re wrong, try to discover that layer and its context. This is a perspective that gets more valuable the more senior you get
Rotate onto other teams. If you work in platform engineering, work with your manager to spend three months on a product engineering team that uses your platform. Do this every few years to build your understanding of how different teams perceive the same situations
Join sales calls and review support tickets. Stepping outside of the engineering perspective to directly understand your end user is a powerful way to step outside of the context layer where you spend the majority of your time
Work in different sorts of companies and industries. There are many benefits to specializing in a given vertical–e.g. fintech or marketplaces–but it’s equally valuable to see a few different industries in your career. By seeing other verticals you’ll come to better understand what’s special about the one you spend the most time in. This is equally true for joining a larger company to better understand what makes startups special, or vice-versa
Finally, build a broad network. Developing a wide network of peers is the easiest way to borrow the hard-won context of others without the confounding mess of a company’s internal tensions and politics. Particularly mine for reasons why your perspective on a given topic might be wrong, rather than looking for reasons you might be right

These things take time, and to be entirely honest it took me a solid decade before I got good at perceiving and navigating context layers. Indeed, it was the biggest gap that prevented me from reaching more senior roles in my first forays up the org chart.

Passion can be blinding

Like many foundational leadership skills, perceiving across context layers is an obvious idea, but a lot of folks struggle with implementation. Lack of curiosity is the most common challenge I see preventing folks from figuring this out, but the most difficult blocker is a bit unintuitive: caring too much.

I’ve run into many very bright engineers who care so deeply about solving a given problem in a certain way–generally a way that perfectly solves the context layer they exist in–that they are entirely incapable of recognizing that other context layers exist. For example, I worked with a senior engineering manager who was persistently upset that they didn’t get promoted, but also threatened to quit if we didn’t introduce a new note taking tool they preferred. We already have a proliferation of notes across a number of knowledge bases, and introducing a new one would fragment our knowledge further–a recurring top three problem in our developer productivity surveys–but this individual believed so strongly about a specific note taking tool that none of that registered to them at all.

As someone who used to struggle greatly with this, I’ve found it valuable to approach problems in three phases:

Focus exclusively on understanding the perspective of the other parties
Enter the mode of academic evaluation where I try very hard to think about the problem from a purely intellectual basis
Only after finishing both those approaches, only do I bring my own feelings into the decision making–what do I actually think is the best approach?

The point of this approach isn’t to reject my feelings and perspective, as I know those are important parts of making effective decisions, instead it’s ensuring that I don’t allow my feelings to cloud my sense of what’s possible. Increasingly I believe that most implied trade offs are artificial–you really can get your cake and eat it too–as long as you take the time to understand the situation at hand. This approach helps me maximize my energy solving the entire problem rather than engaging in conflict among the problem’s participants.

Obvious or invisible

If you find the idea that there are many context layers is too obvious to consider, then maybe you’re already quite good at considering the perspectives at hand. However, if you frequently find yourself at odds with peers or leadership, then take some time to test this idea against some of your recent conflicts and see if it might be at their root. For some exceptionally talented folks I’ve worked with, this is the last lesson they needed to learn before thriving as a senior leader.

Those five spare hours each week.

Sun, 14 Jan 2024 05:00:00 -0600

One of the recurring debates about senior engineering leadership roles is whether Chief Technology Officers should actively write code. There are a lot of strongly held positions, from “Real CTOs code.” at one end of the spectrum, to “Low ego managers know they contribute more by focusing on leadership work rather than coding.” There are, of course, adherents at every point between those two extremes. It’s hard to take these arguments too seriously, because these values correlate so strongly with holders’ identities: folks who believe they are strong programmers argue that CTOs must code, and people who don’t feel comfortable writing software take the opposite view.

There’s another question that I find more applicable in my own career: If I have five spare hours each week, how should I invest that time? It’s hard to answer that without specifying your investment goal, so a more complete version of the question is “If I have five spare hours each week, how should I invest that time to maximize my impact and long-term engagement?”

In most CTO roles, you have roughly four options:

Build deep context – Write code, or otherwise engage in the detailed work within the Engineering organization you lead
Build broad context – Expand your context outside of Engineering, better understanding your peer function’s work, goals and obstacles. This might be talking with customers, discussing product tradeoffs, understanding how marketing is driving interesting, etc
Improve current systems and process – Work on your engineering strategy, planning process, or pretty much any existing element of how your Engineering organization operates
Build relationships – Expand your internal or industry networks

These are all valid ways to invest your time, and picking among them for your last five hours depends on what your role needs from you, and what you need from your role. You should be wary – and honestly somewhat weary – of anyone who tells you, context-free, what’s important for you and your role. You should likely be capable of doing all of those, but there are many ways to do them, and what’s optimal in your circumstances is deeply context-specific.

There are some general rules. Smaller and pre-market fit companies are more likely to need their executives to build deep context. Larger and multi-business unit companies are more likely to benefit from broad context or improvements to existing systems and processes. They’re just generalized rules though, make the decisions for yourself.

Predictability.

Mon, 01 Jan 2024 05:00:00 -0600

Right now I’m reading Michael S. Malone’s The Big Score, and one thing that I love about it is how much it believes that key individuals drive and create industries. It’s an infectious belief, and a necessary one to write a concise, coherent narrative story about the origins of Silicon Valley. It’s something I’ve thought about a lot as well in my career, and also while writing my upcoming book on operating as an engineering executive–how much do good executives really matter?

My ego’s too frail to sustain a proclamation like, “Executives don’t matter!” Further, that doesn’t reflect my lived experience: I think executive quality matters a great deal. That said, I do think that there are non-obvious ways that seemingly mediocre executives outperform the sum of their capabilities, and seemingly brilliant executives underperform their talents. One of those ways is the extent that they create a predictable environment.

Uber gives a clear example and counterexample of predictability:

At Uber, our CTO strongly preferred letting teams work through disagreements themselves. The exception was existential issues (“our database will run out of space in six months”) or CEO decrees (“we will build a datacenter in China in the next six months”), where one of the CTO’s trusted advisors would select a top-down plan.

Many folks disagreed with both the mostly bottom-up approach (“it’s just politics”) or the trusted advisors (“why does he listen to those folks?”). Many disagreed with the specific decisions. However, it was predictable how decisions would get made, and that made it easy for teams to plan around. Teams knew how to make forward progress, even if they often disagreed.
Later, Uber hired an engineering executive beneath the CTO who started rapidly changing a number of technology decisions without much input. He actively avoided seeking input because he was convinced the existing team’s context was irrelevant due to their relative lack of experience. Decision making became unpredictable, both in terms of who was expected to make which kinds of decisions, and which decisions would be reached.

Many folks believed the specific decisions being made were better than previous choices, but decision making became extremely unpredictable. Teams got stuck, unsure how to make forward progress, even those teams that agreed with the vast majority of decisions.

Although I never thought about predictability directly, much of my onboarding approach as an executive is around increasing predictability while I come to understand the business, team and technology enough to make more context-specific decisions. In the first several months, it’s difficult to decide whether to shut down a business unit, but you can absolutely increase predictability by leading with policy rather than exceptions and explicitly documenting the engineering strategy that the organization already follows.

That said, the moral of the story here is that predictability is valuable, not that it’s a universal cure. A mediocre but predictable executive will likely outperform an extraordinary but unpredictable executive, but both are unlikely to be successful in the long-run.

2023 in review.

Mon, 18 Dec 2023 05:00:00 -0700

This was an eventful year. My son went to preschool, I joined Carta, left Calm, and wrote my third book. It was also a logistically intensive year, with our toddler heading to preschool, more work travel, and a bunch of other little bits and pieces. Here is my year in review summary.

I love to read other folks year-in writeups – if you write one, please send it my way!

Previously: 2022, 2021, 2020, 2019, 2018, 2017

Goals

Evaluating my goals for the year:

[Completed] Write at least four good blog posts each year.

I wrote a lot this year, including adding five to my popular posts page, including: Writing an engineering strategy, Measuring an engineering organization, Setting organizational values, and Writers who operate.
[Completed] Write another book about engineering or leadership.

I did this: The Engineering Executive’s Primer goes to print in late January, and should be available for purchase in February. The complete digital version will be available via O’Reilly in January.

I am currently brainstorming a bit on a fourth book, very likely my last on the topic of engineering leadership, although it’ll take a bit of time to decide whether and when to take that on. Right now mostly thining about the topic of engineering strategy.
[Mixed] Do something substantial and new every year that provides new perspective or deeper practice.

Like clockwork, I struggle to give myself a passing grade on this one. Joining Carta has greatly expanded my perspective on executive leadership. I also worked with a new publisher, O’Reilly, which provided a different view into the book creation process than self-publishing or working with Stripe Press. This was also the first year I gave a keynote, this one at QCon, which maybe qualifies?
[In progress] 20+ folks who I’ve managed or meaningfully supported move into VPE or CTO roles at 50+ person or $100M+ valuation companies.

This goal is due in 2029. Without spending much time thinking this through, there are at least five folks who qualify here, and I bet I could get to at least ten if I spent long enough digging into it
[Completed] Work towards a clear goal for physical exercise. (Hitting the goal isn’t important.)

Discussed a bit more below, but I reset my running habbit and worked back up to doing a few eight mile runs. I’m mostly doing four milers now that I’m working full-time again, but it was very validating to stretch milleage a bit!

Carta & Calm

I left Calm earlier this year. I planned to take a year off, but ended up joining Carta after a couple months. When I explain this to folks, particularly those who I’d already told that I wasn’t going to go back to work immediately, what I tell them is: I felt confident that I would regret declining the offer to join Carta.

That’s still how I feel ~nine months into the job. Personally, learning and impact are the two things I value most in my work, and Carta remains the highest indexing job I’ve ever had on both counts.

An Engineering Executive’s Primer

I started and finished An Engineering Executive’s Primer this year. Coming into the year, I expected to write another book this decade, but it wasn’t this book, instead it was Infrastructure Engineering, which I ended up not making much progress on. I wrote up notes on writing Primer, and altogether I’m proud of the book and how quickly it came together.

Other books

It feels good to finish #3, and I think I could put down the pen at this point and not feel like a fraud to consider myself a writer, but I’m not done quite yet. I still have at least one more topic I want to spend some words on, engineering strategy. (I have no idea if I’ll ever get back to the Infrastructure Engineering book, I’m finding it hard to marshall the focus onto a topic that I’m not working on directly day to day.)

My first two books, An Elegant Puzzle and Staff Engineer are both doing well. Have been translated a few more times and so on, but nothing too wild. As I mentioned last year, I’m working hard to focus on the new things I do, and not to spend much time thinking about stuff I’ve already done, hence not reporting on book sales and such anymore.

First keynote

I gave my first first keynote, Solving the Engineering Strategy crisis at QCon SF. You can see a video recording of that talk on Youtube. I’m mostly avoiding conference talks these days, but it was impossible to pass up the opportunity to give my first keynote, particularly a keynote that didn’t require traveling for the conference.

There aren’t any conference talks on my schedule for 2024 but if I do some it’ll probably be focused on the topic of engineering strategy.

Advent of Code

I made it through day twelve of Advent of Code this year before deciding I needed to bail out. Some years ago I read Tanya Reilly’s ode to Advent of Code, and attempted it that year before getting busy, and decided to try again this year as quite a few work, professional and friend groups participated. I really enjoy working on these, but they’re competing for writing project time, and that’s just hard to fit in with the work travel, family visits, and so on that happen around the holidays. Maybe next year.

Angel investments

I did four angel investments this year, and invested in one fund as a limited partner. This is, give or take, roughly the sort of angel investing year I expect to have most years going forward, but it’s not a goal or a priority. I just evaluate the interesting things that come my way and occasionally invest. (I am mostly interested in developer experience, productivity, and infrastruture startups, as it’s the space I understand best and where I think my input is most useful.)

Reading

After finishing up Primer, I’ve been doing a bunch of professional reading. Much of this has been related to collecting my thoughts on engineering strategy, but some of it has been mining for ideas (including structural and presentation ideas) both as a leader and as an author who writes books.

The professional book I’ve read in the last few months are:

Year of personal admin

In addition to various work stuff, this was also a year of personal admin for me, where I tried to catch up on a few years of neglected tasks and ambitions around the house and my body.

Wearing glasses

I wore glasses until I was 13 or so, then I stopped wearing them, essentially on a whim. My vision was good enough for most purposes, including getting a drivers license, so I just didn’t think about it much for the following 20-plus years. Not thinking about it was nice.

When I started my new job, I started getting frequent migraines. Trying to diagnose what might be going wrong, it was clear that I was spending more time looking directly at a computer monitor that I had for several years, and I tried wearing a pair of old glasses that I had made about fifteen years ago as a last resort if I needed them to pass the vision portion of the California driving exam.

It turned out, this worked very well, and my eyes and head have felt much better since returning to wearing glasses. I don’t wear them all the time, but I do wear them whenever I sit down to do more than a few minutes of work on the computer. Age is, I suppose, more than just a number.

Running

Since graduating college, I’ve always been a frequent runner, although rarely been a serious runner. More concretely, other than a detour for a stress fracture, I’ve gone on 2-3 runs a week, averaging 3-4 miles, for quite a long time. In roughly 2013, I ramped up my runs for a while, building up to 6-8 mile runs twice a week for a few months, before ramping back down to my shorter runs. The shorter runs are nice because they take less time, and they also put a bit less strain on my knees which have at times been a bit unreliable. Plus, I generally prefer to stress my knees playing basketball instead of running.

This year, I wanted to build up my running distance and pace, with the goal of reestablishing a higher fitness baseline. Starting from my 3-4 mile runs, I rebuilt up to 8 mile runs, with my fastest average pace at 8 miles being 8 minutes and 42 seconds. I intended to spend more time working on my pace doing short runs, but I got distracted by the new job.

Relatedly, I’ve long been on the fence about buying an Apple Watch, but decided to buy one to help track my runs, and it’s been a surprisingly delightful piece of hardware. (I specifically bought the Apple Watch Ultra.) If I hadn’t bought it, I would absolutely not know how much I’d run, or the pace I ran at. It’s safe to say that I wouldn’t have even tried to do a pace goal, which would have considerably reduced the impact of my running workouts (e.g. only doing slow, long runs, rather than a mix of slow/long and fast/short) and results.

These are, on an absolute scale, not particularly big achievements. I know many runners who are much faster, go much longer, and are even much faster while going much longer, but it still felt good for me! I have no ambitions to be a competitive racer, I just like to push myself a bit occasionally, and particularly to continue pushing myself as I get older to remember that physical decline is in many ways a sum of choices rather than an inevitability.

Invisalign

In January, I started on Invisalign to improve parts of my bite, along with crowding in my lower front teeth. The plan was that I’d only wear them for four months, but twelve months later I’m still wearing them as the original set of trays didn’t fully work out. I’m scheduled to finish in February now, and am looking forward to no longer timing coffee consumption quite so carefully.

That’s my annual year in review for 2023! If you’re writing one, please send it my way! Love to hear what folks are working on and thinking about over the course of years.

Notes on How Big Things Get Done

Fri, 15 Dec 2023 06:00:00 -0600

How Big Things Get Done by Bent Flyvbjerg and Dan Gardner is a fascinating look at why some megaprojects fail so resoundingly and why others succeed under budget and under schedule. It’s an exploration of planning methods, the role of expertise, the value of benchmarking similar projects, and much more. Not directly software engineering related, but very relevant to the work. Also, just well written.

“Think slow, act fast”

It’s fine for planning to be slow (p17), as long as delivery is fast. Each moment during delivery (the actual execution of a task) is a moment something can go wrong, so condensing that timeline is essential to reduce risk. That is of course, actually condensing the timeline, not just lying about it as discussed in the “Honest Numbers” section below.

Planning phase is preferable to delivery phase because (p18) “the costs of iteration are relatively low” during planning. The example of Pixar is used, where they storyboard films up to eight times before moving into delivery phase. This is a large investment, but it’s a much cheaper investment than making a bad film.

It’s also much easier to avoid “lock in”, which is premature commitment (p42) if you plan extensively before moving to delivery. Once you begin delivery, modifying the plan is quite challenging. To make this point, the authors make an extensive comparison between the building of the Sydney Opera House and the Bilbao Guggenheim Museum. The former changed plans frequently with massive delays, the later delivered ahead of time and under budget. (In part due to significant use of modeling for the Bilbao, discussed in “Pixar Planning” section below.)

Also a good discussion of good planning starting from the end and reasoning backwards. Have a clear sense of why you’re doing something before you try to solve it. There’s a mention of Amazon’s Press Releases (p52)–write a future internal press release as a mechanism to pitch your project–as one mechanism to support reasoning backwards.

“Pixar planning”

The book argues that good planning is “Pixar planning” (p60), where you’re able to iterate quickly and cheaply. The average Pixar film is storyboarded 8 times (p70) to cheaply explore improvements.

This means that good planning requires modeling techniques, including modeling software(p68) as one technique, to support rapid, cheap exploration. The example of Frank Gehry extensively modeling out his buildings in simulation software is used to explore how he was able to deliver the Bilbao Guggenheim Museum so effectively. (A few years ago I played around with creating the systems library for modeling systems thinking problems, which was one of my experiments towards this end.)

Finally, the book also observes that learning happens not only within projects but also across projects (p159). Solar and wind projects are significantly less risky than nuclear projects in part because solar and wind projects deploy hundreds or thousands modular units, rather than one very large unit. Even if some wind turbines are poorly designed or installed, they can learn from than for the next ones. Learning to build nuclear power plants is much harder, since so few of the projects occur.

Dataset of projects

One fascinating idea, mentioned a number of times but not deeply explored, is that the authors have a “database of big projects” (p4) where they track the scope and outcome of various projects. This was initially 259 projects (p111), growing up to 16,000 projects over time.

This is a remarkable resource because it makes it possible to benchmark projects against similar projects, referred to as “reference-class forecasting” (p109), or at least benchmark against something, “reference point” (p111). I’ve been thinking a lot about benchmarking recently, and this is definitely something that furthere my interest. (This book also mentions Superforecasting a handful of times, so I’ve ordered a copy of that to take notes from as well.)

Things can be inexperienced

This book says something I’ve understood for a while but never articulated clearly, which is that things can be inexperienced, such like people can (p86). They use the example of a potato peeler that cuts your fingers when you use it, which you replace with iterations of better potatoe peelers than are less likely to cut your fingers. The final edition is an experienced thing, whose design incorporates significant learning into it.

You could probably write an entire book on just that idea alone. Perhaps combined with the observation that we often lose sight of why things work. Perhaps that book is The Design of Everyday Things.

“Honest Numbers”

I also appreciated the discussion of “honest numbers” (p3), which is really a discussion about dishonest numbers and how they justify many projects. A recurring theme in the book is that many leaders deliberately misinform stakeholders about potential costs in order to build commitment, reach a point of no return, and then acknowledge the full costs.

This is eloquently captured in a quote from Willie Brown (p35):

In the world of civic projects, the first budget is really just a down payment. If people knew the real cost from the start, nothing would ever be approved."

This idea, termed “strategic misrepresentation” (p26), reminds me of a poor joke I sometimes tell, which is that “Vice Presidents never miss their targets, they just move the targets to what they accomplish.” Holding the powerful to account is difficult, even if they are acting in good faith, and when they’re acting in bad faith, then it’s remarkably challenging.

This is an important issue, because often the parties who make the commitment aren’t the ones who are stuck paying it off (p38):

Drapeau got his Olypmics. And although it took more than thirty years for Montreal to pay off the mountain of debt, the onus was on the taxpayers of Montreal and Quebec. Drapeau wasn’t even voted out of office.

Incentives are hard, and harder still when there’s not possibility for accountability, as is often the case for politics.

Altogether, a quick and interesting book. Well worth a read.

Writers who operate.

Thu, 07 Dec 2023 11:00:00 -0600

Occasionally folks tell me that I should “write full time.” I’ve thought about this a lot, and have rejected that option because I believe that writers who operate (e.g. write concurrently with holding a non-writing industry role) are best positioned to keep writing valuable work that advances the industry. This is a lightly controversial view, so I wanted to pull together my full set of thoughts on the topic.

The themes I want to work through are:

Evaluating believability for operators is much easier than for non-operators
The pursuit of distribution changes what and how authors write (e.g. pulls towards topics that are trending)
How writing full-time anchors you on writers and audiences, whereas part-time writing allows a third balancing perspective (the folks you work with in the context of your industry work)
Invalidation events happen in industry (e.g. move from ZIRP to post-ZIRP management environment) but it’s difficult for non-operators to understand implications with conviction
Operating is an endless source of new topics (e.g. the topics in An Engineering Executive’s Primer are the direct outcome of my operating)
Part-time writers can still get better at writing, although maybe slower than full-time writers

I’m not particularly interested in convincing someone else whether this is the right choice for them, but hopefully at the end you’ll understand my perspective a bit.

Examples

There are many writers out there who fit into the “writers who operate” archetype. A few examples: Charity Majors, Dan Na, Eugene Yan, Hunter Walk, Tanya Reilly.

Venture capitalists use “operators” to indicate folks who’ve worked in industry as opposed to in venture, but I don’t make that nuance here–working in venture capital is “operating” in my usage. Similarly, you could try to cohort various writers by the volume of their writing, but that’s not too important to me–someone who hasn’t written anything in the past three years probably isn’t who we’re talking about, but generally this is a broad church.

Believability

Believability is a Ray Dalio and Bridgewater idea, and experiencing some public scrutiny of late (e.g. Bridgewater Had Believability Issues), but at its core the observation still rings very true: we should weigh advice more heavily from folks who we have reason to believe. Cedric Chen has a few tech-centric pieces on Believability, that are interesting reads: Believability in Practice and Verifying Believability.

First and foremost, I appreciate writers who operate because they directly experience the consequences of their choices. Cedric’s second piece tells the story of “Q”, a widely read tech leader who’s had a mixed career, as an example of needing to verify believability. I agree with that observation, but the only reason we’re able to evaluate the advice at all is because that writer is an operator. If they weren’t an operator, we wouldn’t be able to evaluate their believability at all.

Operating is, for me, remaining accountable for what I write. What I write is a pretty direct reflection of what I believe and how I operate at the time that I write it.

Distribution shapes writing

As you watch new writers come onto the “scene,” you’ll often notice a shift from a genuine passion in a given niche to engaging in topical events and controversy. The reality is that it’s exceptionally hard to write something that generates a lot of discussion, and it’s even harder to repeat that formula consistently. After folks have the experience of writing a popular piece, they often get sucked into the desire to produce more, and this ultimately means seeking wider distribution.

Reliable distribution is a hard thing to find on the internet, and one of the most obvious opportunities for distribution is to engage in controversy. Write something controversial, engaging in an existing controversy, subtweet someone who did something dumb, whatever. The problem with this is that it pulls you out of picking topics, and instead towards picking positions.

Ultimately, I don’t believe you can say anything particularly novel or interesting in reaction to a trending topic. There are certainly takes that are more or less nuanced, but mobilizing the base is not advancing the industry.

This problem is even more acute when you’re trying to make a financial living out of your writing, because matching your message to your audience becomes that much more important. You’re going to spend even more time tuning your messaging to resonate with what the audience currently believes than you are on writing something new.

Taste is tribal

A year or two back, Brie Wolfson wrote a very compelling take on taste, Notes on Taste. Reading those notes, I want nothing more than to identify as someone with taste. However, perhaps out of jealousy, I’m a bit of a taste-skeptic. I view taste principally as tribal, and find that identity-through-taste is a frequent driver of boring takes and perspectives.

As an example, think about Marc Andreessen’s recent The Techno-Optimist Manifesto. Regardless of how you personally feel about the manifesto, I’m confident that you know exactly how you’re supposed to feel about it within each of the various tribes you participate in. Further, I’m certain that you knew what you’re supposed to feel about it without even reading it. That’s not a recipe for interesting discussion.

This is particularly hard to navigate as a full-time writer, because you’ll become more focused on the tribes of other writers and the tribes of your audiences, and your standing in both is important to your success. As an operator, those tribes will matter to you, but fitting into their expectations is not essential to your success (and your survival, if this is your primary source of financial stability). There are, of course, other tribes you have to pay attention to from your operating work, but those tribes will vary across writers, such that in aggregate they allow for a broader expression.

Invalidation events

In 2020, Ranjan Roy wrote ZIRP explains the world, which is an interesting dive into how zero interest rate policy was shaping so many dimensions of the economy. Among other things, ZIRP created the conditions for hypergrowth companies and funded the industry’s shift towards larger teams driving revenue growth rather than margins. People operating in the industry today have felt this transition in layoffs, a slower hiring process, and a notable shift in the dynamic between employees and employers.

When I meet with industry peers, we spend most of our time discussing either tactical problems related to this shift (e.g. how do we benchmark costs properly to justify engineering headcount) or wondering if we should hide in a hole for several years hoping that the industry reverts to kinder time. Despite that, I see a large swath of folks pitching ZIRP-era content and strategies to struggling leaders.

The folks still making their ZIRP-era talking points aren’t bad people, but they are giving bad advice, and it’s because they’ve failed to recognize an “invalidation event.” Good advice is grounded in accurately diagnosing circumstances, and folks operating in the industry are best positioned to update their advice because they’re directly experience the industry’s changes rather than observing them from a distance.

It’s not that non-operators don’t detect these shifts, they certainly do, but it’s exceptionally challenging to quickly build confidence in a large change when operating on second hand information. Operators get a lot wrong too, but it’s my experience that self-aware operators will get direct information earlier and be in a better position to evaluate it.

Endless topics

Writing as an operator, I have a constant source of new topics. More than just any topics, these topics are the most challenging topics that engineering organizations and companies encounter. All three of my books are directly grounded in the topics I was struggling with at the time. An Elegant Puzzle focused on the challenge of managing within a hypergrowth company. Staff Engineer documented the various ways that senior engineers were finding leadership impact outside of management roles. An Engineering Executive’s Primer tracks what I’ve learned from operating in executive roles. There’s no way that I personally could have written these without the benefit of operating in those environments.

Conversely, I see folks who leave operating roles often fall into a rut of repeating topics. They want to say something, but they’re not encountering new problems, so they fall back onto their fixed experiences in the industry and come back with the same ideas.

Writing well and frequently

Occasionally folks make the assertion that it’s hard to improve as a writer if you’re only writing part-time. There’s a kernel of truth in this observation: writing up my notes on finishing my 3rd book, Primer, I described each book that I write as a separate education. Even on my third book, I’m still learning so much about how to write books. I’m not sure the ideas are getting better, but the books containing those ideas certainly are.

That being said, I’ve found that having the space to explore in my writing has created so much room for improvement that I wouldn’t have found writing under a structured publishing schedule. Free-form writing has allowed me to write when and where I have energy, and to stop writing where I don’t have much energy (e.g. I starting work on Infrastructure Engineering and then subsequently paused it). It’s also allowed me to experiment with formats and mediums: I’ve written this blog, written books, spoken at conferences, done a YouTube recording, and so on. If I was focused on very specific outcomes, I’d likely be experimenting less and trying to “exploit” the mediums more, which would focus my learning.

It’s possible I would have improved more as a writer if I did it full-time, but I’m confident that I’m not a meaningfully worse writer due to the part-time nature of my writing. I also lightly hold the belief that I’m a better writer as a result of not writing full-time. Writing on a schedule is, in my opinion, not at all fun. Further, most of my best writing is stuff that I originally think isn’t even worth writing down, which would translate poorly into a world where I need to predictably write good stuff.

Echoing my earlier comments, not trying to convince anyone to switch sides on this topic, and many non-operating writers are quite good. There are many techniques you can use to address the above topics (e.g. maintaining an active network in industry), but generally those techniques apply equally (or better) to writers who operator (e.g. writers can probably get access to any company in the industry, but you couldn’t convince me that’s not equally true for operators outside of–maybe–getting visibility into a small pool of direct competitors).

Advancing the industry.

Thu, 07 Dec 2023 08:00:00 -0600

Early in my career, I navigated most decisions by simple hill climbing: if it was a more prestigious opportunity and paid more, I took it. As I got further, and my personal obligations grew, I started to think about navigating a 40-year career, where a given job might value pace rather than prestige. Over the last few years, what I’ve come to appreciate is that there’s another phase: purpose.

Purpose isn’t intrinsically the third phase of a career, but it certainly has been for me, as I was fixated on financial stability for most of my first decade in the industry, and then by controlling my career’s pace as we had our first child. It was only after figuring out, to a certain degree, the financial and pacing pieces that I felt like I had enough room in my life to think about purpose at all.

In my “2023 year in review” post, I mentioned the idea of “advancing the industry.” Increasingly, I believe that I have a small but real platform to improve how engineering organizations operate, and that it’s worthwhile to steer my career and hobbies such that I deliberately use that platform for good. I don’t take myself too seriously here–most of what I do on any given day doesn’t advance the industry in any way–but it’s a guiding principle for me when I think about larger professional questions like, “Should I take this job?” or “What theme should I write my next book on?”

Demonstrating how this principle played into a few decent decisions:

Before joining Carta, I thought a long time about what kind of role gives me the most leverage to impact the industry. Some friends believe that I could impact the industry more as a full-time writer, but I personally don’t believe that. First, I believe that successful engineering organizations spread their practices widely across the industry, which makes organizational leadership extremely impactful. Second, I believe that leadership roles allow you to change individuals lives around you in ways that quietly propagate across the industry. Finally, I believe that writers who operate have a unique, powerful voice in the industry.
Staff Engineer and An Engineering Executive’s Primer are both books whose potential readership is relatively constrained, but that readership is also positioned to have an exceptional impact on the industry. There’s probably an alternate topic I could have written about that sold more copies (and consequently made more money), but I don’t think there are alternate topics that would have impacted the industry more

I certainly don’t apply the lens to every decision I make, but I do apply it to most long-term professional decisions, and I find it quite helpful. Even if I go against where this principle steers me, it’s worthwhile to understand why I’m going against it.

Notes on Enterprise Architecture as Strategy

Thu, 07 Dec 2023 07:00:00 -0600

Enterprise Architecture as Strategy by Jeanne W. Ross, Peter Weill, and David C Robertson is an interesting read on how integrating technology across business units shifts the company’sstrategy landscape. Written in 2006, case studies are not particularly current but the ideas remains relevant.

The technology industry is simultaneously grasped by the optimism that things are changing constantly–your skills from last year are already out of date!–and the worry that nothing particularly important has changed since the 1970s when the unix epoch began. I opened Enterprise Architecture as Strategy by Jeanne W. Ross, Peter Weill, and David C Robertson, published in 2006, with both of those ideas firmly in mind.

Despite the age, I think this is one of the better strategy books I’ve read recently. In oarticular, it has some very relevant, structured thoughts on managing coordination across business units within a given company, which is something I’ve been thinking about quite a bit as the CTO for a company with a number of sophsticated business lines.

Core concepts

The book’s core premise (summarized on pages 8-9) is that every company should build a foundation for execution composed of:

Operating model – business process integration and standardization across business units in a company (e.g. where do you select technologies for a business unit?)
Enterprise architecture – organizing logic for business process and IT infrastructure. Essentially, how do you service shared concerns (e.g. customer database)
IT engagement model – the system of governance mechanisms that dictate how the business and IT work together to solve problems

This isn’t groundbreaking, but it is a useful framework. Most companies that I’ve worked within fail to set the rules for decision making, and instead allow ambiguous, semi-political systems to drive outcomes. That’s less true for companies as they grow larger, and the ongoing friction within multi-business unit companies eventually forces clearer rules, but this book suggests we could just specify the answers to these predictable problems instead of discovering them anew at each company.

These ideas are relatively less interesting in the context of a single business line company, where most of these concerns don’t show up nearly as often. (Although, any acquisition does introduce the questions, sometimes very abruptly.)

Operating models

The book proposes four operating models, based on a 2 by 2 grid of two concepts: high and low standardization, and high and low integration. Standardization is running different business units in the same way. Integration is depending on the availability, accuracy and timeliness of other business units’ data.

Coordination (low standardiation, high integration) – few shared implementations, but highly shared data
Unification (high standardization, high integration) – shared implementation and heavy coupling of data across business units
Diversification (low standardization, low integration) – very little alignment across business units, maybe some shared services
Replication (high standardization, low integration) – shared implementation but little shared data across business units that serve distinct and unshared customers

I’d never seen this breakdown before reading this book, and I find it a very useful vocabulary to discuss some of the challenges I have seen across business units in a company. Specifically, it’s helpful for diagnosing why two pairs of business units behave so differently from one another. One pair has low integration, and the other has high integration, but we’ve been hoping to reason about them in the same way. The friction was obvious, but how we might modify the playbook was less obvious without this vocabulary.

I particularly appreciated this quote (p43):

A poor choice of operating model–one that is not viable in a given market–will have dire consequences. But not choosing an operating model is just as risky.

Many leadership teams are so failure-averse that they try to preserve optionality by not making decisions, but generally those decisions get made anyway, at lower levels of your organization, while you sit around and pretend that you’re studying the situation at hand.

Four stages of maturity

The book introduces (p71) four stage of enterprsie architecture maturity: business silos, standardized technology, optimized core, and business modularity. I find the stages specifically a bit hard to map into my experience, likely due to the sorts of companies I’ve worked in, but it’s an interesting lens. Further, they introduce the concept of these four stages as a progression, and their belief that it’s impossible to skip phases: you most go phase by phase from left to right.

They also introduce (p105-109) a series of practices to adopt within each phase. For example, IT ownership is decentralized in phase 2 (business silos), but should be own by a single executive in phase 3 (standardized technology).

Potentially my issue is that most startups and scaleups operate in phase 3 doing their early years, and only reach phase 4 late (if ever). That said, I’m not particularly convinced that the 4th phase is an improvement over the 3rd. More generally, I didn’t find this vocabulary particularly helpful.

Leadership agenda

This sort of book inevitably feels obligated to end with recommended steps for leaders to implement their ideas, and this book is no exception. Towards the end (p195), it proposes a set of common steps such as " Analyze your existing foundation for execution." I don’t find those super helpful, as they largely recap the previous sections of the book.

They also have a handful of principles to keep in mind while implementing these changes. The three of those principles that I find most usefula re:

“Initiate Change From the Top” – the politics and stakeholder management to make these changes is immense. Conversely, many technical leaders–and even many engineers–want to anchor on the concept of bottoms-up leadership, that rejects making these sorts of top-down decisions. That, in my experience, simply doesn’t work beyond a couple hundred people, and we should focus more on good top-down leadership instead for larger teams. Don’t get me wrong – bottoms up leadership is extremely desirable when it works, but I think the industry spends too much time pretending we’re supporting bottoms-up leadership when infact we’re just absconding from our professional duties
“Don’t Skip Stages” – in the maturity model (e.g. from business silos to business modularity), there’s a natural desire to simply skip to the last stage of maturity. The observation that skipping stages generally doesn’t work is an interesting one, and something I’ll need to ponder a bit
“Implement the Foundation One Project at a Time” – the desire for transformational change often overpowers our senses, leading us to concurrent migrations that we know are very unlikely to succeed. Throttling the approach to ensure it succeeds is a recurring leadership lesson for me, and certainly resonates

Altogether, this section is worth a quick skim. It was notably less dense than the preceeding ones, but I recognize how they get edited in to make the research “more actionable.”

Case studies & surveys

This thing that impresses me the most about this book is how much data it’s built on (page ix), relying on 50+ case studies and 200+ surveys, and operating in a field of study the three authors focused on for a decade-plus.

Beyond simply the number of case studies, there’s the quality and level of detail in the case studies as well, which is very high. There simply aren’t enough books written this way, because they take so much effort to write, and I find it very inspiring to see the extent of research that went into the book.

Final thoughts

This was a facsinating read. Some of its biggest focuses were slightly dated by being almost 20 years old, but many of the core challenges still resonate, particularly needing an explicit operating model to navigate decisions across business units.

Create technical leverage: workflow improvements & product capabilities

Fri, 01 Dec 2023 05:00:00 -0600

More than a decade ago, I typed up a few paragraphs of notes, titled it “Building Technical Leverage,” and proceeded to forget about it. Those notes were from a meeting with Kevin Scott, then SVP Engineering at LinkedIn, while we wandered the Valley trying to convince potential acquirers to buy Digg. It was only this morning that I remembered that the post exists when I started trying to title this post on the same topic.

A decade later, I have accumulated more thoughts on the matter. Starting with some definitions:

Technical leverage here means “solving problems using software and software systems.” It is a subset of leverage which would also include solving problems using things like training, improving process, communication and so on
There are two major categories of technical leverage that I see in industry: workflow improvements and product capabilities
Workflow improvements are generally about improving efficiency (e.g. new code is deployed more quickly, database migrations are less likely to break production)
Product capabilities make something possible that was previously impossible, or at least an order of magnitude faster (an example of the former is a machine-learning optimized landing page that optimizes content for a given user rather than globally, an example of the later is replacing a time-intensive manual process to upload content with a wholly automated tool)

With those baseline definitions, let’s explore the topic a bit.

Workflow improvements

Workflow improvements improve your team or company’s efficiency. This can be literally making it faster (e.g. faster build times) or it might be making something slower but removing the need for human attention (e.g. canary deploys might slow down deployments but make it possible for deploys to rollout more safely without a human monitoring them).

You can often find workflow improvements by modeling them with systems thinking. (Here is an example of modeling an example system with systems thinking.)

Examples:

At both Calm and Stripe, we experimented with canary deploys such that our deployment was slower from a machine perspective, but humans were able to stop paying attention more quickly because they knew obviously bad deploys would only go out to a small number of machines and would revert automatically
At Uber, we built a system to support self-service provisioning of services, which replaced a system where services were requested and then provisioned by SRE by hand. We retained control over scaling compute resources beyond a certain threshold in production, allow us to control what we were most concerned about without slowing experimentation
At Calm, we moved to use feature flags for gating access, rather than deployments to gate access, allowing us to instantly release and revert functionality without requiring a (relatively slow) deployment

Failure modes:

Different but not better: sometimes folks convince themselves that a new solution is better, but it’s really just different. This happens most frequently when teams reason backwards from a goal (“I want to use Redis”) rather than reason forwards from a problem (“querying this infrequently changing data is overloading our primary database”)
Now you have N+1 solutions: a new solution is indeed better in some cases, but isn’t better in many other cases, such that a subset of users have a better experience, but most do not, and you’re stuck maintaining yet another solution. (This is one of many variants of a failed migration.)

Product capabilities

Product capabilities are making something possible within your product that previously wasn’t possible or make something currently possible an order-of-magnitude more efficient. This kind of innovation requires identifying something meaningfully new, investing in it to completion, and convincing users it’s worth adopting–even internally–a rare trifecta indeed.

Examples:

At one point, launching new pieces of content at Calm required significant coordination across Content, Product and Engineering teams. This meant that new product development was often interrupted by the work of launching content. We built tooling and workflows to wholly extract Product and Engineering from launching new pieces of content, while also significantly speeding up the Content team’s workflows. Before the project, much of our company’s energy was focused on releasing content. After the project, only the Content team’s energy was focused on releasing content
At one point, Calm’s Growth, Product and Content teams argued over the manual placement of new pieces of content. Placement significantly impacted content performance, Teams had conflicting goals (performance of all content vs performance of a given piece of content), which created ongoing debate around positioning content. We replaced that with a machine-learning powered system which optimized content for each user, with a content testing mechanism for new content, which allowed us to give good, new content even more reach without compromising overall performance, and it did this without human debate. We were able to get a better outcome for all parties while also eliminating a major source of coordination and tension

Failure modes:

Building capabilities for nonexistent problems: generally because the platform hopes to generate new demand for a solution as opposed to servicing existing demand generated by a current problem (e.g. content management was a source of ongoing friction at Calm, and instantly had demand for our solution; conversely at SocialCode we built a web scraping service that misdiagnosed the problem because it was driven from a technology-first perspective, solving the crawling configuration problem rather than token management which was the real source of demand)
Failing to deliver before funding dries up: usually because the approach is poorly architectured to support incremental support. Again, this often occurs because you don’t have a concrete user to build for, where you can validate approach with the specific subset they need as opposed to building the entirety to support abstract future adoption
Failing to drive adoption: there are many useful tools that are never adopted. Sometimes that is for good reasons (not reliable, too expensive), and sometimes that is for bad reasons (the two involved executives didn’t like one another). Either variety of non-adoption kills your product capability

Workflow vs capabilities

Both workflow improvements and product capabilities are valuable. Teams should select between them based on expected ROI and an honest assessment of their risk budget. If you can’t take much risk, focus on workflow improvements. Even if you can take some risk, don’t experiment concurrently with too many product capabilities–they tend to have a high failure rate, and you want to learn that quickly so you can move on to the next one.

My experience is that most Engineering organizations deeply struggle to complete the necessary trifecta of tasks to launch a product capability: identify, fund delivery, and drive adoption. There are techniques that help with this: \

Identify opportunity: core techniques of product discovery
Fund delivery: identify incremental deliverables, often by solving for specific users with limited use cases before you’re able to fund the full project
Drive adoption: build early for some of your hardest customers to derisk possibility that your solution simply won’t work for them

My rule is that product capabilities are only possible with a strong technical lead, engineering executive support, and a broader executive team that trusts the engineering executive. Without all three, very few product capabilities are delivered successfully.

Prioritize with caution

Engineering organizations should generally invest more into technical leverage, but only if they have a track record of doing so successfully. If you don’t have a track record of success, make a focused start, and build confidence that you can finish this sort of work and that it’s useful.

If you invest too much before your organization understands how to select and implement this sort of work, you’re more likely to create an ocean of technical debt than a transformational improvement in your tooling or product. The good news is that getting better is straightforward. These projects tend to fail for very boring reasons: taking on too much before delivering something for feedback, building for non-existent users, building things that are interesting instead of valuable.

Pay attention to those risks, expand your budget slowly over time, and you’ll get a feel for it. Get distracted by interesting projects that don’t solve clear problems for clear people, and you’ll have a fun quarter followed by years of cleanup.

Navigators

Fri, 24 Nov 2023 14:00:00 -0600

In Staff Engineer’s chapter on Managing Technical Quality, one of the very last suggestions is creating a centralized process to curate technical changes:

Curate technology change using architecture reviews, investment strategies, and a structured process for adopting new tools. Most misalignment comes from missing context, and these are the organizational leverage points to inject context into decision-making. Many organizations start here, but it’s the last box of tools that I recommend opening. How can you provide consistent architecture reviews without an articulated vision? Why tell folks your strategy after they’ve designed something rather than in their onboarding process?

As the quote mentions, this was the last suggestion, but not because it’s the ultimate option. Instead, it’s because I find that centralized groups–typically called the Architecture Team or something along those lines–tend to perform poorly in practice. Sometimes they’re inefficient, sometimes they make bad choices, sometimes they just don’t do much at all. (I documented some of the most common challenges these architecture teams face in Scaling Consistency.)

After I joined Carta earlier this year, we spent a while exploring areas where we could make it easier for engineers to do highly productive work. One topic that came up was technical decision making within engineering:

Many engineers disagreed with decisions being made elsewhere in the organization, particularly feeling that decisions were made inconsistently
It was open to interpretation who was accountable for technical decisions in any given part of the business (people roughly agreed, but if you asked four people who was responsible for making decisions regarding Something Important, you’d generally get at least two names)
When two teams disagreed on a technical decision, it was unclear how to resolve the issue short of escalating it to the CTO

The standard playbook here is to first document best practices, hope that’s sufficient, and then reluctantly establish an Architecture Team to manage escalations. At a company like Carta, with ~400 engineers, it’s further true that aligning on best practices is an escalation-laden exercise. That’s because best practices are context dependent, and there is no universal context in an organization of that size. Best practices are a balance between global and local optimization, and some of the decided upon best practices will certainly be worse locally for some teams.

That meant that we either needed to establish working groups for deciding each of the areas of best practice, or decide on an Architecture Team to serve as the “standing working group” for the full set of practices we wanted to codify. Carta has a large bench of very talented engineers, so we certainly could have established an Architecture Team.

There was, however, one little problem: me. I rather dislike Architecture Teams.

It’s not that Architecture teams can’t be good, I’ve worked with a number of very good ones, but even the best are slowed by consensus, and suffer the downsides of consensus-driven decision making (particularly, optimizing for acceptable decisions rather than best decisions). I’ve also found that consensus-driven groups are slow to improve, even with direct coaching, because it’s generally difficult to figure out who to coach, and accountability can be a bit amorphous.

So, instead we went with a pattern that I’ve experimented with a few times: Navigators. The core structure of Navigators are along these lines:

Each major area of the business has one Navigator (we started with about ten). That Navigator is an active contributor to the software written and operated within that area
There is exactly one Navigator for a given area; Navigators do not overlap
Each Navigator is wholly accountable for the technical decisions made in their area. More than merely being accountable, they can make decisions. This includes interpreting organizational strategy to apply it correctly within their context
In practice, most real issues are an intersection of technical, prioritization and people constraints. Navigators are responsible for aligning with the people leadership in their area when making technical decisions
Each Navigator is wholly accountable to the CTO in their decision making and interpretation of organizational strategy
Navigators are the escalation point for technical consideration within their area, and a pair of Navigators is the escalation point for cross-area issues
If a pair of Navigators are unable to resolve a cross-area concern, or a Navigator and people-manager unable to resolve a technical-social concern, then the CTO is the escalation point

We’ve been running in this mode for about six months, and I must admit that I think it’s working quite well. It’s very powerful to have a clearly accountable party that everyone agrees is accountable. There have been some messy conversations, but we know who is responsible for what, and we’ve been able to sit down and have those conversations. Afterwards, we’ve been able to improve, and do better for the next set of conversations. (I’ve rarely seen Architecture Teams improve quickly, but I have seen Navigators improve quickly.)

The Navigator pattern is so obvious, and in my experience so much better than the alternatives, that it’s worth asking ourselves why we don’t see it very often. My short answer is that I think the industry continually underestimates what senior engineers can do if we give them the necessary context and remit to do it. It’s considered a radical act to trust your engineers to make truly important decisions, but it doesn’t need to be that way. As long as we’re willing to hold engineers accountable for carrying important roles, then trusting them is no more radical than trusting anyone else in your organization: there’s some risk, but it’s a risk well worth taking.

Irrational Exuberance

Notes on how to use LLMs in your product.

Rebuild your mental model

Revamp workflows

Retrieval Augmented Generation (RAG)

Rate of innovation

Human-in-the-Loop (HITL)

Hallucinations and legal liability

“Zero to one” versus “One to N”

Copyright law

Data Processing Agreements

Provider availability

Ex-technology companies.

Leadership requires taking some risk.

When should you take risks?

Owning the risk

Obfuscated capacity

Leadership requires some risks

Friction isn't velocity.

More (self-)publishing thoughts.

Digital release of Engineering Executive's Primer.

Thesis on value accumulation in AI.

High-Context Triad.

Useful tradeoffs are multi-dimensional.

Examples

How to add dimensions

Late-career abilities

Navigating ambiguity.

Navigation process

Is the time ripe?

Do you have an executive sponsor?

Don’t get stuck

Layers of context.

How to see across layers

Passion can be blinding

Obvious or invisible

Those five spare hours each week.

Predictability.

2023 in review.

Goals

Carta & Calm

An Engineering Executive’s Primer

Other books

First keynote

Advent of Code

Angel investments

Reading

Year of personal admin

Wearing glasses

Running

Invisalign

Notes on How Big Things Get Done

“Think slow, act fast”

“Pixar planning”

Dataset of projects

Things can be inexperienced

“Honest Numbers”

Writers who operate.

Examples

Believability

Distribution shapes writing

Taste is tribal

Invalidation events

Endless topics

Writing well and frequently

Advancing the industry.

Notes on Enterprise Architecture as Strategy

Core concepts

Operating models

Four stages of maturity

Leadership agenda

Case studies & surveys

Final thoughts

Create technical leverage: workflow improvements & product capabilities

Workflow improvements

Product capabilities

Workflow vs capabilities

Prioritize with caution

Navigators