Irrational Exuberancehttps://lethain.com/Recent content on Irrational ExuberanceHugo -- gohugo.ioen-usWill LarsonThu, 18 Dec 2025 10:00:00 -07002025 in review.https://lethain.com/2025-in-review/Thu, 18 Dec 2025 10:00:00 -0700https://lethain.com/2025-in-review/<p>Yet another edition of my annual recap! This year brought my son to kindergarten, me to forty and to a new job at Imprint, my fourth book to bookstores, and a lot more time in the weeds of developing software.</p> <hr> <p><em>Previously:</em> <em><a href="https://lethain.com/2024-in-review/">2024</a>, <a href="https://lethain.com/2023-in-review/">2023</a>, <a href="https://lethain.com/2022-in-review/">2022</a>, <a href="https://lethain.com/2021-in-review/">2021</a>,</em> <em><a href="https://lethain.com/2020-in-review/">2020</a>, <a href="https://lethain.com/2019-in-review/">2019</a>, <a href="https://lethain.com/2018-in-review/">2018</a>, <a href="https://lethain.com/things-learned-in-2017/">2017</a></em></p> <h2 id="goals">Goals</h2> <p>Evaluating my goals for this year and decade:</p> <ul> <li> <p><strong>[Completed]</strong> <em>Write at least four good blog posts each year.</em></p> <p><a href="https://lethain.com/orchestration-heavy-leadership-heavy/">Moving from an orchestration-heavy to leadership-heavy management role</a>, <a href="https://lethain.com/good-eng-mgmt-is-a-fad/">Good engineering management is a fad</a>, <a href="https://lethain.com/competitive-advantage-author-llms/">What is the competitive advantage of authors in the age of LLMs?</a>, <a href="https://lethain.com/company-ai-adoption/">Facilitating AI adoption at Imprint</a></p> </li> <li> <p><strong>[Completed]</strong> <em>Write three books about engineering or leadership in 2020s.</em></p> <p>This year I finished <a href="https://craftingengstrategy.com/"><em>Crafting Engineering Strategy</em></a> with O&rsquo;Reilly. This is my third engineering book in the 2020s. More about this in the <em>Writing</em> section below.</p> </li> <li> <p><strong>[Completed]</strong> <em>Do something substantial and new every year that provides new perspective or deeper practice.</em></p> <p>After almost a decade of not submitting a substantial pull request at work, <a href="https://lethain.com/coding-at-work/">I&rsquo;ve been back in the mix since joining Imprint</a>. I&rsquo;ve submitted a solid handful of real pull requests that implement production features, and have used Claude Code widely in their creation. I&rsquo;ve missed this a lot, and have learned a bunch about developing software with LLMs.</p> </li> <li> <p><strong>[In progress]</strong> <em>20+ folks who I’ve managed or meaningfully supported move into VPE or CTO roles at 50+ person or $100M+ valuation companies.</em></p> <p>This is a decade goal ending in 2029. I previously increased the goal in 2022 from <code>3-5</code> to <code>20</code>. In 2024, the count was at <code>10</code>. Things haven&rsquo;t moved too much since then, but I&rsquo;ll refresh next year.</p> <p>I think that I&rsquo;m on track, but I will say that I think getting into these roles is markedly harder than it was three years ago. There are just fewer of these roles available recently, and they tend to be both more demanding and more difficult than the standard VPE/CTO role a few years ago.</p> </li> </ul> <p>For backstory on these goals: I originally <a href="https://lethain.com/2019-in-review/">set them in 2019</a>, and then <a href="https://lethain.com/2022-in-review/">revised them in 2022</a>. I&rsquo;ve come to believe that I should be revising these every year, but also that it&rsquo;s not that interesting to revise them every year. I&rsquo;ll revise them again in a few years.</p> <h2 id="writing">Writing</h2> <p>I finished my fourth book, <a href="https://www.amazon.com/Crafting-Engineering-Strategy-Thoughtful-Decisions/dp/B0FBRJY116"><em>Crafting Engineering Strategy</em></a>, and wrote <a href="https://lethain.com/crafting-engineering-strategy/">some notes on writing it</a>. I&rsquo;m really excited for this book to be done, because I think it&rsquo;s been a missing book in the industry, and I hope it will change how the industry thinks about &ldquo;engineering strategy.&rdquo; In particular, I hope it&rsquo;ll pull us away from the frequent gripe that &ldquo;we have no engineering strategy!&rdquo; You <em>do</em> have an engineering strategy, it&rsquo;s just not written down yet.</p> <p>As part of finishing this book, I&rsquo;ve also recognized that if I write another book, it will be far into the future. After publishing four books in six years, I&rsquo;m booked out, and I&rsquo;m pretty sure I&rsquo;ve tapped out my last decade&rsquo;s path of <a href="https://lethain.com/advancing-the-industry-pt2/">writing books to advance the industry</a>. I&rsquo;ll definitely keep writing, but it&rsquo;ll be posts focused on the stuff I&rsquo;m concretely working on, without trying to map them into a larger book structure.</p> <p>(Last year I mentioned adding <a href="https://lethain.com/high-context-triad/">The High-Context Triad</a> to a second edition of <em>Staff Engineer</em>, which I still plan to do, but I&rsquo;m not quite sure when. Probably in a few years.)</p> <h2 id="work">Work</h2> <p>I <a href="https://lethain.com/stuff-learned-at-carta/">left Carta in May</a> after two years there, and joined Imprint. Imprint has just been a lot of fun for me. I&rsquo;ve written a small number of real pull requests that implement meaningful things. That&rsquo;s something I haven&rsquo;t done since working at Uber, and aligns with my desire to be working in the details again. There&rsquo;s nothing more energizing to me than getting to solve real, concrete problems, and that&rsquo;s exactly the sort of job Imprint has been for me. I just haven&rsquo;t been spenting time on stuff like <a href="https://lethain.com/notion-agent/">implementing internal workflow agents</a> or <a href="https://lethain.com/dependabot-auto-merge/">automatically merging Dependabot pull requests</a> in a long time, and I missed it.</p> <p>It&rsquo;s also, after some years spent on making teams more efficient, been an opportunity to really hire again, which I haven&rsquo;t gotten to do since my first couple years at Calm. It&rsquo;s <a href="https://lethain.com/productivity-in-the-age-of-hypergrowth/">never easy working at a fast growing company</a>, but you do learn a lot, and quite quickly.</p> <h2 id="family">Family</h2> <p>My son entered kindergarten this year. I turned 40. My wife is starting to explore the world of fractional software development, and she&rsquo;s figuring out its rules. We&rsquo;ve had a fair amount of health issues in the immediate and extended family, but altogether everything is going well.</p> <h2 id="speaking">Speaking</h2> <p>I didn&rsquo;t do much public speaking, although I spoke on <a href="https://www.youtube.com/watch?v=RBPtGtMY8bE">Book Overflow about Staff Engineer</a>, which was a fun discussion.</p> <p>I also spoke at several private events, and recorded practice runs on YouTube of <a href="https://www.youtube.com/watch?v=IJlrX4Z4QWs&amp;t=968s&amp;pp=0gcJCTwKAYcqIYzv">Good engineering management is a fad</a> and <a href="https://www.youtube.com/watch?v=2Q98TAMoiMI">CTOs must earn the right to specialize</a>. Those are very similar talks, where I&rsquo;ve been iterating on the core idea of how engineering managers need to adapt to the current era.</p> <h2 id="reading">Reading</h2> <p>In 2024, I read 27 profession-adjacent books. In 2023, I read 11. I&rsquo;m not quite sure how many I read in 2022, because I put together a <a href="https://lethain.com/books-2022/">2019-2022 professional reading recap</a>, but it was about 50 over four years. This year I didn&rsquo;t do much professional reading, mostly because I was too busy with the new job and polishing my most recent book.</p> <p>What I did read was:</p> <ol> <li><em><a href="https://www.amazon.com/AI-Engineering-Building-Applications-Foundation/dp/1098166302">AI Engineering</a></em> by Chip Huyen</li> <li><em><a href="https://www.recodingamerica.us/">Recoding America</a></em> by Jennifer Pahlka</li> <li><em><a href="https://www.amazon.com/Facilitating-Software-Architecture-Empowering-Architectural-ebook/dp/B0DMHGWCPN/">Facilitating Software Architecture</a></em> by Andrew Harmel-Law</li> <li><em><a href="https://www.amazon.com/Turning-Flywheel-Monograph-Accompany-Great/dp/0062933795">Turning the Flywheel</a></em> by Jim Collins</li> </ol> <p>It&rsquo;s interesting to note the drop in volume, but I feel fine about it. I don&rsquo;t read to hit a goal, I read to learn or understand a particular problem, and found myself mostly working on topics that didn&rsquo;t align well with that approach this year.</p> <hr> <p>If you&rsquo;ve written something about your year, send it my way!</p>Automatically merging dependabot PRshttps://lethain.com/dependabot-auto-merge/Thu, 18 Dec 2025 08:00:00 -0700https://lethain.com/dependabot-auto-merge/<p>One of the recurring themes of software development is patching security issues. Most repository hosting services have fairly good issue reporting at this point, but many organizations still struggle to apply those fixes in a timely fashion. This past week we were discussing how to reduce the overhead of this process, and I was curious: can you just auto-merge <a href="https://docs.github.com/en/code-security/getting-started/dependabot-quickstart-guide">Github Dependabot pull-requests</a>?</p> <p>It turns out, [the answer is yes], and it works pretty well. You get control over which types of updates (patches, minor updates, major updates, etc) you want to auto-merge, and it will also respect your automated checks. If you have great CI/CD that runs blocking linting, typing and tests, then this works particularly well. If you don&rsquo;t, then, well, this will be an effective mechanism to get you to having good linting, typing, and tests afer traversing a small ocean of tears.</p> <p>I got this running for about a dozen repositories at work over the past few days, but I&rsquo;ll show an example of setting up the same mechanism for my blog.</p> <p>First, add a <code>.github/workflows/dependabot-auto-merge.yml</code> file to your repository that looks like this:</p> <pre tabindex="0"><code># Automatically approve and merge Dependabot PRs for minor and patch updates name: Dependabot auto-merge on: pull_request permissions: contents: write pull-requests: write jobs: dependabot: runs-on: ubuntu-latest if: github.event.pull_request.user.login == &#39;dependabot[bot]&#39; &amp;&amp; github.repository == &#39;lethain/irrational_hugo&#39; steps: - name: Dependabot metadata id: metadata uses: dependabot/fetch-metadata@v2 with: github-token: &#34;${{ secrets.GITHUB_TOKEN }}&#34; - name: Approve Dependabot PR run: gh pr review --approve &#34;$PR_URL&#34; env: PR_URL: ${{ github.event.pull_request.html_url }} GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - name: Enable auto-merge for Dependabot PRs if: steps.metadata.outputs.update-type == &#39;version-update:semver-patch&#39; || steps.metadata.outputs.update-type == &#39;version-update:semver-minor&#39; run: gh pr merge --auto --squash &#34;$PR_URL&#34; env: PR_URL: ${{ github.event.pull_request.html_url }} GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} </code></pre><p>Then go to your repository settings (something like <code>https://github.com/lethain/irrational_hugo/settings</code>), and enable auto-merging for your repository. This still respects all required branch rules, like required test passes or approvals, etc.</p> <div class="ba b--light-gray"> <p><img src="https://lethain.com/static/blog/2025/dep-automerge.png" alt=""></p> </div> <p>Then make sure you have appropriate status checks for whatever linting, typing and tests you have in your repository.</p> <div class="ba b--light-gray"> <p><img src="https://lethain.com/static/blog/2025/dep-status-checks.png" alt=""></p> </div> <p>Then enable Dependabot (something like <code>https://github.com/lethain/irrational_hugo/settings/security_analysis</code>). Even the default settings are just find.</p> <div class="ba b--light-gray"> <p><img src="https://lethain.com/static/blog/2025/dep-enable-prs.png" alt=""></p> </div> <p>Then you&rsquo;re done. The PRs from dependabot will automatically merge going forward. There are lots of nuances here&ndash;I already found one issues that automatically merged despite an issue because of a missing test&ndash;but ultimately I think that&rsquo;s valuable pressure to improve the testing quality, rather than a reason to avoid, or backtrack, on the approach.</p>Facilitating AI adoption at Imprinthttps://lethain.com/company-ai-adoption/Sun, 07 Dec 2025 08:00:00 -0700https://lethain.com/company-ai-adoption/<p>I&rsquo;ve been working on internal &ldquo;AI&rdquo; adoption, which is really LLM-tooling and agent adoption, for the past 18 months or so. This is a problem that I think is, at minimum, a side-quest for every engineering leader in the current era. Given the sheer number of folks working on this problem within their own company, I wanted to write up my “working notes” of what I’ve learned.</p> <p>This isn&rsquo;t a recommendation about what you should do, merely a recap of how I&rsquo;ve approached the problem thus far, and what I&rsquo;ve learned through ongoing iteration. I hope the thinking here will be useful to you, or at least validates some of what you&rsquo;re experiencing in your rollout. The further you read, the more specific this will get, ending with cheap-turpentine-esque topics like getting agents to reliably translate human-readable text representations of Slack entities into <code>mrkdwn</code> formatting of the correct underlying entity.</p> <div class="bg-light-gray br4 ph3 pv1"> <p><strong>I am hiring:</strong> If you&rsquo;re interested in working together with me on internal agent and AI adoption at Imprint, we are hiring our founding <a href="https://jobs.ashbyhq.com/imprint/2c83c162-40d4-4b27-91e5-a1a2e81262ab">Senior Software Engineer, AI</a>. The ideal candidate is a product engineer who&rsquo;s spent some time experimenting with agents, and wants to spend the next year or two digging into this space.</p> </div> <h2 id="prework-building-my-intuition">Prework: building my intuition</h2> <p>As technologists, I think one of the basics we owe our teams is spending time working directly with new tools to develop an intuition for how they do, and don&rsquo;t work. AI adoption is no different.</p> <p>Towards that end, I started with a bit of reading, especially Chip Huyen&rsquo;s <em><a href="https://www.amazon.com/AI-Engineering-Building-Applications-Foundation/dp/1098166302">AI Engineering</a></em>, and then dove in a handful of bounded projects: building <a href="https://lethain.com/our-own-agents-our-own-tools/">my rudimentary own agent platform</a> using Claude code for implementation, creating <a href="https://lethain.com/library-mcp/">a trivial MCP for searching my blog posts</a>, and <a href="https://lethain.com/notion-agent/">an agent to comment on Notion documents</a>.</p> <p>Each of these projects was two to ten hours, and extremely clarifying. Tool use is, in particular, something that seemed like magic until I implemented a simple tool-using agent, at which point it become something extremely non-magical that I could reason about and understand.</p> <h2 id="our-ai-adoption-strategy">Our AI adoption strategy</h2> <p>Imprint&rsquo;s general approach to refining AI adoption is <a href="https://lethain.com/testing-strategy-iterative-refinement/">strategy testing</a>: identify a few goals, pick an initial approach, and then iterate rapidly in the details until the approach genuinely works. In an era of crushing optics, senior leaders immersing themselves in the details is one of our few defenses.</p> <div class="ba b--light-gray"> <p><img src="https://lethain.com/static/blog/2025/ai-strategy.png" alt="First draft of Imprint&rsquo;s strategy for AI adoption"></p> </div> <p>Shortly after joining, I partnered with the executive team to draft the above strategy for AI adoption. After a modest amount of debate, the pillars we landed on were:</p> <ol> <li><strong>Pave the path to adoption</strong> by removing obstacles to adoption, especially things like having to explicitly request access to tooling. There&rsquo;s significant internal and industry excitemetn for AI adoption, and we should believe in our teams. If they aren&rsquo;t adopting tooling, we predominantly focus on making it <em>easier</em> rather than spending time being skeptical or dismissive of their efforts towards adoption.</li> <li><strong>Opportunity for adoption is everywhere</strong>, rather than being isolated to engineering, customer service, or what not. To become a company that widely benefits from AI, we need to be solving the problem of adoption across all teams. It&rsquo;s not that I believe we should take the same approach everywhere, but we need <em>some</em> applicable approach for each team.</li> <li><strong>Senior leadership leads from the front</strong> to ensure what we&rsquo;re doing is genuinely useful, rather than getting caught up in what we&rsquo;re measuring.</li> </ol> <p>As you see from those principles, and my earlier comment, my biggest fear for AI adoption is that they can focus on creating the impression of adopting AI, rather than focusing on creating additional productivity. Optics are a core part of any work, but almost all interesting work occurs where optics and reality intersect, which these pillars aimed to support.</p> <hr> <p>As an aside, in terms of the <a href="https://lethain.com/components-of-eng-strategy/">components of strategy</a> in <em><a href="https://craftingengstrategy.com/">Crafting Engineering Strategy</a></em>, this is really just the <a href="https://lethain.com/policy-for-strategy/">strategy&rsquo;s policy</a>. In addition, we used <a href="https://lethain.com/testing-strategy-iterative-refinement/">strategy testing</a> to refine our approach, defined a concrete set of initial actions to operationalize it (they&rsquo;re a bit too specific to share externally), and did some brief <a href="https://lethain.com/exploring-for-strategy/">exploration</a> to make sure I wasn&rsquo;t overfitting on my prior work at Carta.</p> <h2 id="documenting-tips--tricks">Documenting tips &amp; tricks</h2> <p>My first step towards adoption was collecting as many internal examples of tips and tricks as possible into a single Notion database. I took a very broad view on what qualified, with the belief that showing many different examples of using tools&ndash;especially across different functions&ndash;is both useful and inspiring.</p> <div class="ba b--light-gray"> <p><img src="https://lethain.com/static/blog/2025/ai-tips.png" alt="The image is a table listing AI tips and trainings with columns for the name of the tip and the relevant team. It includes topics like using Claude Code, adding Slack bots, and employing ChatGPT for marketing."></p> </div> <p>I&rsquo;ve continued extending this, with contributions from across the company, and it&rsquo;s become a useful resource for both humans and bots alike to provide suggestions on approaching problems with AI tooling.</p> <h2 id="centralizing-our-prompts">Centralizing our prompts</h2> <p>One of my core beliefs in our approach is that making prompts discoverable within the company is extremely valuable. Discoverability solves four distinct problems:</p> <ol> <li>Creating visibility into what prompt&rsquo;s can do (so others can be inspired to use them in similar scenarios). For example, that you can use our agents to comment on a Notion doc when it&rsquo;s created, respond in Slack channels effectively, triage Jira tickets, etc</li> <li>Showing what a good prompt looks like (so others can improve their prompts). For example, you can start moving complex configuration into tables and out of lists which are harder to read and accurately modify</li> <li>Serving as a repository of copy-able sections to reuse across prompts. For example, you can copy one of our existing &ldquo;Jira-issue triaging prompts&rdquo; to start triaging a new Jira project</li> <li>Prompts are joint property of a team or function, not the immutable construct of one person. For example, anyone on our Helpdesk team can improve the prompt responding to Helpdesk requests, not just one person with access to the prompt, and it&rsquo;s not locked behind being comfortable with Git or Github (although I do imagine we&rsquo;ll end up with more restrictions around editing our most important internal agents over time)</li> <li>Identifying repeating prompt sub-components that imply missing or hard to use tools. For example, earlier versions of our prompts had a lot of confusion around how to specify Slack users and channels, which I got comfortable working around, but others did not</li> </ol> <p>My core approach is that <em>every</em> agent&rsquo;s prompt is stored in a single Notion database which is readable by everyone in the company. <em>Most</em> prompts are editable by everyone, but some have editing restrictions.</p> <p>Here&rsquo;s an example of a prompt we use for routing incoming Jira issues from Customer Support to the correct engineering team.</p> <div class="ba b--light-gray"> <p><img src="https://lethain.com/static/blog/2025/ai-routing-prompt.png" alt="The image provides instructions for triaging Jira tickets, detailing steps for retrieving comments, updating labels, and determining responsible teams. It includes guidelines for using Slack for communication and references, and lists teams with their on-call aliases and areas of responsibility."></p> </div> <p>Here&rsquo;s a second example, this time of responding to requests in our Infrastructure Engineering team&rsquo;s request channel.</p> <div class="ba b--light-gray"> <p><img src="https://lethain.com/static/blog/2025/ai-infra-prompt.png" alt="The image contains detailed instructions for service desk agents on handling Slack messages related to access requests for tools such as AWS, VPN, NPM, and more. It provides step-by-step guidelines for different scenarios, including retrieving user IDs, handling specific requests, and directing users to appropriate resources or teams."></p> </div> <p>Pretty much all prompts end with an instruction to include a link to the prompt in the generated message. This ensures it&rsquo;s easy to go from a mediocre response to the prompt-driving the response, so that you can fix it.</p> <h2 id="adopting-standard-platform">Adopting standard platform</h2> <p>In addition to collecting tips and prompts, the next obvious step for AI adoption is identifying a standard AI platform to be used within the company, e.g. ChatGPT, Claude, Gemini or what not.</p> <p>We&rsquo;ve gone with OpenAI for everyone. In addition to standardizing on a platform, we made sure account provisioning was automatic and in place on day one. To the surprise of no one who&rsquo;s worked in or adjacent to IT, a lot of revolutionary general AI adoption is&hellip; really just account provisioning and access controls. These are the little details that can so easily derail the broader plan if you don&rsquo;t dive into them.</p> <p>Within Engineering, we also provide both Cursor and Claude. That said, the vast majority of our Claude usage is done via AWS Bedrock, which we use to power Claude Code&hellip; and we use Claude Code quite a bit.</p> <h2 id="other-ai-tooling">Other AI tooling</h2> <p>While there&rsquo;s a general industry push towards adopting more AI tooling, I find that a significant majority of &ldquo;AI tools&rdquo; are just SaaS vendors that talk about AI in their marketing pitches. We have continued to adopt vendors, but have worked internally to help teams evaluate which &ldquo;AI tools&rdquo; are meaningful.</p> <p>We&rsquo;ve spent a fair amount of time going deep on integrating with AI tooling for chat and IVR tooling, but that&rsquo;s a different post entirely.</p> <h2 id="metrics">Metrics</h2> <p>Measuring AI adoption is, like all measurement topics, fraught. Altogether, I&rsquo;ve found measuring tool adoption very useful for identifying the right questions to ask. Why <em>haven&rsquo;t</em> you used Cursor? Or Claude Code? Or whatever? These are fascinating questions to dig into. I try to look at usage data at least once a month, with a particular focus on two questions:</p> <ol> <li>For power adopters, what are they actually doing? Why do they find it useful?</li> <li>For low or non-adopters, why aren&rsquo;t they using the tooling? How could we help solve that for them?</li> </ol> <p>At the core, I believe folks who aren&rsquo;t adopting tools are rational non-adopters, and spending some time understanding the (appearance of) resistance goes further than top-down mandate. I think it&rsquo;s often an education gap that is bridged easily enough. Conceivably, at some point I&rsquo;ll discover a point of diminishing returns, where the lack of progress is stymied on folks who are rejecting AI tooling&ndash;or because the AI tooling isn&rsquo;t genuinely useful&ndash;but I haven&rsquo;t found that point yet.</p> <h2 id="building-internal-agents">Building internal agents</h2> <p>The next few sections are about building internal agents. The core implementation is a single stateless lambda which handles a wide variety of HTTP requests, similar-ish to Zapier. This is currently implemented in Python, and is roughly 3,000 lines of code, much of it dedicated to oddities like formatting Slack messages, etc.</p> <p>For the record, I did originally attempt to do this <a href="https://lethain.com/commenting-notion-open-ai-zapier/">within Zapier</a>, but I found that Zapier simply doesn&rsquo;t facilitate the precision I believe is necessary to do this effectively. I also think that Zapier isn&rsquo;t particularly approachable for a non-engineering audience.</p> <h3 id="what-has-fueled-adoption-especially-for-agents">What has fueled adoption (especially for agents)</h3> <p>As someone who spent a long time working in platform engineering, I still <em>want to believe</em> that you can build a platform, and users will come. Indeed, I think it&rsquo;s true that a small number of early adopters <em>will come</em>, if the problem is sufficiently painful for them, as was the case for <a href="https://lethain.com/uber-service-migration-strategy/">Uber&rsquo;s service migration (2014)</a>.</p> <p>However, what we&rsquo;ve found effective for driving adoption is basically the opposite of that. What&rsquo;s really worked is the intersection of platform engineering and old-fashioned product engineering:</p> <ol> <li>(product eng) find a workflow with a lot of challenges or potential impact</li> <li>(product eng) work closely with domain experts to get the first version working</li> <li>(platform eng) ensure that working solution is extensible by the team using it</li> <li>(both) monitor adoption as indicator of problem-solution fit, or lack thereof</li> </ol> <p>Some examples of the projects where we&rsquo;ve gotten traction internally:</p> <ul> <li>Writing software with effective <code>AGENTS.md</code> files guiding use of tests, typechecking and linting</li> <li>Powering initial customer questions through chat and IVR</li> <li>Routing chat bots to steer questions to solve the problem, provide the the answer, or notify the correct responder</li> <li>Issue triaging for incoming tickets: tagging them, and assigning them to the appropriate teams</li> <li>Providing real-time initial feedback on routine compliance and legal questions (e.g. questions which occur frequently and with little deviation)</li> <li>Writing weekly priorities updates after pulling a wide range of resources (Git commits, Slack messages, etc)</li> </ul> <p>For all of these projects that have worked, the formula has been the opposite of &ldquo;build a platform and they will come.&rdquo; Instead it&rsquo;s required deep partnership from folks with experience building AI agents and using AI tooling to make progress. The learning curve for effective AI adoption in important or production-like workflows remains meaningfully high.</p> <h3 id="configuring-agents">Configuring agents</h3> <p>Agents that use powerful tools represent a complex configuration problem. First, exposing too many tools&ndash;especially tools that the prompt author doesn&rsquo;t effectively understand&ndash;makes it very difficult to create reliable workflows. For example, we have an <code>exit_early</code> command that allows terminating the agent early: this is very effective in many cases, but is also easy to break your bot. Similarly, we have a <code>slack_chat</code> command that allows posting across channels, which can support a variety of useful workflows (e.g. warm-handoffs of a question in one channel into a more appropriate alternative), but can also spam folks. Second, as tools get more powerful, they can introduce complex security scenarios.</p> <p>To address both of these, we currently store configuration in a code-reviewed Git repository. Here&rsquo;s an example of a JIRA project.</p> <div class="ba b--light-gray"> <p><img src="https://lethain.com/static/blog/2025/ai-jira-config.png" alt="This image shows a configuration script for a Jira setup with specified project keys, a prompt ID, a list of allowed tools such as &ldquo;notion_search&rdquo; and &ldquo;slack_chat,&rdquo; and a model set to &ldquo;gpt-4.1&rdquo;. The configuration also has a setting &ldquo;respond_to_issue&rdquo; set to False."></p> </div> <p>Here&rsquo;s another for specifying a Slack responder bot.</p> <div class="ba b--light-gray"> <p><img src="https://lethain.com/static/blog/2025/ai-slack-config.png" alt="This image shows a code snippet configuring a channel in Slack for &ldquo;eng-new-hires,&rdquo; with specified Slack channel IDs, a Notion prompt ID, and a list of allowed tools like &ldquo;notion_search&rdquo; and &ldquo;jira_search_jql.&rdquo; The model specified is &ldquo;gpt-4.1.&rdquo;"></p> </div> <p>Compared to a JSON file, we can statically type the configuration, and it&rsquo;s easy to extend over time. For example, we might want to extend <code>slack_chat</code> to restrict which channels a given bot is allowed to publish into, which would be easy enough. For most agents today, the one thing not under Git-version control is the prompts themselves, which are versioned by Notion. However, we can easily require specific agents to use prompts within the Git-managed repository for sensitive scenarios.</p> <p>After passing tests, linting and typechecking, the configurations are automatically deployed.</p> <h3 id="resolving-foreign-keys">Resolving foreign keys</h3> <p>It&rsquo;s sort of funny to mention, but one thing that has in practice really interfered with easily writing effective prompts is making it easy to write things like <code>@Will Larson</code> which is then translated into <code>&lt;@U12345&gt;</code> or whatever the appropriate Slack identifier is for a given user, channel, or user group. The same problem exists for Jira groups, Notion pages and databases, and so on.</p> <p>This is a good example of where centralizing prompts is useful. I got comfortable pulling the unique identifiers myself, but it became evident that most others were not. This eventually ended with three tools for Slack resolution: <code>slack_lookup</code> which takes a list of references to lookup, <code>slack_lookup_prefix</code> which finds all Slack entities that start with a given prefix (useful to pull all channels or groups starting with <code>@oncall-</code>, for example, rather than having to hard-code the list in your prompt), and <code>slack_search_name</code> which uses string-distance to find potential matches (again, useful for dealing with typos).</p> <p>If this sounds bewildering, it&rsquo;s largely the result of Slack not exposing relevant APIs for this sort of lookup. Slack&rsquo;s APIs want to use IDs to retrieve users, groups and channels, so you have to maintain your own cache of these items to perform a lookup. Performing the lookups, especially for users, is itself messy. Slack users have a minimum of three ways they might be referenced: <code>user.profile.display_name</code>, <code>user.name</code>, and <code>user.real_name</code>, only a subset of which are set for any given user. The correct logic here is, as best I can tell, to find a match against <code>user.profile.display_name</code>, then use that if it exists. Then do the same for <code>user.name</code> and finally <code>user.real_name</code>. If you take the first user that matches one of those three, you&rsquo;ll use the wrong user in some scenarios.</p> <p>In addition to providing tools to LLMs for resolving names, I also have a final mandatory check for each response to ensure the returned references refer to real items. If not, I inject which ones are invalid into the context window and perform an additional agent loop with only entity-resolution tools available. This feels absurd, but it was only at this point that things really started working consistently.</p> <hr> <p>As an aside, I was embarassed by these screenshots, and earlier today I made the same changes for Notion pages and databases as I had previously for Slack.</p> <h2 id="formatting">Formatting</h2> <p>Similarly to foreign entity resolution, there&rsquo;s a similar problem with <a href="https://docs.slack.dev/messaging/formatting-message-text/">Slack&rsquo;s <code>mrkdwn</code> variant of Markdown</a> and JIRA&rsquo;s <a href="https://developer.atlassian.com/cloud/jira/platform/apis/document/structure/">Atlassian Document Format</a>: they&rsquo;re both strict.</p> <p>The tools that call into those APIs now have strict instructions on formatting. These <em>had</em> been contained in individual prompts, but they started showing up in every prompt, so I knew I needed to bring them into the agent framework itself rather than forcing every prompt-author to understand the problem.</p> <p>My guess is that I need to add a validation step similar to the one I added for entity-resolution, and that until I do so, I&rsquo;ll continue to have a small number of very infrequent but annoying rendering issues, To be honest, I personally don&rsquo;t mind the rendering issues, but that creates a lot of uncertainty for others using agents, so I think solving them is a requirement.</p> <h3 id="logging-and-debugging">Logging and debugging</h3> <p>Today, all logs, especially tool usage, are fed into two places. First, it goes into Datadog for full logging visibility. Second, and perhaps more usefully for non-engineers, they feed into a Slack channel, <code>#ai-logs</code> which create visibility into which tools are used and with which (potentially truncated) parameters.</p> <p>Longer term, I imagine this will be exposed via a dedicated internal web UX, but generally speaking I&rsquo;ve found that the subset of folks who are actively developing agents are pretty willing to deal with a bit of cruft. Similarly the folks who aren&rsquo;t developing agents directly don&rsquo;t really care, they want it to work perfectly every time, and aren&rsquo;t spending time looking at logs.</p> <h2 id="biggest-remaining-gap-universal-platform-for-accessing-user-scope-mcp-servers">Biggest remaining gap: universal platform for accessing user-scope MCP servers</h2> <p>The biggest internal opportunity that I see today is figuring out how to get non-engineers an experience equivalent to running Claude Code locally with all their favorite MCP servers plugged in. I&rsquo;ve wanted ChatGPT or Claude.ai to provide this, but they don&rsquo;t <em>really</em> quite get there, Claude Desktop is close, but is somewhat messy to configure as we think about finding a tool that we can easily allow everyone internally to customize and use on a daily basis.</p> <p>I&rsquo;m still looking for what the right tool is here. If anyone has any great suggestions that we can be somewhat confident will still exist in two years, and don&rsquo;t require sending a bunch of internal data to a very early stage company, then I&rsquo;m curious to hear!</p> <h2 id="whats-next">What&rsquo;s next?</h2> <p>You&rsquo;re supposed to start a good conclusion with some sort of punchy anecdote that illuminates your overall thesis in a new way. I&rsquo;m not sure if I can quite meet that bar, but the four most important ideas for me are:</p> <ol> <li>We are still very early on AI adoption, so focusing on rate of learning is more valuable than anything else</li> <li>If you want to lead an internal AI initiative, you simply <em>must</em> be using the tools, and <em>not</em> just ChatGPT, but building your own tool-using agent using only an LLM API</li> <li>My experience is that <em>real</em> AI adoption on <em>real</em> problems is a complex blend of: domain context on the problem, domain experience with AI tooling, and old-fashioned IT issues. I&rsquo;m deeply skeptical of any initiative for internal AI adoption that doesn&rsquo;t anchor on all three of those. This is an advantage of earlier stage companies, because you can often find aspects of all three of those in a single person, or at least across two people. In larger companies, you need three different <em>organizations</em> doing this work together, this is just objectively hard</li> <li>I think model selection matters a lot, but there are only 2-3 models you need at any given moment in time, and someone can just tell you what those 2-3 models are at any given moment. For example, <a href="https://platform.openai.com/docs/models/gpt-4.1">GPT-4.1</a> is just exceptionally good at following rules quickly. It&rsquo;s a <em>great</em> model for most latency-sensitive agents</li> </ol> <p>I&rsquo;m curious what other folks are finding!</p>Coding at work (after a decade away).https://lethain.com/coding-at-work/Wed, 19 Nov 2025 08:00:00 -0700https://lethain.com/coding-at-work/<p>Since joining Imprint a bit over six months ago as the CTO of a ~50 engineer team, I&rsquo;ve merged 104 pull requests, which is slightly over four per week. Many of them are very minimal configuration and documentation tweaks, and none were <em>the hardest</em> or even <em>most time-sensitive</em> task available at any given time; I&rsquo;m much more of a pull request scavenger finding opportunities that don&rsquo;t disrupt the operating teams&rsquo; rhythms.</p> <p>That said, a decent chunk represent meaningful software development tasks, and these 104 pull requests are more pull requests than I&rsquo;ve completed <em>combined</em> in my prior decade of work, where Uber was the last time I was substantially submitting pull requests (well, whatever Phabricator called its pull request equvialent). Since then, I&rsquo;ve been predominantly managing managers, and <em>not</em> writing software with my own hands at work.</p> <p>This has been a fascinating shift, and I wanted to write dome some thoughts on whether this is good for software, good for me, whether it&rsquo;s fun, and how I&rsquo;ve personally worked to adapt to <a href="https://lethain.com/good-eng-mgmt-is-a-fad/">the current era</a>.</p> <hr> <p><em>Unrelatedly, I did enjoy Peter Seibel&rsquo;s <a href="https://codersatwork.com/">Coders at Work</a> (2009).</em></p> <h2 id="dubious-return-on-effort-of-manager-coding">Dubious return-on-effort of manager coding</h2> <p>I was a computer science major, and worked as a software engineer prior to becoming an engineering manager. I continued to write software when I was a line manager, and continued to write small projects in my free time after becoming a manager. The idea of writing software at work has always appealing to me.</p> <p>However, when I became a manager of managers, I stopped writing software at work. I <em>wanted</em> to write software, but it simply felt like a lower return on time than doing something else. For example, if I focusd my time hiring another engineer onto the team, they would undoubtedly do more than I would. Similarly, if I spent time improving our plans, that would have a higher impact than me writing a piece of software. Ultimately, I got stuck in the trap that it was always clear that writing software would be valuable, but it would always be less valuable than another endeavor. Each hour I spent writing software was bad for the business overall, and a sign of questionable judgment.</p> <p>The same wasn&rsquo;t true when it came to <em>understanding</em> the codebase, and I&rsquo;ve tried&ndash;to various degrees of success&ndash;to build a workable degree of awareness of the codebases I&rsquo;ve managed. That being said, since I left line management roles, I&rsquo;ve never been truly successful at doing this beyond the superficial level. My experience is that only writing software can build a truly effective understanding of <a href="https://lethain.com/reclaim-unreasonable-software/">unreasonable software</a>, and that most startup software is unreasonable due to its origin of being iteratively designed by a shifting cast of characters with an evolving understanding of the domain they are modeling.</p> <p>You can get very far with a minimal understanding of a codebase and surrounding yourself with strong engineers who you can ask useful questions to and get honest answers. For the big things, this is generally sufficient, but it means you can&rsquo;t really engage with the small questions without spending engineers&rsquo; time on it or making low-judgment decisions.</p> <p>Whether manager coding is valuable comes down to whether you believe making large decisions more quickly with fewer interrupts for context pulling&ndash;and the ability to make numerous small decisions that are otherwise too expensive to make effectively&ndash;builds into meaningfully more impactful management. This isn&rsquo;t a proclimation for others, but for me, I&rsquo;ve really enjoyed getting back to writing software at work, in large part because the time commitment to do so has dropped significantly over the past couple years.</p> <h2 id="finding-small-pockets-of-time-to-write-software">Finding small pockets of time to write software</h2> <p>The biggest specific challenge for me when it came to writing software as a manager was finding blocks of thinking time to either understand a problem, or to implement the solution. Even simple fixes require an effective mental model of the codebase being worked on.</p> <p>This becomes inescapably true when you&rsquo;re focused on long-term impact of your work. For example, adding a bunch of tests <em>might</em> be useful, but it&rsquo;s often the case that poorly designed tests get throw away over time because they overlap with other existing tests, are flaky, or are slow in aggregate. I think traditionally, a lot of manager coding has fallen into this bucket of optically useful with somewhat dubious long-term value. Doing high quality work simply requires too complete a mental model for folks jumping in and out of writing software.</p> <p>The new wave of AI tooling like Claude Code or OpenAI Codex are extremely susceptive to creating low-quality commits, but my experience is that used effectively they also provide several opportunities for creating useful code contributions are well. They are effective at:</p> <ol> <li>answering questions into a codebase, e.g. &ldquo;what are our most common patterns for working with authn and authz? what are good examples of each?&rdquo;</li> <li>writing code that fits the existing codebase&rsquo;s patterns and structure, particularly with a well-written <code>AGENTS.md</code>&rsquo;s guidance</li> <li>taking general feedback to revise the approach, e.g. &ldquo;look for existing utilities that solve date math within our codebase and reuse those&rdquo;</li> </ol> <p>Most importantly, you can do each of those in a few minutes at a time. Between meetings at work, I generally pop back into one of several Claude Code sessions to see where it got to on a given task, review the code, and suggest next steps.</p> <p>It&rsquo;s worth acknowledging that there&rsquo;s a significant learning curve to doing this well. I&rsquo;ve spent a meaningful amount of time in the last year learning to write software this way, and each month there are new caveats that I&rsquo;ve had to understand. Slowly but surely, I&rsquo;ve built a mental model of both how writing software with AI works, and how Imprint&rsquo;s codebases work.</p> <p>As I&rsquo;ve gotten more knowledgable at both, I&rsquo;ve refound my ability to write software at work because I can make progress in the small chunks of time between other projects, combined with an hour or two over the week to think more deeply about my approach to more complex issues.</p> <h2 id="judgment--problem-selection">Judgment / problem selection</h2> <p>While new AI tools make it easier than ever for managers to write useful software at work, they also make it easier than ever to write unhelpful software at work. Particularly in senior roles, it&rsquo;s very easy to write software that makes you feel helpful, but is genuinely unhelpful to the team, and leaves them busier than they were before.</p> <p>The handful of rules that I&rsquo;ve found useful for myself here are:</p> <ol> <li> <p>Never contribute to something that is truly time sensitive unless it&rsquo;s a very constrained thing that I can solve end-to-end <em>today</em>. When that isn&rsquo;t the case, I&rsquo;m usually going to slow things down despite trying to help.</p> </li> <li> <p>Prioritize projects that are hard for teams to get to, but are obviously valuable over time. For example, technical debt clean up, small user-requested features, or missing instrumentation.</p> <p>Infrequently take on a strategic company project which doesn&rsquo;t have an owner, and which I am able to complete <em>myself</em>. A real example of this has been writing our first pass of a new <a href="https://en.wikipedia.org/wiki/Interactive_voice_response">Interactive Voice Response</a> built on an AI agent, which was clearly valuable but difficult to prioritize over the team&rsquo;s existing work.</p> </li> <li> <p>Hold myself to a higher bar than I would hold others in terms of fully releasing my software, monitoring its release, and solving bugs it creates. If I don&rsquo;t have time to do those things, then I am stealing time from the team rather than helping them.</p> </li> <li> <p>Err towards implementing feedback in pull requests, even if I consider it generally neutral. The cost of giving feedback to me is so high for folks, that I&rsquo;m responsible for going out of my way to incorporate it when it is given.</p> </li> </ol> <p>These rules <em>have</em> meant that I didn&rsquo;t work on some projects that I wanted to, but I think they&rsquo;ve done a fairly good job of allowing me to build my judgment about how our software works without getting in the way of the teams who are doing the vast majority of the heavy lifting. I&rsquo;m sure there are better versions of these rules, but in general I&rsquo;d guess that managers ignoring them are close to the border of being unhelpful.</p> <h2 id="should-you-be-coding-at-work">Should you be coding at work?</h2> <p>I&rsquo;m pretty sure that &ldquo;Should managers be coding at work?&rdquo; isn&rsquo;t nearly as interesting a question as folks want it to be, and a meaningful answer depends on each situation&rsquo;s details. However, what&rsquo;s been clearly true for me is that the overhead of writing software at work is substantially lower than it was a few years ago. If you weren&rsquo;t writing software at work because it simply took too much time away from managing the team directly: the constraints have shifted in a profound way after you learn the new wave of tooling.</p> <p>The learning loop for writing software with agents is definitely not zero, but it&rsquo;s something you can learn for a few dollars of tokens and a couple dozen hours spent writing personal projects that are safe to throw away afterwards. That feels like a worthwhile investment to remain effective in one&rsquo;s chosen profession.</p>"Good engineering management" is a fadhttps://lethain.com/good-eng-mgmt-is-a-fad/Sun, 26 Oct 2025 04:00:00 -0700https://lethain.com/good-eng-mgmt-is-a-fad/<p>As I get older, I increasingly think about whether I&rsquo;m spending my time the right way to advance my career and my life. This is also a question that your company asks about you every performance cycle: is this engineering manager spending their time effectively to advance the company or their organization?</p> <p>Confusingly, in my experience, answering these nominally similar questions has surprisingly little in common. This piece spends some time exploring both questions in the particularly odd moment we live in today, where managers are being told they&rsquo;ve spent the last decade doing the wrong things, and need to engage with a new model of engineering management in order to be valued by the latest iteration of the industry.</p> <div class="bg-light-gray br4 ph3 pv1"> <p>If you&rsquo;d be more interested in a video version of this, here is the recording of a practice run I gave for a talk centered on these same ideas (<a href="https://docs.google.com/presentation/d/17lTreuVdYMNOr7k2XLzrshEJnB-StaNUzAyh9tE0b5w/edit?slide=id.g39f551c2725_0_0#slide=id.g39f551c2725_0_0">slides from talk</a>).</p> <iframe width="560" height="315" src="https://www.youtube.com/embed/IJlrX4Z4QWs?si=m9HgL7y0AvdOSUq6" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> </div> <h2 id="good-leadership-is-a-fad">Good leadership is a fad</h2> <p>When I started my software career at Yahoo in the late 2000s, I had two 1:1s with my manager over the course of two years. The first one came a few months after I started, and he mostly asked me about a colleague&rsquo;s work quality. The second came when I gave notice that I was leaving to <a href="https://lethain.com/digg-v4/">join Digg</a>. A modern evaluation of this manager would be scathing, but his management style closely resembled that of the team leader in <a href="https://www.amazon.com/Soul-New-Machine-Tracy-Kidder/dp/0316491977"><em>The Soul of A New Machine</em></a>: identifying an important opportunity for the team, and navigating the broader organization that might impede progress towards that goal. He was, in the context we were working in, an effective manager.</p> <p>Compare that leadership style to the expectations of the 2010s, where attracting, retaining, and motivating engineers was emphasized as the most important leadership criteria in many organizations. This made sense in <a href="https://lethain.com/productivity-in-the-age-of-hypergrowth/">the era of hypergrowth</a>, where budgets were uncapped and many companies viewed hiring strong engineers as their constraint on growth. This was an era where managers were explicitly told to stop writing software as the first step of their transition into management, and it was good advice! Looking back we can argue it was bad guidance by today&rsquo;s standards, but it aligned the managers with the leadership expectations of the moment.</p> <p>Then think about our current era, that started in late 2022, where higher interest rates killed <a href="https://www.readmargins.com/p/zirp-explains-the-world">zero-interest-rate-policy (ZIRP)</a> and productized large language models are positioned as killing deep Engineering organizations. We&rsquo;ve flattened Engineering organizations where many roles that previously focused on coordination are now expected to be hands-on keyboard, working deep in the details. Once again, the best managers of the prior era&ndash;who did exactly what the industry asked them to do&ndash;are now reframed as bureaucrats rather than integral leaders.</p> <p>In each of these transitions, the business environment shifted, leading to a new formulation of ideal leadership. That makes a lot of sense: of course we want leaders to fit the necessary patterns of today. Where things get weird is that in each case a morality tale was subsequently superimposed on top of the transition:</p> <ul> <li>In the 2010s, the morality tale was that it was all about empowering engineers as a fundamental good. Sure, I can get excited for that, but I don&rsquo;t really believe that narrative: it happened because hiring was competitive.</li> <li>In the 2020s, the morality tale is that bureaucratic middle management have made organizations stale and inefficient. The lack of experts has crippled organizational efficiency. Once again, I can get behind that&ndash;there&rsquo;s truth here&ndash;but the much larger drivers aren&rsquo;t about morality, it&rsquo;s about ZIRP-ending and optimism about productivity gains from AI tooling.</li> </ul> <p>The conclusion here is clear: the industry will want different things from you as it evolves, and it will tell you that each of those shifts is because of some complex moral change, but it&rsquo;s pretty much always about business realities changing. If you take any current morality tale as true, then you&rsquo;re setting yourself up to be severely out of position when the industry shifts again in a few years, because &ldquo;good leadership&rdquo; is just a fad.</p> <div class="bg-light-gray br4 ph3 pv1"> <p>Earlier this summer, I also gave a presentation at the YCombinator CTO summit on this specific topic of the evolution of engineering management. You can watch a <a href="https://www.youtube.com/watch?v=2Q98TAMoiMI">recorded practice run of <strong>that</strong> talk on YouTube</a> as well, and <a href="https://docs.google.com/presentation/d/1Du-oNGoN92mMQWj8EeS36ktOrDqxZbbhAmNexYkjod4/edit?slide=id.g24afdb76500_0_415#slide=id.g24afdb76500_0_415">see the slides</a>.</p> </div> <h2 id="self-development-across-leadership-fads">Self-development across leadership fads</h2> <p>If you accept the argument that the specifically desired leadership skills of today are the result of fads that frequently shift, then it leads to an important followup question: what are the right skills to develop in to be effective today and to be impactful across fads?</p> <p>Having been and worked with engineering managers for some time, I think there are <a href="https://lethain.com/categories-engineering-leadership/">eight foundational engineering management skills</a>, which I want to personally group into two clusters: core skills that are essential to operate in all roles (including entry-level management roles), and growth skills whose presence&ndash;or absence&ndash;determines how far you can go in your career.</p> <p>The core skills are:</p> <ol> <li> <p><strong>Execution</strong>: lead team to deliver expected tangible and intangible work. Fundamentally, management is about getting things done, and you&rsquo;ll neither get an opportunity to begin managing, nor stay long as a manager, if your teams don&rsquo;t execute.</p> <p><em>Examples</em>: ship projects, manage on-call rotation, sprint planning, manage incidents</p> </li> <li> <p><strong>Team</strong>: shape the team and the environment such that they succeed. This is <em>not</em> working for the team, nor is it working for your leadership, it is finding the balance between the two that works for both.</p> <p><em>Examples</em>: hiring, coaching, performance management, advocate with your management</p> </li> <li> <p><strong>Ownership</strong>: navigate reality to make consistent progress, even when reality is difficult Finding a way to get things done, rather than finding a way that it not getting done is someone else&rsquo;s fault.</p> <p><em>Examples</em>: doing hard things, showing up when it&rsquo;s uncomfortable, being accountable despite systemic issues</p> </li> <li> <p><strong>Alignment</strong>: build shared understanding across leadership, stakeholders, your team, and the problem space. Finding a realistic plan that meets the moment, without surprising or being surprised by those around you.</p> <p><em>Examples</em>: document and share top problems, and updates during crises</p> </li> </ol> <p>The growth skills are:</p> <ol> <li> <p><strong>Taste</strong>: exercise discerning judgment about what &ldquo;good&rdquo; looks like—technically, in business terms, and in process/strategy. Taste is a broadchurch, and my experience is that broad taste is an somewhat universal criteria for truly senior roles. In some ways, taste is a prerequisite to Amazon&rsquo;s <a href="https://www.amazon.jobs/content/en/our-workplace/leadership-principles">Are Right, A Lot</a>.</p> <p><em>Examples</em>: refine proposed product concept, avoid high-risk rewrite, find usability issues in team’s work</p> </li> <li> <p><strong>Clarity</strong>: your team, stakeholders, and leadership know what you&rsquo;re doing and why, and agree that it makes sense. In particular, they understand how you are overcoming your biggest problems. So clarity is not, &ldquo;Struggling with scalability issues&rdquo; but instead &ldquo;Sharding the user logins database in a new cluster to reduce load.&rdquo;</p> <p><em>Examples</em>: identify levers to progress, create plan to exit a crisis, show progress on implementing that plan</p> </li> <li> <p><strong>Navigating ambiguity</strong>: work from complex problem to opinionated, viable approach. If you&rsquo;re given an extremely messy, open-ended problem, can you still find a way to make progress? (I&rsquo;ve <a href="https://lethain.com/navigating-ambiguity/">written previously about this topic</a>.)</p> <p><em>Examples</em>: launching a new business line, improving developer experience, going from 1 to N cloud regions</p> </li> <li> <p><strong>Working across timescales</strong>: ensure your areas of responsibility make progress across both the short and long term. There are many ways to appear successful by cutting corners today, that end in disaster tomorrow. Success requires understanding, and being accountable for, how different timescales interact.</p> <p><em>Examples</em>: have an explicit destination, ensure short-term work steers towards it, be long-term rigid and short-term flexible</p> </li> </ol> <p>Having spent a fair amount of time pressure testing these, I&rsquo;m pretty sure most effective managers, and manager archetypes, can be fit into these boxes.</p> <h3 id="self-assessing-on-these-skills">Self-assessing on these skills</h3> <p>There&rsquo;s no perfect way to measure anything complex, but here are some thinking questions for you to spend time with as you assess where you stand on each of these skills:</p> <ol> <li><strong>Execution</strong> <ul> <li>When did your team last have friction delivering work? Is that a recurring issue?</li> <li>What’s something hard you shipped that went really, really well?</li> <li>When were you last pulled onto solving a time-sensitive, executive-visible project?</li> </ul> </li> <li><strong>Team</strong> <ul> <li>Who was the last strong performer you hired?</li> <li>Have you retained your strongest performers?</li> <li>What strong performers want to join your team?</li> <li>Which peers consider your team highly effective?</li> <li>When did an executive describe your team as exceptional?</li> </ul> </li> <li><strong>Ownership</strong> <ul> <li>When did you or your team overcome the odds to deliver something important? (Would your stakeholders agree?)</li> <li>What’s the last difficult problem you solved that stayed solved (rather than reoccurring)?</li> <li>When did you last solve the problem first before addressing cross-team gaps?</li> </ul> </li> <li><strong>Alignment</strong> <ul> <li>When was the last time you were surprised by a stakeholder? What could you do to prevent that reoccuring?</li> <li>How does a new stakeholder understand your prioritization tradeoffs (incl rationale)?</li> <li>When did you last disappoint a stakeholder without damaging your relationship?</li> <li>What stakeholders would join your company because they trust you?</li> </ul> </li> <li><strong>Taste</strong> <ul> <li>What’s a recent decision that is meaningfully better because you were present?</li> <li>If your product counterpart left, what decisions would you struggle to make?</li> <li>Where’s a subtle clarification that significantly changed a design or launch?</li> <li>How have you inflected team’s outcomes by seeing around corners?</li> </ul> </li> <li><strong>Clarity</strong> <ul> <li>What’s a difficult trade-off you recently helped your team make?</li> <li>How could you enable them to make that same trade-off without your direct participation?</li> <li>What’s a recent decision you made that was undone? How?</li> </ul> </li> <li><strong>Navigating ambiguity</strong> <ul> <li>What problem have you worked on that was stuck before assisted, and unstuck afterwards?</li> <li>How did you unstick it?</li> <li>Do senior leaders bring ambiguous problems to you? Why?</li> </ul> </li> <li><strong>Working across timescales</strong> <ul> <li>What’s a recent trade off you made between short and long-term priorities?</li> <li>How do you inform these tradeoffs across timescales?</li> <li>What long-term goals are you protecting at significant short-term cost?</li> </ul> </li> </ol> <p>Most of these questions stand on their own, but it&rsquo;s worth briefly explaining the &ldquo;Have you ever been pulled into a SpecificSortOfProject by an executive?&rdquo; questions. My experience is that in most companies, executives will try to poach you onto their most important problems that correspond to your strengths. So if they&rsquo;re never attempting to pull you in then either you&rsquo;re not considered as particularly strong on that dimensions, or you&rsquo;re already very saturated with other work such that it doesn&rsquo;t seem possible to pull you in.</p> <h3 id="are-core-skills-the-same-over-time">Are &ldquo;core skills&rdquo; the same over time?</h3> <p>While those groupings of &ldquo;core&rdquo; and &ldquo;growth&rdquo; skills are obvious groupings to me, what I came to appreciate while writing this is that some skills swap between core to growth as the fads evolve. Where <em>execution</em> is a foundational skill today, it was less of a core skill in the hypergrowth era, and even less in the investor era.</p> <p>This is the fundamentally tricky part of succeeding as an engineering manager across fads: you need a sufficiently broad base across each of these skills to be successful, otherwise you&rsquo;re very likely to be viewed as a weak manager when the eras unpredictably end.</p> <h2 id="stay-energized-to-stay-engaged">Stay energized to stay engaged</h2> <p>The &ldquo;<a href="https://lethain.com/frameworks-decision-making/">Manage your priorities and energy</a>&rdquo; chapter in <a href="https://www.amazon.com/Engineering-Executives-Primer-Impactful-Leadership/dp/1098149483/"><em>The Engineering Executive&rsquo;s Primer</em></a> captures an important reality that took me too long to understand: the perfect allocation of work is not the mathematically ideal allocation that maximizes impact. Instead, it&rsquo;s the balance between that mathematical ideal and doing things that energize you enough to stay motivated over the long haul. If you&rsquo;re someone who loves writing software, that might involve writing a bit more than helpful to your team. If you&rsquo;re someone who loves streamlining an organization, it might be improving a friction-filled process that is a personal affront, even if it&rsquo;s not causing <em>that much</em> overall inefficiency.</p> <h2 id="forty-year-career">Forty-year career</h2> <p>Similarly to the question of prioritizing activities to stay energized, there&rsquo;s also understanding where you are in your career, an idea I explored in <a href="https://lethain.com/forty-year-career/">A forty-year career</a>.</p> <p><img src="https://lethain.com/static/blog/2019/40-year-hero.png" alt="Diagram of different ways to prioritize roles across your career"></p> <p>For each role, you have the chance to prioritize across different dimensions like pace, people, prestige, profit, or learning. There&rsquo;s no &ldquo;right decision,&rdquo; and there are always tradeoffs. The decisions you make early in your career will compound over the following forty years. You also have to operate within the constraints of your life today and your possible lives tomorrow. Early in my career, I had few responsibilities to others, and had the opportunity to work extremely hard at places like Uber. Today, with more family responsibilities, I am unwilling to make the tradeoffs to consistently work that way, which has real implications on how I think about which roles to prioritize over time.</p> <p>Recognizing these tradeoffs, and making them deliberately, is one of the highest value things you can do to shape your career. Most importantly, it&rsquo;s extremely hard to have a career at all if you don&rsquo;t think about these dimensions and have a healthy amount of self-awareness to understand the tradeoffs that will allow you to stay engaged over half a lifetime.</p>Crafting Engineering Strategy!https://lethain.com/crafting-engineering-strategy/Sat, 25 Oct 2025 04:00:00 -0700https://lethain.com/crafting-engineering-strategy/<p>On November 3rd, 2023, I posted <a href="https://lethain.com/publishing-eng-execs-primer/">Thoughts on writing and publishing <em>Primer</em></a> to celebrate the completion of my work on my prior book, <em><a href="https://www.amazon.com/Engineering-Executives-Primer-Impactful-Leadership/dp/1098149483/">The Engineering Executive&rsquo;s Primer</a></em>. Three weeks later, I posted <a href="https://lethain.com/strategy-notes/">Engineering strategy notes</a> on November 21st, 2023, as I started to pull together thoughts to write my upcoming book, <em><a href="https://craftingengstrategy.com/">Crafting Engineering Strategy</a></em>.</p> <p>Those initial thoughts turned into my first chapter draft, <a href="https://lethain.com/llm-adoption-strategy/">How should you adopt LLMs?</a> on May 14th, 2024. Writing continued all the way through the <a href="https://lethain.com/api-deprecation-strategy/">Stripe API deprecation strategy</a>, which was my final <em>draft</em>, completed the afternoon of April 5th, 2025. In between there were another 35 chapters: two of which were <a href="https://lethain.com/wardley-mapping/">Wardley maps</a>, four of which are <a href="https://lethain.com/strategy-systems-modeling/">systems models</a>, and two of which are short section introductions.</p> <p>This post is a collection of notes on writing <em>Crafting Engineering Strategy</em>, how I decided to publish it, ways that I did and did not use foundational models in writing it, and so on.</p> <div class="bg-light-gray br4 ph3 pv1"> <p><em>Buy on <a href="https://www.amazon.com/dp/B0FBRJY116">Amazon</a>. Read online on <a href="https://craftingengstrategy.com/">craftingengstrategy.com</a>.</em></p> </div> <h2 id="why-write-this-book">Why write this book?</h2> <p>One of my decade goals for the 2020s, last updated in my <a href="https://lethain.com/2024-in-review/">2024 year in review</a>, is to write three books on some aspect of engineering. I published <em>An Elegant Puzzle</em> in 2019, so that one doesn&rsquo;t count, but have two other books published in the 2020s: <em>Staff Engineer</em> in 2021, and <em>The Engineering Executive&rsquo;s Primer</em> in 2024. So I knew I needed one more to complete this decade goal. At one point, I was planning on finishing <em><a href="https://infraeng.dev/">Infrastructure Engineering</a></em> as my fourth book, but honestly I&rsquo;ve lost traction on that topic a bit over the past few years.</p> <p>Instead, the thing I&rsquo;d been thinking about a lot instead was engineering strategy. I&rsquo;ve written chapters on this topic in my last two books, and each of those chapter was tortured by the sheer volume of things I wanted to write. By the end of 2024, I&rsquo;d done enough strategy work myself that I was pretty confident I could take the best ideas from <em>Staff Engineer</em>&ndash;anchoring the essays in amazing stories&ndash;and the topic I&rsquo;d spent so much time on over the past there years, and turn it into a pretty good book.</p> <p>It&rsquo;s also a topic that, like <em>Staff Engineer</em> in 2020 when I started writing it, was missing an anchor book pulling the ideas together for discussion. There are <em>so many</em> great books about strategy out there. There <em>are</em> some great books on engineering strategy, but few of them are widely read. My aspiration in writing this book was to potentially write that anchor book. It&rsquo;ll take a decade or two to determine whether or not I&rsquo;ve succeeded. At the minimum I think this book will either become that anchor or annoy someone enough that they write a better anchor instead. Either way, I&rsquo;ll mark it as a win for advancing the industry&rsquo;s discussion a bit.</p> <h2 id="llm-optimized-edition">LLM-optimized edition</h2> <div class="bg-light-gray br4 ph3 pv1"> <p><em>Buy the <a href="https://www.amazon.com/dp/B0FXN2J4PJ">AI Companion to Crafting Engineering Strategy</a> on Amazon.</em></p> </div> <p>While I was writing this book, I was also increasingly using foundational models at work. Then I was using them to write tools for managing this book. Over time, I became increasingly fascinated with the idea of having a version of this book optimized for usage with large language models.</p> <p>For this book in particular, the idea that you could collaborate with it on creating your own engineering strategy was a very appealing idea. I&rsquo;m excited that this ultimately translated into an LLM-optimized edition that <a href="https://www.amazon.com/dp/B0FXN2J4PJ">you can also purchase on Amazon</a>, or you can read the <a href="https://craftingengstrategy.com/aic/preface/">AI companion&rsquo;s text on craftingengineeringstrategy.com</a>, although you&rsquo;ll have to buy the actual book to get the foundational model optimized file itself (e.g. a markdown version of the book that plays well with foundational models).</p> <p>This is the culmination of <a href="https://lethain.com/competitive-advantage-author-llms/">my thinking about the advantage of authors in the age of LLMs</a>, where I see a path to book turning into things that are both read and also collaborated with.</p> <p>Regarding the foundational model optimized-version, at its core, this is just running <a href="https://repomix.com/">repomix</a> against the repository of Markdown files, but there are a number of interesting optimizations for a better product. For example, it&rsquo;s adding absolute paths to every referenced file and link, including adding a link to each chapter at its beginning to help models detect to them as references.</p> <p>It&rsquo;s also translating images into text descriptions. For example, here&rsquo;s an image from one of the chapters.</p> <p><img src="https://lethain.com/static/blog/strategy/uber-provis-model-errors.png" alt="Original image of a systems model"></p> <p>Then here is the representation after I wrote a script to pull out every image and replace it with a description.</p> <p><img src="https://lethain.com/static/blog/2025/txt-description-image-llm.png" alt="Example of gpt-4o translating an image from book into text description"></p> <p>My guess is that many books are going to move in the direction of having an LLM-optimized edition, and I&rsquo;m quite excited to see where this goes. I&rsquo;ll likely work on an LLM-optimized edition of <em>Staff Engineer</em> at some point in the near-ish future as well.</p> <h2 id="role-of-llms">Role of LLMs</h2> <p>When non-authors talk about LLMs making it easier to write books, they often think about LLMs <em>literally writing books</em>. As a fairly experienced author, I have absolutely no interest in an LLM writing any part of my book. If I don&rsquo;t bring something unique to the book, the sort of thing that an LLM would not bring, then generally either it isn&rsquo;t a book worth writing or I am the wrong author to write it. As a result, I wrote every word in this book. I didn&rsquo;t use LLMs to expand bullets into paragraphs. I didn&rsquo;t use LLMs to outline or assemble topics. I didn&rsquo;t use LLMs to find examples to substantiate my approach. Again, this is what I know how to do fairly well at this point.</p> <p>However, there are many things that I&rsquo;m not that good at, where I relied heavily on an LLM. In particular, I used LLMs to copy-edit for typos, grammatical errors and hard-to-read sentences. I also used LLMs to write several scripts that I used in writing this book:</p> <ol> <li> <p><code>grammar.py</code> which sent one or more chapters to an LLM for grammatical and spelling correction, returning the identified errors as regular expressions that I could individually accept or reject</p> </li> <li> <p><code>import.py</code> which translated from my blog post version of the posts into the <a href="https://craftingengstrategy.com/">craftingengstrategy.com</a> optimized version</p> </li> <li> <p><a href="https://lethain.com/links-script-book/"><code>links.py</code></a> which I used for standardizing format of chapter references, and balancing the frequency that strategies were referenced</p> </li> <li> <p>Generated <a href="https://craftingengstrategy.com/faq/">craftingengstrategy.com&rsquo;s FAQ</a> by attaching full LLM-optimized version to Anthropic Claude 3.7 thinking and running this prompt:</p> <pre><code> I want to make a frequently asked questions featuring topics from this book. What are twenty good question and answer pairs based on this book, formatted as markdown (questions should be H2, and answers in paragraphs). Answers should include links to relevant chapters at the end of each answer. Some example questions that I think would be helpful are: 1. Is engineering strategy a real thing? 2. How is engineering strategy different from strategy in general? 3. What are examples of engineering strategy? 4. What template should I use to create an engineering strategy? 5. Can engineers do engineering strategy? Is engineering strategy only for executives? 6. How to get better at engineering strategy? 7. Are there jobs in engineering strategy? 8. What are other engineering strategy resources? Please directly answer those 8 questions, and then include another 10-15 question/answer pairs as well. </code></pre> <p>This worked quite well, in my opinion. Generating a better FAQ than the one I created for <a href="https://staffeng.com/faq/">Staff Engineer&rsquo;s FAQ</a> in a very small amount of time. SEO seems well and truly &ldquo;cooked&rdquo; based on this experience.</p> </li> </ol> <p>Just like I don&rsquo;t see software engineers being replaced by LLMs, I don&rsquo;t see authors being replaced either.</p> <h2 id="craftingengstrategycom">craftingengstrategy.com</h2> <p>For <em>Staff Engineer</em>, I put up <a href="https://staffeng.com/">staffeng.com</a>. For <em>The Engineering Executive&rsquo;s Primer</em> I just posted things on this blog, without making a dedicated, standalone site. For <em>Crafting Engineering Strategy</em>, I decided to put together a dedicated site again, which maybe deserves an explanation.</p> <p>My writing over the past few years is anchored in my goal of <a href="https://lethain.com/advancing-the-industry/">advancing the industry</a>, and I think that having a well-structured, referencable version of the book online is a valuable part of that. If people want to recommend someone read <em>Staff Engineer</em>, they can simply point them to staffeng.com, and they can read the whole book. They might also buy it, which is great, but it&rsquo;s fine if they don&rsquo;t. <em>The Engineering Executive&rsquo;s Primer</em> is just much less referencable online, so someone ultimately has to decide to buy it before testing the content, which undermines its ability to advance the industry.</p> <p>For the record, that wasn&rsquo;t an O&rsquo;Reilly limitation, just poor planning on my part. <em>An Elegant Puzzle</em> didn&rsquo;t require a standalone site, so I thought that <em>Primer</em> wouldn&rsquo;t either, but I think that&rsquo;s more a reflection of <em>An Elegant Puzzle</em> being a collection of this blog&rsquo;s writing in the 2010s rather than the best way to support the typical book.</p> <p>At this point, my experience is that most books benefit from having a dedicated site, and that it doesn&rsquo;t detract from sales numbers. Rather, if properly done with clear calls to action on the site, it supports sales very effectively.</p> <h2 id="decision-to-publish-vs-self-publish">Decision to publish vs self-publish</h2> <p>I worked with O&rsquo;Reilly to publish this book, same as I did for my previous book, <em>The Engineering Executive&rsquo;s Primer</em>. My continued experience is that it&rsquo;s harder to create a book with a publisher, the financial outcomes are significantly muted, but the book itself is almost always a much better book than you would have created on your own.</p> <p>For me, that was the right tradeoff for this book.</p> <h2 id="my-last-book-of-2020s">My last book of 2020s</h2> <p>I&rsquo;m not at all sure this is the last book that I&rsquo;ll ever write, but I&rsquo;ve completed my decade writing goal for the 2020s, and I&rsquo;m committed to this being my last book of the 2020s. For the past seven years I have been continually writing, editing or promoting a book, and I am exhausted with that process. I&rsquo;ve <em>loved</em> getting to do it, and am grateful for having had the chance to do this four times. But, I&rsquo;m still exhausted with it!</p> <p>I could imagine eventually having another book to write, and that is definitely not now. Instead I want to spend more time with my family, my son, personally writing software professionally (rather than exclusively leading teams doing the writing), and other projects that are anything but writing another book.</p>