Irrational Exuberance content on Irrational ExuberanceHugo -- gohugo.ioen-usWill LarsonMon, 07 Jun 2021 07:00:00 -0700Can senior leaders make friends at work?, 07 Jun 2021 07:00:00 -0700 <p>Chatting with a friend recently, they asked a question that I’ve spent time wondering about as well, “Can senior leaders have friends at work?” It’s reductive to pretend there’s one universal answer to this question, but most folks find it increasingly challenging to have friends at work as you get more senior. There are multiple factors at work, and it’s interesting to dig into it a bit.</p> <p>The biggest reason is that your peers have already developed a full life outside of work. Earlier in their career folks tend to look for their work to serve a large role in their lives (employment, friends, a source of identity), but work typically plays a narrower role in their lives after they’ve been working for ten-plus years. They’re open to developing friendships at work, they just tend to get squeezed out by existing commitments.</p> <p>I’ve also found that tenured managers eventually run into perspective-altering events at work that make them cautious about developing work friendships. A few from my own experience: an acquaintance who interviewed with us, wasn’t given an offer, and took their life shortly thereafter; a friend who I managed, started giving performance feedback, a brief punctuation when they threw their resignation paper in my face, and never speaking again; hiring your mentor, them struggling, giving them feedback, and them abruptly disengaging from you; ending up managing a former peer who takes it poorly. Each of these taught me a bit about the obstacles that work friendships face where other friendships generally wouldn’t.</p> <p>Even when they are interested and have time to build friendships at work, leadership roles often come with circumstances that make friendships difficult to build. Rapid growth changes the folks you’re working with frequently. Many leaders foster teams that compete with each other for individual success rather than collaborating together for joint success. Some teams are filled with genuinely kind, helpful folks who nonetheless are fairly transaction rather than relationship driven. All of these can make it difficult to form friendships, even if everyone involved is lovely.</p> <p>If you’ve read that and are still hoping to develop more friendships, most of the time you can cultivate an environment that supports developing more work friendships. Schedule more team events. Prioritize collaborative work together. Just be deliberate about it, and be careful to avoid putting too much energy into it if it doesn’t work. My experience is that you’ll find you’re still developing friends at work, just at a slower rate and inconsistent pace.</p>An interlude., 11 May 2021 21:58:09 -0700 <p>A few months ago around nine on a Wednesday evening, my vision blurred, and I lost my sense of balance. It seemed like a fine time to go to bed. When I attempted to explain my predicament to my wife, it turned out I’d also lost my ability to communicate verbally. I was having a stroke. My wife&ndash;wiser than I and besieged by the mismatched words that I believed constituted communication&ndash;collected our infant son, coaxed me into the car, and we were at the nearby hospital a few minutes later.</p> <p>With Covid protocols in effect, we separated at intake, and I spent the evening alone as I drifted from the emergency room to MRI machine and back again a few times. The situation felt surprisingly abstract. Probably because I wasn’t thinking particularly well, my wife had a different experience. Our son had his own experience, frustrated to be woken up ten hours before his opportunity to wake up us.</p> <p>Slowly I felt a bit better, and tapped out a few messages to my wife. It also felt very important to explain my imminent absence from work, and I spent an unknowable duration tapping, erasing, and retapping a message before finally sending it. Writing these texts was a challenge. In one sense, this was because my brain was in the early stages of recovering from asphyxiation, but in a more concrete sense it was because my left thumb kept sliding from the space bar to the return key.</p> <p>With the morning shift change, I was moved into the neuro floor, passed a swallow test, and control of language slowly returned to me. Our son was with his nanny, and my wife took off work to stay with me during visiting hours. I was at the hospital for two and a half days getting worked up with a bit of everything, including another MRI, several ultrasounds, and a spinal tap. One perk of being at a teaching hospital was receiving the doctor’s very first spinal tap. Which didn’t require much bravery from me&ndash;no one asked my opinion about it&ndash;but impressed me with the true spectacle of learning to be a doctor: imagine learning so much in public!</p> <p>My test count grew due to an absence rather than a risk. Results were pretty normal for someone of my age, an age that is rather young for experiencing a stroke. As they ran out of tests to run, they let me go home. There are few joys like having the last IV removed from your arm after several days in the hospital. Although I’ll admit walking out of the hospital with your wife is a similar joy. Another is going home to your son after spending your first nights sleeping more than a wall removed.</p> <p>The early days at home were rough. This was mostly due to the spinal tap taking a bit longer to close, which made keeping food down difficult for three or fours days. The mental piece was hard too. I read bad fiction on my phone. I struggled, mostly failed, to parent. A few days later I tried using a computer and was overwhelmed. So many windows, all filled with different things. It ended up taking a bit over a week at home before I could start using a computer again. Once I was back at it, I did the smallest bits of writing. I slowly migrated my blog to a simpler setup over the course of three days. After two weeks, I went back to work.</p> <p>I could have taken longer to recuperate, but I think folks who suggest taking longer as a solution miss one of the hardest parts of recovering from a stroke: you want to know how much of your previous life remains available. Will I still be able to do my job? Will I still be able to write? Will I have lost that conceptual spark behind all of it that I’ve wasted so much time being proud and vain about? Will I become a burden to my family? The answer to all these questions is only discoverable by returning.</p> <p>Perhaps most frustrating is that your own confidence and belief regarding the outcome have a large influence in deciding your outcome. You have to take the rest you need. Then you simply have to decide that you aren’t changed. To decide that you’re still the capable person you were before. Well, that’s the beautiful dream, anyway. Some would suggest it’s difficult to retain a vivid memory of your own perfection while also spending every afternoon sleeping your first week back because you’re too exhausted to continue working. But it got better. Each week, a bit better. Fewer forgotten names. Energy levels returning. Better.</p> <p>At some point the question shifts away from returning to who you were, and instead to loving whoever you are now. I’m fortunate that the difference between the past and present feels faint, although in some ways I wonder if that’s simply a newfound inability to perceive the distance.</p> <p>It’s hard to say what someone should take from this experience! There’s a lot of merit in taking nothing away from it: random events happen. Personally, I’ve tried to simply be grateful for all the blessings around me: my wife, my son, my family, access to health care and insurance, a supportive team and workplace, financial stability to weather some of the potential storms. Ultimately, there’s no right way to come to terms with life’s events, we all have to find our own way forward.</p> <p>As part of finding my way forward, I&rsquo;ve been writing less since my stroke. I don&rsquo;t think that will last forever, hopefully it won&rsquo;t even last long, but for now most of my energy is going towards the places it&rsquo;s needed the most.</p>Mailbag: Should we just call them architects?, 26 Mar 2021 05:00:00 -0700 <p>A couple of days ago, I got another question about <em><a href="">Staff Engineer</a></em>, which felt worth digging into a bit:</p> <blockquote> <p>I started reading your new book <em>Staff Engineer</em> and wondering if you can write about your thoughts on the difference between what we know as Technical Architect vs Staff Engineer. It looks like the big companies ditched the &ldquo;Architect&rdquo; title and invented the &ldquo;Staff Engineer.&rdquo; The more I read your book, the more I feel like Staff Engineers are plain old tech architects. Having set up an architect group that reports to the CTO office based on this: <a href=""></a> , I am now confused if I should rename the team :)</p> </blockquote> <p>I hadn&rsquo;t read the linked <a href="">Who Needs an Architect</a> by Martin Fowler, so I started by working through that. The core arguments I read were:</p> <ul> <li>The term &ldquo;software architect&rdquo; means different things to different people</li> <li>Despite that term confusion, there&rsquo;s still software architecture to be done</li> <li>Some &ldquo;architects&rdquo; prioritize mentorship, others decision-making</li> <li>Despite the numerous definitions of &ldquo;Architect,&rdquo; the term is generally understood to refer to the decision-making oriented variety</li> </ul> <p>I agree with all those points, and eighteen years later (the article was written in 2003!), the tactics are clearer: use the Staff-plus engineer titles rather than the architect title.</p> <p>When I worked at Yahoo!, the career ladder went from Senior Engineer to Architect, and from there, the architect titles got increasingly fancy. Software Architect? Sure. Senior Software Architect? A good step. Distinguished Architect? Yeah, most definitely. These titles do a good job of conveying seniority, but they also suffer from artificial specificity. There are many folks out there doing work at the level of a Senior Software Architect who are <em>not</em> doing typical architecture.</p> <p>The Staff-plus sequence of titles decouples seniority from archetype and is more durable for it. You can assess someone&rsquo;s seniority from their title: Staff engineer, Principal engineer, Distinguished engineer, perhaps with some varieties like Senior Staff engineer thrown in for very large organizations. You can separately understand someone&rsquo;s mode of operation by documenting someone&rsquo;s <a href="">archetype</a>: solver, architect, tech lead, or right hand.</p> <p>If you were feeling argumentative, you could have a title for every archetype, but I don&rsquo;t think that would work well in practice. There are two reasons why this is hard. First, archetypes create clarity by creating distinctions, but reality tends to be blurry. Very few folks mix &ldquo;tech lead&rdquo; and &ldquo;right hand&rdquo;, but many &ldquo;tech leads&rdquo; spend some time on doing &ldquo;architect&rdquo; work. This makes evaluating folks a nuanced activity. Second. creating four <em>useful</em> career ladders is surprisingly challenging. It&rsquo;s very possible to add specializing paragraphs to a couple of blocks explaining how a &ldquo;solver&rdquo; might create impact differently than an &ldquo;architect.&rdquo; However, I don&rsquo;t know of <em>any</em> company that operates different ladders for each archetype successfully. Performance ratings seem simple but are nuanced and frustrating at scale across thousands of people.</p> <p>Winding back to the specific question, I would ask the current tech architect team what they want. Long term, I think you&rsquo;d be better off with the Staff-plus titles, but that sort of change needs to happen at the organization level and might take a while. Short term, it&rsquo;s probably not worthwhile to change titles again if the change happened recently unless the team is already excited for the change. If you don&rsquo;t make the change now, slot it into the organizational backlog and figure out when it might make sense later.</p>RSS feed changing! Migrating blog in next few days., 25 Mar 2021 06:00:00 -0700

<p><strong>tl;dr - subscribe to RSS via <code>/feeds.xml</code> instead of <code>/feeds/</code></strong></p> <p>The current version of this blog has been running for about four years, since I wrote <a href="">Notes from fifth blog rewrite</a>. It&rsquo;s been running as a Golang service that loaded posts and files from S3, which has worked out surprisingly well for most purposes. However, there are a few things that have taken a bit of effort to keep working. In particular have spent a few hours each year on the GCP Kubernetes cluster, the Docker builds, and LetsEncrypt for SSL. My actual implementation ended up being quite similar to <a href="">Hugo</a>, and over the last bit I decided to bite the bullet to migrate to Hugo.</p> <p>The new implementation is pure Hugo, served over SSL on <a href=""></a>, hosted via a Github page, and continuously deploying via Github pages. I&rsquo;ve always practiced a <a href="">Cool URIs don&rsquo;t change</a> strategy with this blog, and I&rsquo;ve done my best to keep the URIs consistent in this migration as well. There are a few URIs that don&rsquo;t make much sense anymore, for example <code>/all-posts/</code> will redirect to <code>/</code> since all posts are now available on <code>/</code> directly, but generally things are still where they were.</p> <p>The one exception that&rsquo;s been tricky is RSS feed, which more than a decade ago I decided to host at <code>/feeds/</code>, in addition to supporting a number of filters like <code>/feeds/tags/os-x</code> to only get an RSS feed restricted to a given tag. I stopped supporting the filtered RSS feeds some time ago, but still served the general RSS feed to any URI starting with <code>/feeds/</code>.</p> <p>My migration plan is roughly:</p> <ol> <li>Update the existing Golang service to serve the RSS at <code>/feeds.xml</code> in addition to the existing locations</li> <li>Post this entry explaining the RSS feed has moved. My hope is this will support motivated folks to switch to <code>/feeds.xml</code></li> <li>Wait a day for various RSS feeds to retrieve this post, so at least they&rsquo;ll have an explanation if this breaks</li> <li>Hack the Hugo version to copy <code>/feeds.xml</code> to serve at <code>/feeds/</code> and <code>/feeds/all/</code>. Redirect other known variants of <code>/feeds/*</code> to <code>/feeds.xml</code></li> <li>Live with the (hopefully minimal) consequences</li> </ol> <p>I&rsquo;m optimsitic this will work well enough, but I am hopeful that diligent readers will be able to read this as the last entry in my RSS feed and fix it manually. Apologies for the inconvience, and here&rsquo;s to another decade or writing!</p> <p>The overall look and feel is <em>mostly</em> an extension of the existing site, with some ideas borrowed from <a href="">Julia Evans</a> site as well on the story pages.</p>
Mailbag: How to deal with unhappy users on your Internal platform?, 03 Mar 2021 05:00:00 -0700 <p>I&rsquo;m in the early stages of working with my friend, <a href="">Rachael Stedman</a>, on an &ldquo;Ask an engineering leadership&rdquo; project where we try to answer folks challenging engineering leadership questions. This is one of the questions that came in that wasn&rsquo;t a perfect fit for that project (we&rsquo;re still callibrating a bit on what <em>is</em> a perfect fit), but still a question I wanted to take a stab at answering.</p> <blockquote> <p>I work as an Engineering Manager at a large organization, supporting team managers across four teams who work on the shared infrastructure and platform for the various web products. The department is new, and is a result of a strategic shift to complement existing engineering departments which traditionally managed their own separate infrastructure and stacks. My joining this new department about nine months ago also coincided with me stepping up into the manager-of-managers role after a number of years as a team manager in another organization.</p> <p>The nascent platform was initially developed from a ‘spike’, and I have inherited a set of teams, responsible for different layers in the platform stack. These teams were existing teams who have come together from various other departments. I’ve faced the challenge of building cohesion between these teams to ensure we are building a cohesive platform. We’re also now seeing the number of teams adopting our platform accelerate significantly, with projections of 250+ engineers developing on our platform in the coming months.</p> <p>Our tenant teams are now facing into the frustrations of the shared platform, which inevitably constrains them more than their old world. I am confident our strategy lies in building self-service tooling, and we have the promise of being able to accelerate all of these teams. However, the rapid adoption means these benefits lie beyond the maturity of the platform as it stands just now and the frustrations are real and present. This frustration is starting to cause significant friction, with lots of door stepping for support requests and a sense amongst tenant teams of my teams acting as ‘the police’ due to more human intervention than we’d like.</p> <p>The friction distracts my team from their strategic goal of maturing the platform. I fear that tenant teams will circumvent the platform in order to avoid this friction. If this happens, then we will never realise the potential benefits of the shared platform. How can we build buy-in from the wider organization and ensure we are seen as an accelerator amongst the tenant engineering teams - while also maturing the platform?</p> </blockquote> <p>The first thing I&rsquo;d say is that this is a fundamental tension that all internal platform teams experience. It&rsquo;s not just you, this is a common growing pain. With that said, there&rsquo;s a lot you can do to reduce the pain.</p> <p>It sounds like you&rsquo;re in the midst of a <a href="">prolonged migration</a>, and one of the core tenents of the migration is to drive fit before you drive adoption. Many platform teams have goals that incentize migrating as many users onto your platform as possible, even if it&rsquo;s an unhappy adoption. This will spike your early adoption numbers, but stall out fairly quickly. Instead the preferrable approach is to bring on one or two challenging users and iterate with them until the platform solves their needs. In parallel, bring on easier users at a fixed rate with a focus of streamlining the onboarding process. (Absolutely not the goal of driving up platform adoption.) Until you have some complex teams who have successfully adopted the platform and are happy using it, driving adoption will only lead to sorrow.</p> <p>It sounds like you are <em>in</em> that sorrow right now, which is totally understandable: most platform teams are there are some point. If <em>all</em> of your users are unhappy, it&rsquo;s possible that you started adoption a bit too early and you&rsquo;re just going to have a rough go of it, but at least that means that there are only a few things you need to solve before most folks will be happier. If only <em>some</em> of your users are unhappy, maybe you can fire them for the time being. Yes, it&rsquo;s embarassing to have them churn off your platform, but your goal is maximizing <em>long-term adoption</em> not maximizing <em>short-term adoption</em>, and this is a case where you can sacrifice your long-term goal if you are afraid of letting a few particularly challenging users churn off your platform (back to whatever they used before).</p> <p>Often when folks are frustrated with lack of power, it turns out to be an interface design issue, and I started to focus on the idea of <a href="">providing pierceable abstractions</a> to allow users to bypass the abstractions as necessary to reach another layer of complexity (e.g. from container to VM). It&rsquo;s not that you <em>want</em> them to bypass the abstraction, just that it&rsquo;s hard to prioritize every teams' specific needs, and letting them bypass allows you to address their problem on <em>your</em> schedule rather than <em>their</em> schedule.</p> <p>Once you identify a strategy to allow the teams to solve their own problems, whether it&rsquo;s a pierceable abstraction or the self-service tools you&rsquo;ve described, then it&rsquo;s a matter of scraping together enough engineering time to actually get there. This is a case where I think what feels fair and what works are a bit different, and in my opinion what works is actually the fairest thing you can do since it avoids the scenario of everyone being miserable together forever. More concretely what this means is that I&rsquo;d recommend having a subset of the team, it can be a rotation or whatever, who simply soak up the incoming requests and do their best to support them. They&rsquo;re going to be overloaded and going to do a mediocre job. They&rsquo;re going to be annoyed about it. The rest of the team has to then be <em>completely</em> focused on the work that will relieve the load on the folks soaking up the incoming requests. I&rsquo;d recommend using a <a href="">service cookbook</a> to make tracking requests even easier to ensure you&rsquo;re prioritizing the right work.</p> <p>The above scenario is basically what I was hired into at Uber with a four engineer Core Operations team that was trying to support 200 engineers and falling further behind every day. We dug out through ruthless prioritization, some pretty tough quarters, and building a fantastic self-service platform that eventually offloaded all the manual work we were doing. Then folks went off and built ten thousand services using it, which is a story for another day.</p> <h2 id="resources">Resources</h2> <p>Some of the stuff I&rsquo;ve written related to this topic:</p> <ul> <li><a href="">Migrations</a> talks about technology migrations</li> <li><a href="">Magnitudes of exploration</a> talks about the tradeoffs between freedom and standardization</li> <li><a href="">How to invest in technical infrastructure</a> talks about a series of approaches to platform investment</li> <li><a href="">Providing pierceable abstractions</a> talks about designing interfaces to be bypassed by power users</li> </ul>Measures of engineering impact., 27 Feb 2021 05:00:00 -0700 <p>My engineering leadership circle met yesterday, and like usual, we talked about our current challenges before segueing into a deeper discussion. This time, <a href="">Jack Danger</a> brought up the challenge of measuring engineering impact, which is a fascinating topic that most engineering leaders have to tackle.</p> <p>Measures of engineering impact are <strong>not</strong> <a href="">Accelerate’s measures of developer productivity</a>: lead time, batch size, failure rate, and time to revert. Those measures support understanding and optimizing your development process but aren’t very effective at grading business impact. That’s partially because there are many ways to score highly against those measures without creating much business impact, and partially because they don’t resonate much with folks outside of the engineering organization. (Conflating efficiency with impact is also my general frustration with the current crop of manager tooling attempting to measure <a href="">developer meta-productivity</a>.)</p> <p>Some examples of how organizations measure engineering impact:</p> <ul> <li>Amazon: # <a href="">press releases</a> (<a href="">h/t Jack Danger</a>)</li> <li>Square: # new billable features (<a href="">h/t Jack Danger</a>)</li> <li>Gusto: # competitive advantages created, # competitive advantages improved, # table stake features (<a href="">h/t Jack Danger</a>)</li> <li>Uber: # projects* shipped per quarter per team. Projects are not just product-focused: migrations, tech debt removal all count as projects, as long as they have an impact on the team (<a href="">h/t Gergely</a>)</li> </ul> <p>Looking at the Amazon, Square, Gusto, and Uber impact measures, there are a few characteristics that make them effective:</p> <ol> <li>They’re straightforward&ndash;if you buy into the trope that leadership is mostly about repetition, then these are the sorts of measures that you can recite at the start of every org meeting without taking up too much time. They don’t take much explanation to convey</li> <li>They center on continued innovation, which is the implicit religion of growth-oriented businesses. There are many valid concerns with the innovation emphasis, but these goals correctly align with their companies’ values that emphasize creation over sustenance</li> <li>They are relatively difficult to “game” in dysfunctional ways as long as they’re used by a hands-on leadership team. Most of those would be easy to detect over time&ndash;if you release a very mediocre press release, people will notice that and take corrective action</li> </ol> <p>They’re not perfect, but I’m pretty confident any engineering organization with more than one layer of management would be better off having something along these lines than not having them. This sort of goal is to some extent your minimum viable <a href="">engineering strategy</a>.</p> <p>Personally, I’ve also spent a bunch of time thinking on this topic, and within <a href="">Calm Engineering</a>, we’ve been doing something similar for the past year, with a focus on:</p> <ul> <li># big bets - these are new, differentiated features released to users. While some parts of this are a shared goal across pretty much all organizations within the company, we hold ourselves accountable as one of (if not the) core constraints to delivery</li> <li># experiments - how many features and optimizations have we tested, with a target distribution for winning, neutral, and losing outcomes. There is a set target to help balance quantity and ambition</li> <li># fires - how many incidents have we had? We’re not trying to drive this to a target, rather are trying to avoid a slope change. Overall, I do believe fires are important to factor into overall productivity as a countervailing measure</li> <li>1-2 technical investments: how many technical investments or explorations are we doing? Organizations have a limited bandwidth to absorb technical changes, so we grade ourselves on having a few but not many</li> </ul> <p>Comparing Calm’s to those used by the other companies above, the obvious flaw is that they’re simply too many and too complex. I think this is mostly just a matter of optics, as Square and Amazon obviously track more than one measure of impact, it’s just a matter of different granularities.</p> <p>Shifting topics a bit, it’s also interesting to think about how measuring engineering impact does or doesn’t simplify the problem of measuring infrastructure engineering impact. The challenge of measuring infrastructure impact consumed much of my last two years at Stripe, as I pursued the goal of right-sizing infrastructure investment relative to infrastructure impact.</p> <p>That topic led to several explorations on how to measure impact within an organization whose primary contribution is enablement:</p> <ul> <li><a href="">Metrics for the unmeasurable</a></li> <li><a href="">How to invest in technical infrastructure</a></li> <li><a href="">Writing a reliability strategy</a></li> </ul> <p>One way or another, all of these approaches have rough edges. Some folks argue that infrastructure organizations should measure themselves as if they were Heroku: user adoption, reliability, cost to serve, etc. I’ve tried that: it’s a beautiful dream and parts of it work. The biggest challenge is that infrastructure groups typically become their company’s vehicle for enforcing various global trade offs like GDPR, data locality, security, high-availability, and so on.</p> <p>In organizations that reward or tolerate local optimization, infrastructure groups become the unwilling avatar of global optimization and are frequently the intermediary between company goals (e.g. GDPR compliance) and misaligned pockets of locally optimizing users. You can grade that infrastructure group on user satisfaction, but in practice these are users that you would fire as an independent company (or more likely would have never bought your services to begin with), so it’s a bit of an awkward exercise.</p>Digital gardening at Exuberant Sketches., 25 Feb 2021 05:00:00 -0700 <p>Last year I read <a href="">Weinberg on Writing</a>, and I’ve been thinking about its ideas a lot since. It focuses on the concept of “The Fieldstone Method” of writing, which I’d summarize as (1) writing things that energize you and (2) approaching writing organically rather than linearly. Lose the excitement for your current project? Work on something else for a while. Find a fantastic idea that doesn’t fit into this article? Add it to a pile of related concepts to come back to later.</p> <p>Some of Weinberg’s specific suggestions don’t mesh with my writing process, but many of them do. It helped me finally articulate something that’s been increasingly clear over the past couple of years. There’s an increasing gap between the kind of writing I find energizing and the writing that folks expect to read on <em>Irrational Exuberance</em>. The gap isn’t huge by any stretch, but there’s really no way to convince folks reading your blog to stop reporting typos. Even thinking that idea feels ungrateful to the time folks spend letting me know about my mistakes.</p> <p>Digging into the mismatch between the kinds of things I want to write and the kinds of things that <em>Irrational Exuberance</em> readers want to read, I identified three rough categories:</p> <ol> <li>Content I create for a larger project or preparing for a talk, e.g., most stuff with the tag <a href="">staff-plus</a> was for <em><a href="">Staff Engineer</a></em></li> <li>Articles I hope are actually good, matching most folks expectations of a professional’s blog, e.g. <a href="">A forty year career</a> and <a href="">Your first 90 days as CTO or VP Engineering.</a></li> <li>Writing something down to get it out of my head without forgetting it entirely. This is essentially using the blog as a <a href="">digital garden</a>. These are generally rough, quick takes on things like <a href="">Why not create a StaffEng Slack or Discord?</a> And <a href="">My skepticism towards current developer meta-productivity tools</a>.</li> </ol> <p>This mix has felt intuitive to me but has very reasonably not been intuitive to a vocal subset of well-intentioned readers. I’ve been pondering how to make this simpler. One idea that I almost went with was excluding wiki content from the mailing list or RSS feed, but that’s not quite right: it’s not that I don’t want folks engaging with the wiki content; it’s just that I want a different expectation of completeness.</p> <p>I’m going to continue writing the first two kinds on <em>Irrational Exuberance</em>. I’m moving the third category to a new area, <em><a href="">Exuberant Sketches</a></em>. There’s an <a href="">RSS feed</a> but no mailing list, and the stuff up there so far is a total mess (mostly exports of a dozen google docs into Markdown). It’s totally possible that it won’t go anywhere, but it’ll be a fun experiment either way and as part of that experiment, I’ll try to remember to spell-check stuff I post here going forward. (No guarantees.)</p>Self-publishing Staff Engineer., 17 Feb 2021 05:00:00 -0700 <p>After I published <em><a href="">An Elegant Puzzle</a></em>, I consolidated my notes into <a href="">What I learned writing a book</a>. That was my first book writing and publishing experience, and I learned a great deal. When I started working on <em><a href="">Staff Engineer</a></em>, I sort of knew what I wanted to do, but because I ended up self-publishing, I ended up learning at least as much as with my first book.</p> <p>There are a bunch of great articles on writing technical books out there like <a href="">The Economics of Writing a Technical Book</a>, <a href="">Writing a book: is it worth it?</a>, and <a href="">this thread on the Orange site</a>. My experience isn’t incredibly unique, but perhaps it will be useful for folks debating publishing a book, and either way, I will <a href="">learn a lot from writing this</a>.</p> <p>Finally, before getting into the details, two huge “thank yous.” First to <a href="">Gergely Orosz</a> who was a month or two ahead of me working on _<a href="">The Tech Resume</a> _and saved me many dozens of hours along the way. Second to the <a href="">TechWriters community</a> who provided a great deal of feedback, suggestions, and publishing experience.</p> <h2 id="beginnings">Beginnings</h2> <p>I started thinking about my next writing project in October 2019, about six months after <em>An Elegant Puzzle</em> was published. I wasn’t excited to write another book about engineering management but did enjoy the prospect of publishing another book. There were a handful of <a href="">books I could imagine writing</a>, but the one that felt most natural was one exploring the plight of the Staff engineer. Staff engineers are the core of large technology companies. Still, at many companies, neither the Staff engineers nor the engineering managers have a clear vision for what the role actually entails.</p> <p>Once I had a clear idea, I started figuring out how to approach the book.</p> <h2 id="schedule">Schedule</h2> <p>This project’s overall schedule was roughly:</p> <ul> <li>Started the project in October 2019</li> <li>Released <a href=""></a>, June 2020</li> <li>Cover started, December 10th, 2020</li> <li>Cover completed, December 20th, 2020</li> <li>Editing started, November 2020</li> <li>Foreword completed, January 20th, 2021</li> <li>Editing completed, January 27th, 2021</li> <li><a href="">Early Edition</a> for sale on Gumroad, January 28th, 2021</li> <li><a href="">Full Release on Amazon and Gumroad</a>, including paperback, February 5th, 2021</li> <li>Audiobook, estimated late February 2021 to early March 2021</li> </ul> <p>These dates are slightly rough in a few places; not everything has a clear starting or stopping date but is more or less accurate.</p> <h2 id="budget">Budget</h2> <p>I didn’t keep a particularly clear budget and some items are slightly arbitrary to allocate against this project versus other projects (for example, Mailchimp is costing about $80/month now, but half of that is due to my blog’s mailing list). That said, to the best of my sleuthing, this is a somewhat accurate budget:</p> <ul> <li>Cover art: $500</li> <li>Mailchimp: $480 = $40/m * 12m</li> <li>Canva: $78 = $13/m * 6m</li> <li>Gumroad = $60 = $10/m * 6m (excluding processing fees)</li> <li>Audiobook narration &amp; production: estimated $3,000</li> <li>ISBNs: $295 for 10 ISBNs (cheapest size for 2 codes)</li> <li>Frontendor: $59 (styling for marketing site)</li> <li>Airtable: $288 = $24/m * 12m</li> </ul> <p>So all in, that’s $1,760 for everything excluding the audiobook and $4,760 including it.</p> <h2 id="sales">Sales</h2> <p>I’m trying not to spend too much time fixating on sales, although I have to admit I like numbers that go up. I’ll occasionally be updating sales numbers on <a href="">the Exuberant Sketches page for Staff Engineer</a>. As of 2/15 (10 days after Amazon launch, about 15 days after Gumroad launch), the sales are 1618 copies via Gumroad, 944 paperbacks, and 762 Kindle. That’s a total of 3,324 copies. It’s still very early days on whether the sales remain steady or quickly taper off, but either way, I’m trying to focus more on the folks saying the book has helped them understand their role (or get promoted) rather than the raw sales.</p> <p>I’ve come to believe that folks don’t value things that they don’t pay for, so I wanted to charge a reasonable price even if folks could still get most of the content for free on <a href=""></a>. Initially, I was thinking of charging $10 everywhere, but that felt like it would undercut the market in an uncharitable way to folks trying to make a living off writing technical books, so I decided to go with $25 for both paperback and digital copies.</p> <p>Money moves slowly (especially money from Amazon), but from that, I’ve been able to donate $9,000 so far across Black Organizing Project, Resilient Coders, /dev/color, and Black Girls Code. I will be excited to donate more as it comes in.</p> <h2 id="process">Process</h2> <p>For <em>An Elegant Puzzle</em>, I was able to draw from my own experiences as a manager, but I knew I wasn’t going to be able to do that for a book about Staff engineering. I decided to start by interviewing a handful of folks.</p> <p>These interviews became <a href=""></a>, and served not only as a great way to ground the book in real experiences but also an effective mechanism to build up an email list of folks interested in the topic. I put together the site in <a href="">Gatsby</a> in a few days, although knowing what I know now, I wish I had used <a href="">Hugo</a> (much simpler, fewer moving pieces, easier to get CI/CD working on Github Pages). From there, I found a good domain, created the Github repository, set up Github Pages to host it, and created a new MailChimp mailing list.</p> <p>I stored up about ten interviews before starting to release them, with the first being from <a href="">Keavy McMinn</a>. Around the same time, I also began writing the guides and releasing them on this blog, with the first being <a href="">Learn to never be wrong.</a> I dripped out the stories and guides, sharing them via mailing list and promoting the stories a bit on Twitter and LinkedIn.</p> <p>That write-promote loop became the project’s overall cadence. I started with a goal of twenty interviews (recently hit with <a href="">Aaron Suggs</a> as the twentieth), and with an initial slate of guides. Both the goal and the guides got reworked a number of times as I went, and the project became clearer in my mind.</p> <h2 id="formatting">Formatting</h2> <p>In September 2020, I started working on the actual book itself. The book is a private repository on Github that pulls in content from the public <a href="">lethain/staff-eng</a> repository as part of its build. It is a 438 line Python script, a 48 line makefile, and a bunch of <a href="">Pandoc</a> configurations. It’s a one-line command to rebuild the epub, digital pdf, and print pdf versions of the book and runs in about two minutes (most of the time is spent in LaTeX generating the pdfs).</p> <p>You can see a <a href="">redacted version of the build script on Github</a>.</p> <p>This is pretty similar to how <em>An Elegant Puzzle</em> worked, except my goal there was simply to produce something good enough to hand over to <a href="">Stripe Press</a>, whereas my goal here was to create something to send to users directly. It was easier than I expected to get something pretty good.</p> <p>The popular opinion is that any time spent creating tools is wasted, but I sort of disagree. It took a couple of hours to <a href="">add QR codes for every reference into the print copy</a>, and I just wouldn’t have been able to do that without creating my own tooling.</p> <p>Digital formatting was pretty straightforward, but I did run into a bit of a learning curve getting the paperback working under <a href="">Kindle Direct Publishing</a>. It took three rounds of printed proofs before it turned outright, and I gave up entirely on KDP Cover Creator&ndash;the final cover was actually made in Omnigraffle, which I find pretty funny.</p> <h2 id="cover">Cover</h2> <p>The cover image was illustrated by an artist I found on <a href="">Fiverr</a>, and then I created the full cover image on <a href="">Canva</a>. I was very intimidated by this process, but it ended up being pretty painless, and I love how the cover turned out.</p> <p>It took less than two weeks from creating my Fiverr account to having a completed cover, and cost about $500. Next time I will try to find an artist to work with directly. Fiverr was a great way to ease the learning curve, but the compensation for the artist’s work was low. The biggest issue I had was accidentally using the approval copy of the cover illustration rather than the final copy and realizing later that it had a subtle watermark. This meant that I had to go back and remake all the various assets, but really that’s pretty minor.</p> <h2 id="editing">Editing</h2> <p>I was able to find about a dozen folks from Twitter who were willing to review book chapters. This was immensely helpful and greatly improved the quality of those sections. I used <a href="">Grammarly</a> to do an initial pass on grammar and typos, and my wife <a href="">Laurel</a> was kind enough to do a final close editing pass for other issues. I explored getting a line editor, but ultimately the timing didn’t quite work out.</p> <h2 id="audiobook">Audiobook</h2> <p>While I’d always considered making an audiobook for <em>Staff Engineer</em>, I was leaning heavily towards not making one by the time I finished the digital release. However, enough folks asked about it that I decided to look into it. I went into it hoping that I might be able to record the book myself, but life with an eight-month-old kid in a small house just didn’t mesh with the logistics of recording an audiobook: 20+ hours to record, and the need for a very quiet space.</p> <p>So then I gave up on the audiobook again, until someone else asked for it, at which point I dug a little further into Audible’s audiobook creation platform, <a href="">ACX</a>. Then I gave up again because it felt a bit too complex and complained about it on Twitter, which led to <a href="">Ian Joshua</a>, a product manager at Audible, demystifying the process over a Twitter DM.</p> <p>A few days later and I’d listened to thirty narrator auditions, submitted an offer, and the audiobook was underway. I won’t have a precise cost to create the audiobook until it’s fully finished, but my guess is it’ll be around $3,000. Audible payments are fairly inscrutable, so I honestly don’t have a clear sense of how this will pan out financially. I’m hoping to at least break even and get a free education on audiobook creation.</p> <h2 id="foreword">Foreword</h2> <p>I&rsquo;m pretty sure that I met <a href="">Tanya Reilly</a> while asking her if she’d be willing to do an interview for this project, which led instead to a discussion of an adjacent project she was working on. We kept in touch a few times afterwards, and when I started thinking about my dream author to write <em>Staff Engineer</em>’s foreword, she was absolutely the first person on the list: Tanya’s talk and post on <a href="">Being Glue</a> is the seminal work on Staff-plus leadership.</p> <p>As such, it’s hard to exaggerate how excited I am that she agreed to write the amazing foreword that she did, and I hope to return the favor (I imagine more figuratively than literally, although literally would be fun, too).</p> <p>For some reason, I never even considered adding a foreword to <em>An Elegant Puzzle</em>, probably because my personal network was much smaller at that point and an entirely unproven author to boot. I’m really glad I didn’t go that route this time. This is kind of a funny statement because that was only two years ago, but that&rsquo;s genuinely my experience of it.</p> <h2 id="blurbs">Blurbs</h2> <p>Carrying on in that vein, I was very intimidated by the process of finding blurbs when I wrote <em>An Elegant Puzzle</em>, but it was much easier this time. There were a number of folks who I met while recording interviews but who weren’t able to make the interviews work from a timing perspective. Of those, Amy Unger and Nicky Wrightson were kind enough to provide fantastic blurbs. In addition to Amy and Nicky, several others were generous enough to review the work, including Padmini Pyapali, Kevin Stewart, Uma Chingunde, George Orosz, Emily Robinson, and Shawn Wang.</p> <p>These blurbs ended up on <a href=""></a>, the <a href="">Gumroad page</a>, <a href="">Amazon page</a>, and even <a href="">the printed book</a>.</p> <h2 id="selling">Selling</h2> <p>I decided to sell via <a href="">Gumroad</a> and <a href="">Amazon</a>. Both have gone well.</p> <p>Gumroad has been fantastic. I initially hoped to go with a direct Stripe integration, but it would have required a bit too much tool building when I knew I should be focused on finishing the book itself. I also added a hosted Gumroad store at <a href=""></a>, which seemed like a good idea at the time, although I haven’t found much of a use for it yet.</p> <p>Amazon has been good as well, and I’ve found the paperback experience with <a href="">Amazon KDP</a> to be surprisingly great. The print quality of all the paperbacks I’ve received has been quite good, although at least one person has submitted an Amazon review with a poor-quality print, which is quite a shame and hard to track down as they use multiple printers in each region. That said, the quality has been quite high for most folks, and having a physical copy was an important goal for me.</p> <h2 id="isbns">ISBNs</h2> <p>I wanted to have the same ISBN for Gumroad and Amazon, which meant that I spent $250 for ten unique identifiers from <a href="">Bowker</a>. ISBN pricing feels like a scam, but fortunately, the rest of my selling experience was great. In retrospect, I’m not quite sure why I did this. I probably should have just not bought ISBNs because I don’t anticipate <em>Staff Engineer</em> being sold in any contexts that would actually need them.</p> <h2 id="marketing">Marketing</h2> <p>Someone once told me that books are sold rather than bought, and I tried to keep that in mind when I started marketing <em>Staff Engineer</em>. I did a variety of things here:</p> <ul> <li>Sent out two email blasts to StaffEng mailing list (one for early release, one for full release)</li> <li>Featured the release in my <em>Irrational Exuberance</em> mailing list</li> <li>Created a sales page at <a href=""></a> using <a href="">Frontendor</a> (ty Gergely for the suggestion!)</li> <li>Did a couple of podcasts, <a href="">Software Engineering Daily</a> and <a href="">Career Chats</a>, and working on a few more</li> <li><a href="">Added to Goodreads</a></li> <li>Updated <a href="">Amazon author page</a> from what was leftover from previous book</li> <li>Worked within the available formatting options for the product page on Amazon</li> <li><a href="">Styled the Gumroads page</a> to the extent possible</li> <li>Added the book to the “right rail” of my blog</li> <li>Updated my <a href="">blog’s about me page</a></li> <li>Updated my <a href="">LinkedIn profile</a></li> <li>Updated my <a href="">Twitter profile</a></li> <li>Did some tweets</li> </ul> <p>That was pretty much it, I think. For <em>An Elegant Puzzle</em> I spoke at a couple in-person conferences and went to a handful of private company events to talk about the book, but I’m skeptical that those did too much. The only thing I really regret not being able to easily recreate was <a href="">the feature with First Round Review</a> which created a surprisingly large amount of visibility.</p> <hr> <p>Btw, Gergely’s <a href="">Want to Start a Startup, as a Software Engineer? Sell Something Online</a> is a great piece on book marketing.</p> <h2 id="amazon-advertising">Amazon advertising</h2> <p>On the advice of Alex Xu, author of <em><a href=";keywords=system+design+interview&amp;qid=1611683996&amp;sr=8-1">System Design Interview</a></em>, I decided to start experimenting with a small amount of Amazon advertising. Alex’s experience was good conversion, reasonable clicks, and very hard to scale up volume. So far, I’m seeing a couple of dollars in spend a day out of a $10 daily budget, and am seeing roughly $0.40 per click, but too early to say whether this is going to drive many sales.</p> <h2 id="thats-a-wrap">That’s a wrap</h2> <p>When I launched <em>An Elegant Puzzle</em>, Brianna Wolfson told me that books are sold over the first six months, not just the first week, and that really stuck with me. It was also, with the exception of a week where Amazon pulled <em>An Elegant Puzzle</em>’s digital version for two weeks due to a communication misfire over quality reports, true. <em>An Elegant Puzzle</em> sold <a href="">~13k copies in the first five months</a>. If <em>Staff Engineer</em> sells that many copies in its lifetime I’ll be pretty happy.</p> <p>If <em>Staff Engineer</em> ends up being helpful to you, please <a href="">drop me a note</a>!</p>Mailbag: Building alignment around a new strategy., 16 Feb 2021 05:00:00 -0700 <p>Another question regarding <a href="">Staff Engineer</a> that came in recently:</p> <blockquote> <p>There was a bit in your book where you mentioned after you finished writing your vision - that you shared it out because you were very excited (and then trying not to be let down by not hearing much back in terms of response). Was curious if you could share more about how you think about sharing larger technical visions or strategies. Do you start with smaller circles? Do you send out to everyone at once? Do you introduce it at an All Hands meeting? Do you have comments turned on? We’re experimenting with some different ways right now, but don’t think we’ve nailed mass cross-organizational consensus building - at least not efficiently. Maybe that’s not possible?</p> </blockquote> <p>For context, this question is referencing the end of <a href="">Writing engineering strategy</a>:</p> <blockquote> <p>After you finish writing your vision, the first step folks usually take is sharing it widely across the engineering organization. There is so much work behind the vision&ndash;five design docs for each strategy, five strategies for one vision&ndash;it&rsquo;s hard not to get excited when you&rsquo;re done. So excited that it&rsquo;s easy to get discouraged, then, when the response to your strategy will almost always be muted. There are a few reasons for the muted response. First, the core audience for your vision is folks writing strategies, which is a relatively small cohort. Second, a great vision is usually so obvious that it bores more than it excites.</p> </blockquote> <blockquote> </blockquote> <blockquote> <p>Don&rsquo;t measure vision by the initial excitement it creates. Instead, measure it by reading a design document from two years ago and then one from last week; if there&rsquo;s marked improvement, then your vision is good.</p> </blockquote> <p>There’s really no way to avoid the truth here: making decisions in a large engineering organization can be a real grind. One of the techniques I’ve developed to address this is <a href="">the working group</a>, specifically with a structured process for gathering feedback and identifying working group members. <a href="">Mark wrote a good post about how we use this model at Calm</a>.</p> <p>The structured working group model does a great job of building alignment, but over time I’ve come to appreciate that sometimes they don’t generate great work. <a href="">Groups usually make worse decisions than individuals</a>, even for activities where you’ve likely been taught the opposite, <a href="">like brainstorming</a>. This isn’t a refutation of the working group model, it’s more a tweak in how they work. Do the initial research as a group, and iterate on the proposal as a group, but don’t try to write the proposal as a group. Instead identify one member to take the lead on synthesizing the research into a proposal, delegating specific sections to other working group members as necessary.</p> <p>The most common way that working groups fail is that they encounter a vocal minority that filibusters their proposal. Consensus doesn’t scale to very large groups, but it’s surprising how many organizations implicitly rely on consensus for overarching decisions. Organizations either develop a practice of genuinely following disagree-and-commit, or they end up relying on frequent escalations to make forward progress (sometimes these are lateral escalations to blessed decision owners in a given area like a data architect for data modeling decisions, etc). Awkwardly, companies that depend on frequent escalations almost never acknowledge this reliance, which makes many folks working there ineffective, because they continue to view escalations as a failure of diplomancy rather than simply the way the organization works.</p> <p>Pulling all these threads together to directly answer the question, my general approach to these sorts of problems is:</p> <ol> <li>Write a clear problem statement, soliciting feedback from stakeholders to ensure it’s genuinely the right problem statement</li> <li>Create a working group around solving that problem statement, including the right folks to build consensus among key stakeholders</li> <li>Proactively solicit feedback and input for the working group, focusing on mining for dissent from the folks who are most likely to have objections</li> <li>Have one person within the group synthesize all the research into a proposal</li> <li>Iterate within the working group until members are aligned on approach</li> <li>Share a non-commentable document with the organization, along with an invite to a Q&amp;A session about the proposal. Also solicit written feedback as documents rather than comments (document comments encourage nit-picks and minute feedback rather than more useful big picture feedback)</li> <li>Spend extra time with the folks who are really frustrated, trying to understand their frustration. Figure out who within the working group is best positioned to address each concerned party</li> <li>Once the rate of useful feedback slows, incorporate what you’ve gotten into the proposal</li> <li>Align with all stakeholders in a single thread to ensure there’s joint commitment to the proposal. If there’s a small disagreement, iterate a bit until it’s resolved. If it’s a structural disagreement, escalate to someone who can effectively resolve the disagreement</li> <li>Share the approach with the organization in Slack. In an email. At the All Hands. At new hire onboarding. Basically, never stop sharing it if it’s something you want folks to actually remember. How far you go will depend on the size of the strategy or decision, but generally speaking these things are undershared. This is doubly true in a fast growing organization where new folks won&rsquo;t know about whatever decision you made last quarter</li> </ol> <p>This isn’t a perfect approach, but it’s good enough in my experience.</p>Mailbag: How to encourage good documents rather than perfect documents?, 15 Feb 2021 05:00:00 -0700 <p>A great question on <em><a href="">Staff Engineer</a></em> that I received recently:</p> <blockquote> <p>When you’re writing docs (and reviewing them) I really resonated with the idea of pushing for them to be good and not perfect. Was curious how you go about fostering that culture beyond yourself? I feel like since we’ve shifted to a remote culture, doc grooming has reached an all time high. Also curious if that’s something you’ve felt at all?</p> </blockquote> <p>There’s an increasingly pervasive belief that written communication is the “obviously best” communication culture. This was initially driven by the <a href="">cultural export of Amazon’s memo culture</a>, and has been bolstered recently by tailwinds from the pandemic-driven shift to remote work. I’m quite partial to written communication, but it’s a challenging shift. This question hits on one way this shift can go awry: an emphasis on perfection rather than good slowing down discussions.</p> <p>This is a fairly familiar problem for engineering teams. It comes up frequently in system design, in the tension between <a href="">YAGNI</a> and not wanting to rewrite the system in six months. Even more commonly, most engineers have been on a team where the perception of nit picking feedback during code review leads to only sharing pull requests for review after the approach has calcified beyond alteration.</p> <p>The example of code review is probably the most familiar, so we can think about how the tactics for debugging a flailing code review process can be applied to writing culture:</p> <ol> <li><strong>People hide work because they’re uncomfortable.</strong> If documents are being shared late, it’s because some aspect of the sharing process is making authors uncomfortable. Usually this is because the feedback they’re receiving isn’t helping them accomplish their goal. Maybe there’s too much feedback and folks are waiting until late to share to minimize responses. Maybe the feedback isn’t useful, so folks don’t bother requesting feedback until they’re past the point of cheaply incorporating new ideas.</li> <li><strong>There’s probably four people at your company who make documents uncomfortable.</strong> Before you start thinking about a systems level solution to this problem, I’d guess that there are a very small number of people who need to be given direct coaching on how to provide feedback on documents. If you hold them accountable for how they engage with other folks documents, that conceivably might be enough to solve the entire problem.</li> <li><strong>Provide a structured template and document lifecycle.</strong> A flexible document template will eliminate debate on structural issues. A document lifecycle process will make it clear when to provide which kinds of feedback. The template and lifecycle act like a code linter in code review, moving feedback away from style and towards substance.</li> <li><strong>There’s a teachable skill to reviewing documents.</strong> If you ask someone to edit a document, a surprising number of folks will immediately gravitate to tweaking words or providing grammatical feedback. Sometimes this is useful, but it usually isn’t. While engineers tend to become better at code review with frequent repetition, most don’t do enough document review to get good at it. Fortunately, this is a teachable skill. Have folks read the entire document once before leaving any comments (avoiding the dreaded “but this is addressed later&hellip;” response). Focus feedback on the ideas rather than the format.</li> <li><strong>Provide a designated friendly reviewer / writing coach.</strong> Going further, most folks don’t have much experience writing documents either! You can reduce the perceived risk of sharing a document by providing a writing coach, probably an experienced engineer in their area, who can provide very early feedback. This coach will also be in a great position to nudge folks to ship their document when it’s “good enough” rather than continuing to polish it. This person also provide “social proof” for the person sharing the document, loaning their privilege as a senior engineer to the author that their document is indeed ready to share.</li> <li><strong>Provide a repository of “golden documents” with the right quality level.</strong> I bet you already have a list of “example documents” that folks should emulate, and I bet they are absolutely incredible documents. Which is to say, that they are absolutely not the example you want folks to follow if there is too much emphasis on perfection. Create a new list of “golden documents” which aren’t necessarily the best documents but instead those at the correct level of quality. Ideally you should even acknowledge the documents which are “too good.” Yes, they’re beautiful, but no, that’s not what we’re trying to do here.</li> </ol> <p>Of all those things, I think the most effective approaches are continuing to model “good rather than perfect” behavior yourself, pushing other leaders <a href="">to model that behavior</a> as well, and lending your privilege for imperfection to authors who take a larger risk when publishing imperfect work.</p>