Someone once told me that people tend to move between companies
using the flying wedge formation: a scout shows up
to learn the culture and opportunities, and then an increasingly
large contingent of previous coworkers and friends stream in.
This jives with my experience, likely the direct consequence of
most hiring systems' heavy reliance on referrals (incidentally, I
think what Slack is attempting to do to move their hiring pipeline
away from referal dependence is one of the more interesting ideas
in tech recruiting today).
One of the fascinating side-effects of group movement is that
they tend to come with unexpected stowaways: their favorite technology stack.
This causes certain technology stacks to propagate from company
Who has used Clusto? Well, there is essentially no chance you’ve
even heard of this tool unless your company inherited
Digg’s infrastructure stack (I believe Linden Lab is one of the few
companies that adopted it organically).
If you have worked at such a company, especially in the 2011 to 2014 range,
then you very likely did use it, or at least strongly considered it!
(If so, you may have developed feelings about Clusto.)
Clusto’s just the tip of the inherited stack thought!
The full Digg infrastructure stack looks like:
- Python for scripting.
- Clusto for infrastructure metadata.
- Puppet for role configuration.
- FAI (aka Fully Automatic Installed) for provisiong servers.
- HAProxy running locally on every node as a load balancer.
- A healthy distrust of DNS.
I’m optimistic that there aren’t many new companies getting
founded with this specific technology stack, but if you looked at the companies
where ex-Digg engineers joined as early engineers, you’ll continue to see
some of the hallmarks propagate, dilluting over time.
This isn’t unique to Digg (if you’ve used NSQ,
your chances of having a coworker from Bitly is rather high), but part of a
larger pattern that you’ll see everywhere as you start looking for it.
I suspect it’s even an optimal behavior, in terms of picking tools you’re
familiar and comfortable with, instead of investing in technology
decisions that are fairly unlikely to make or break a young company.
For some technologies though, and I think Clusto is likely in this category in 2017,
it does create a hazy second-life for open source technologies that never achieve organic adoption.
They end up with adoption returns that mirror venture capitalists financial returns: probably 90%
of the engineers using Clusto are at Uber (even though I’m pretty sure there aren’t
any ex-Digg engineers there anymore), with limited survival in the broader marketplace of ideas.
(Consul and EtcD, along with the
Puppet, SaltStack and Chef specific tooling, feel like they’re today’s winners.)
Whatis your team’s technology inheritance?