John Kodumal, Founder/CTO of LaunchDarklyJohn Kodumal joined the CTO Guild on 8/6 for a special session where he shared a new approach called “constraint-based scaling” and how he uses it as a guiding principle as theLaunchDarkly org grows. John is the CTO and founder of LaunchDarkly.. Prior toLaunchDarkly, John was a development manager at Atlassian, where he led engineering for the Atlassian Marketplace. Before that, he worked as a researcher and architect at Coverity.

Constraint-Based Scaling (CBS)

What is CBS?​ This is a simple, yet powerful concept that John uses to be a lot more explicit in the way that they plan work and think about investments at LaunchDarkly.

When to apply it?​ Around the ~20 FTE mark it starts to make sense and only gets more impactful as the org scales. (LD is now ~200 FTE).

Why use it? ​At a prior company, they went through a sudden series of public outages, lots of customer churn, and employees moving to competitor brands. This escalated to the point of requiring a public apology. Eventually, the CTO called a war room and required every team to assess engineer availability, identify which metrics were out of whack, and which investments needed to be made. The org spent countless weeks putting this together and presenting to executives and eventually working through the issues. It ended with a full cessation of all new product work and a requirement to rework & retask availability. The three problems John identified here were:

(1) It’s very expensive to gather the information needed to fix the problem

(2) Not addressing underlying problems on a continuous basis leads to a pile up

(3) Retasking teams after a crisis (vs. continuously) leads to severe drops in engineer productivity. That drop org-wide of all of your timelines impacts marketing, revenue, etc.

Keys to Successfully Implementing CBS

(1) Define an operating constraint, it is very important that that constraint is both a quantitative & objective metric.

(2) Have the ability to continuously monitor the constraint. This is essential to avoid that situation where you can get so out of kilter that you've dug yourself into a huge hole and you need to make a massive painful investment to get yourself back in line with strength.

(3) Operate assuming that the constraints are not violated.

(4) Revisit the parameters of that constraint periodically, the constraints that you define when you're a series A or Series B startup are not going to be the same constraints that you define your pre-IPO company.

Core Benefits of CBS

  • Reduces biased or short-term decision making vs. long-term projects
  • Amortizes “unglamorous” work vs.packing ‘keep the light on’ work into focus weeks
  • Reduces “design space” around many decisions -- i.e. if things are working as designed, but still see problems, helps you understand why

Where do good constraints come from for this type of team composition example?

Start off with some of those invariants that are hard to argue against that can give you some initial structure for your team. For example:

  • Team leads​ (50/50 player coach) should only manage 6-8 individuals—past that they’re spending too much time on 1:1s
  • Engineering managers​ should only manage ~2-3 squads (18-24 engineers)

Another area where CBS helps? Tech debt.​ ​

Technical debt is notoriously qualitative. “It’ll be so painful to do X, Y, Z.” While not all technical debt can be defined in a quantitative manner, it is possible to carve out chunks of it by using constraint-based scaling principles. John defined a few operating constraints around LD’s technical debt:

Finally, two closing notes from John on how to make CBS work in your organization. The first is to document constraints clearly and communicate and reference them often. The second is to shine a light on them whenever possible. On the latter point, John manifests this as a quarterlyState of Product Delivery report that summarizes everything that’s going on and it's fully transparent to the entire org. The report has six sections:

(1) Service health​: overview of incidents, service uptime, & key operational metrics. This is where they specifically track against eng org goals & operational constraints

(2) COGS​: Infra spend (goal to stay under 20%)

(3) Delivery​: What they shipped to customers & partners

(4) OKRs​: How they tracked against large OKR projects that quarter

(5) Adoption​: Key product metrics (funnel, usage, adoption)

(6) People​: Health & makeup of the team (current squads, open headcount)

Bonus​: For an in-depth dive into similar concepts, John recommended reading ​An Elegant Puzzle: Systems of Engineering Management​ by Will Larsen

How To Share Best Practices Across Pods

Siloing can be good and bad. On one hand (+) it fosters a sense of ownership. On the other hand (-) there are product & implementation inconsistency issues and you can have duplicative effort.

How to mitigate​: Give squads the ability to swap members on a quarterly basis. Implement cross-squad projects and/or principles: e.g. “URLs should be beautiful and easily shareable.” The latter can be wrapped up in a product deliverable guideline booklet to be shared org wide.

How To Think About Engineering Velocity

Instead of thinking about velocity, John stresses thinking about predictability instead.We use a triad model: PM, designer, eng lead who work together to build the spec, then start the implementation phase; they stay on the product -- never seen handoffs work well. “The longer an engineering project lasts, the more things can go badly.Break them into smaller chunks.” At LD, the typical project tends to go 20%+ longer than estimated -- John’s comfortable with that.

Attracting Engineering Talent

Pre-COVID, our biggest advantage was being a big fish in a small pond(geographically). We’re Oakland based (90% in physical HQ) so we used that as a draw.One non-obvious thing we’ve done is build out our profile on keyvalues.com—a site geared specifically for engineers to fill out their value/culture preferences and then it’ll surface specific companies that match that profile. We’ve seen great success there as well. We’ve also found that our radical transparency (e.g. full dashboard publicly available for all levels of the org to constantly have access to) + continuous monitoring of pulse surveys has led to only 2 regrettable engineers leaving over the entire cours eof the company life to date

‍