On Engineering Consensus

The main reason science works is, of course, the scientific method. It forces rigor into the process and it’s what began to fork hard science from philosophy. Engineering, while not a science per se, is a sibling discipline. It too benefits from the scientific method (though more so in the realm of testing) and the engineering method is similar:

1Ask a question“Does fire destroy matter?”Consider a problem“How can I seal a container?”
2Form a hypothesis“Burning stuff in a sealed container should tell us”Design a solution“Maybe putting silicone on a jar lid will do it”
3Make a prediction“If the container weighs the same, the matter just turned to gas”Implement the solutionPut the silicone on the lid
4Run a testBurn some stuff in a sealed containerRun a test Burn some stuff in the sealed jar
5Analyze the resultsSee if it weighs the same or is lighterAnalyze the results See if any smoke got out, and if there were any bad side effects, like melting

We can generalize them both to something like:

  1. Have an idea
  2. Figure out what to do about it
  3. Do that thing
  4. See if it worked

In any case, they’re related. And one overlooked aspect of science is that it’s not all that cut and dry. Rare is the experiment that produces unequivocable results that are obvious to any layman. Lavoisier’s experiments on the conservation of matter seem straightforward to us now, but the test could’ve been wrong in a lot of ways: the sealed glass vessel could’ve had a microscopic leak, the scales might not have been sensitive enough to detect that some matter was destroyed, and Lavoisier himself could’ve had his finger on the scale!

Antoine Lavoisier

This is why science relies on not just the method, but equally so, also peer review. Other people had to read about Lavoisier’s experiment or maybe observe it in person. People that were experts enough in the field that they understood all of the details about creating a sealed vessel and about the accuracy of scales and other aspects of the experiment. And eventually, other people recreated his experiment and got the same result, and only then was the scientific consensus attained that no — matter cannot be destroyed.

Good science works because regardless of the prestige of the scientist or the seeming quality of the experiment, the certification of the finding is independently verified by other experts in the field, peers who know what to look for and what notions to accept as scientific fact.

Good engineering works the same way. Instead of peer review of research papers we do peer review of design documents and pull requests, and if that step of achieving engineering consensus is missing, the quality of the work suffers.

“But,” you say, “the testing will prove the quality of the work!” Except that there’s a fine distinction there: tests will prove that the solution works as intended; it says nothing about how well it’s built. It could be held together by duct tape, it could be an overly complicated Rube Goldberg device that’s impossible to maintain, or it could be a pile of spaghetti that’s impossible to refactor. In engineering as in life, the ends rarely justify the means. And so we need consensus on whether those means are good.

Cosensus is a tricky thing though. I dread submitting my code for review, even when I’m very happy and confident with it, because I know there are things I might have missed, and as much as I want to embrace learning from my mistakes, I really don’t like to make mistakes. However, when certain smart, experienced people are on vacation, I don’t mind it so much, because I know I haven’t made any mistakes junior developers are likely to catch, and I can make a good argument if there are questions on my approaches. But those arguments might not fly past more senior developers, who might have insight that I’m lacking and the experience to know what works and doesn’t to back their stance with.

So it’s not enough to just get the consensus of any two people: for quality consensus, it has to be two (or more) of your peers. Developers operating on your level or higher, who not only have the general experience and skills to recognize whether your work is good, but also have the specific experience with the surrounding landscape — be it the type of thing you’re designing, if it’s a design document, or the codebase that you’re changing, if it’s code.

And it should be people who aren’t afraid to speak up. Some talented engineers that would otherwise be good peer reviewers might be intimidated by a Bob that’s less talented. Maybe this Bob is higher up the totem pole, or maybe he bullies, badgers, or simply exhausts all opposition.

The choice of peer reviewers should be truly peers. People who are:

  1. Technical equals
  2. Organizational equals
  3. Up for a debate
  4. On good rapport

(That last one is to avoid a situation where Chelsea always nitpicks Bob’s code because she thinks he’s the worst.)

This is a kind of ideal engineering consensus to strive for, and for better or worse, in practice there are two reasons why it wouldn’t happen all the time:

  1. Most teams are small and there’s no equal to the tech lead, either technically or organizationally.
  2. Most things aren’t important enough to spent a lot of time reaching consensus

Which exposes an important point in technical leadership: one of the reasons having good leaders matters is for the times when consensus matters. Good tech leaders have good rapport with the team, they mentor and build expertise in others, they set high standards for excellence, and encourage healthy debates from the team members. Over time, good leadership results in exactly the kind of savy, comfortable team that generates worthwile consensi. Consensuses? Consensi. Please excuse me while I look up the consensus on this matter.