The Art of Interviewing

The job interview process is a high-stakes dance that’s notoriously difficult and full of missteps — on both sides. (At least, in software engineering it is. If you care about another industry, <Jedi hand wave/> this is not the article you’re looking for.) Avoiding the missteps is an art, just like it is in dancing; we get better with practice, just like dancing; it’s a lot of fun when it goes well, just like dancing; and leading with dexterity is paramount. This article focuses on that last bit: not on what questions to ask or how to structure the time, but rather on artfully leading an interview. First, however — because it’s a crucial part of the interview context — we should start with how the job candidate views the process. Just like dancing.

Because while for Larry the Interviewer, it’s just a small fraction of his day — maybe an annoying one that pulls him from the dynamic programming problem he was definitely not working on — for Jane the candidate, it’s one of the most important events of her life. The median job tenure is about 4 years, so people will only have somewhere around 10 jobs their whole professional life. And each of those jobs is going to significantly affect Jane’s life both during, and after, it. The CV builds on the shoulders of the previous job, yes, but work also where we spend the bulk of our weekday, make friends, and develop a large part of our identity.

So what does Jane go into this high stakes dance equipped with? If the company is large, maybe she can get a sense of the general culture from Glassdoor, or news articles, or social media. Otherwise, maybe a sense of how it wants to be seen, from its marketing. But even then: what will her daily life be like there? Would she like her new boss? Her teammates? The bureaucracy? The tasks she’ll be working on over the next year? Four years? Unless she knows someone on the inside, these questions mostly won’t get answered until after the start date. Sure, she’ll get to peek in now and then through cracks in the process and through the five minutes of each interview in which she can ask stuff, but: largely unanswered. Which means it all adds up to a big gamble.

The gamble is better for more mercurial and adaptable personalities than those more averse to change, but your resume can only tolerate a couple of quick stints before it starts getting tossed aside. So it’s a big gamble for everyone. Or rather, I should say “for every job searcher”, because well… it’s a small gamble for Larry the Interviewer. Worst case for him? It isn’t even giving his thumbs up to a terrible employee — because someone truly terrible will get fired before too long. No, the worst case is that he gives his thumbs up to a really annoying Taylor that ends up on his team and really annoys him for the next 4 years. And also produces mediocre work that Larry then has to constantly deal with. But Taylor isn’t annoying enough and the work just isn’t bad enough to get fired or even be put on a PIP. Taylor kind of coasts just baaaarely on the good side of the policy. And in doing so, makes Larry’s work life feel like a chirping smoke alarm that he can never find. That is the worst case for Larry.

The best case? He ends up with a really awesome coworker. Which, depending on the existing coworkers, may or may not matter a lot. But in any case except for the narrow and unlikely one of Taylor, the stakes for Larry are much, much, much lower than for Jane. And yet, this is who decides Jane’s fate.

The irony is that all of us will — at different times — be the Jane, but most of us will also be the Larry at other times. And when we are the Larry, interviewing a job candidate, do we act as the interviewer we’d like to have interview us, when we’re searching for a job? When I grudgingly leave the house, I usually drive, and sometimes I walk, and sometimes I bike. And what I find fascinating is that when I’m a driver, I get mad at other drivers, when I’m a cyclist I get mad at drivers, and when I’m a pedestrian, I get mad at everyone. Even though I know exactly what challenges they’re all going through.

Through this lens, how should Larry — all of us, when we are the Larry — conduct his interviews? How can we be the Larry we want to see in the world?

Larry does have a concrete output from the interview; he needs to answer one simple question: is Jane likely to be successful in this role? “But wait,” you ask, “can Jane’s ideal Larry even answer that question? Can he both be empathetic and figure out if she should be hired?” To which I say: “not only can he, but that’s the best way to do go about it!” Let me illustrate by looking at some questions Larry should not be trying to answer:

  1. Did Jane answer all my questions correctly?
  2. Was Jane quick on her feet and graceful under pressure?
  3. Did Jane pick up on my algorithmic hints?
  4. Was her solution complete?
  5. Do I like Jane, as a human being? Could we be friends?

Those are certainly signals one can get in an interview, but are they relevant to her success in the role? Does her answering all of Larry’s questions correctly mean there’s a good chance she’ll get a high performance rating next year? It depends on the questions, right? Well then, what about the inverse: does getting answers wrong correlate with bad performance reviews? Or does it correlate with nervousness? Or miscommunication? Or ignorance of a concept that Jane could learn in the first week on the job?

The hard truth is that while most interviews test something, that something is more likely to be “ability to pass our interview” than “ability to do the job well”. Which — if the abstract interview is indeed a conscious choice, which more often it is not — then the argument for it goes like this:

We want people that will do what they need to in order to succeed; if they study hard for our arbitrary and irrelevant interview process and pass it, it means they (a) really want to work here, and (b) can do the same for a real-world project

Even for well-known companies that can attract enough people to run their gauntlet, this is of dubious value, because they’re filtering for a specific criteria: studiousness; and simultaneously filtering out many that are at odds with it, starting with people that don’t have the luxury of time to study for their interview.

Instead, I think interview questions should be relevant for the job and tailored to illuminate whether Jane will be successful in it. Does it matter if Jane came up with a coding solution in 10 minutes versus 20? How much time would she have in the course of her job for that problem? Does the job require her to be well-versed in algorithms? Even if Stack Overflow didn’t exist. Is it relevant that she didn’t finish the last part, even though she explained how it would work? Are the people we like more productive than the ones we don’t? Do these rhetorical questions make my point?

Which is that the interview is not a test and it’s certainly not an interrogation. It is above anything else a conversation, ideally between equals, in which both parties are trying to figure out if an employment arrangement would be a win/win scenario. And as the interviewer, by leading it with empathy, you accomplish two things:

  1. Have a much better chance at arriving at the truth
  2. Leave the candidate with a great impression

So how should Larry approach the interview? I like to think about role models, because we humans are great at mimicking, and in this case I think the right archetype is a podcaster. They have a guest on their show and they generally try to make the experience pleasant, to keep the content interesting, and to really get at what makes their guest tick in their particular way. If they have an actor on they try to figure out what makes them a great actor, if it’s a business tycoon, what makes them great at business, if it’s a scientist, what makes them great at science. And similarly, Larry’s job is to figure out what makes his guest great at programming.

In order to successfully do that, the guest has to first and foremost be comfortable. People won’t let you in unless they’re comfortable. And if they won’t let you in, it is a giant barrier to understanding them. So Larry should spend some time in the beginning of the interview breaking that ice. He should make small talk — genuine small talk, not awkward conversation about the weather. Tell Jane a little about himself: what he does at the company, what he’s passionate about in his work, and what his background is. Things that will give Jane an understanding of him and help them find common ground. It’s not only worth the investment, but it’s what makes the rest of the interview worthwhile.

Once a good rapport has been established — or Larry’s given up on such happening — only then should he go into the topics he needs to cover. And he should always remember to treat Jane as he would a valued guest in his home. To be kind, to be forgiving, and to give her the benefit of the doubt. If she says something that sounds incorrect, he should make sure it’s not due to a simple mistake or misunderstanding. He should ask polite questions that help him understand how much Jane knows and understands about graph traversal or whatever — not merely that she does (or doesn’t) know that those words are the answer to his question. Because in the end, Larry’s job isn’t that of a proctor, to simply grade Jane on her performance as if this were an audition or a midterm exam. Larry’s job is actually much more difficult: it’s to create, in 45 to 60 minutes, a sorely incomplete mental model of Jane from a certain angle — be it programming ability or cultural fit or leadership style or what have you — and to then decide if that mental model is a good match to fill the open role.

Again: this is hard and it takes a lot of practice to do well, just like dancing. But taking shortcuts will just lead to lots of false positives or negatives. Tech companies love to industrialize the process and create complex questions with “objective” answers and rubrics that tally up those answers into a simple pass/fail exam and then also point to that process when talking about fairness. But in truth, there’s nothing fair about standardized tests — which is why higher education is finally moving away from the SATs.

They add a veneer of objectivity on top an industrialized process that answers not so much what a person understands, but what they’ve been exposed to and what they can quickly recall in a stressful situation. It’s like being tested for ticking time bomb diffusal for a job making watches. And the opportunities for subjectivity still abound: from how comfortable or nervous the candidate is, to how many hints they’re given, to how much sleep they got the night before, to whether they’ve seen a similar question recently, and to the leniency of the proctor.

What sets Oxford and Cambridge apart from most other universities is that they use the Tutorial System, in which students learn the subject matter in whatever way makes sense, meet in very small groups with a tutor and have a discussion — in which it will become readily apparent how well they understand the subject.

It has been argued that the tutorial system has great value as a pedagogic model because it creates learning and assessment opportunities which are highly authentic and difficult to fake.

from Tutorial System at Wikipedia

Software interviews of a similar ilk have the same value.

But the one thing to remember is that regardless of the questions we ask when we’re being the Larry — no matter how good or fair or comprehensive they are — the questions are just a means to an end, and not the end. They are a conversation starter, and it’s up to us to guide that conversation in such a way that will allow us to understand our guest to such a degree that we can answer the only question that matters: “is this person likely to be successful in this role?”

Lessons Learned on the Job Search

I’m starting what I’m sure will be a great new job at an awesome company soon, but before I got that offer (and four other great ones), I was on the interview circuit for a few months. When I started looking, I figured that in this boiling job market (of mid-late 2021) it would take like around a month. For better or worse though, I hit a string of bad luck on what turned out to be an already overly-optimistic timeline: two hiring freezes (both at the offer stage in the interview loops) and a conflict of interest (after an offer).

I say “for better”, because this bad luck did come with a silver lining: it prolonged the process enough that I ended up in a lot of interviews: 79, over 34 interview loops. Which, on the one hand, was exhausting; but on the other hand, it gave me huge exposure to the current landscape of interview processes for engineering leadership at tech companies. I’ll write another article on how I think these processes can be improved, but this one is about how to deal with them — as a job seeker. Specifically, an engineering leader, such as management, or staff-level IC and higher.

This experience was so radically eye-opening for me, because aside from a random interview now and then, I’d never been exposed to it on this level. And in the pre-pandemic world, this gauntlet would’ve been impossible, due to the travel alone. My previous jobs came to me — which is a completely different dynamic — so I’m writing this to share what I’ve learned, mainly for others that are looking for a change, and are as unfamiliar with the landscape as I was.

The main takeaway: interviewing is a skill onto itself. It’s largely unrelated to the day-to-day of the job, it matters a great deal, and it’s absolutely something that can be learned. It’s like dating that way: the first few dates are crucial, and making a good impression is paramount, but the impressions you get on those early dates contain a lot of not only false signals — like being nervous or interested in everything your date loves — but also superfluous signals, that don’t matter a year into the relationship — like your favorite band.

And unfortunately, both dating and interviewing are optimized for charming (and good looking) people, who will always win out if everything else is equal. Fortunately, being a polished interviewee is a skill that I think most people can master with practice. Not unlike the premise of the movie Hitch. Because you could be the most amazing engineering leader in the world but, if you don’t interview well, no one will ever know that — aside from the people who’ve worked with you.

Before we get into the details, some more context: I interviewed with a wide range of companies, from Big Tech like Meta/Facebook and Amazon to smaller ones like like Etsy and Reddit, to startups at various stages. I applied for only remote roles, roughly evenly split between Engineering Manager (EM), Sr. EM, and Director — with a few senior IC roles added in — and all my interviews were remote, via Zoom or the like.

And I kept statistics.

But don’t worry, there’s a TL;DR section at the end that you can always skip to, if it gets too dry.

Anatomy of a Loop

Most companies have a 4±1 step process:

  1. Screening call with an internal recruiter
    • 30% of my interview loops skipped this step
  2. A first round, which is generally a role-fit discussion with the hiring manager (HM)
    • I consider this to be the first round of the loop
    • For me, 76% of the 25 of these I had were just a casual discussion with the HM
    • Of the remaining six:
      • four were a panel (two of which had technical components)
      • one was a coding challenge
      • one was a design challenge
  3. A second round, consisting of one or more people, sometimes as a panel (meaning multiple interviewers are in the video call), sometimes as a string of video calls with individual interviewers. But all pre-scheduled together.
    • Eight of the 13 I went through had a technical bent:
      • four included a technical chat
      • three design challenges
      • one coding challenge
    • Six of them were also panels:
      • one of the design challenges, and three technical chats
  4. A third round, which is usually the opposite of the second: if they did a panel first, then this is not a panel; and vice-versa. If they have a technical screen and the second round was technical, then this won’t be; and vice-versa.
    • 29% of my interview loops skipped this step
    • Three of my five were panels, and one of those was technical
    • The other two were informal chats with a CTO or VP
  5. Hopefully, an offer
    • Most of my interview loops skipped this step 🙂
    • All five of the offers I got had a fairly consistent base, plus 10-20% bonus, but big variation on equity — not just in terms of value, but also type: from no equity to private options to public RSUs. Equity ended up being the deciding factor for me.

For me, the average time from resume submission to some kind of decision was 32 days, but the max was 93 days, and 18% took over 45 days. Most of this was just waiting a long time for the initial response, and then for the next steps to be scheduled, each of which generally took a week or two.

The average time from resume submission to some kind of response was 17 days, but the max was 70.

What Worked at Each Stage

Resume

As you can see above, this was the biggest rejection stage, by a Grand Canyon-sized margin. A full quarter of my resume submissions were rejected, but even more (39%) were ghosted, and I never heard anything about them. I put a lot of work on my resume — and it always helps for it to be short (one page, if possible, because no one has time to screen them), to include the buzz words you’re skilled with and interested in (to make it past recruiters), and to be visually attractive (you can find lots of great templates for Google Docs on Etsy, for under 5$) — but that’s not what made the difference, as I found:

It basically all hinged on referrals, which increased my chances of getting to a recruiter or hiring manager from 5% to a whopping 63%! That’s a 12x improvement, which is impossible to get any other way. I’m sure it helped that my resume looked sharp, but I’m also sure that even with my previous, plain and long-ish resume, the referral success rate would still be multiples of the cold application.

So if you’re looking for a job: tell everyone you know in the industry, and hopefully they know of an opening or know someone who does and even if they can’t vouch for you personally: if all they do is pass your resume along with a “this person might be good for this role”, that itself does wonders. Also, if you find a job posting you really like, try to find a recruiter or someone in your network from that company. If you don’t know anyone directly, look at 2nd connections on LinkedIn, and then ask your mutual connection to introduce you.

Ironically though, in spite of all of this, I ended up at one of the 3 companies in the bar graph above where I had no network connection. And for all three of those, I wrote a thoughtful message in the freespace of the application. Now, I also did that for probably at least a dozen others, so not a great success rate there, but: if you’re applying cold, it definitely helps to write the recruiter a note about why they should choose your resume out of the pile. None of my other cold applications made it out alive.

The other big category is getting recruited. Most of the ones that reached out to me were either Big Tech companies or startups I’d never heard of, and most of those were clearly automated messages. I entertained almost all of them, if for nothing other than the interview experience. And here, LinkedIn is key: using the appropriate phrasing and buzz words and highlighting the experience relevant for the kind of job you want. Recruiters use various tools to search through the hundreds of thousands of potential candidates, and a little SEO goes a long way.

Recruiter Screen

At this stage I did pretty well: 3 rejections out of 18 conversations, for a pass rate of 83%.

What the recruiter wants is to make sure you’re not going to embarass them. They’ve seen your resume and decided that based on your history and so forth, you’d probably be a good candidate. So all you really have to do is sound competent, back up your resume with your voice, and fit within the logistical parameters for compensation, location, time zone, etc.

Hiring Manager Screen

Of the 19 HM screens I was in, my pass rate was 68%, and I have to say that most of the rejections at this stage were surprising to me. I thought almost all of them went well, and in fact I had at least two of the HMs tell me they’d like to move me forward, only to get a Dear John email a few days later from the recruiter. I imagine they talked to someone they liked more the next day, but one of the more frustrating parts of the process is that rejection feedback is exceedingly rare. The unfortunate truth is that the overwhelming majority of companies do not provide feedback, largely for legal reasons.

But like the recruiter, the HM largely wants to make sure the person behind the resume is a solid candidate, that they seem personable and would fit on the team, and at least part of this conversation is them selling you on the job. They ask more poigniant questions than the recruiter obviously, and have more details around the position and what they’re looking for and have a better sense of how you’d succeed in the position, but largely this stage shouldn’t be hard, and if you don’t pass, you probably wouldn’t have fit in on the team anyway — not that you’re not qualified, but team dynamic is a real thing.

Behavioral Interview

The vast majority of leadership interviews I was in were what has become the norm the tech industry: storytelling. I can’t tell you how much PTSD I have over the phrase “tell me about a time when …”. The idea is that an experienced manager can (a) demonstrate that experience by recollecting tales from a storied career, and (b) knows to emphasize the parts that the interviewer wants them to emphasize. In the beginning of my job search, I would fail miserably at the second part with the mind-reading, because I didn’t realize there was a hidden agenda to the question.

So they would ask me to tell them about a time when I promoted someone, and I’d literally tell them about such a thing and move on. When in fact, they also wanted me to tell them about my philosophy (a.k.a framework) around promotions, and maybe even around career development in general, and emphasize how that’s an important part of a manager’s job, etc, etc. And because of that, my pass rate in the second round (where behavioral interviews would often take place) fell to 54%, and was my lowest. And that’s after I cracked the code. It helped when reviewers would nudge me in a certain direction, but that didn’t happen often, and certainly not with Big Tech, where the rubric is king.

If you’re interested in seeing examples of this tactic, watch some videos on ExponentTV (and paying for Exponent is probably a great return on investment as well) to get a better sense of the kind of answer interviewers are looking for. And make a list of stories you can cycle through quickly, on the spot. The answers are supposed to given in the STAR format. There are various lists out there, but here are some of the ones I actually got:

Tell me about a time when…

  1. you had to manage someone out
  2. you had to deal with a difficult employee
  3. you had a disagreement with a peer
  4. you promoted someone
  5. your project was late
  6. you failed
  7. you dealt with a DEI issue
  8. you motivated your team
  9. you affected change without using authority
  10. you were part of a large undertaking

One prevalent theme was essentially #6 above, and that was also something I wasn’t quite sure how to answer. It felt like the old joke to me, about the interviewer asking what the candidate’s greatest weakness is, who replies that he works too hard.

But the reason people ask this is twofold: first, because experience implies failure, since no one has a perfect batting average. And so the idea is that if you’re able to talk about a time you’ve failed, you’re more likely to be an experienced leader. Second, being able to talk about failure in a way that shows humility is important in that it demonstrates some good leadership qualities such as empathy, the ability to learn from mistakes, and the ability to turn things arounds.

And herein is why getting good at interviewing and storytelling is so important, because it’s not enough to merely have those qualities: you have to realize when you’re being subtly prompted to reveal them and to do so in a tactful way. But again: it’s definitely a skill that can be learned.

Technical Interview

Only 20% of my post-recruiter-screen interviews were rigorous technical ones, and I think about half of the loops had such a stage. So it’s definitely possible to get a job without running into one, but they’re very common in Big Tech.

Of my nine technical interviews, five were design questions — and one of those was a take-home exercise. Another two were presentations of a project I’d done in the past, and the last two were coding challenges.

One of the coding challenges consisted of two leetcode-type algorithmic/data structure questions. There’s nothing to be done about this except grinding on leetcode until that part of the brain that lay dormant since your CS college courses has sprung back awake. Yes, it has nothing to do with anything in a real job, but such is life.

For the presentations, I gave essentially the same one both times: passed one, failed the other.

The design questions generally ask you to design some kind of system you’ve likely dealt with before: a search engine, a messaging service, a URL shortener. Here, it’s important to start at a high level — the main components and connections — make sure you’ve covered enough there (and take hints in the form of questions from the interviewer) and then go a level lower and talk about technologies, protocols, tradeoffs, and so on. Anything to make the interviewer feel comfortable that you have the technical chops to design things. Again, Exponent has some great videos to give you an idea of what a good one looks like.

TL;DR

Interviewing processes are very broken at most companies, and as a job seeker, there’s not much you can do about it except refusing to take part in broken processes — such as live coding challenges — if you have the luxury of doing so. For many people though, especially in the new remote world of SF/NY salaries, this would mean walking away from a significant pay raise.

Otherwise, the things I wish I had known at the onset:

  1. It will likely be a multi-month process, due to the speed at which most companies operate; I would prepare for roughly 3 months
  2. The best thing to get your resume noticed is to make use of your network
  3. When you do have to cold apply, writing a paragraph to the recruiter is worth the investment
  4. A polished, short, beautiful resume is worth the investment
  5. Optimizing your LinkedIn profile for recruiter searches is worth the investment
  6. Feedback, about why you were rejected, is an endangered bald eagle: rare to come across and wondrous to behold
  7. Practicing storytelling for the behavioral interview, and looking for the question behind the question, are both crucial
  8. Practicing leetcode is needed for many interviews at Big Tech companies
  9. Good PowerPoint skillz might come in handy
  10. There are lots of mock interviews available to watch — both behavioral and design — and they help a lot

All of this can be boiled down to, as Patrick McKenzie put it, “becoming aware of how power operates (versus how virtue is generally described) and choosing to join it”. Like with many concepts in the tech world (microservices, cloud computing, Agile project management) this fairly narrow interview process has evolved and largely caught on, and being familiar with it is a shibboleth that’s as important as anything else in a job search, in the early-2020s tech world anyway.