allenc allencheung

Conducting Developer Interviews

Some comments from my last post and on Hacker News brought up an interesting point: it’s not easy to conduct software engineering interviews. If you’ve ever looked for a job as a developer, chances are you have been unfairly grilled by bad interviewers, and hopefully been pleasantly surprised by a few good interviews as well.

With so many companies looking to hire “rockstar developers”, the increased movement in the job market has resulted in more developer interviews, so getting the process right makes a bigger impact now more than ever. And while plenty of literature has been written on how to prepare the interviewee to face the onslaught of riddles, mental jumping jacks and algorithmic conundrums, prepping the interviewer is usually left up to the company, which for most startups and small firms is non-existent.

These are some ideas and routines I’ve tried out. I’m not claiming to be some great interviewer here; I was just tasked, for the past ~7 years, to interview engineering candidates of all stripes, and by rote repetition have worked out a system that works for me. That said, every interviewer has his or her own style, so you may find something useful to incorporate into your interviews, and I’d be interested to hear about other engineers’ techniques as well.

I’m just going to list things in chronological order here.

Doing your homework

Before you step into the interview room, you should have prepared yourself. The obvious part is making sure you have a set of interesting technical challenges for the interviewee, and while I’ve seen some interviewers prepare 8-10 quick-fire question sets, I personally prefer having one or two questions that are open enough for us to follow up in multiple areas.

The slightly less obvious, but still pretty discernible piece of preparation is going over the interviewee’s resume. I like to look at the interviewee’s work experience{{1}}, familiarity with areas of software development, and at least glance at their educational background{{2}}; these data points let me tailor my questions to the candidate and try to test him on familiar subjects. Some might argue that this is coddling the candidate and that they should answer to any of your questions; I say that at this point, they’ve probably been vetted by someone else to be a potential fit for the position and it’s in your best interest to bring out his best – regardless of whether said best will pass your hiring bar.

Warming up

Interviewing sucks. Stuck inside a small room with poor ventilation, with a progressively judgmental stream of engineers sneering at you as they walk through the door, the sweat from your timid handshake barely dry before they ask you to solve Traveling Salesman in constant time. If your nervous quota has been low this year, a tough technical interview or two will shoot it right back up.

So I always try to start off with something non-technical, driven by the interviewee, so they get a slight confidence boost in being able to talk about something they’re intimately familiar with: a previous project, a favorite language or framework, an article they just read. I totally sympathize with interview butterflies, so getting them at ease is an important starting point.

Gauging the Depth (on a phone screen)

So far, anything with a pulse has aced the interview. At this point, if I’m on a phone screen, I like to start probing at their depth of knowledge based on their background. A new CS grad – should be familiar with algorithms, big-O notation, and data structures; web developer – can either go the UI route and ask for various CSS properties, or focus on AJAX and XHR usage; backend engineer – can ask about database sharding, TCP/IP implementations, and threading models, depending on their field of specialization. Some interviewers like to spend their entire time going through these quick-fire phone screen questions{{3}}; I only like doing one or two, as a gauge of their competence, but also to see what kind of a role I should take for the rest of the interview.

Most of the time, I’m the one asking the questions, so I ought to know the answers; I’m comparing their work to what I already know is correct, and if they get stuck I should be able to drive the interview forward. Once in a while, though, I’ll be talking to someone coming from something I’m completely unfamiliar with, or someone way more senior – and I realize, besides that HR has again screwed up interview placements, that if we were to hire this candidate my interactions with this engineer will involve him teaching me what he knows. In that case, I’d better make sure he can teach.

The Question

Alright, 15 minutes in, time for the real challenge.

Now, I know a lot of interviewers like asking specific coding question, implementing a sorting function that would be in any core library, except that rats stole our linked list implementation so we’re stuck using stacks and our hardware was built in Africa so we have to run in O(ln(ln(n)) time. There’s usually a brute-force solution that would incinerate the hardware if we actually tried running it, a slightly more optimized solution, and a clever answer that tells the interviewer you did a good job studying interview questions.

Me, I prefer asking a more open-ended question that starts at a systems design level, and has enough complex parts to be drilled down, coding- and design-wise, as time permits. This way, I can be completely flexible and can adjust based on need (e.g., his previous interviews had him code, let’s stick with design and testing methodologies) as well as candidate strength (e.g., I know this guy probably won’t pass, let’s stay at a high level and not have him get frustrated at trying to implement it). Better yet if I can distill a recent problem I had to solve into a more general question; I’m familiar with a solution, the parameters are much more realistic, and there’s an obvious application.

I realize this may not be possible for all engineering subspecialties – front-end development just happens to encompass wide-enough of an area for me to talk about anything from UI design to data latencies. Though, I still like the idea of asking rich questions that can open up into multiple areas, each of which can be explored in gritty technical detail if need be. I also realize that sometimes, you’re the last interviewer, the last three guys did exactly this and just really needs to know whether this guy can code, so you throw all this away and ask him to implement Traveling Salesman in O(ln(ln(n)) time.

Keeping a Diary

Once upon a time, I would walk into interviews, talk with a candidate for 45 minutes, and remember enough of it that 3 hours later the engineers would reconvene and I would give my impressions.

I’ve since given up that practice.

Taking notes during an interview can be disruptive and distracting, but I can think of no better way to objectively record what was actually said, uncolored by the funny quip the candidate uttered as you walk out of the room or the blank whiteboard after the candidate quickly erased his solution. Writing it down lets you sit on the feedback, then come back to it in a few hours or days and reflect on his or her performance. And although I usually give my interview feedback right after, I still find it valuable in reminding me of certain mannerisms, quirks, behaviors, and coding details. Write stuff down.

Reversing the Roles

At the end of an interview, I always leave a few minutes for the candidate to ask me about my job, the company, or whether the food is really as good as they say. It’s only fair that they get to grill you just a tiny bit, and it’s another data point revealing what they feel is important and what they actually care about. At the same time, it’s a way to just wind down the interview, make a quick pitch, and make them feel good about navigating your grueling technical gauntlet.

Hopefully, this has helped someone out there give better interviews. Sure, interviewing is an art and so there’s no right-or-wrong way to go about it, but then again, there’s a lot of ugly art out there. Again, I’m far from an expert on this stuff, so if you think I’m completely wrong or have additional pointers, feel free to share.

[[1]]The words they use to describe their role can be especially telling, vague “contributed to project” claims are a red flag.[[1]] [[2]]This is more to check whether they studied computer science; some of the best developers I know didn’t attend CS courses in school, and when I encounter a non-CS major I make sure to avoid CS-centric topics like runtime analysis.[[2]] [[3]]Incidentally, does anybody actually whip out a formula and try to calculate a candidate’s score when they read through their list of questions? I mean, if you give good two-word answers to, say, 6 out of 9 questions asked, does that mean you pass the screen?[[3]]
By allen
allenc allencheung