Some comments from my last post and on Hacker News brought up an interesting point: it’s not easy to conduct software engineering interviews. If you’ve ever looked for a job as a developer, chances are you have been unfairly grilled by bad interviewers, and hopefully been pleasantly surprised by a few good interviews as well.
With so many companies looking to hire “rockstar developers”, the increased movement in the job market has resulted in more developer interviews, so getting the process right makes a bigger impact now more than ever. And while plenty of literature has been written on how to prepare the interviewee to face the onslaught of riddles, mental jumping jacks and algorithmic conundrums, prepping the interviewer is usually left up to the company, which for most startups and small firms is non-existent.
These are some ideas and routines I’ve tried out. I’m not claiming to be some great interviewer here; I was just tasked, for the past ~7 years, to interview engineering candidates of all stripes, and by rote repetition have worked out a system that works for me. That said, every interviewer has his or her own style, so you may find something useful to incorporate into your interviews, and I’d be interested to hear about other engineers’ techniques as well.
I’m just going to list things in chronological order here.
Doing your homework
Before you step into the interview room, you should have prepared yourself. The obvious part is making sure you have a set of interesting technical challenges for the interviewee, and while I’ve seen some interviewers prepare 8-10 quick-fire question sets, I personally prefer having one or two questions that are open enough for us to follow up in multiple areas.
The slightly less obvious, but still pretty discernible piece of preparation is going over the interviewee’s resume. I like to look at the interviewee’s work experience1, familiarity with areas of software development, and at least glance at their educational background2; these data points let me tailor my questions to the candidate and try to test him on familiar subjects.
Some might argue that this is coddling the candidate and that they should answer to any of your questions; I say that at this point, they’ve probably been vetted by someone else to be a potential fit for the position and it’s in your best interest to bring out his best – regardless of whether said best will pass your hiring bar.
Warming up
Interviewing sucks. Stuck inside a small room with poor ventilation, with a progressively judgmental stream of engineers sneering at you as they walk through the door, the sweat from your timid handshake barely dry before they ask you to solve Traveling Salesman in constant time. If your nervous quota has been low this year, a tough technical interview or two will shoot it right back up.
So I always try to start off with something non-technical, driven by the interviewee, so they get a slight confidence boost in being able to talk about something they’re intimately familiar with: a previous project, a favorite language or framework, an article they just read. I totally sympathize with interview butterflies, so getting them at ease is an important starting point.
Gauging the Depth (on a phone screen)
So far, anything with a pulse has aced the interview. At this point, if I’m on a phone screen, I like to start probing at their depth of knowledge based on their background. A new CS grad – should be familiar with algorithms, big-O notation, and data structures; web developer – can either go the UI route and ask for various CSS properties, or focus on AJAX and XHR usage; backend engineer – can ask about database sharding, TCP/IP implementations, and threading models, depending on their field of specialization. Some interviewers like to spend their entire time going through these quick-fire phone screen questions3; I only like doing one or two, as a gauge of their competence, but also to see what kind of a role I should take for the rest of the interview.
Most of the time, I’m the one asking the questions, so I ought to know the answers; I’m comparing their work to what I already know is correct, and if they get stuck I should be able to drive the interview forward. Once in a while, though, I’ll be talking to someone coming from something I’m completely unfamiliar with, or someone way more senior – and I realize, besides that HR has again screwed up interview placements, that if we were to hire this candidate my interactions with this engineer will involve him teaching me what he knows. In that case, I’d better make sure he can teach.
The Question
Alright, 15 minutes in, time for the real challenge.
Now, I know a lot of interviewers like asking specific coding question, implementing a sorting function that would be in any core library, except that rats stole our linked list implementation so we’re stuck using stacks and our hardware was built in Africa so we have to run in O(ln(ln(n)) time. There’s usually a brute-force solution that would incinerate the hardware if we actually tried running it, a slightly more optimized solution, and a clever answer that tells the interviewer you did a good job studying interview questions.
Me, I prefer asking a more open-ended question that starts at a systems design level, and has enough complex parts to be drilled down, coding- and design-wise, as time permits. This way, I can be completely flexible and can adjust based on need (e.g., his previous interviews had him code, let’s stick with design and testing methodologies) as well as candidate strength (e.g., I know this guy probably won’t pass, let’s stay at a high level and not have him get frustrated at trying to implement it). Better yet if I can distill a recent problem I had to solve into a more general question; I’m familiar with a solution, the parameters are much more realistic, and there’s an obvious application.
I realize this may not be possible for all engineering subspecialties – front-end development just happens to encompass wide-enough of an area for me to talk about anything from UI design to data latencies. Though, I still like the idea of asking rich questions that can open up into multiple areas, each of which can be explored in gritty technical detail if need be. I also realize that sometimes, you’re the last interviewer, the last three guys did exactly this and just really needs to know whether this guy can code, so you throw all this away and ask him to implement Traveling Salesman in O(ln(ln(n)) time.
Keeping a Diary
Once upon a time, I would walk into interviews, talk with a candidate for 45 minutes, and remember enough of it that 3 hours later the engineers would reconvene and I would give my impressions.
I’ve since given up that practice.
Taking notes during an interview can be disruptive and distracting, but I can think of no better way to objectively record what was actually said, uncolored by the funny quip the candidate uttered as you walk out of the room or the blank whiteboard after the candidate quickly erased his solution. Writing it down lets you sit on the feedback, then come back to it in a few hours or days and reflect on his or her performance. And although I usually give my interview feedback right after, I still find it valuable in reminding me of certain mannerisms, quirks, behaviors, and coding details. Write stuff down.
Reversing the Roles
At the end of an interview, I always leave a few minutes for the candidate to ask me about my job, the company, or whether the food is really as good as they say. It’s only fair that they get to grill you just a tiny bit, and it’s another data point revealing what they feel is important and what they actually care about. At the same time, it’s a way to just wind down the interview, make a quick pitch, and make them feel good about navigating your grueling technical gauntlet.
Hopefully, this has helped someone out there give better interviews. Sure, interviewing is an art and so there’s no right-or-wrong way to go about it, but then again, there’s a lot of ugly art out there. Again, I’m far from an expert on this stuff, so if you think I’m completely wrong or have additional pointers, feel free to share.
The words they use to describe their role can be especially telling, vague “contributed to project” claims are a red flag.↩
This is more to check whether they studied computer science; some of the best developers I know didn’t attend CS courses in school, and when I encounter a non-CS major I make sure to avoid CS-centric topics like runtime analysis.↩
Incidentally, does anybody actually whip out a formula and try to calculate a candidate’s score when they read through their list of questions? I mean, if you give good two-word answers to, say, 6 out of 9 questions asked, does that mean you pass the screen?↩
So from your piece here; you are looking for ‘rockstar developers’ that have NOT ‘contributed to project’; but of course…
Not at all. I simply say that while many companies are indeed looking for engineers, there’s a giant filter that is the interview process to go through, and I personally look at resumes that claim an engineer has “contributed to project x” as a red flag because of its vagueness and ambiguity. Of course, I’ll still talk to them, but from my exp. my initial suspicions turn out to be true.
Well they can say they worked on Project X or they can have a 5 page resume
Not necessarily. You ever hear the expression “less is more”? Just put down, in concrete terms, your most impressive accomplishments at a job, stuff like “increased conversion rates by 800% via AB-testing” or “Rewrote backend pipeline for CRM data”. Not be able to do that tells me you didn’t actually do that much in the project.
Incidentally, I hate people with 3+ page resumes (NOT a CV, however). It’s like they think they can impress recruiters with length and if they’ve looked at a line of Java code they’ll go into detail the 3 lines they wrote.
lolz, well maybe instead of putting all that effort into a resume that almost no one will read they should put that effort into a blog that almost no one will read…
Just wondering if Google has done any research on how effective their hiring methods are. What strikes me is that they may, in fact, be counterproductive. When you take a look at Microsoft, this is a company that for many years could hire the cream of the crop. It had a rigorous hiring process not all that different from what Google uses. Yet they consistently have problems just getting products out the door and the products they do get out the door are not terribly successful.
I sometimes wonder if this incredibly involved hiring process at places like Google and Microsoft isn’t more a kind of status symbol designed mainly to improve the social status of people who work there as opposed to serving a real purpose.
I mean is there any real difference in ability between the new college grads hired by Google and a random sample of people from top CS programs with 3.25 or better GPAs? My guess is the answer is no. So why bother with all the hoopla.
You’ve hit upon the fact that a good product requires more than just a group of talented engineers. Google’s process and culture optimizes for engineering quality, but obviously other aspects, such as project mgmt, QA, UI, marketing, etc. take a hit and it takes all of those to produce successful products.