The Tech Interview Is Broken!

At least, the entry level tech interview is.  Experienced software engineering candidates are, well, experienced:  They should have specialized skills which lend themselves to detailed, specific technical interviews which can identify an ideal candidate with an ideal set of skills.  Entry-level software engineers, on the other hand, generally do not have specialized skill sets, so their interviews must be much more general.  Over the past couple of decades, the professional software engineering community has developed and solidified widespread industry standards for entry-level technical interviews.  In many cases, the interviews using these standards do not efficiently identify the best candidates for entry-level software engineering positions.

Disclaimer

Before I get started, let me clear the air by saying that I’m not a salty wannabe sick of being rejected!  I work as a Software Engineer II (L4 for those on the Levels system), and I interned at four other companies before graduation.  I’ve passed and failed many entry-level technical interviews and have even given out a few of my own.  I’m not a hiring manager, but I know the tech interview will, and I know what it takes to succeed as an SE0/1/2.

The Average Interview

The modern entry-level technical interview consists of one near-universal component—the algorithmic question(s)—and one or more of a handful of optional components:

• Simple implementation in the primary language / framework of the position
• General problem solving (not coding)
• Technical knowledge discussion
• Test case creation
• Basic architecting

More specialized positions—like iOS app developer—are more likely to include a simple implementation question to confirm that the candidate at least understands the absolute basics of developing in that framework.  The tech giants—intent on appearing highly selective—are more likely to include a general problem solving question to confirm that candidates can think on their feet.  Some companies include a technical knowledge discussion, ensuring candidates can “speak the language” of the trade, others include test case creation to make candidates prove their skill at spotting edge cases and thinking outside the box, and still others include a simple architecting question if the interview is for a backend position.  All in all, if you are about to jump into an entry-level technical interview, you can be certain you will encounter an algorithmic question, and you can expect any number of the other five sections; only in rare cases will you encounter something else entirely.

False Negatives

The common criticism of this format is that its central component—the algorithmic question—is not very pertinent to the actual work done by most software engineers, leading to false negatives.  I mostly agree with this view.  I do not write complex algorithms in my job; rather, that work is abstracted away for me in importable open source packages.  If you work in software, in practically any subdomain except embedded systems, then I am sure it is the same for you.  Algorithm questions, then—by their nature of not directly representing the job—absolutely exclude people who would otherwise be capable engineers.

The common counterargument to this is that algorithm questions are just the best we’ve got to quickly evaluate problem-solving and coding skill.  There is some truth to this.  I don’t think writing algorithms is the best way to evaluate coding skill (we’ll discuss alternatives later), but it is the simplest to set up and administer.  I don’t need to do anything to prepare to ask a candidate to write code to search lists, detect palindromes, or invert binary trees, aside from ensuring that I can identify effective and ineffective solutions.  Algorithms don’t need any starter code, they can be written in any programming language, and they directly communicate candidates’ skills in certain areas.  However, they only test coding skill and problem-solving secondarily; primarily algorithmic questions test knowledge of algorithms, and that’s just not a skill you need.

False Positives

Less concerning to candidates, but more concerning to companies, focusing on algorithms as the core piece of the technical interview also produces false positives.  As discussed above, knowledge of algorithms will not be useful to most software engineers day-to-day, so having that knowledge is not necessarily indicative of an engineer’s day-to-day abilities.  In theory, engineers with natural talent in algorithms are effective problem solvers, and engineers who practice their algorithmic skills are dedicated to their trade, so the correlation between engineers with algorithmic skill and successful professional engineers should be strong.  However, companies must rely on this correlation holding true, and that’s simply not always the case.  Ineffective engineers can pass algorithm questions too.  So, if testing proxy skills and hoping they translate is flawed, why not test the most important software engineering skills directly?

The Solution

We’ve talked a lot about what isn’t core to a software engineer’s job, but what is?  Well, for engineers working on large systems, the most important technical skill is that of navigating and working within the complicated, separate-but-interrelated pieces which make up these systems.  Most engineers almost never write code in a vacuum, instead having to poke, prod, and stretch multi-thousand-line codebases to implement even the simplest feature.  So, let’s test this skill!  Give candidates large, complicated, deprecated (if your code isn’t open source) systems and ask them to add or modify a feature.  If it is a smaller system and feature, the interviewer can watch in on the candidate’s process of learning and working on the system, or, if it is larger, the candidate can be given a day to work asynchronously to produce a pull request.  Then, after each interview, the pull request can be deleted, leaving the system clean and ready for the next candidate.  Have multiple candidates at once?  Fork the system for each.  Interviewees will have to learn the codebase(s), find the right spot(s) to implement their feature, write their code without breaking other features, and hopefully round it out with a unit test or two.  And they’ll have to do it all under a time crunch to prove their efficiency.  Ultimately, this is nearly identical to a software engineer’s normal workflow.

This true test of a candidate’s working skill should be the basis of the technical interview, and the algorithmic question should be relegated into the group of optional sections.  Yes, preparing a real codebase for interviews can be complicated, and yes, ensuring the candidate has a working development environment may be difficult, but the result will be a more fair and accurate interview.  By deprioritizing algorithmic skills, candidates lacking in this area but excelling in real systems development will no longer be incorrectly weeded out, and candidates who grind HackerRank but can’t quickly produce functional code within large systems will no longer slip in unnoticed.  Let’s stop pretending that an abstracted test of problem-solving skills is the best we can muster, and let’s start finding employees who can navigate the massive webs of code on which most of us work.