Coding Interviews and AI: Are Traditional Methods Still Relevant?

The big question

In today’s rapidly evolving tech industry, traditional coding interviews are increasingly under scrutiny. Are they still relevant, especially with the rise of AI technologies?

I am genuinely unsure. I have feelings about traditional coding interviews, but those are not appropriate for this forum. I am specifically talking about coding interviews when a candidate has to code on a whiteboard or some sort of text editor, that is not their preferred IDE.

First principles

Here are my thoughts. From first principles, the goal of an interview is to assess how well a person can perform the job they are applying for. So, whatever you ask during an interview should be the entrance criteria. Imagine it as defining a ‘shape’: if a candidate passes through this shape, they can be hired. In practical terms, if your job description emphasizes ‘TDD, BDD, CI/CD’, but your interview focuses on solving Depth First Search on a Binary Tree, your ‘shape’ is misaligned.

FAANG companies often use such methods out of necessity. With tens of thousands of developers, they need a consistent entry bar. They opt for the biggest common denominator: ‘you can do the coding interview in any language known to humanity,’ using a whiteboard or Google Docs to equalize the tooling bar. This isn’t because it’s the most effective method, but because they don’t have much choice. However, if your company isn’t FAANG, you don’t have to follow this approach.

Imagine if you were to hire a carpenter, and ask candidates to build a table using stone stone axe only. What would you learn from this? What if you let that carpenter use their tools, or give them tools they will use on the job?

I have used my IDE for 11 years now (I used another one for the first 5 years of my career). I have read every release note for the last 11 years and mastered it inside and out. My productivity with my IDE versus without it is incomparable. I also use AI effectively, which significantly contributes to my value as a professional, beyond just writing code. Is that irrelevant?

Modern reality through personal stories

Today this approach is even more detached from reality. Here are a few stories I wanted to share.

Using AI to pass coding interview

I came across an interesting position at Pulley. In the job description they had this 🚨 NOTE: We've gotten an influx of applications recently, and are offering an open coding challenge to anyone who's interested in expediting their process. You can find it at https://ciphersprint.pulley.com/; it will give you all instructions needed. I genuinely love this kind of approach to a tech interview, "Here is the problem, solve it in any way you want." The thing is, that the position is for a Staff Software Engineer, AI, so why would I code the solution myself? I should build an AI system that does it for me. If you are interested here is the code https://gist.github.com/oantoshchenko/460abfb676daba7c65c1e869187fae73.

The moral of this story is not that I solved a particular problem using AI, by asking ChatGPT to solve every challenge for me. I built a multi-agent AI system that progressively writes and adapts code for multiple challenges, executes it, and submits the solution autonomously. I did not even look past the first 2 challenges (there are 6); the AI system I built did all that for me.

Do you think I have cheated? Do you still care if I can solve those problems myself? If so, why? The analogy for this would be if I solved this problem using, say, Java, and you asked me to implement the solution in Assembly, because you want to see that I really understand how the fundamentals work. Aren't we past that today?

We build AI driven coding tool, but you can't use it

I also had a chat with a company that builds a coding tool that uses AI. They asked me to do the coding interview in Google Docs. So naturally I asked if I could use their tool instead, and they said "no." The irony was lost on them somehow. I am not even talking about "dogfooding" your own product, but this is the whole proposition of your business. Just... Why?

Questions and conclusions

AI-generated code is just a tool, like an IDE or programming language. Dismissing it because ‘I tried ChatGPT once and didn’t like the outcome’ is shortsighted. The real question is how effectively these tools can be integrated into our workflow. And as such, they should be part of an interview process. Shouldn't they?

Why do small companies, that can ask very specific questions, and can define the "shape" of their interview very precisely opt for a "square?" You can see how productive or unproductive people are in their "natural" environment. So why use Google Docs?

I know I might sound polarized and convinced, but I am not. I genuinely believe that I am missing some deep wisdom. Please let me know what you think!