What Are We Really Testing in Tech Interviews?

My worst moments

I was on a tech interview once, and I forgot that the @Autowired annotation exists in Spring. It's so basic, like breathing, to Java developers. You don't forget to breathe... Unless you're under immediate pressure. Like on tech interviews...

That was a very cringe moment for me that I don't love recalling, and afterwards, I felt so stupid. But it got me thinking. I actually know these things. If I'm not sure of something, I'll look it up immediately. And I'm not bad at my job. Actually... I'm doing really well. Especially considering I haven't been a software developer since the Stone Age, like some people. I made a total career switch just a few years ago. I ship reliable code. I'm not afraid to take on new challenges. I love learning new things and exploring new technologies. I frequently receive feedback from different client representatives on my clear communication. Many colleagues trust me with more than just code.

I thought about the interview: how I failed to answer some basic questions, yet later felt confident about more complicated ones. I have seen this kind of trivia testing pattern in interviews more than once, but what are we really testing for, though?

Once, I forgot my PIN code in the middle of paying at the store because, for security reasons, their card terminal displayed the numbers in an unusual order. My brain completely froze. I usually automatically enter my PIN using the pattern of number placement. I actually know the numbers too, but at that moment, I forgot everything. Does this mean I'm stupid and shouldn't be able to pay with my own card with my own money? No, it means that human psychology and physiology work in mysterious ways. We don't have control over everything our body and nervous system decides to do.

The real-life devs I admire

I started thinking about my own strengths and the qualities I secretly admire in senior devs I know. It's never about how many LeetCode interview questions they can memorize, or how much of the syntax they can correctly write by heart on paper. What I've seen effectively working in real-life software projects is a different story than just a nice database of tech knowledge in human form.

The best devs are the ones who understand how everything works as one system, who can foresee problems on the other side of the system when you make changes to one part. The best devs are the ones who don't make reckless changes or refactoring, but who also don't passively wait for someone to tell them exactly what to do. The best devs are people with integrity, critical thinking, initiative, ownership, a broad perspective, and strong empathy and communication skills. And the ones who know how to fill the information gaps best and use the right tools for specific tasks. These are some of my strengths, too, but these qualities rarely get tested in interviews.

The senior developers I admire and would hire in an instant are all good communicators, not just technically adequate. They never make me feel dumb, even when I forget something. They seem to love teaching, guiding, and supporting. They also listen. They know they don't know everything either, and they really listen. So when I come up with a good idea or solution, they don't relentlessly push their own original opinion on me - they adapt and help to make the outcome awesome. They also listen when someone is in trouble or overwhelmed and quietly strengthen their support.

Seriously. I've seen it in action. And people feel it. People feel part of the team and like everyone is working towards the same goals. No one has to feel alone with a monstrous problem. At least that's how it is in good teams. I've seen the other side as well.

Not so admirable examples

Because I come from a medical field and have some background in psychology, I'm naturally inclined to observe people. I've seen developers who are expert-level in some technologies or domains but almost always fail to state their opinion when it matters. When it's time to make decisions about future developments, they keep their mouths shut. Only later, when everything is in motion and already falling apart, it turns out they could have already predicted the outcome, but just failed to communicate it to the team. Why? There are many possibilities, but it's not because they're introverts, as some people like to hint. Please let's differentiate introversion from social anxiety and poor communication skills. Introverts recharge when alone. That's it. Extroverts can also have social anxiety and under-the-line communication skills.

I've also seen "expert" devs force their opinions on others, illicitly rewrite someone else's code for no good reason, and make others feel small and unappreciated. Or making reckless changes that break everything for everyone else, creating extra work and time pressure. Without proper communication, of course, too.

I'm sure some of these later-mentioned devs would have aced that interview if they had been in my place. They wouldn't have suddenly forgotten the damn basics. But who are companies really looking to hire?

What exactly are we hiring for?

I don't really know. Perhaps I'm an idealist in a tech world, but I also know people and systems, and I've seen enough by now to make my own conclusions.

If I were hiring and interviewing new devs for my fictional company, instead of asking people to recite details about OAuth2 with pen and paper, I would create some short stories for them to discuss. Firstly, that would take the pressure off the candidates to perform in a certain way. Secondly, you can't memorize having a good character. You can't train online to be a good team player. You can't practice caring. And you can't really Google or ChatGPT your way into those things either.

What I would do to select truly valuable candidates

1. Problem-solving, not trivia

Instead of: Which annotations are used for dependency injection?

I would ask: You join a project, and the codebase is a mess: tight coupling, no tests, weird side effects. You get a medium-sized feature. Please tell me how you'd approach this situation from the first read of the ticket to merging the change.

From this question, I could learn:

• They explore first or just start coding?
• Do they write tests or at least think about them?
• Do they refactor a reasonable amount of code or just add to the chaos?
• Do they communicate, ask questions, and clarify expectations?

2. Teamwork and ownership

Instead of: What's your experience with Kafka?

I would ask: Tell me about a time something important was dropped or unclear in a project. What did you do? And what did the rest of the team do?

From this question, I could learn:

• Do they blame and whine or take some ownership?
• Can they see system-level problems like work processes and communication?
• Do they have any ideas for improving things instead of just complaining?

3. Handling uncertainty

I would ask: Here's a tiny spec for a feature in an unfamiliar domain. Show me how you would start working on it? What questions would you ask? What would you test first?

From this question, I could learn:

• Are they comfortable saying "I don't know yet"?
• Do they think in iterations, not in "one perfect shot"?
• Can they prioritize what to figure out now vs later?

4. Emotional and interpersonal awareness

I would ask: Tell me about a time when you worked with someone much more senior than you and someone much more junior than you. What was hard? What did you learn from each?

From this question, I would learn:

• Can they handle hierarchies without sucking up or bullying down?
• Do they show basic empathy or only talk about code?
• Can they learn from people or just from tutorials?

5. Realistic tech knowledge questions

Instead of: Explain OAuth2 deeply.

I would ask: Imagine you inherit an app that uses OAuth2. You get a bug: users are suddenly logged out after 5 minutes. How would you debug that? What would you look at first, second, third?

From this question, I would learn:

• Can they use docs, logs, and experiments in finding solutions?
• Can they reason with partial understanding/information?
• Do they panic or stay methodical?