As a software consultant, I usually get interviewed around 2-3 times a year, but I have also interviewed quite a few software engineers for clients of mine over the years.
I am quite fortunate in that sense, and having been on both sides of the table on so many occasions has helped me greatly to be a better interviewee and an interviewer.
As an interviewee, I am usually able to prepare quite well by putting myself in the shoes of the interviewer and probing myself with questions about skills and areas that I think the company should care about when hiring.
Equally, when interviewing software engineers, I have great respect for their time and also appreciate that it might be a stressful period for them. So I try not to waste their time with exercises and questions that I don't think are relevant for the given position. At the same time, I have my client's best interests at heart, and I want them to succeed in finding the right candidate and building a successful team.
What to look for
As a software engineer myself and having worked on tens of projects over my career, I've come to appreciate certain skills and behaviours of great engineers, which I think result in real, tangible value for their company and their company's customers. It's quite a list, but at the top of it are things like:
Following test-driven development, which increases the likelihood of having a good software design and provides adequate test coverage to ensure that future breaking changes are caught on time.
Striving for simplicity: simple code can be easily read and understood by the majority of engineers, not just those well versed in certain design patterns or libraries.
Ability to design and build code with good separation of concerns
Understanding that tests should test behaviour and not implementation, as this can really hinder future refactoring efforts.
Strong desire to automate things
Not being afraid to ask for help as opposed to working in a silo—this can result in a lot of wasted effort and time.
Discussing and communicating software/coding ideas clearly
These are the kinds of behaviours and ideas that result in maintainable code, which can easily outlive the engineer's tenure at the company and allow other people to get easily onboarded and eventually take over and continue to evolve the project.
Let's face it, people come and go, and as a company, you don't want to have huge amounts of expertise concentrated in a few individuals, it's too risky! Good engineers try to make themselves redundant by automating things and sharing knowledge, which, as a matter of fact, makes them even more valuable to the company as they can now focus their efforts elsewhere and improve their team's productivity even further.
How you approach a specific problem as a software engineer does matter! It can also have a serious impact on your project's testability and whether it lends itself to being easily refactored and improved in the future. For example, asking somebody in a coding exercise to traverse a graph or solve some other purely algorithmic problem will tell you nothing about their expertise in building and testing a complex multi-layered software service that has some sort of presentation layer, a database, and publishes events to a messaging broker, for example. At the same time, these are the types of projects that can accumulate serious amounts of tech debt for your company and result in a significant drain on engineering time.
So give your company the best chance in finding the person who can hit the ground running the right way and start adding value, not tech debt!
How to do it
How the candidate approaches the coding exercise, i.e., whether they start by writing a test first or they jump right into implementing a solution, can tell you a lot about their habits and discipline.
I usually consider it a positive sign when the interviewee asks me for help during the coding interview. That tells me that there's no ego at play; they have a team mindset to problem solving, and they are mature enough to ask for help when it would get them to solving the problem at hand the fastest. That's way better than just ploughing ahead and implementing the wrong thing. I've seen too many people wrestle with issues for way too long before they finally dare to ask for help and realise that the solution was quite simple in the first place. They could've asked for help, got help, and moved on, thus improving their knowledge on the subject much faster.
They're not familiar with the specific framework or have not worked with it in a while? Not a problem! That should not stop you from gaining valuable insight into their ways of thinking as engineers. As an interviewer, you should be helping them with the trivial parts of the pairing exercise, as this is how it will unfold in real life too - the person with the right knowledge gives suggestions and helps the pair move forward. Being able to follow your guidance is just as important and tells you if they are well versed in the language and general engineering concepts.
All you really want to do is simulate an environment as close as possible to reality.
If you needed the indoor walls of your house painted, you wouldn't find any painter and ask them to show you how they paint a fence, would you? You'd want them to show you how they paint corners and edges and if they have a steady hand, right? At the end of the day, this is what they'll be doing when you hire them, so why test them on anything else?
So if you need someone to represent your company at the national informatics olympiad, then you probably should be probing their knowledge on data structures and algorithms. For the majority of companies, knowing when and how to use a given data structure is probably good enough.
Why aren't companies doing it?
So why aren't companies' using tech tests which are tailored to their particular needs?
It's hard!
You have to come up with a project, plan how it will evolve during the interview, and plant potential problems to present the candidate with. These problems should be important enough to reveal the candidate's ability of coding and designing good software, but at same time you have limited time at your disposal. The interviewer should also be able to separate good from bad decisions and allow for compromises with the goal of progressing to the next problem that they want to test the candidate on. So this is not a straight-forward task by any means and requires time and effort.
Let me know if you need guidance with your interview process, and I'll be happy to share my experience with you!