Clojurians Log v2

@beshatku There are problematic aspects with the "take home test" on both sides. First off, it assumes people can carve out that amount of time at home -- which can be very difficult for some people with family responsibilities etc. Second, it assumes people have a development-capable machine at home on which to do the work. It's a privilege for either of those to be true. It's a lot of privilege for both of those to be true. Third, some companies use this as "free work" and expect quite a commitment of the candidate's time with no compensation -- again, not something everyone can accommodate and, in my opinion, not something everyone should be expected to accommodate. Finally, from the company's side, there's no guarantee the candidate actually wrote the code. There's always been the possibility of a candidate getting someone to help them write it -- and now, with ChatGPT etc, a lot of "simple" test problems can be solved with almost zero knowledge. Technical tests -- whether conducted live in an interview or via some sort of offline process -- are not a good way to measure ability.

"Coding interviews" (where candidates are asked to write code in a live interview) are almost universally panned. Take-home tests are also almost universally panned. Not everyone has a portfolio of code examples, and it's impractical (to say the least) for a candidate to produce example code from their current or former employer's systems. What is one to do? To Sean's point, asking a candidate to produce a small pile of code is not a reliable measure of ability. There is a lot more to software engineering than writing code. However, coding really is a significant part of a the job, so it seems necessary to establish at least some basic capability. That said, I wouldn't hire an engineer from example code alone. In my experience, the two approaches to the technical screen that seem (to me) to work best are: • A take-home assignment of moderate difficulty, that should take a candidate at the level you are seeking an hour or two to complete, with a follow-up live interview to have the candidate walk through the code. Optionally, add a twist where the candidate would need to adapt their solution to incorporate a new constraint or requirement during this interview. • A relatively simple code review and refactoring exercise, where you ask the candidate to read some prepared code of relatively low complexity and ask them to critique it, and to implement a couple of their own suggested improvements. No need for a working IDE in this one - I've done this using just a Google Doc. In both these cases, we avoid the worst offense of many coding interviews, which is the unreasonable difficulty. Most systems I've worked on (or been in a position to hire for) are mostly line-of-business applications, and not novel inventions with intense algorithmic complexity. The take-home assignment is relatively low-stress compared to live coding, but we avoid the issue of "cheating" by incorporating the live portion where the candidate needs to have a deep understanding of the code (the kind that one would get from having written it) in order not to stumble with a basic change. Some candidates like to "strut their stuff" with something a bit meatier, and the take-home format gives space to do that. On the other hand, it can consume some time, and requires access to a personal machine with an IDE. If time is the biggest constraint, then my favorite is the refactoring exercise. It's usually low-stress because it's not even their code! Professional engineers spend way more time reading code than writing it, after all.

I think that "code review and refactoring" exercise is reasonable -- if you feel you truly must have a code-based test in your process. It was the format I used for one place I worked in the UK, in the early '90s, that absolutely insisted we use code to screen candidates. My retort was to insist that we subject all existing members of the team to the test, in order to get a baseline to fairly evaluate candidates: most team members performed very poorly on it(!). At least it's a collaborative exercise that better reflects how devs work in a team environment. My comments above apply to your other approach: even if you believe you are only asking for "an hour or two", you're still assuming they have a dev-capable computer at home and can spare that time out of a busy family schedule. I'll try to resist repeating this point any further... 🙂

No need to repeat the indictment against take-home tests 😊 I'm (genuinely!) curious about your allusion to not needing/wanting code-based evaluation whatsoever during candidate screening. Can you elaborate on how one can assess coding skills without reading or writing code? Or are you suggesting coding ability is immaterial when hiring engineers?

I suppose one possible option is to skip the coding assessment until the candidate is literally on the job... but that would seem to have its own set of serious problems.

I can get a really good sense of someone's technical ability from having a guided conversation with them about past projects (I've posted my mind map a few times which I use as a rough guide for topics). Actual coding can be taught/mentored and each company has their own house style and preferred set of tooling/frameworks/etc.

If someone genuinely can't code, that will show through when you asking them to talk about how they solved problems on past projects and how code reviews and testing went for them.

(so far, in about thirty years as a hiring manager, I've never had to fire someone for an inability to code, that I've hired through a no-code-test process)

That's a great track record 🙂 I don't have nearly that much experience hiring folks. That said, I also feel like I can sniff out an imposter pretty easily through simple questioning, but I have been surprised a few times in the past by people who interview really well, but utterly fail to be productive developers. Perhaps I'm just not that good at it after all 😅 . I will say, of the 150 or so candidates I've had the pleasure of evaluating over the years (most of which with a mandated code screen), I've actually seen far more candidates who can code, but ultimately did not pass because they could not communicate effectively or handle constructive criticism.

Anyway, I appreciate your insights, Sean!

As an industry, we're generally poor at conducting interviews because no one teaches us to do it because, deep down, we don't seem to view interviewing as a valuable skill for developers (or even managers) to have 😞

> I've actually seen far more candidates who can code, but ultimately did not pass because they could not communicate effectively or handle constructive criticism. Yeah, and I think that's because we don't focus enough on the so-called "soft skills" in interviews, unfortunately.

To clarify, most of these were caught prior to the offer stage, during a multi-interview process. Pass code with flying colors, and fail the collaboration/communication screens, or visa versa.

WRT interview skills, ie "the engineer evaluation problem", I've seen a number of startups try to solve this problem, and ultimately fail and regress to simply yet-another-job-board. Shame.

It's not a technology problem, it's a people problem 🙂

Even that, though. For example, TripleByte started off as a vetting site that relied heavily on a live interview. Then it pivoted to purely quiz-based. Now it's basically just a job board

(or at least, before it was picked up by Karat)

Great points @seancorfield! And you're right, most interviewers (especially technical folks) suck at interviewing as it's not part of their main job. One point that is being missed is that most interviews are not able whether you're able to do the job or not, but whether you're able to sell your skills or experience to what the hiring manager expects.

As a developer most of us are great at coding, but fall short when it comes to selling ourselves sometimes. We can be good at communicating ourselves on the technical level, but may lack the practise in communicating or selling to the audience (in this case the one doing the hire)

Well, my view is that a good interviewer can and should be able to draw that out of a candidate -- I don't really want a "salesperson" trying to sell me their skills and experience (and too much "selling" can be a red flag for me, as far as how they'll interact with other team members). But, yeah, if you're trying to get a poor interviewer to accept you are a good candidate, a certain degree of selling can be necessary 🙂

Agreed, when I mean "selling" your skills, I don't mean to boast or to over-exaggerate your skills, but to craft your answers to who is interviewing you. For example, if a senior executive is interviewing you, do not just answer what you can do, or how you can code this or that, but how that translate to a company's product or business strategic goals.

I think interviewing is a skill. No matter your self-worth (or market value) as an engineer, if you cannot present yourself well, it will be difficult to secure job offers that reflect your "value". I mean this mostly in a monetary sense, but also in the sense of the responsibilities of the role (which are often correlated anyway). I'm also not just talking about your ability to solve LeetCode problems in front of a live audience, but your ability to articulate who you are, what you want, what you are capable of, and connecting that to the goals of the interviewer. The only way I know to develop the skills and confidence to perform well in this arena is practice.

> your ability to articulate who you are, what you want, what you are capable of, and connecting that to the goals of the interviewer point_up::skin-tone-2 This, absolutely yes. Communication is a key part of being a software engineer -- in my view, it is the paramount part since software development is a team effort and you may also interact with non-software folks. Any gaps in the tech side of your ability can be taught/mentored but if you can't communicate, it's always a "no" in my book when I'm interviewing.

I completely agree that communication is the key skill necessary to be an effective engineer. That said, I know some great engineers (and therefore, great communicators) who fall apart in an interview, because they are overly nervous or uncomfortable talking about themselves.

Whilst I agree about communication, it has to be from the interviewers too. I have had far too many interviews where the interviewer was completely unprepared or unable to communicate honestly what the job was or what challenges they were trying to solve. Worse still where several interviewers from the same company everyone had a different idea what the job was because it was obvious they hadn't communicated with each other.

I spent a whole afternoon at one company interviewing with several different groups of people. The last interview was with one person who had contrived a cunning test that was so badly communicated I spent more time wondering if I was applying for the wrong job or if this person really fitted in with the rest of the company. Maybe after 5 hours of interviews I was being slow on the uptake, but it was months before I thought it might have been a test. In the meantime I was quite relieved I didn't get the job (got a nicer one instead after a quick chat. The interviewer already understood my value proposition from my CV and so we just had one meeting and I was hired. I worked there successfully for 5 years and even got several promotions.

I should write a book on all my bad interview experiences (perhaps after I retire otherwise no one will ever interview me again 😭)

You should go on a sabbatical just taking interviews everywhere and collecting the horror stories

I would buy that book!

We all know the interview process is broken and awful. But what are the solutions? Triplebyte et al doesn't seem to work

As a candidate, there's little anyone can do. As a hiring manager, you can push back. But it's going to take a systemic effort to fix this... And not enough people are on board yet, unfortunately.

Yeah, as a person who's been on the hiring and the one being hired, what works best in my opinion is a simple take-home test initially. First, it allows the candidate time to think and reflect, as not everybody can code well "live" with somebody watching over you. If it looks good, then a follow on call to let somebody walk through the code or make some simple modifications to it. At least it will filter out people like this: https://www.youtube.com/watch?v=_P9JpJRY_QA

Video above is somebody trying to cheat through an interview using somebody answering the questions through a phone

Given that most tech interviews are awful and no real measure of anyone's ability, I'm not sure that would even be helpful?

Pretty much all the "self-test" sites online are junk too...

If you define worth in terms of potential income, then I don't think there's any substitute for going around and seeing how much you can convince someone to pay you 🙂

Keeping in mind both people and thus markets are not always rational

I am pretty sure securing a job through most technical interviews is solely a mixture of quantum physics and chaos theory. A persons self worth is largely irrelevant. Self worth can only truly come from yourself. If you're anything like me, then you're the most critical person of your own ability but also more likely to spend enough time to actually appreciate your strengths when you step back and consider your experiences I do find applying for various roles can be quite insightful and a way to appreciate your experience on your own terms. I write a specific CV for each role, limited to 2 pages, so it's an interesting challenge to convey what I believe to be my skills and experience that are relevant to a particular role and company. My full history is on LinkedIn if people care to spend an hour reading it all If you do take interviews, please base your self worth on your own evaluation of the interview and not the persons interviewing. Consider any feedback given during the interview with the appreciation that you have no way to understand all the biases (unconscious or not) an interviewer is applying to you. They are unlikely to fully appreciate that themselves. Unfortunately the rest of the interview process tends to rapidly go down hill from writing the CV, although initial chats can be of value. I try not to ghost companies, but if there process is so terrible I find a way to quickly and politely withdraw. I actually switched off 10 minutes into an interview because it was so bad, it was in-person and I only stayed for the full hour out of sheer politeness and to see how bad things would really get. If I care enough about the company then I'll try give some constructive feedback, but only if there is a sign that it would be considered. I am pretty sure most companies have used the same old, rapidly thrown together years ago, technical test. Usually because they haven't spent any time thinking about how to evaluate potential candidates. Or there's some mistaken rite of passage that everyone has to do the same test (which just leads to a very limited set of thinking within the company). Or they just didn't make time to invest sufficiently.

@paulocuneo it's not 100% clear to me if the question is about "evaluating ourselves" or "evaluating others" (to maybe hire them)

@mario.giampietri I meant "ourselves", while considering a job offer salary or contract wage, or when biding on part time/hourly work. But I guess hiring someone, is the other side of the same coin. I'm not sure all companies have the same objective when hiring, I guess it depends on their budget/credit and if the company is growing or shrinking or if they need to build a pool of workers to stabilize work throughput. From management literature bigger/older companies would prefer to have predictable work throughput, and younger companies prefer to just get things done.

Thank you all for your insights!

I've recently been working through all the 4clojure problems. I have had multiple interviews where the interview started out with pair programming on a random puzzle from the site. This didn't go well either time, as I got tied up in some knots with some of the problems - even though I'm mostly capable. So I recommend knocking them out as a baseline. Thinking through recursion with someone else looking over your shoulder takes practice.

The 4Clojure challenges are a great way to learn Clojure but many challenges are terrible as an evaluation of someone's skills during a live interview. The challenges could be useful for a proper pairing session where the interviewer didn't already know how to solve the problem and actively helped solve it. At the Clojure dojo events in London, most challenges seemed simple but unless you got lucky one challenge could easily take more than an hour with several people working on it. Most of the time I thought of a much more elegant solution on the journey home from the dojo (which is also often the case with interviews)