NOEL:  Today on the show, we're talking about hiring. We have a pretty large crew of guests so I'll let them all introduce themselves on the show, rather than take up your time here. But we do talk about the entire developer hiring process from how to advertise your company to potential candidates, through coding tests and interviews, and all the way to the final decision process. It's a great conversation with a lot of different perspectives and a lot of good advice. I think you'll like it. 

We'd like to hear from you. What do you look for when hiring developers? What do you think works and doesn't work in your company's hiring process? Let us know at TechDoneRight.io/56 or on Twitter at @Tech_Done_Right.

Before we start the show, one quick message. Table XI offers training for developers and product teams. If you want me to come to your place of business and run an interactive hands on workshop, I would very much like to do that. We can help your developer team or learn topics like testing Rails or Rails and JavaScript or managing legacy code or we can help your entire product team improve their Agile process. If you're in the Chicago area, be on the lookout for public workshops starting in early 2019. As you hear this, we will have just run our first 'How to Buy Software' workshop and keep an eye out for more to come. For more information, email us at Workshops@TableXI.com or hit us on the web at TableXI.com/Workshops and now, here's the show.

Hi, everybody. We're here on the show this week to talk about hiring and we have a lot of guests this week and I'm going to ask them introduce themselves one by one. Jennifer, would you like to start?

JENNIFER:  Sure. Hi. My name is Jennifer Tu and I'm one of the co-founders of Cohere. We are a small software consulting company that focuses on both creating software and helping the teams that deliver that software. Last year at RailsConf, I gave a workshop on Interviewer Skills.

NOEL:  Great. And Zee?

ZEE:  I'm Zee Spencer. I work with Jennifer at Cohere and a slight reframe on who we are is we are a technology and management education consultancy. We don't do a lot of typing code for money. We do a lot of pairing with existing managers and existing engineering leaders to help them be more effective and the myriad of challenges and tensions that you face as an engineering leader or a manager or an executive.

NOEL:  Okay and we have Thayer.

THAYER:  Hello. I'm Thayer. I built Team Prime about 15 years ago after coming to the dark side and leaving programming behind and trying to fix the recruitment industry. I mostly welcome building inclusive teams and help people understand why diversity is really useful in technology and I'm from London.

NOEL:  And Matt.

MATT:  I'm Matt Patterson. I'm a software consultant and I run software teams in a large organization like the BBC and I've taking over a small startup. I've been hiring in both the situations and I could have some feelings about that.

NOEL:  Let's start by asking what do you think the biggest mistake people make when they're trying to hire developers or a mistake you see really commonly.

THAYER:  I'll dive in. For me, one of the biggest mistakes anybody makes in hiring at all is hiring people they like and that they want to work with 'cause they're nice, as opposed to hiring against a spec of what the work you supposed to be doing and what a job should be fulfilled with the human's capabilities. More often than not, I see hires because people enjoyed meeting somebody at the interview, rather than actually looking at how well they do against the skills needed for the job.

JENNIFER:  I feel like I see something very similar but coming at it from a different angle, which is teams that hire for this platonic ideal of someone who is incredibly technically talented and isn't actually bringing the skills that that team needs in order to fulfill whatever their mission is.

MATT:  I think I see something similar to Jennifer, which is teams hiring against the interview but the interview is not actually a real test of what they need, so they get people who can do well in interview but can't actually do well at the job.

ZEE:  The biggest mistake I see is a decoupling of the incentives around hiring and I think this manifests in the ways that all of you have mentioned but when we disconnect the control and the autonomy around hiring from the team that is doing the hiring and the team that has bringing on this new member, we make a mistake with how the incentives work. Suddenly it becomes a 'we need to fill 20 headcount' and it becomes less 'we need to build a team that is capable of accomplishing the goals that the organization has set forward.'

When we fall into that trap, we create these proxies for what a competent team member is. Maybe they feel really good or we like the conversation or like Jennifer says, "They maybe have the technical skills but they're not bringing a healthy mindset to the team," or like Matt has said like, "We have these rules about how we interview that are decoupled from the day-to-day work and that decoupling causes our hiring practices to identify candidates who aren't as valuable or as useful in the current context as they might be."

NOEL:  Interviews are pretty established as the way that we find people to hire. How do we work around these tendencies and create a hiring procedure that identifies and allows you to make decisions on the people that are really going to be able to fill your needs?

THAYER:  I've been working on hiring procedures with companies for quite a while now and one of the things I found over the last decade that seems to really work well is to do a bunch of different types of interview. Your first sort of an initial ones that I think most people are used to, such as a telephone interview to make sure expectations are set or face to face to make sure that people understand that depth and breadth of the job and make sure that they're enthused to keep going with the interview process.

But then the real crux of it for me that I found working with people tends to be doing a half day on site so that's where you'd actually work with two or three people in different teams. A lot of the companies I work with work in product teams or Agile teams, so if you're a developer, for example you'd actually end up working during that half day, maybe an hour with a product person, listening to their brief and seeing that you understand and how you communicate but maybe also working with a junior designer and that way, also making sure that you're working with people that seem different to you, look different.

If you're a white male, hopefully either work with a person of color, one of those in twos or you'd work with the female or with somebody junior to you or somebody senior, so you get that range of communications within a kind of four-hour block and what it gives the person who's interviewing is the ability to actually understand what that job feels like in reality, not just meeting people that they're trying to impress in an interview because one of the things, I think people struggle with an interview is they spend so much time trying to impress them to get the job that they don't actually get to understand what the workplace is like for them and that's super important.

NOEL:  One issue that I have with that particular process for us -- we're a consulting company and a lot of our stuff is client work and often, bringing people into a client project that's not our own project, that might have its own set of NDAs, is sometimes challenging. We work around that in slightly different ways. Jennifer and Zee, what do you think are the kinds of processes that you like to see people build in their interview process?

JENNIFER:  I was just working with another consulting company around their hiring process and the approach that we took was to center what their values were and what they needed to find that was compatible with the way they wanted to exist as a team and that meant that we needed to start by digging into what these values were and helping them figure things out.

This is another one of this classic mistakes that people make. There are so many teams out there that when you ask them what they want out of their new developer, their new teammate will say, "We want someone who is smart and learns quickly and isn't an asshole." That doesn't really tell you what they're looking for because to one team, a jerk might be something completely different. What I mean by that is for some people, valuing whether or not you are direct is a quality of not being a jerk but for other people, valuing someone else's feelings as you give them feedback and prioritizing that over the efficiency of giving them that feedback is what makes not a jerk.

NOEL:  Yeah. Matt, what have you've seen being successful in your experience in hiring?

MATT:  I find hiring very difficult for different reasons in the startup context, I found lots of problems around uncertainty about future funding and those kinds of problems where you're asking people to commit to you and you're not sure how viable over the kind of medium term your commitment to them would be and then the other problem is getting people to actually find out that you want to hire, particularly in a place like Berlin, which is a tech hub and there's a lot of companies, a lot of them have a lot of money and a lot of budget to tell people that they're hiring and if you don't, actually just getting anyone to actually hear that you are lovely place to work and you have rewarding and challenging things to do with a lovely team, you're getting that message out to enough people that you will get responses and get enough responses that some of them will be people that would, in turn, be worth working with is actually really hard.

That was the biggest problem with the startup, whereas in larger organizations, the bigger problems are just shear weight of getting through the process -- very strongly established processes. Some aspect of it are really, really good and some aspects of it were less good. I think it kind of swings around to that but there's kind of two big problems. One is actually the advertising part and the other is making sure you can cope with the work of doing the interviewing and doing the sorting through applications and doing all the ton of the work around hiring, which if you're busy and you got people leaving you, it's very hard to actually give that the attention that it deserves and then you wonder why aren't I getting quality talents and why does anyone responding to my job ad that makes this place sound like a gulag, those kinds of things.

NOEL:  One thing I definitely learned when I was in charge of hiring as it is an incredibly time consuming to do well. Jennifer was there something you want to add?

JENNIFER:  Yeah. Matt, I was kind of wondering when you have a problem of how to get your company's name out there, did you find anything in particular to that work?

MATT:  The thing that really worked for us was word of mouth -- recommendations for people through other people who already knew us, so somebody who was wanting to leave London and move to Berlin were like, "Our friends here are looking for a job," and that actually can work really well if you're quite small. If you're a magical unicorn, and you need to fill 500 jobs, then that's not going to work you but you have one or two and you have a good network of people that not want work for you. In which case, maintaining that network is really important.

THAYER:  I would flag that a little bit because I think actually this is a number one diversity mistake that all companies make, especially startups and it's one that's totally understandable but also entirely avoidable. What happens is you have a startup week at your first seed round and you start hiring and nobody really knows how to hire, so it's always somebody else's job. What you do is you ask your network, you ask friends of friends and you find people and you find people that look like you and then usually, when people get to about 10, 20 sets of people big, they start freaking out that actually it's all white guys or they're all from a certain area or from a certain college and then they have to pay quite a lot of money to have people come in and tell them how to try to diversify. Of course, the problem there is actually retrofitting inclusion is really hard. It's something you need to think about at the beginning and actually, I don't think any of us have a big enough bubble to have truly inclusive teams.

Although I think it's a good strategy, it should be part of your strategy and I think that alongside asking people in your network and networks of networks, we should also be breaking into communities that don't look like us and don't sound like us or went in the same schools as us and don't have the same color skin, etcetera.

ZEE:  I just wanted to "yes, and" with Thayer there because I have made that mistake in the past. I have fallen into the trap of friends and family hiring is kind of what I call it, like when you go and get your seed round, go to your friends and family. That makes sense because they're the ones who trust you. They're the ones who are like. They're the ones who are willing to give you the resources. But when you are reaching out to make your hires, it's really, really important to advertise in communities that are not necessarily the kind of communities that you are used to being in.

One of the ways that I have mitigated this in the past and again, this is not a panacea, is to spend most of my social activity around networking, focusing on communities that don't look like me and doing it in a way that I hope and my friends, hopefully will correct me when I am out of line here, that is supportive of their initiatives.

There's this amazing video of this notion of a first follower -- someone who comes in behind someone and adds validity to their activity and what I've spent a lot of my time and attention doing in the last five years has been being that person for communities that aren't just white dudes, for communities that aren't just self-taught programmers, communities that aren't just MIT grads and part of my intentional act of inclusivity is to minimize myself and minimize who I am, not in a unhealthy way of self-loathing but so that I can fit safely into spaces that are predominantly person-of-color led or predominately women-led and not overpower with my traditional energy that can sometimes just overwhelm a space.

MATT:  That has always been part of my approach. I completely agree with Thayer. It can't be all that you do but the quality of your network in terms of how many different kinds of people are in it, that's super important if you want to successfully get a team that doesn't just look like you and I think it's really important to do that. I think it's really important to do it as early as possible because the cost of fixing that later on are huge and the opportunity cost of the perspectives you lose during that time is huge that's why I think it's really a terrible waste.

I think when we hired that way, we were quite lucky when we didn't just hire other white dudes but we got lucky and I desperately wouldn't want to have to rely on that again, to try to fill the position. I want to have the skills to do a better job, which is why I would talk to someone like Thayer in the future, to start to add at the very beginning when it was going because it's really hard to hire. That's one of the thing we heard pretty much everyone here. If youre core skill is not hiring, chances are you're not going to be the best person at it.

THAYER:  I completely agree and I've got a question for everyone actually because I've just been having this discussion with a few people over the last couple years and I would love for everybody's input. I have a blog post brewing in me recruitment is a C level word, which is an English joke, kind of a swear word that I shan't say but the idea being for me is that quite often, I get called in to companies that have grown 50, 60, even a hundred plus people big without hiring anybody in looking after people. Like a people leader. I don't like the word 'HR,' so I'm not going to use HR manager, although that's a separate skill set, I think anyways. But it is really weird.

I guess my question to everybody is that my feeling is that really, as soon as you get your seed round, as soon as you're going to grow more than one or two or three people in the next year, I truly believe that you need to be engaging either with somebody part-time to work on your people side of things to take that off you and make sure that things being done really well and delightfully both internally and outternally, or if you are hiring 10 plus in a year that should consider getting a permanent full time head of people type role.

For me, head of people, just to be quite clear, is all the recruiting up into your company than onboarding and looking after but it's not the traditional HR role of like reviews and contracts and the legal side. I see that as a separate role as well. I just wanted from everybody what your thoughts are on that.

NOEL:  That's actually wound up being my role for about a year and a half. It was essentially everything that was traditional HR except benefits, so that included recruiting and onboarding and also reviews. I spent a lot of time at relatively small consulting companies and there's definitely a difference between the ones that invest in meta-personnel: personnel about their people early and the ones that don't and yeah, I agree completely that I was not in any way trained to do interview stuff beyond being a developer and having a lot of opinions about developer interviews.

ZEE:  I am all in favor of bringing someone on to manage the recruiting pipeline early. I was the fifth hire at a startup a few years ago and the C-suite was like, "We need to grow to 40 people the next year," and I was like, "We need to hire someone who is actually good at this because I am not and I am technically just a senior engineer on paper and that's what I want to do. Please don't make me do this," and of course that didn't happen because it's a startup and really, "everyone just goes through the friend and family, Anyways, Zee, it's fine."

I'm glad I got the opportunity to do that because I learned a lot about hiring, I learned a lot about managing a recruiting funnel and making sure that as people progress through the funnel, we're reflecting regularly on is the funnel excluding people based upon non-significant factors, such as ethnicity or race or gender or age. Those factors are not blockers for your competency.

NOEL:  We did an audit of our funnel to make sure that every step was representative of the population that was getting to that step, was represented by the population going out of that step. It's a good idea to do.

JENNIFER:  How did you do that audit?

NOEL:  Approximately. We didn't have all the information we needed to do it but we were hoping that we would be able to catch a large scale effect of like all the women dropping out at the code sample or something like that.

JENNIFER:  Did you find anything?

NOEL:  We were able to convince ourselves that we did not have such an effect and then therefore, our problems were in our input pipeline and not in our through pipeline.

ZEE:  I have an amazing anecdote about that kind of thing. One of the first things I noticed was that when we sent a take home back with people, the percentage based upon perceived gender, which is really all I could really look at, people who came back with a take home was approximately 20% not masculine-presenting and 80% masculine-presenting. That take home, the fact that we were sending this out was the biggest blocker from a gender axis for inclusively in our pipeline for us and that just like blew my mind. When I did the math to look through, I'm like, "Oh, yeah. We put 60 people through this step and we've been pretty good about bringing people in who are not white dudes and reaching out to them and getting them to apply," but the fact was our design was removing people disproportionately along that axis at the very beginning. It's just like taking the time to do the math and look into the system and approach it not from a judgmental perspective of like, "We're just not inclusive enough," but from a 'how do we make this a more inclusive environment and a more effective hiring funnel for everyone.', shifted our dynamic so much that we went to approximately 50% masculine-feminine-presenting kind of workplace in the next three to four months.

NOEL:  I specifically wanted to talk about that kind of take home exercise because we do one but we do a lot of stuff in the way we present it, designed to try and avoid the kind of problems that Zee, you were talking about because we were aware of those problems and tried to work around them. I was kind of wondering what everyone else on here thinks of that kind of exercise as a way of ascertaining somebody's technical level. Is it useful? Is it more trouble than it's worth? How do you feel?

MATT:  It's a good way of ascertaining how much free time they have. It's not the most you can say about it but it's really problematic because not everyone has a huge amount of time and not everyone has the same amount of time at all points in their lives. You know, I have a very young child and now, doing something that takes an hour or two in the evening is an hour more than I have, really and certainly, something that requires really serious thought, I couldn't really do that. There's a whole slate of people who have a whole bunch of other responsibilities outside of work that means that really, since you're asking to do a take home exercise, particular one that has a significant time component, they can't do that.

JENNIFER:  I would add to Matt. It's a good judge of how much spare time your candidate has that it's also a good judge of if your candidate approaches an ambiguous situation in an identical way to what you would do yourself. If this is what you're aiming for, that what you want is people who are sufficiently homogenous that they can anticipate how everyone would behave in an ambiguous situation, then you would be achieving that goal but you would be achieving at the sacrifice of goals around being able to approach problems from a more holistic perspective that could bring you to a better solution.

ZEE:  The way we shifted it is, first off we acknowledged that each person has a different mode of operation and one of the things that we had agreed on as a team, outside of the interviewing, is that we wanted to support both high pairing time and non-pairing time because some of us love pairing and we do it six hours a day and some of us were like, "If I pair for more than two hours a week, I will literally collapse into a ball." It's not quite that bad but if we took that as the dichotomy and so we're like, "In order to be a healthy organization, where all the people who already participate can also healthily participate in the future, we need to design our hiring system such that we are not disqualifying people based upon their willingness to pair or not pair or their willingness to do this in the way that we intend versus do this in a way we don't and instead give them a bunch of options that they can opt into but do so in such a way that the output is the same.

The output from our take home was working software. We sent them a Rails application and we said, "If you want, you can add a very small feature and we outlined a feature in the email. You can submit a pull request to this private repository that we just forked to you and you can send us comments and questions through that," or what you can do is schedule a two-hour pairing session with us where we will help you solve this feature together and submit it as a patch and then what we do is we would look at the pull requests, we would look at the comments, we'd look at the commits and we would use that time-boxed time period basically, to determine whether or not, they are capable of shipping features in a somewhat ambiguous but well-supported environment in a manner that is compatible with our existing team and in a manner that we could expand the team to encompass their cultural needs but also, where that becomes visible early but doesn't overpower the signal that comes from the session.

NOEL:  We wound up doing some similar kinds of things. We made it pretty clear our take home was to structure a very simple Rails app and we made it really clear that we were not going to be evaluating, for example, design, which takes out a lot of what people spend a lot of fiddly time on, a lot of the how much spare time does a person have, and we also tried to make it very clear that if somebody was unable to do this in a take home kind of situation, that we would allow them to come in and pair, that we would jump directly to the paring session, which was the next part of it. Not very many people took us up on that but as far as we can tell, this seems to helped us avoid some of the problems with this but it's kind of hard to tell.

MATT:  My question is, what is the most useful aspect of a take home test? Because from my perspective, a working code is not the most useful output of a process like this. The most useful thing is to know how someone approaches a problem like this one that's defined, perhaps quite ambiguously or perhaps, very unambiguously but how they approach a problem and how they figure out what they're going to do and why they're going to do it. That's one of the things you can't get from the final artifact, which is one of the big problems I have with take home tests, so how do you deal with that to make your take home useful?

JENNIFER:  There's a saying by Sandi Metz, where she says, "The cost of the code is not in the writing of it but in the reading of it," and I feel like that's one of the mistakes that teams make around take home is they feel like the cost of assessing this candidate comes from having to talk with them for an hour and pair with them or watch them code and that hour, they feel is a loss. That's ignoring the cost of spending the time to read the code that would be submitted as part of take home and understand and evaluate it. It takes much longer to do that than it would to spend the hour to sit with a candidate but because you don't have that time allocated, it's much harder for teams to see and recognize the problem.

ZEE:  I would push back a little bit on the notion that the artifact, the output, is not the most important part of the interview process. Keep in mind, when I talk about take homes, I'm not talking about "can you solve this problem". I'm talking about "can you submit a patch to a functioning piece of code that implements a feature and can you do in a way that is aligned with our norms and behaviors".

If someone pushes up a pull request and it's full of jargon or one letter variable names or whatever, it is the responsibility of the interviewer to be like, "I really appreciate you taking the time to send this out. Here is some refactoring suggestions I would make --" just like we arre doing a code review within our existing workplace, "-- and I was hoping that maybe if you could take another small period of time to either work through that or you can schedule an hour or two hours to pair with me as we refactor through this together and what that does is it gives another seam.

If you think about software development, you want to get seams to your system architecture, so that you can inject the appropriate behavior for the appropriate context, this gives you a seam for your recruiting system to allow your organization to get richer or more interesting information in a way that is mutually respectful of one another's time. You're not saying, "I'm not going to accept this until you fix this," you're saying, "If you have the time and attention, one thing that would be beneficial would be this, I am willing to take that step with you. I am willing to be on a call with you and do this full in real time because your time is valuable," and that's a pretty significant shift from, "You submitted the take home, I'm going to go through it and judge you on eight axes and be like, 'I'm sorry. Your variable names are terrible,'" because that's just not healthy.

JENNIFER:  Is that still mutually respectful given that now you've introduced a possibility of expanding out the interview process and are you going to do that for every candidate who is like that? Is it okay that you are judging one candidate based off of 10 hours of work and a different candidate on three hours of work?

NOEL:  The next step in our process historically has been a pairing session on the take home sample and we generally are pretty generous about who gets to that step. The pairing session is usually the step that is the tough hurdle and one of the things we do in that is to try to ascertain how the person responds to feedback about their code. That's often the interesting bit of information, especially for an entry level position where what they do on their own is probably not representative of how they're going to be able to write code after having been on a team in our place for a couple weeks.

ZEE:  Yeah, that feels really good to me. Setting the expectation that they will get time with you once they send the artifact regardless of how they do it, I feel like that helps address Jennifer's concern about the playing field being not quite level and having to figure out how to balance the different time investments that candidates have put in versus just running with it. That's a really nice adaptation.

MATT:  One of the ways I approach interviewing as the candidate is that I regard the interview process as a way for me to assess, whether the company is worth my time. It seems like one of the really great things about pairing is that it's a very two-way process: you learn about me in what I do and I learn about you in the way you think, in the way that you do. Whereas in a take home, the only communication from you to me is the kind of metaphorical piece of paper that you gave me that says, "Make a thing that does X." How careful do you have to be with that if to actually communicate the kind of the things that someone really wants to know about the organization? Do you make sure that you use a test that's related to your products or the way that you work on projects or do you kind of pick one of the algorithm examples or something like that?

NOEL:  We moved away from algorithm example because it wasn't giving us the information that we needed and we moved to something that was a little bit more aligned with our actual projects. 

NOEL: I do want to ask, what do you think that you can do in an interview process once you are talking to the candidate? What are good uses of that interview time and what are bad uses of that interview time?

JENNIFER:  A bad use of time is to say, "Tell me about yourself."

THAYER:  I completely agree with Jennifer. I think it was one of the things I started off venting a little bit about at the beginning of the podcast is not about working with people you like. It's about working with people who are great for the job and can do that job. When you talk about what are good things to talk about an interview, I always say to -- I've been doing this for 15 years and I still to do this. I take a crib sheet for myself that keeps a very specific questions about the role, about that person that I've already looked at the CV and want to delve in about. Usually, over the course of half an hour to 45 minutes and the first thing I do is let them know exactly what's going to happen. I'd say, "This interview is approximately 30 to 45 minutes long. I'm going be asking you around five questions is there anything you want to know about the process before we start," just to make everybody comfortable and then dig into these questions but stay on topic.

"I love long distance running," for an example, so somebody happens to mention for some reason that they like running  not to suddenly, "Oh, my God. Me too. What's your 10k time?" because you can waste a good five minutes of that 30 minutes on something that's got nothing to do with a job but this happens quite frequently and says, "Oh, cool, you're running. That's great but tell me about the project that you managed in 2016, the biggest client that you worked with and what went wrong and how did you solve it?" you know, to bring it straight back on.

The other massive thing, I think that a lot of people still seem to subscribe to, which upsets me greatly is trying to off put people in interviews or make them in any way feel like they've got to jump through hoops. I'm completely the other way around with this and I advise all of my clients and anybody listening really, to do the exact opposite. Our job is to make this person feel as liked and as comfortable as possible. If they are going to fail the interview, they are going to fail the interview based on the skills or experience not being appropriate has nothing to do with how well they perform. If they can't keep eye contact with you, for example, it got nothing to do with the fact that, "They don't like me. They can't even look me in the eye." I hear this a lot.

We have an awful lot of people in our industry who are introverts where eye contact is actually quite hard, especially in a situation where we're not the one in power, so it's all about focusing on the questions, the skill sets, and making the best use of that time against the job, not about, "What do we think about this person? Are they a good runner? Do they hold eye contact? Oh, my God, is she really wearing that?" These are bad things. Just keep it directly to the job.

JENNIFER:  Thayer, when you do your interviews or when you give advice to your clients on how to interviews, do you recommend having any consistency in the questions that you ask your candidates?

THAYER:  Yeah, absolutely. I think that's really important and also, we review them and get candidates to feedback on them and then, we review them ourselves as well and see how they work because sometimes, we'll come up with a question that we think is really relevant to the role but actually, the answers we're getting aren't demonstrating the sort of thing we want quite consistently, so then we know it's our question, rather than anything else.

I think it's a really good way of working. I first came across this in a really strict way when I was hiring for the UK government and they actually have sort of a standards-based recruiting procedure whereby you have to go in with a set of questions on a matrix that you then score people against between one and five and you have a pane. It's quite old school and archaic. At first, I really hated it and I still don't like panels by the way. I think it puts the power on the wrong foot but that aside, what it was really good at was actually taking out some biases, which we haven't talked about at all yet, which we're probably not going to have time to but I found that really fascinating because it starts people saying, "I really like this person because X, Y, Z."

Again, if you find yourself using the word 'like,' something's gone wrong. It was more about, "Can they do this? Have they demonstrated the experience to do it," and say, "Yeah, absolutely." You literally add up scores and then work out which one scored the most. That was fascinating to me and it really does work. Standardization really helps remove bias.

MATT:  Absolutely. When I was hiring at the BBC, we used the exactly the same process. It was literally the same process, the same standard and it felt very weird at the time but the more time goes by, the more I look back on that and think, that was a really... a lot of good attributes, but particularly the crib sheet, with these are the questions you can ask. You can't ask other questions. You have to understand how these questions map to the job requirements. You put in your job advertisement. If it wasn't in your job ad, you could your question about it and then yes, you add up scores and I think that can work really.

There are some other issues around exactly how you develop these questions but I think the kind of core idea there is to have this and a really good way of making things explicit which will help you with some of the biases, not all of them but quite a lot of them. I think it's brilliant. I would do that again.

NOEL:  One thing that I've always wondered about the right way to do is what to do at the end of the process. You have a bunch of people who have opinions on the candidate and you have to have some way of turning that into a yes/no decision. I felt like all of the ways that I tried to do that were inadequate in one way or another. What do you recommend that you feel like is a good process there?

JENNIFER:  My favorite thing to do is to make sure that everyone writes down their impressions before they talk with anyone else and by impressions, I mean write down specific things that happen. Don't just write down, "Oh, this candidate was good," or, "This candidate was bad." Right down specifically what you observed to support the conclusion you come to. I also really like a one-to-five system for how much you think the candidate is a hire or no-hire. Pick one at the end of the spectrum for 'definitely not,' pick the other end for 'definitely yes.'

Get everyone to record their thoughts separately and then have a synchronous meeting in which everyone simultaneously reveals what their one through five change was. If you are doing this in person or on video call, you can do this by holding up a fist in the air and on count three, flashing a number of fingers.

ZEE:  Yeah, we call that fist to five. I learned it from --

JENNIFER:  There's a name?

ZEE:  Yes, there's a name and I learned it from Activism, where you basically hold up your fist and when you reveal your number, you can leave your hand as a fist to say -- and by the way, if this moves forward, I'm out, like I'm leaving this conversation because I can't, in good faith, continue to participate, which I think is a really powerful mechanism for consensus and consensual decision making but it might be a little bit of a derail for this conversation.

NOEL:  My experience with that kind of meeting is where you get in a mixed case or where it's just borderline and I definitely have had people who say like, "If it's borderline, then it's no."

JENNIFER:  Yes. Hard yes or no. If it is borderline, don't do it.

NOEL:  You get in a situation where like you talk about it until one side basically is exhausted and then you call the result consensus and that's generally how these things go for me but 'if it's borderline, it's no' is a rule that we more or less wind up using. It can derail or it can prevent a lot of fairly contentious meetings about that.

THAYER:  One thing I just want to jump in on the "borderline is no" thing. I had a couple of instances where I'd been working as an interim people lead or something at employers and interestingly, I've had hiring managers who are interviewers who come to me and ask me some really interesting questions about how to make up their mind. I'll give you an exact case scenario. And I'll leave out everybody's names, obviously. Somebody came to me because they felt they shouldn't employ somebody because they had revealed in their interview that they were bankrupt and this person, I was working for a security company at the time, was worried that because they were bankrupt, they may have people that come after them and therefore, they may sell secrets from the company.

I know this sounds really bizarre but this was a genuine concern for this person, although they hit all the other skills and I was super glad they came to me because when they came to me and talked about it, I could explain to them that's not really our decision to have. It's got nothing to do with the job. It was admittedly strange that that person would talk about in the interview. However, did they hit all the things to the job, you know, keeping back to that, then that person kind of realized, "Oh, yeah, they did."

It depends would be my answer and if you can and you really are on a fence, I would try and talk to somebody who does work, either in a kind of people management role, where they've done quite a few interviews and just bounced me off because it could be something you don't know about like that person has genuinely felt that was a reason not to employ that person, when actually when you bounced it off somebody else, it becomes quite obvious that that's not really a reason we should take into account.

I also had a company and this is illegal, by the way, just to completely underline this, that didn't hire someone because their child was in a different country. That's illegal. You can't do that. However, they made the decision on behalf of the candidate that it would be wrong to take that candidate away from the child.

JENNIFER:  Was that candidate a women, by any chance?

THAYER:  Interestingly, in this situation it was not. Although the company in question did a lot of things against women. Still illegal. It was actually a man and the child was going to stay with the mother and he was going to move for the job and they made the decision not to hire him.

ZEE:  I just want to vehemently agree with Thayer here and I get the desire for "borderline equals no" to be the default. Unfortunately, our cognitive biases are so often below the surface that a borderline may be -- and this is not a 'will be' but a may be -- indicative of some subconscious permutations that we don't really realize are influencing our decision about the candidate.

I just love how Thayer talked about bringing this back to that job because if you don't have that, if you don't have 'this is what the job is, this is what we're hiring for, and this is why' it's really hard to have the conversation that is, "Can you point to the particularly thing in the job that you believe this person is a maybe or a borderline on, so we can discuss that?" It becomes a much more healthy conversation than, "It's borderline, so we're going to say no," or, "It's borderline but we're going to say yes because we want to err on the side of opportunities for people who may not aligned directly with us," because that conversation is so critical to making good decision.

I agree with Noel that you don't want to be in a spot where you're just using that conversation to beat the opponent into submission. That's unhealthy and terrible but the conversation is still really valuable and having that, as a possibility and making space for that conversation to happen, is critical in my opinion.

MATT:  Interviewing as a candidate is still a skill, no matter how much we do to maybe put people as at ease as possible, it's still a skill. It becomes this 'some people are better than others.' I think really important, at the end of interview process, if someone is a no for whatever reason that you make a genuine offer to provide good quality feedback to that person. If that person's wants feedback, then you provide them with constructive feedback.

In my expense, I had a case where someone did a really bad job in an interview, asked for feedback, in the process of giving them feedback. It became obvious that actually they were a fabulous candidate and with the feedback, then they kind of applied again for another job that came up later on and sailed through and that was brilliant.

I think, particularly with someone's first interview or they never interviewed at a company of that size or whatever, it's a whole other things that make people really nervous, particularly in the face to face stuff and so, actually having an explicit offer of like, "We will tell you constructively what didn't work in this person," can be super useful for candidates and for helping those candidates come back better and more reflective in their true skills later on.

NOEL:  Great. I think that's a really good note to end up on. If people want to continue this conversation with you online, where can they find you, Jennifer?

JENNIFER:  You can find me on Twitter at @JTu.

NOEL:  Zee?

ZEE:  You can find me on Twitter as @ZSpencer or you can mention WeCohere, which is our organization at @WeCohere and I watch both of those far too much, so you'll get ahold of me pretty easily there.

NOEL:  Thayer?

THAYER:  Oh, yeah. You can also find me on Twitter on @Thayer or you can check out my company site, which is Team-Prime.com.

NOEL:  And Matt?

MATT:  You can find me on Twitter @fidothe.

NOEL:  Great. Thank you for being here. This was great and I'm glad we all made it and thank you very much.

Tech Done Right is a production of Table XI and you can find us on the web at TechDoneRight.io and on Twitter at @Tech_Done_Right. You can find Table XI on Twitter at @TableXI and on the web at TableXI.com. I'm Noel Rappin. You can find me on Twitter at @NoelRap.

The show is edited by Mandy Moore. She's on Twitter at @TheRubyRep and of course, if you like the show, tell a friend, a colleague, your social media network, tell me, tell a pet, tell your manager, that would all be very, very helpful and a review on Apple Podcasts helps people find the show.

Table XI is a UX design and software development company in Chicago, with a 15-year history of building websites, mobile applications and custom digital experiences for everyone from startups to storied brands. Find us at TableXI.com where you can learn more about working with us or working for us and we'll be back in a couple of weeks with the next episode of Tech Done Right.