Fairness in Hiring

0:03 Mike Boufford, CTO de Greenhouse, nous parlera d'équité en matière d'embauche lors de son talk. Bonne conférence, Mike! I'd like to start off thanking all of the organizers here at Tech.Rocks and to each of you for spending the time with me today to explore fairness in hiring. My name is Mike Boufford. I'm the CTO and was founding engineer at Greenhouse Software. For those of you who don't know what Greenhouse is, we're the makers of hiring software, ATS, CRM, and onboarding software used by more than 5,500 companies across the world, from startups to the Fortune 500. Having hired hundreds of people over the past years, and having worked in hiring best practices for most of the past decade, I felt this was a talk that I was uniquely positioned to give to all of you. It's also an important thing for us to be discussing. As hiring managers, we carry a huge amount of responsibility to ensure fairness in the economy. It's a big deal, and it's something that we should all be focused on getting better at every day. By the end of this talk, I hope you walk away with a deeper understanding of fairness in general and a few things that you can do to bake fairness into the hiring process.

1:11 This talk will be about half philosophy and half practical, but let's start with something fun. Prepare your brain. I'm going to make you think for about five minutes. So splash a little bit of coffee on your face. We're going to talk about fairness through the lens of game rules. Fairness game number one, accumulating power. The objective of the game is to accumulate more than half of all the tokens. Three rounds will be played. In round one, all players start with just one token. In round two, players start with roughly half the tokens that they had in their prior game, had earned in their prior game. And in round three, all players start with half of their tokens earned during that prior game in round two. So in round one, it certainly feels fair. Everyone's got the same starting point. It feels consistent. By round two, if you're a player in the game, it starts to feel a little bit less fair because there are players who have an edge. And there are players like you, if you didn't accumulate a bunch of tokens, who might still only have one token at the beginning.

2:13 By round three, if you hadn't accumulated enough tokens in prior games, you are all but certain to lose. The rules are consistent, they're equally applied to all players, so is this game fair? As we'll see time and again, it depends on how you scope the question. If everyone starts at round one, yeah, it probably is fair even if it's not very fun to play in round two and round three. But if you start in round three, it doesn't feel fair at all. Babies born today into our world are born into round three. Accumulated power has already accumulated for a bunch of people, and most people's starting point is something around one token. Fairness game number two, equal pay, fair pay. The objective of this game is to maximize profit earned for 100% employee-owned All players have equal experience, rank, and work the same hours. All players have an equal share of the business. Blue players always produce $10 an hour profit, red players $5, green players $1.

3:14 So what's the fair division of profits among players? Would it be fair to divide profits proportionally based upon performance? Would it be fair to divide profits equally among shareholders despite differences in their individual contributions to profits? Are we paying for time? Are we paying for results? Those are questions the company might ask and make policy around. If we choose time and another company chooses results, might blue players leave for a deal that optimizes for what feels fair to them? When we take it into the market scope, it changes how fair a particular policy feels. And when we take the subjective experience of one player versus another, that further changes how it feels from a fairness perspective. What feels fair or unfair is fundamentally subjective, and in academia, definitions of fairness anchor on concepts like envy-freeness, where no other person with equal rights feels envious of another person's share. Obviously, if I come along and I give each of you $5, you don't feel envious of anyone else's receipt of the money.

4:15 But let's look at a subtly more interesting example. We might offer two working arrangements to employees. One where you can work 40 hours per week and be paid 100% of a salary. Another where you work 32 hours per week and are paid 80% Since both options are available and there's pay parity per hour worked, you might be able to argue that this is an envy-free work arrangement, as long as it's up to the employee to choose how much time they commit to work. Now, let's say that the person working 32 hours per week just happens to be better at their job, and they produce the same amount of output as the person working 40 hours per week in less time. And suddenly the arrangement starts to feel less fair to the person being paid 80%, even though they're the person being paid 80%. They're paid the same per hour wage. They're not paid per unit of output. So if they were paid per unit of output and not per hour, then the person working more hours may feel it is unfair that they are paid the same as someone else putting in less time. Again, fairness is fundamentally a subjective question.

5:17 Fairness game number three, disparate impact. Let's start with 1,000 players evenly distributed between red, blue, and green, and they're vying for 100 well-paying jobs. All players have equal potential to pass the interview. However, the job does require a certificate to interview. 15% of blue players have a certificate, 21% of red players, and 35% of green players. So how does this one requirement affect outcomes for each group of players? When we look at this chart on the bottom, you can see what the outcome is from just this one requirement. As you can see, green players are far more likely to get the job than blue players applying at the top of the funnel. The rules are consistent for all groups of players, but the starting point is not. To create equal odds for red and blue players to be hired as for green, you'd need to start the game with 467 blue players, 333 red players, and just 200 green players, not the even distribution that we started with. These percentages represent real numbers for college graduation rates in the United States among several race-based demographic groups.

6:24 Note how this one requirement, just one, all other things being equal, results in wildly different outcomes for each group of players. If it's not predictive of hiring success for a given role, is it fair to include a requirement? Fairness game number four, the original position. And just before we get started, I want to make sure credit goes to where it's due. The philosopher John Rawls came up with this concept of the original position. So I've stolen it from him for the sake of this talk. Now diving in. Imagine we create a new society, 10,000 people, and they have broad-based diversity. They have different abilities. They have different demographics, different ways of thinking, different socioeconomic starting points. And your objective is to design the rules and norms by which society will be governed. That's a big ask. So. Imagine this. You're you, but you haven't been born yet. So you don't exist. But you could be born into any one of those 10,000 bodies.

7:25 You wake up one day and you are that person. You could be smart or not so smart. You could have exceptional or limited mobility. You could start rich. You could start poor, have dark skin or light, speak the native tongue or struggle to learn it. From the day you take your first breath, you might have won the genetic lottery or be fated to wrestle with constant insurmountable challenges. So given that you have no idea who you'll become, how should society work? Should access to health care, education, and housing be equitable for all? Better for some? If it is better for some, what traits should result in more access or better access? Should people of higher intelligence be paid more? Should there be a tax on wealth to avoid power accumulation? This exercise helps us to apply a more objective lens. Uniquely, it allows us to both think selfishly, what will be best for me, and think about what's best for everyone at the same time. So does this lens change any of your beliefs about what might be fair? So what do you think about when you think about fairness in hiring?

8:28 Is it consistent decision-making criteria? When we do talk about fair process, we're usually just talking about consistency. To design a consistent process for hiring, you want to figure out what somebody needs to be like in order to be successful in the role, what criteria they need to meet. And we want to figure out how we're going to do that. to assess those criteria. Those are sort of the foundational elements in order to build a consistent hiring process. And this is called structured interviewing. It's the thing my company brought to market as software in 2013, but it's been around for a really long time and it's been studied since even the 60s and 70s. Everyone used the same criteria, questions, and assessments, so it's repeatable and reliable. And certainly it makes hiring more fair than an inconsistent process. But is it enough on its own to call hiring a hiring process fair? What if we set criteria like a college degree, which produces outcomes that feel unfair, or at least unfair by demographic group, like we saw in our earlier fairness game, demonstrating disparate impact? The other way we tend to evaluate whether a process is fair is to look at whether we see equal outcomes.

9:34 Demographics are interesting in part because whatever your race or gender is, is completely irrelevant to whether you're going to do well in a job. So if we're seeing differences in outcomes among different groups, might there be some other type of unfairness at work? Many times there is. There's implicit and explicit biases that cause differences in demographic outcomes, not just things like requirements. And how as interviewers we perceive differences in culture, communication, style, and even physical appearance can trigger unconscious bias and produce disparate outcomes for different demographic groups. And I'm eight. This is all to say that fair process and fair outcomes are often two different things. Fair process is about creating consistency, and fair outcomes is about producing consistent results. Now, we spent a lot of time with our heads in the clouds talking about fairness philosophically. In this next section, let's start applying some of that thinking to the hiring process. This is a simplified view of the entire hiring funnel.

10:37 Sourcing, finding candidates, screening, whittling down the pile, interviewing, deeply assessing specific skills. Offer, hoping that they accept the job based upon your proposed terms. In this next section, we'll look at each part of the funnel and talk through the types of design choices we make when hiring and the effects on both fairness and equitability of outcomes. So what is sourcing? It's fundamentally finding people to devote the time and effort to go through your interview process. This includes tons of different things. Job boards, agencies, how you market the role into the world, events, cold outreach, referrals, internal job boards, and more. All of these present choices which affect how accessible your roles are. Who do you tell about the job? Who do you let apply? When I was hired at Greenhouse before it had a name or the first lines of code were written, I was hired because someone I had worked for in the past had referred me. And that person knew about the job because... Someone else had referred him. So like in many startups, despite being 10 years younger, I was remarkably like the founders. I was a white, college-educated, Jewish straight man living in Manhattan, networked in the nascent New York startup ecosystem.

11:43 And I got the job. And if I'm totally honest with myself, it was a life-altering career opportunity, and it might have been for just about anyone. An opportunity that not everyone had equal access to, because they never knew the opportunity existed in the first place. So this story is repeated time and time again in hiring, and we must be mindful that this type of access issue, where people don't know about the job, reinforces existing inequities in our society. By this, I don't mean to say that referrals or any other specific sourcing strategy is particularly bad or intrinsically unfair, but if we do strive to build demographically diverse teams, we must ensure that a diverse group of people know about the job in the first place. Building a sourcing plan that strives for equal access across demographics is still a challenge and requires a lot of work in order to get it right. When we're encouraging people to apply to a job, we get to choose who to invite to the party and what the admission criteria are. Many companies are working hard to diversify their teams, trying to drive improvements in business outcomes, helping to attract and retain great people.

12:45 And the way companies do this is often by looking at photos, names, and affiliations and guessing at somebody's ethnicity or gender. Though many pieces of software do now allow you to specify gender pronouns, Greenhouse included, a number of A number of companies have started creating sourcing tools to go find candidates that leverage AI to do the same thing. They're guessing at race or gender by running photographs and names through a model. This does mean that companies are using race and gender at scale as selection criteria to invite someone to apply to a job, which can be an emotionally charged ethical and legal gray area. This is a topic our own ethics committee has discussed in depth, and our view is that the ends just don't justify the means, given the significant moral hazards posed by facial recognition technology. We believe that self-identification is the right approach, and carries with it less ethical and legal risk than ML-based approaches. We also believe looking at population level data to choose sources, for example, job boards which attract applicants, which would add diversity to the team,

13:49 provides a path to build diverse applicant pools without explicitly using race or gender at the individual level in assessing fitness for a role. Which brings us to the question, should we use demographic data as part of our sourcing strategy? Again, it depends on the scope we use in thinking about fairness. At the individual level, it is discriminatory to use race or gender as explicit hiring criteria. At the societal level, ignoring demographics distributes opportunity unevenly, favoring those groups who already have power, and leading to unfair outcomes for this generation and the next. Screening is the part of the hiring process where we whittle down the pile. We usually screen in phases. review, assessments, phone screens, and the goal of the screening process is to ensure that the company is only investing time in interviewing those who are most likely to be hired into the role. As a candidate, if you stand no chance, it's better to get knocked out early than waste your time on a hiring process. There are jobs for which there are literally a thousand people or more vying for the position.

14:51 To whittle down a thousand people to just one requires filtering through people, a lot of people. And it's often true at the very top of the funnel where candidates apply, recruiters are under-resourced to sort through all the candidates one by one. They certainly don't have time to get on a phone call with a thousand people. So they make some judgments about which things make it more likely someone will be hired successfully into a role. Common things might be requisite skills and experience or tenure, prestige of previous employers, the recruiter gut feel. Some are assessed by a real human being, and some of these things might be addressed by AI or ML. Real humans tend to make these judgments by reading resumes and trying to match keywords, and so do computers. They make judgments about the quality of the applicant based upon proxies, like did they go to a top school, or are they a job hopper? ML-based candidate ranking solutions tend to mimic the prioritization that humans use. But ranking is fraud, and most ML-based ranking solutions today are black box. It's not obvious which features of an applicant's resume are used to provide the ranking.

15:54 Famously, Amazon built a black box algorithm to mimic recruiter behavior without sufficient feature engineering, and it started to show a preference for men in engineering roles. The preference likely existed, perhaps subconsciously, in how recruiters made decisions. And predicting what a recruiter would do accurately was likely how they measured success for the algorithm. That said, a set of undesirable behaviors turned into an algorithm has the potential to institutionalize unfairness at scale, so we should be very wary of ranking which lacks explainability. Let's talk about the Fair Credit and Reporting Act for a minute, because I think it provides an interesting... model to think about this sort of thing and how regulation might be applied in order to create more explainable hiring decisions. So once upon a time, banks built risk models which took into account bad criteria like, does this person live in a zip code where default happens more frequently? This often meant that despite what might be a perfect individual credit track record, a consumer might be denied access to credit based on criteria which correlated with default, like living in a low-income neighborhood.

16:57 To avoid this type of issue, the FCRA was created to require that if a credit line is denied, the bank provides the underlying reason behind the denial. If you've ever been rejected for a credit card, you got some type of explanation. It said not enough credit history or late payment 90 days, at least in the U.S. For other consequential decisions in life, like hiring decisions, this remains largely unregulated, and it's up to the company's discretion. Most people never even hear back when they send in an application, nonetheless understand why they were rejected in the first place. A fair screening process uses explicit, explainable criteria, just like in structured interviewing, to ensure consistent and fair treatment of candidates. This should be true with or without ML in the mix. There are regulations which try to ensure protected classes are not discriminated against, but they don't go as far as the FCRA in trying to ensure all hiring decisions are made using fairness criteria. While screening was about saving time assessing people who are unlikely to get the job, interviewing is about deeply understanding how well the candidate can do the job and collaborate with her team.

18:01 Ultimately, there may be a bunch of people we find are qualified at the end of the interview process, but we may still be forced to whittle down the pile again to the one, the candidate we ultimately hire. And earlier in this talk, we spoke about consistency as a key part of how to think about fairness in hiring. In this next section, we'll get into the mechanics of how to create a consistent process. Let's talk about the most important anti-bias tool we'll discuss today, and I mentioned its name earlier, structured interviewing. It's our best tool in the fight against capriciousness in hiring. It's the only approach validated time and time again in academic research to reliably produce better hires and decrease the effect of bias in the interview process. A structured interview process is the set of answers to the questions, what needs to be true about this candidate to succeed in the role, and how will we evaluate each of those criteria. In practice, this means starting with thinking through what's important and turning those criteria into a scorecard. Then figuring out what questions or tests would evaluate whether each of those criteria are true. By writing all of this stuff down, it becomes repeatable.

19:03 So the interview can be conducted by anyone, and the candidate will have a consistent, mostly fair experience. Being explicit about criteria and questions does not eliminate subjectivity, and therefore unfair differences in evaluation. But it does make things much more fair than they would be if left up to each interviewer to just choose their favorite gotcha questions. In structured interviewing, your process has decision points. Should this person be advanced? Interviews and assessments? Does this person have the skills to do the job well and a scorecard that gets progressively filled out by the team? To make a technical analogy, your interview process is a function that takes a collection of candidates and returns a hire. The quality of what gets returned by that function rests almost entirely on how carefully you design the algorithm. Designing a structured interview process requires a significant investment of time and energy to get right, but it produces better hires, hired more fairly. There are two reasons that not everyone does this. One, they might not be aware. And number two, it's a lot of work. It really is.

20:04 It's a lot of work. To speak to that latter bit, I just want to say it's worth it. You know what's way worse than building interview plans? Building a performance improvement plan. Winding up with the wrong person. Or having to let someone go. It's awful for you as a manager, but it's way worse for them. You just committed to bring them on and give them an exciting new job and they find themselves jobless months later because you weren't careful enough in how you decided to hire. It's not just a way to produce more fair outcomes to you structured interviewing and create a plan. It respects the gravity of the decision to make a hire on your life and on the company's. A well-structured interview process tells you things you need to know in order to make a hire across a bunch of different dimensions. Do they have the technical skills? Are they a great collaborator? Do they want the job? Are they strong written communicators? We're testing different aspects of a person throughout the process, so it shouldn't be surprising that some interviewers rate the candidate differently than others. As they were likely asking different questions meant to assess different qualities. It's possible that they were all right.

21:05 The image represents a parable called the blind men and the elephant. Each feels a different part of the animal and describes it differently as a result. The one touching the leg thinks it's like a tree. The one touching the trunk thinks it's like a snake. All of them are of course right, but each had a narrow perception of the whole. That would be true in interviewing as well. Interviewing is a team sport and we must trust each other's assessments. Assessments become a lot more more trustworthy when it's clear what's being asked and how. And that makes the who a lot less relevant. Another critical aspect of fairness in hiring is to eliminate the influence of factors which are irrelevant to a person's ability to succeed in the role. This means trying to level the playing field for as many of those relevant factors as possible. My bad lighting or audio effect how a video interview goes? Send some tips beforehand. Interviewing someone hearing impaired? Be sure to turn on closed captioning in Zoom. What's your office dress code? Let the candidate know. I once showed up for a software engineering interview about 12 years ago in a three-piece suit and it felt weird to the team wearing t-shirts.

22:08 The only silver bullet for inclusivity is thoughtfulness, actively thinking about how to eliminate irrelevant factors for the candidate. One final topic I want to discuss in interviewing is the role of funnel analysis and how it affects hiring outcomes across demographic groups. What you see here is a report stolen from Greenhouse that shows the pass-through rates of candidates from different racial and ethnic backgrounds from, let's call it, Company X. And as you can see, in this process, we see that the proportion of South Asian applicants progressing to hire is lower than Latinx. Now, at the top of the funnel, you see significantly more South Asian folks than you do Latinx. But by the time you get to the bottom of the funnel, it's four times more likely that a Latinx person will be hired than a South Asian person. So from these numbers, we can tell that something is going on. The hard work of fairness in hiring often comes down to How far we're willing to go to understand why. Finally, let's talk about offers. You've sourced hundreds of candidates, you've slogged through months of interviews, and you've found them the one, and you go to make your offer and they've rejected you.

23:12 They found another offer with higher pay. They rejected you. They found another offer with a more aligned mission, a better vacation policy. You did so much work to get to this stage, and now you're back to square one. Candidates evaluate the fairness of offers according to three, and maybe more, different lenses. One, how does this compensation compare to the market as a whole? How does compensation compare to my current deal? Is this going to be better than what I've got today? And number three, how do I value all the intangibles, the mission, the benefits? For them to feel that the offer is fair, they'd like it to be true that you're paying market. They'd like to be paid more than they were in their previous role. But it's also true that things like greater flexibility in their working arrangements, a mission that they really care about, or challenging work, great coworkers, all of these are part of the value proposition that your company puts out there when you send an offer to a candidate. And though pay is critically important, it's not the entire picture. Important to keep in mind. Revisiting a concept from earlier, the perception of fairness, is this actually a fair deal, is about reducing envy when we compare our lot to others.

24:16 If we work for a company that feels like it has a much better mission, we might be willing to give up a few dollars in the process to be there. Negotiation is part of arriving at any fair agreement between two parties, ensuring that each is satisfied and the with the arrangement before signing on the dotted line. But a fair deal between two parties may create systemic unfairness, and this is no better highlighted than as part of the gender wage gap. There's way too much going on here for me to explain in this talk, and I certainly don't know all of the answers. There's in transgender roles, professions chosen, inequity in childcare responsibilities, and the list goes on and on. But it's also been shown in studies that men are more likely to negotiate than women. In a 2018 study by Robert Half, 68% of men And 45% of women negotiated the terms of their offer. Men are 50% more likely to negotiate, and that's a big deal. If we assume negotiation makes it more likely that you'll be paid more, then as a cohort, men are, based on this one difference alone, likely to be paid more than women. So how do we make negotiation more fair?

25:19 One thing we can do is try to set some clear bounds. This is the upper limit. This is the lower limit that we're willing to discuss. An explicit offer to discuss terms in the first place with the candidate provides an opening for people who are less assertive. But ultimately, the optimal strategy for fairness might be not to negotiate pay at all. And that means paying enough to close the candidates that you need without negotiating. Some might walk away, but for those who accept your offers, you'll know that your pay is consistent with what you believe about the market, not determined by a candidate's level of assertiveness. For most roles, assertiveness is not what you're hiring for, and it's an irrelevant trait to pay extra for. That all said, we don't live in a perfect world where no one budges on offers and pays top of market. We're running real businesses, and that sometimes does mean negotiation. But we should be wary of how a reactive policy on negotiation creates disparate impact, as we may contribute unwittingly to inequality by doing so. So that's really the bulk of it. A few takeaways. Fairness is both complex and subjective.

26:21 Fair process and fair outcomes are not always the same thing. And fairness issues creep into every step of the hiring process. So be mindful. As leaders, We are the hiring market, and together we can make hiring more fair. Thank you.