Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Monitoring hiring discrimination through online recruitment platforms

Abstract

Women (compared to men) and individuals from minority ethnic groups (compared to the majority group) face unfavourable labour market outcomes in many economies1,2, but the extent to which discrimination is responsible for these effects, and the channels through which they occur, remain unclear3,4. Although correspondence tests5—in which researchers send fictitious CVs that are identical except for the randomized minority trait to be tested (for example, names that are deemed to sound ‘Black’ versus those deemed to sound ‘white’)—are an increasingly popular method to quantify discrimination in hiring practices6,7, they can usually consider only a few applicant characteristics in select occupations at a particular point in time. To overcome these limitations, here we develop an approach to investigate hiring discrimination that combines tracking of the search behaviour of recruiters on employment websites and supervised machine learning to control for all relevant jobseeker characteristics that are visible to recruiters. We apply this methodology to the online recruitment platform of the Swiss public employment service and find that rates of contact by recruiters are 4–19% lower for individuals from immigrant and minority ethnic groups, depending on their country of origin, than for citizens from the majority group. Women experience a penalty of 7% in professions that are dominated by men, and the opposite pattern emerges for men in professions that are dominated by women. We find no evidence that recruiters spend less time evaluating the profiles of individuals from minority ethnic groups. Our methodology provides a widely applicable, non-intrusive and cost-efficient tool that researchers and policy-makers can use to continuously monitor hiring discrimination, to identify some of the drivers of discrimination and to inform approaches to counter it.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Effects of the ethnicity of jobseekers on contact rate and time spent by recruiters on jobseekers’ profile.
Fig. 2: Variation in the time that recruiters spend on profiles and ethnic penalties during the workday.
Fig. 3: Gender-based hiring disparities across occupations.

Data availability

To obtain access to the anonymized data, researchers have to sign a data sharing agreement with the KOF Micro Data Centre at ETH Zurich and the Labour Directorate of SECO. Source data are provided with this paper.

Code availability

Code to replicate all analyses presented here is available at the publicly accessible Harvard Dataverse: https://doi.org/10.7910/DVN/GGENFB.

References

  1. 1.

    Altonji, J. G. & Blank, R. M. Race and gender in the labor market. Handbook of Labor Economics 3, 3143–3259 (1999).

    Article  Google Scholar 

  2. 2.

    Blau, F. D. & Kahn, L. M. The gender wage gap: extent, trends, and explanations. J. Econ. Lit. 55, 789–865 (2017).

    Article  Google Scholar 

  3. 3.

    Neumark, D. Experimental research on labor market discrimination. J. Econ. Lit. 56, 799–866 (2018).

    Article  Google Scholar 

  4. 4.

    Bartoš, V., Bauer, M., Chytilová, J. & Matějka, F. Attention discrimination: theory and field experiments with monitoring information acquisition. Am. Econ. Rev. 106, 1437–1475 (2016).

    Article  Google Scholar 

  5. 5.

    Bertrand, M. & Mullainathan, S. Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. Am. Econ. Rev. 94, 991–1013 (2004).

    Article  Google Scholar 

  6. 6.

    Bertrand, M. & Duflo, E. in Handbook of Economic Field Experiments Vol. 1 (eds Banerjee, A. V. & Duflo, E.) 309–393 (Elsevier, 2017).

  7. 7.

    Baert, S. in Audit Studies: Behind the Scenes with Theory, Method, and Nuance (ed. Gaddis, S. M.) 63–77 (Springer, 2018).

  8. 8.

    Kuhn, P. & Shen, K. Gender discrimination in job ads: evidence from China. Q. J. Econ. 128, 287–336 (2013).

    Article  Google Scholar 

  9. 9.

    Horton, J. J. & Tambe, P. Labor economists get their microscope: big data and labor market analysis. Big Data 3, 130–137 (2015).

    Article  Google Scholar 

  10. 10.

    Chan, J. & Wang, J. Hiring preferences in online labor markets: evidence of a female hiring bias. Manage. Sci. 64, 2973–2994 (2018).

    Article  Google Scholar 

  11. 11.

    Eriksson, S. & Lagerström, J. Detecting discrimination in the hiring process: evidence from an internet-based search channel. Empir. Econ. 43, 537–563 (2012).

    Article  Google Scholar 

  12. 12.

    Tibshirani, R. Regression shrinkage and selection via the lasso. J. R. Stat. Soc. B 58, 267–288 (1996).

    MathSciNet  MATH  Google Scholar 

  13. 13.

    Belloni, A., Chernozhukov, V. & Hansen, C. Inference on treatment effects after selection among high-dimensional controls. Rev. Econ. Stud. 81, 608–650 (2014).

    MathSciNet  Article  Google Scholar 

  14. 14.

    Zschirnt, E. & Fibbi, R. Do Swiss Citizens of Immigrant Origin Face Hiring Discrimination in the Labour Market? Working paper 20 (Université de Neuchâtel, 2019).

  15. 15.

    Hainmueller, J. & Hangartner, D. Who gets a Swiss passport? A natural experiment in immigrant discrimination. Am. Polit. Sci. Rev. 107, 159–187 (2013).

    Article  Google Scholar 

  16. 16.

    Greenwald, A. G. & Banaji, M. R. Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychol. Rev. 102, 4–27 (1995).

    CAS  Article  Google Scholar 

  17. 17.

    Chugh, D. Societal and managerial implications of implicit social cognition: why milliseconds matter. Soc. Justice Res. 17, 203–222 (2004).

    Article  Google Scholar 

  18. 18.

    Muraven, M. & Baumeister, R. F. Self-regulation and depletion of limited resources: does self-control resemble a muscle? Psychol. Bull. 126, 247–259 (2000).

    CAS  Article  Google Scholar 

  19. 19.

    Pocheptsova, A., Amir, O., Dhar, R. & Baumeister, R. F. Deciding without resources: resource depletion and choice in context. J. Mark. Res. 46, 344–355 (2009).

    Article  Google Scholar 

  20. 20.

    Danziger, S., Levav, J. & Avnaim-Pesso, L. Extraneous factors in judicial decisions. Proc. Natl Acad. Sci. USA 108, 6889–6892 (2011).

    ADS  CAS  Article  Google Scholar 

  21. 21.

    Olivetti, C. & Petrongolo, B. The evolution of gender gaps in industrialized countries. Annu. Rev. Econ. 8, 405–434 (2016).

    Article  Google Scholar 

  22. 22.

    Azmat, G. & Petrongolo, B. Gender and the labor market: what have we learned from field and lab experiments? Labour Econ. 30, 32–40 (2014).

    Article  Google Scholar 

  23. 23.

    Riach, P. A. & Rich, J. An experimental investigation of sexual discrimination in hiring in the English labor market. Adv. Econ. Anal. Policy 5, https://doi.org/10.2202/1538-0637.1416 (2006).

  24. 24.

    Eagly, A. H. & Steffen, V. J. Gender stereotypes stem from the distribution of women and men into social roles. J. Pers. Soc. Psychol. 46, 735–754 (1984).

    Article  Google Scholar 

  25. 25.

    Pager, D. & Shepherd, H. The sociology of discrimination: racial discrimination in employment, housing, credit, and consumer markets. Annu. Rev. Sociol. 34, 181–209 (2008).

    Article  Google Scholar 

  26. 26.

    Hagendoorn, L. Intergroup biases in multiple group systems: the perception of ethnic hierarchies. Eur. Rev. Soc. Psychol. 6, 199–228 (1995).

    Article  Google Scholar 

  27. 27.

    Phelps, E. S. The statistical theory of racism and sexism. Am. Econ. Rev. 62, 659–661 (1972).

    Google Scholar 

  28. 28.

    Arrow, K. The Theory of Discrimination. Working paper 30A (Princeton University, 1971).

  29. 29.

    Becker, G. S. The Economics of Discrimination (Univ. Chicago Press, 1957).

  30. 30.

    Greenwald, A. G. & Krieger, L. H. Implicit bias: scientific foundations. Calif. Law Rev. 94, 945–967 (2006).

    Article  Google Scholar 

  31. 31.

    Bohnet, I., Van Geen, A. & Bazerman, M. When performance trumps gender bias: joint vs. separate evaluation. Manage. Sci. 62, 1225–1234 (2016).

    Article  Google Scholar 

Download references

Acknowledgements

We acknowledge funding from the Swiss National Science Foundation (grant no. 162620). We are grateful to SECO for collaborating on this project and sharing data, and we thank S. Thöni and M. Bannert for programming support.

Author information

Affiliations

Authors

Contributions

D.H., D.K. and M.S. designed the research; D.H., D.K. and M.S. performed the research and analysed the data; and D.H., D.K. and M.S. wrote the paper.

Corresponding author

Correspondence to Dominik Hangartner.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Nature thanks Philipp Kircher and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended Data Fig. 1 The search process on Job-Room.

The search process on the government-affiliated online recruitment platform of the Swiss public employment service. Left, stylized illustration of the search process; right, screenshots of every step. ac, First the recruiter completes the search mask (a), typically by entering the occupation for which a vacancy exists and place of work, from which a ranked list of candidates who meet these criteria is generated (b). From this list, the recruiter selects the profiles that provide detailed productivity-related information akin to a standard CV about the jobseekers’ education, work experience and skills, as well as gender and markers of ethnicity (c). The bottom of the profile contains a button that reveals the contact information, typically a jobseeker’s phone number and email address, and the contact details of the employment office.

Extended Data Fig. 2 Validation of research design.

a, Percentage point effects and associated cluster-robust 95% confidence intervals of ethnicity on the likelihood that the contact button has been clicked conditional on profile visit (n = 254,975). The sample is restricted to non-registered recruiters who do not observe the nationality and name of jobseekers. Controlling for detailed language skills of jobseekers, these recruiters should not be able to infer candidates’ ethnicity. Standard errors are clustered at recruiter level. The null hypothesis that all ethnicity coefficients are jointly zero is not rejected (P = 0.93 (two-sided F-test)). b, Comparison of our baseline results that are based on the click on the contact button (in blue) with analogous regressions with outcome variables that are more direct signals of contact attempts by recruiters (n = 3,251,303). Dots with horizontal lines indicate point estimates with cluster-robust 95% confidence intervals from OLS regression. The dependent variable in the model ‘mail or print button’ is a binary indicator if a recruiter clicked on the hyperlink of the email address of the candidate or on the button ‘print candidate profile’ available on the profile page (n = 3,251,263 profile views). These click events happen in only 3.3% and 4.8% of profile visits, respectively. The dependent variable in the model ‘contact button (60 sec)’ is a binary indicator equal to one if a recruiter clicked on the contact button and subsequently stayed on the profile of the respective jobseeker for at least 60 s—a sign that the recruiter contacted the jobseeker immediately, possibly via phone or email. This happens in 27.3% of all contact attempts. The coefficients are normalized with the mean of the outcome variables to represent percent effects.

Source data

Extended Data Fig. 3 Discrimination by type of recruiter.

a, Effects of ethnicity on contact rate estimated separately for recruiters with 1 search (n = 62,081 profile views), 2–10 searches (n = 404,203 profile views), 11–50 searches (n = 546,441 profile views) and more than 50 searches (n = 2,238,538 profile views) in the study period. The coefficients are normalized with the mean contact rate of the respective category. Dots with horizontal lines indicate point estimates with cluster-robust 95% confidence intervals from OLS regression. b, Effects of ethnicity on the contact rate estimated separately for registered (n = 2,996,288 profile views) and non-registered (n = 254,975 profile views) recruiters of the platform. Dots with horizontal lines indicate point estimates with cluster-robust 95% confidence intervals from OLS regression. The light red bars indicate minority ethnic groups that are not (or to a lesser extent) identifiable based on their language skills on Job-Room. Supplementary Table 11 provides an overview of the language skills of the different ethnic groups.

Source data

Extended Data Fig. 4 Effects by search duration within a session and for slow and fast deciders.

a, Effect of ethnicity on contact rate depending on the length of a session for immigrant jobseekers from Europe (in blue) and from outside of Europe (in red). Dots with horizontal lines indicate point estimates with cluster-robust 95% confidence intervals from OLS regression (n = 1,378,011 profile views). The estimates are derived from interactions between the indicators of ethnicity and indicators of the time elapsed since the start of the session. A session is defined as the time at which the recruiter logs into Job-Room until he or she logs off or closes the browser window. Longer sessions typically encompass several searches. As recruiters vary in their average session length, and these compositional differences can influence our results, the sample is restricted to sessions that take at least 30 min. The regressions control for session instead of search-fixed effects. Swiss jobseekers are used as the reference category. b, Effects of ethnicity on contact rate depending on the average decision time of a recruiter. Dots with horizontal lines indicate point estimates with cluster-robust 95% confidence intervals from OLS regression. The sample is restricted to recruiters who conduct at least 6 searches and view at least 12 profiles. We use a random subset of one third of searches per recruiter to assign recruiters into one of four groups: fast deciders (lowest quartile), rather fast deciders (second quartile), rather slow deciders (third quartile) and slow deciders (highest quartile). The figure shows the ethnic penalties estimated separately for each group of recruiters using the test sample (the remaining two thirds of searches per recruiter). The average time that recruiters are looking at jobseekers’ profiles in the test sample are 5.7 s for fast deciders (n = 303,934 profile views), 9.9 s for rather fast deciders (n = 545,538 profile views), 15.4 s for rather slow deciders (n = 690,989 profile views) and 31.7 s for slow deciders (n = 364,018 profile views).

Source data

Extended Data Fig. 5 Additional gender analyses.

a, Effect of female gender on the probability of contact (in %) (n = 17,369,372 profiles). The circles show the (dis)advantage that women face compared to men in a given occupation, plotted against the share of women in that occupation. In contrast to Fig. 3 in the main text, the share of women per occupation is calculated from the Swiss earnings structure survey instead of the share of women in the result list. The circumference of the dots denotes the share of searches in each occupation. The solid black line indicates the weighted least squares regression of the estimated gender disparity against the share of female workers in each occupation. The dashed black lines show the associated 95% confidence intervals. The colour of the circles indicates the ISCO-1 level occupation classification. b, Relationship between occupational wages and hiring penalties (advantages) for female versus male jobseekers, grouped by two-digit ISCO occupation (n = 17,369,372 profiles). We estimate occupation-specific wages from the October 2016 wave of the Swiss Earnings Structure Survey. The circles show the (dis)advantage that women face compared to men in a given occupation, plotted against the log average wage in the occupation. The circumference of the dots denotes the share of searches in each occupation. The solid black line indicates the weighted least squares regression of the estimated gender disparity against the average log wage in each occupation. The dashed black lines show the associated 95% confidence intervals. The colour of the circles indicates the ISCO-1 level occupation classification.

Source data

Extended Data Fig. 6 Tests for statistical discrimination.

a, Effects of ethnicity on the contact rate estimated separately for jobseekers with low (n = 332,425 profile views), medium (n = 585,979 profile views) or high (n = 501,506 profile views) unobserved employability. Dots with horizontal lines indicate point estimates with cluster-robust 95% confidence intervals from OLS regression. The construction of the index of unobserved employability exploit that we can observe the age, marital status and pre-unemployment wage of jobseekers. These characteristics are unobserved by recruiters on Job-Room. The construction involves several steps. First, we collapse the data at the jobseeker level and split the sample randomly in a training (50% of the searches) and a test sample (the remaining 50%). Second, we interact the three characteristics with nine occupation dummies (according to the Swiss SBN occupational classification). Third, we regress a dummy variable that a jobseeker leaves unemployment within the first four months after registering at the Swiss public employment service on these interactions, controlling for occupation-period and canton fixed effects. On the basis of these occupation-specific estimates, we predict the likelihood that jobseekers in the test sample leave unemployment within four months. We then assign each of these jobseekers to one of three equally sized groups (high, medium and low unobserved employability). The likelihood that a jobseeker with high unobserved employability leaves unemployment within 4 months is 37%, whereas it is 10% for jobseekers with low unobserved employability. The regressions are based on the test sample. b, Effects of ethnicity on the contact rate by length of the unstructured text field containing information on additional skills of jobseekers (see Supplementary Table 1 for examples). Dots with horizontal lines indicate point estimates of the interaction between length of skill field and ethnicity with cluster-robust 95% confidence intervals from OLS regression (n = 3,251,263 profile views). The reference categories (the hollow dots on the zero line) are Swiss jobseekers who report no additional skills, a skill field with 1 to 10 words and a skill field with more than 10 words, respectively.

Source data

Extended Data Fig. 7 Effect of a click on the contact button on the likelihood of leaving unemployment.

Effect and associated cluster-robust 95% confidence intervals of a click on the contact button on Job-Room on the likelihood that a jobseeker leaves unemployment, exploiting that we can link candidates to the unemployment register. The dependent variables are dummy variables equal to one if a jobseeker leaves unemployment within 30, 60, 90, 120 and 150 days, respectively, after a search. Each coefficient is derived from a separate regression (n = 12,823,811 profiles). The variable of interest is the contact attempt on Job-Room. The sample consists of all individuals that appeared within a result list of a search between March and September 2017. We drop the remaining periods because we observe unemployment exits only up to March 2018. We also exclude all registered unemployed individuals who reach their maximum benefit duration to make sure that exit out of unemployment represents a voluntary decision by the jobseeker. The regressions control for search-specific fixed effects and all variables visible to recruiters on Job-Room including unemployment duration at the time of the search and all relevant first-order interactions. The model contains 2,569 covariates that are predictive for leaving unemployment and the contact button click, selected by the Lasso-based post-double selection method. The regressions thus control for all factors that plausibly influence recruiters’ contact attempts on the platform. The coefficients are identified because certain individuals are contacted on the platform whereas observationally equivalent individuals who appear in the same search are not; for example, because of idiosyncrasies in recruiters’ decisions or because their candidate profile is not visited (for example, as they are ranked low in the result list). The average likelihood of leaving unemployment after 30 (60, 90, 120, 150) days is 17% (30%, 40%, 48%, 54%). The click on the contact button thus increases the likelihood to leave unemployment within 90 days after the search by 2.1% (0.821/40). Standard errors are clustered at recruiter level.

Source data

Extended Data Table 1 Comparison of ethnic penalties with estimates from a contemporaneous correspondence study
Extended Data Table 2 Characteristics of candidates with at least one profile visit by nationality
Extended Data Table 3 Descriptive sample statistics with and without sample restrictions

Supplementary information

Supplementary Information

The Supplementary Information contains supplementary methods and eleven supplementary tables that provide details on the data used in the analysis, the statistical model used to estimate ethnic and gender penalties, robustness and sensitivity analyses for these regressions, and extensions to our main regression results to draw out further implications.

Reporting Summary

Source data

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hangartner, D., Kopp, D. & Siegenthaler, M. Monitoring hiring discrimination through online recruitment platforms. Nature 589, 572–576 (2021). https://doi.org/10.1038/s41586-020-03136-0

Download citation

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Quick links