How Riot Games job postings have affected company gender bias over time
Riot Games ($RIOTGAMES) is officially being investigated by the California Department of Fair Employment and Housing for "alleged unequal pay, sexual harassment, sexual assault, retaliation, and gender discrimination in selection and promotion."
Since the beginning of the reported cultural controversy inside the studio behind the popular video game League of Legends, we have tracked job openings at the company with full descriptions of all available jobs. This allows us to use our gender decoder, an algorithm based on a sociological study on language that may be off-putting to male and female applicants.
According to Riot Games job postings cross-referenced with word lists that indicate masculine or feminine bias, we not only saw job posting language lean away from masculine terms over time, but also a major shift in language used to describe positions.
A rating of 50% is regarded as the least off-putting language among an employer's job listings between men and women. A zero represents a strong lean towards language that would be off-putting to male applicants, and 100% represents a strong amount of language off-putting to female applicants, as determined by the aforementioned study.
In terms of a timeline, here is where key developments in the Riot Games scandal happened, and where the language in the company's job postings leaned:
This article written by Ceclia D'Anastasio at Kotaku highlighted the complex situation at Riot Games, and ultimately labeled the game company as one that had a "bro culture" with sexism present. At that point in time, the gender decoder rated the job opening language at Riot to be at 54.33%, which is 4.33 percentage points above the middle towards more masculine-leaning language.
This rebuttal to the Kotaku investigative report came at a time when, according to job listings data, descriptions were leaning slightly more masculine in language. Specifically, there was a .47 increase in masculine-leaning language among Riot Games' job postings between the Kotaku report and Riot's first reply.
Sep. 29, 2018 — Frances Frei is three weeks into his role as senior advisor for diversity and inclusion
Frances Frei, an ex-Uber SVP who was part of that company's culture turnaround under similar circumstances, was brought into Riot Games on September 12. A few weeks into his job, the gender decoder rated job openings at 56.98%, the largest gap from the middle ground of biased language.
Nov. 5, 2018 — First lawsuit filed against Riot
In between Frances Frei's hiring and Rosen Saba LLP suing Riot on behalf of one current and one former employee, Riot made several statements during its League of Legends World Championship esports event about having "some work to do to be better." Meanwhile, language in job openings started to head back towards the 50% mark and, by the time this lawsuit was filed, the average opening contained language that leaned more towards female applicants.
Dec. 13, 2018 — COO Scott Gelb is suspended without pay
Gelb, who faced several allegations from current and former employees about him touching worker's testicles and making inappropriate comments, got a two month suspension from Riot from December 13 onwards. At the same time, language in job openings at the company was actually leaning more masculine in nature.
Between Gelb's suspension and today, Riot released an update on diversity and inclusion, made several hires indiviative of change, including bringing on Angela Roseboro as its Chief Diversity Officer, went into arbitration over several lawsuits they faced, and also had a few employee walkouts.
All the while, the gender decoder score shifted drastically — from as high as 55.11% to 45.20% today — indicating a drastic change in language as to appeal to more applicants.
During this entire scandal, job openings at Riot Games increased over time.
It also does not seem as if there was a shift in what Riot was hiring for; Riot mainly looks for Engineers on a daily basis, as well as employees for its Publishing team.
Whether this is Riot Games recognizing a fault or saving face, there is a data-proven shift in the way it writes its job openings in an attempt to appeal to a wider base of applicants. Given that the company's scandal is coming up on its one year anniversary, however, better semantics to attract a diverse set of potential employees may be the least of the company's worries.
This article was written in collaboration within Thinknum Media.
About the data and the gender decoder
This gender decoder is based on the research paper "Evidence That Gendered Wording in Job Advertisements Exists and Sustains Gender Inequality" (Journal of Personality and Social Psychology, July 2011, Vol 101(1), p109-28), written by Danielle Gaucher, Justin Friesen, and Aaron C. Kay. In this paper, the three looked at language in job descriptions they found to be "feminine" or "masculine" in nature and whether or not men and women may be off-putted by sample job descriptions.
Thinknum tracks companies using information they post online - jobs, social and web traffic, product sales and app ratings - and creates data sets that measure factors like hiring, revenue and foot traffic. Data sets may not be fully comprehensive (they only account for what is available on the web), but they can be used to gauge performance factors like staffing and sales.