What Happens to Hiring When AI Writes Job Applications?

When AI Enters the Chat

Tuck economics professor Anaïs Galdin and Princeton Ph.D. student Jesse Silbert were early on the trail of AI’s effects on the matching process between employers and job seekers. In January of 2022, they began examining data from Freelancer.com, a platform where employers can find freelance workers for discrete digital gigs, like building a website or coding software. To apply for these jobs, workers write a short proposal and make a bid for the wage they expect to receive. When Galdin and Silbert started analyzing these proposals, they noticed that the ones that were more specific and detailed tended to result in the worker being hired, while the proposals that took less effort were more often unsuccessful. “Already, we were thinking, Okay, these proposals play an important role on this site,” Silbert says. 

Then the world changed. In November of that year, OpenAI released ChatGPT to the public, and people immediately marveled at its ability to write in a derivatively narrative style, aping anyone from Shakespeare to a business coach giving advice on a side-hustle. Freelancer.com joined the fray in April of 2023, introducing an on-platform AI-writing tool that allowed job seekers to generate an application with one click. Its LLM would use the job description and the user’s profile to spin up a proposal, and employers were unable to see which proposals were AI-generated. 

The End of Signaling

For Galdin and Silbert, this moment represented not only an exogenous shock to the system, but also a perfect research opportunity. Traditionally, a written job application and cover letter functioned as a signal of the applicant’s skill and ability. These signals are especially important in a context like Freelancer.com. “Employers are receiving around 60 applications within a few minutes,” Galdin said, “so it’s really hard to figure out who is good and who is bad. Having a bit of information beyond their bid has a high value for employers at the time of selection.”

However, with AI in the mix, anybody can put forth a customized, well-written application. The signal value of writing is therefore, at least theoretically, decimated. Galdin and Silbert wanted to see what the empirical evidence said. They present it in their new working paper, “Making Talk Cheap: Generative AI and Labor Market Signaling.”

They first examined the data before and after generative AI, looking at the signals workers sent and the effort they exerted in doing so. The co-authors measured these by employing an LLM to quantify the extent to which the proposals were customized and relevant to the job posting, and by observing click data that showed how much time each worker spent on each application. Before the platform deployed AI, employers were much more willing to pay for workers who sent customized proposals, and this is attributed to the workers’ signals being predictive of their ability to complete the job successfully. 

After AI, the picture changes drastically. As they write in their paper, “Employer willingness to pay for workers sending higher signals falls sharply, proposals written with the platform’s native AI-writing tool exhibit a negative correlation between effort and signal, and signals no longer predict successful job completion conditional on being hired.” AI, it appeared, had wiped out a critical indicator of a job seeker’s fitness for the task. 

But Galdin and Silbert needed to dig deeper. It was possible that other factors in the post-AI world were impacting the job market. Maybe employers were already outsourcing some of these jobs to AI. Maybe AI changed workers’ behavior around job selection. To isolate the effect of AI on the signaling function of job applications, the co-authors built a structural model that captures the logic inherent in the market. Within this mini laboratory, they asked what would happen to the market if everyone—or no one—could write an expert job application, but AI didn’t change anything else about the market. The result was an alarmingly dystopian race-to-the-bottom. In the simulated post-AI job market, workers “in the bottom quintile of the ability distribution are hired 14% more often, while workers in the top quintile are hired 19% less often,” they write. 

What Happens Next?

What can employers do today to find the best workers? It’s likely going to take more, well, work. For some jobs, especially those that are higher in value, a fuller screening via live talking and maybe even live tests, will be necessary. “Another solution is to allow for more exploratory sorts of temporary contracts,” Galdin said, “where you hire someone to do the first 10% of the job and see how it goes before offering a full-term contract.” 

Ironically, AI could help remedy the problem it created: platforms like Freelancer.com could deploy an AI tool to streamline employers’ search, highlighting the workers who have completed the most jobs that are like the one that is posted. Another possibility is an AI-powered questionnaire that helps employers find the right person based on the details of the job. 

For workers, AI will likely raise the importance of showcasing work experience. The challenge will no longer be using eloquence and rhetoric to prove your worth, but coming up with the hard currency of “what have you done lately?”