Candidates using AI in interviews
Candidates using AI in interviews can change a score when the answer is not watched. Two published studies measure pieces of that. The figures below are theirs. Each paragraph says what the study measured.
In ChatGPT, can you take my job interview? Examining artificial intelligence cheating in the asynchronous video interview (Damian Canagasuriam and Eden-Raye Lukacik, International Journal of Selection and Assessment, first published 24 June 2024), 245 people completed an asynchronous video interview. One group used no ChatGPT. One group read ChatGPT answers word for word. One group edited ChatGPT answers. Mean overall performance, scored out of 5, was 2.52 with no ChatGPT, 3.78 when the answers were read word for word, and 3.62 when the answers were edited. Mean honesty, scored out of 100, was 81.83, 42.77, and 59.40 in that same order. The paper reports that response delivery ratings did not differ across the three groups. These means are video-interview scores from that study. A coding decision uses a watched coding test.
In The Impact of Generative AI-Powered Code Generation Tools on Software Engineer Hiring: Recruiters' Experiences, Perceptions, and Strategies (Alyssia Chen, Dan Port, Timothy Huo, Anthony Peruma, and Yunhee Nam, arXiv, September 2, 2024), 32 people who hire software engineers answered a survey open from February 2024 to May 2024. On official guidelines for AI code-generation tools when evaluating candidates, 21 of 32 (65.63%) said their organization had none, 5 of 32 (15.63%) said it had guidelines, and 6 of 32 (18.75%) were not sure. On a separate question, 1 of 32 (3.13%) said they had permitted candidates to use those tools in a technical interview, 20 of 32 (62.50%) said they had not, and 11 of 32 (34.38%) selected not applicable. The paper reports those counts for this group of 32.