OpenAI confirms it’s looking into text watermarking for ChatGPT that could expose cheating students

Following a report from The Wall Street Journal that claims OpenAI has been sitting on a tool that can spot essays written by ChatGPT with a high degree of accuracy, the company has shared a bit of information about its research into text watermarking — and why it hasn’t released its detection method. According to The Wall Street Journal’s report, debate over whether the tool should be released has kept it from seeing the light of day, despite it being “ready.” In an update published on Sunday to a May blog post, spotted by TechCrunch, OpenAI said, “Our teams have developed a text watermarking method that we continue to consider as we research alternatives.”

The company said watermarking is one of multiple solutions, including classifiers and metadata, that it has looked into as part of “extensive research on the area of text provenance.” According to OpenAI, it “has been highly accurate” in some situations, but doesn’t perform as well when faced with certain forms of tampering, “like using translation systems, rewording with another generative model, or asking the model to insert a special character in between every word and then deleting that character.” And text watermarking could “disproportionately impact some groups,” OpenAI wrote. “For example, it could stigmatize use of AI as a useful writing tool for non-native English speakers.”

Per the blog post, OpenAI has been weighing these risks. The company also wrote that it has prioritized the release of authentication tools for audiovisual content. In a statement to TechCrunch, an OpenAI spokesperson said the company is taking a “deliberate approach” to text provenance because of “the complexities involved and its likely impact on the broader ecosystem beyond OpenAI.”