A shared characteristic of many of the malicious usages of LLMs is that they seek to conceal that the text was LLM-generated. This is notably the case for LLMs employed for the purpose of manipulating individuals on social media or spreading propaganda and misinformation. Or, on a more mundane level, the use of LLMs by pupils and students to expedite various school assignments.
There is currently no reliable method for detecting whether a piece of text was generated through a large language model instead of a human writer. There are, however, a number of so-called watermarking techniques (see for instance Kuditipudi et al, 2024) that can be applied to leave an "invisible" (to a human reader) mark in the selection of tokens, and therefore provide a way to determine the provenance of the text.
Although such techniques have been shown to be somewhat robust to modest edits in the text, it is always possible to modify the initial, LLM-generated text to such an extent that the watermark is no longer detectable. But where does this "tipping point" lie in practice, in particular for texts of moderate size, such as for instance school essays? Can text watermarks e.g. resist the use of automated paraphrasing models ?