Scientific writing with AI

As a researcher, group leader, teacher, reviewer, and editor, I work a lot with text. AI is changing the way I read, revise, and evaluate text. In some ways it’s becoming easier as texts by my students often come in a polished form that makes it easier to parse its information than when being distracted by language imperfections. In other ways, there has also been a rapid transformation in the way I appreciate this nice polished “surface”. The core of the text and its content, not its form, remains its essence. And it has become ever more important that this core lives up to expectations. Everybody can now generate text that appears publication-ready at first sight. Ever more clearly, the distinguishing point has become whether the science behind the text is solid and accurately represented by the text. Here lies the problem.

AI-style text is often generic, comes in hyperbole, and refers to specifics in an unreliable manner. In science, writing is a form of translating data and insights from analyses into words. In doing so, the text must stay true to what has been done and what can be said and is supported by the analysis. Perhaps most clearly in the Methods description of a scientific article, the text is just a form of describing exactly what your analysis scripts did, so that the scripts could be reliably reverse-engineered from your text. AI often generates text that violates the link between the specifics of your analysis and data and how it is presented in text. Instead, we want text that is concise, modest, true to the analysis and data, and is devoid of the typical markers of what comes out of an AI. When I get repeatedly reminded that I am reading the output of an AI, I turn away and view the text more critically than when it is a genuine product of the author personally.

Remember, as the author of a text, you are solely responsible for its content and bloomy words are not a replacement for content. As a teacher and supervisor in the age of AI, I learn to distill the essence and the specifics from the wrapping in polished text. I am becoming less fooled by good writing, hiding poor science. Therefore, much more attention will have to be paid to what matters most: the science on which the text reports. If I have to work hard in stripping down bloomy text from a feeble basis and am left with little that stands, mental load is wasted. Sometimes, the style of text an indicator for how large the gap between the shiny text and the science behind is. Therefore, be very careful with the adoption of AI-styled text! If you expose yourself as being replaced by an AI in excessive adoption of its style of writing, you take yourself out of the game and let yourself (your intellect, your style, your personality) be replaced by an AI. Don’t do that.

Another point is that AI “levels the playground” such that non-native English speakers can overcome a systemic disadvantage in the entirely English-dominated academic publishing world. That’s fair. But it remains important that we all keep control over the interpretation and framing of our science. We cannot delegate this to AI. The challenge will be to find the fine line between using AI as a tool to translate complex science into concise text versus letting AI to the full job of translating numeric insights into the form that is parsed by human brains - language. There is not a clear line. But if the framing and interpretation of the data is not your own and if language has diverged into a form that is clearly no longer your own personal way of writing, then it has gone too far.

The following is an incomplete collection of things to avoid and things to emphasise. If you have suggestions and comments, please direct-message me.

Things that you should do

• Write down all content in bullet points or in full text. Such text may be clumsy, but it must be accurate. Then you may let AI help you transform clumsy text into clear text. While you let AI help you translate numeric insights into comprehensible text, make sure it remains perfectly accurate to what you want it to express. The more complete your text instructions are, the more effectively you avoid AI taking it too far.
• Be as specific as possible. AI often generates text that sounds nice, but is not really referring to your specific analysis or data. Therefore, pay particular attention that the accurate description of specific types of analyses and data.
• Be as precise as possible in what conclusions you draw from your analysis and stay close to what your analysis actually yields. Highlight/explain/justify when generalisations and interpretation of wider validity are made. AI often takes interpretations to a level of generality that is not supported by your analysis. This must be counteracted (often actively) to avoid violation of what is actually a basic principle and old wisdom of scientific writing (AI doesn’t particularly go with old wisdoms): Avoid over-reaching conclusions, that is conclusions that are not supported by the analysis.
• Consistent use of specific terms throughout the manuscript. AI is often used to (re-) write individual paragraphs. Consistent use of terms is often lost by that practice.
• Cross-reference to figures and tables whenever suitable. AI often neglects cross-references.

Things that you should not do

• Avoid text in bullet points.
• Avoid excessive use of the semi-colon and the ‘—’. Instead use full stop ‘.’, commas, and brackets.
• Avoid persistent bold-facing of the first 1-3 words in each paragraph.
• Avoid judgemental language. Report the strength of an effect, or the significance, etc. with numbers and avoid any statement that this is ‘strong’, or ‘surprising’, or ‘interesting’, or ‘remarkable’. That’s an old wisdom in scientific writing, but AI doesn’t really know about old wisdoms, it seems.
• Avoid hyperbole. Use concise, precise, simple, and modest language. Make your text more boring than AI thinks it should be.