Section 3.78 min read

Responsible AI Use in Research

Core summary

Large language models (LLMs) like ChatGPT are increasingly used in research for writing assistance, literature summarization, and code generation. Major journals and institutions have established policies: AI cannot be listed as an author, AI use must be disclosed, and researchers remain fully responsible for all content. AI hallucination — confident but false outputs — is a critical risk requiring human verification of every AI-generated claim.

Detailed explanation

The rapid adoption of AI in research has outpaced policy development, but a consensus is forming. Key principles include: Authorship exclusion — AI systems cannot meet ICMJE criteria (they cannot be accountable), so they cannot be listed as authors. Transparency — researchers must disclose when and how AI tools were used, typically in the Methods section or an acknowledgments statement. Accountability — the human authors bear full responsibility for accuracy, originality, and integrity of all content, including any AI-assisted portions. Verification — AI-generated content must be verified against primary sources. LLMs can 'hallucinate' — generating plausible-sounding but fabricated references, statistics, or claims. Specific risks in research include: fabricated citations (AI may generate references that look real but do not exist), confident but incorrect statistical interpretations, paraphrasing that inadvertently plagiarizes source material, and generation of plausible but false data patterns. Responsible use means treating AI as an assistant, not an authority. Use it for: brainstorming, improving grammar and clarity, generating code scaffolds, and summarizing large texts. Always verify factual claims, check every cited reference exists, and never present AI output as your own expert judgment without verification.

Clinical example

You use an AI tool to help draft the discussion section of your manuscript. The AI generates a paragraph citing three supporting references. When you check: reference 1 exists and is accurate, reference 2 exists but the AI misrepresented its findings, and reference 3 does not exist at all — it was hallucinated. You must verify every single AI-generated citation before submission.

Research example

In 2023-2024, multiple published papers were found to contain telltale AI-generated phrases like 'As of my last knowledge update' or 'Certainly, here is...' — indicating that authors submitted AI-generated text without even basic review. These cases triggered retractions and heightened scrutiny of AI use in academic publishing.

Knowledge check

Q1. Can AI be listed as an author on a research paper?

Q2. What is 'AI hallucination' in the context of research?

Q3. Who bears responsibility for AI-generated content in a manuscript?