Performance evaluation and interpretability of a Qwen LangGraph agent for AI written and AI rewritten text detection
Résumé
Abstract The widespread adoption of large language models has increased the need to distinguish human-authored text from AI-generated content, including cases in which generated text is subsequently rewritten or paraphrased. In practice, many detectors output only a single AI-likelihood score or percentage, offering limited transparency and little indication of whether a passage appears directly AI-written (AIW) or AI-rewritten (AIR). This study presents an agentic detection framework for two connected tasks: binary provenance detection (Human versus AI) and conditional subtype detection (AIW versus AIR). The dataset combined the Human-ChatGPT Comparison Corpus (HC3) and the ai-text-detection-pile with 4,348 pre-generated AIR records produced by a single GPT-based rewriting workflow. The proposed Qwen–LangGraph AI agent was compared to a HashingVectorizer–TF-IDF–SGD classifier, DeBERTa-v3-xsmall, and ELECTRA-small. On the binary test set, DeBERTa-v3-xsmall achieved the best discrimination (99.4995% accuracy and 99.9911% AUROC), whereas the agent achieved 98.8816% accuracy and 99.9187% AUROC. On the AI-only test set (10,769 AIW; 1,274 AIR), the agent obtained the highest threshold-based accuracy (96.7367%), macro-F1 (90.0357%), and AIR F1 (81.8643%), but its AUROC (83.2507%) and AUPR (79.7336%) were lower than those of the DeBERTa-v3-xsmall and ELECTRA-small transformers. Paired DeLong and McNemar analyses with Holm correction showed that the agent improved threshold-based AIR performance over the lexical baseline. Interpretability was operationalised through lexical weights, linguistic and out-of-distribution diagnostics, sentence-level indicators, and route traces. The findings support the use of AI as decision support, not as automated proof of plagiarism or misconduct.
Citer ce document
Exporter : BibTeX · RIS (Zotero, Mendeley, EndNote)
Accès au document
Texte intégral archivé sur SPHAERO, en accès ouvert : lisible gratuitement en ligne après connexion. Le document reste aussi disponible à sa source. Se connecter
Voir l'article sur le site de la revueLicence et provenance
Licence : CC BY
Notice moissonnée depuis OpenAlex le 19/10/2026. Le document reste hébergé par sa source.
Voir le document à la source →