Detecting GPT-Assisted Writing Using Interpretable Stylometric Features

arXiv:2609.26687 · cs.CL, cs.CY · Submitted 2026-09-22 · Read on arXiv

cs.CL, cs.CY

Submitted: 2026-09-22

Updated: 2026-09-22

Comments: 10 pages, 6 figures, 5 tables

Journal ref: Hawaii International Conference on System Sciences (HICSS), 2027

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

The gist: Distinguishing GPT-assisted from independently authored student writing has become a critical challenge in academia.

Terminology

Abstract

Distinguishing GPT-assisted from independently authored student writing has become a critical challenge in academia. This paper evaluates the discriminative capability of interpretable stylometric features extracted solely from submitted text. Using data from 90 participants who wrote both independently and with ChatGPT assistance, we evaluate eight machine learning classifiers while keeping data from the same participant together during validation. On the held-out test set, Random Forest achieved an ROC-AUC of 0.87 and an F1-score of 0.84, with False Positive and False Negative rates of 22.2% and 11.1%, respectively. SHAP analysis shows that lexical and grammatical characteristics drive the resulting predictions. The findings suggest that transparent, text-intrinsic features provide measurable signal for detecting GPT-assisted writing.

Sources

Related papers