Incremental Value of Structured Metadata for Text Based Risk Triage in a Role Play Derived University Counselling Corpus
Résumé
Abstract Automated triage systems for student mental-health support may combine free-text messages with demographic and intake fields, but the incremental value of structured data requires empirical testing. This study compared text-only, structured-only and combined logistic-regression classifiers using a role-play-derived university counselling corpus. The original corpus contained 673 scenarios and the expanded corpus 1,672 scenarios; complete-case analyses used 654 and 1,509 scenarios, respectively. Under five-fold shuffled cross-validation, text-only escalation ROC-AUC was 0.705 (original) and 0.805 (expanded), compared with 0.687 and 0.799 for combined models and 0.633 and 0.752 for structured-only models. Paired bootstrap and repeated-split analyses found no demonstrated advantage from adding structured variables, while structured-only performance remained lower than text-only performance. Problem-category indicators carried most of the structured signal; demographic fields contributed little. At a 0.5 threshold, text-only sensitivity was 62.8% and 66.8%, while Critical-class recall in the four-class model was 25.0% and 64.9%. Achieving 95% escalation sensitivity required flagging 87% and 78% of cases. Structured metadata therefore provided no consistent incremental benefit once message text was available. However, participant-level grouping was unavailable, the corpus was authored through role-play, and no model achieved performance suitable for autonomous safety-critical triage. Independent validation on authentic counselling text is required before operational use.
Citer ce document
Exporter : BibTeX · RIS (Zotero, Mendeley, EndNote)
Accès au document
Texte intégral archivé sur SPHAERO, en accès ouvert : lisible gratuitement en ligne après connexion. Le document reste aussi disponible à sa source. Se connecter
Voir l'article sur le site de la revueLicence et provenance
Licence : CC BY
Notice moissonnée depuis OpenAlex le 09/11/2026. Le document reste hébergé par sa source.
Voir le document à la source →