Accès ouvert · CC BY

A Dual-Layered Evaluation of Geopolitical and Cultural Bias in LLMs

Article scientifique 2025 Sciences économiques et de gestion › Sciences de gestion (OCDE 5.2) Anglais

Résumé

As large language models (LLMs) are increasingly deployed across diverse linguistic and cultural contexts, understanding their behavior in both factual and disputable scenarios is essential-especially when their outputs may shape public opinion or reinforce dominant narratives.In this paper, we define two types of bias in LLMs: model bias (bias stemming from model training) and inference bias (bias induced by the language of the query), through a two-phase evaluation.Phase 1 evaluates LLMs on factual questions where a single verifiable answer exists, assessing whether models maintain consistency across different query languages.Phase 2 expands the scope by probing geopolitically sensitive disputes, where responses may reflect culturally embedded or ideologically aligned perspectives.We construct a manually curated dataset spanning both factual and disputable QA, across four languages and question types.The results show that Phase 1 exhibits query language-induced alignment, while Phase 2 reflects an interplay between the model's training context and query language.This paper offers a structured framework for evaluating LLM behavior across neutral and sensitive topics, providing insights for future LLM deployment and culturally-aware evaluation practices in multilingual contexts.WARNING: this paper covers East Asian issues which may be politically sensitive.

Citer ce document

Kim, S., & Kim, H. J. (2025). A Dual-Layered Evaluation of Geopolitical and Cultural Bias in LLMs. https://doi.org/10.18653/v1/2025.acl-srw.38

Exporter : BibTeX · RIS (Zotero, Mendeley, EndNote)

Accès au document

Texte intégral archivé sur SPHAERO, en accès ouvert : lisible gratuitement en ligne après connexion. Le document reste aussi disponible à sa source. Se connecter

Voir l'article sur le site de la revue

Licence et provenance

Licence : CC BY

Notice moissonnée depuis OpenAlex le 28/09/2026. Le document reste hébergé par sa source.
Voir le document à la source →

Statistiques

Consultations (hors robots) : 6

Téléchargements (hors robots) : 0

Méthode de comptage