lnu.sePublikasjoner
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
How Bias Cues Affect Trust in Large Language Model Responses: An Empirical Study Design and Evaluation
Linnéuniversitetet, Fakulteten för teknik (FTK), Institutionen för datavetenskap och medieteknik (DM).
Linnéuniversitetet, Fakulteten för teknik (FTK), Institutionen för datavetenskap och medieteknik (DM).
2026 (engelsk)Independent thesis Basic level (degree of Bachelor), 10 poäng / 15 hpOppgaveAlternativ tittel
Hur biassignaler påverkar tilliten till svar från stora språkmodeller : En empirisk studiedesign och utvärdering (svensk)
Abstract [en]

Large language models (LLMs) are increasingly integrated into critical decision-making processes, serving as assistants for education, writing, and professional support. This widespread adoption places questions of bias and trust at the center of their responsible deployment. However, much of the existing research on LLM bias focuses on evaluating the internal mechanics and probabilities of models, rather than exploring how users perceive and interact with generated text in real-world scenarios. The core research problem addressed in this thesis is the misalignment between true model capabilities and human perception, often termed the calibration gap. Prior research establishes that state-of-the-art LLMs exhibit persistent social identity and cognitive biases despite rigorous value alignment [1, 2, 3]. Furthermore, users often systematically overestimate LLM accuracy when presented with confident or detailed explanations [4]. Parallel work demonstrates that superficial presentation cues, such as attribution or subtle framing, can systematically shift how information is evaluated [5]. Building upon this theoretical foundation, this thesis investigates how observable bias cues embedded in LLM responses, specifically operationalized as subtle positive or negative framing relative to a neutral baseline, influence readers’ perceived trustworthiness and related judgments. To examine this, we conducted an online survey experiment (N = 42) wherein participants evaluated five realistic AI assistant scenarios. For each scenario, participants were randomized to read one of three response variants and subsequently rated the output on four multidimensional Likert items: trustworthiness, objectivity, decision comfort, and appropriateness of tone. Our quantitative and qualitative analysis reveals that framing cues primarily affected perceptions of objectivity and tone, with both positively and negatively framed variants being rated as less objective and less appropriate than their neutral counterparts. Interestingly, the direct effects on overall factual trustworthiness were highlyscenario-dependent, yielding statistically significant differences in only two out of the five scenarios. Additionally, we found that a user’s baseline trust in AI correlated strongly and positively with their trustworthiness ratings across all scenarios(r = 0.51, p = 0.001). Taken together, the findings suggest that users often decouple factual trustworthiness from perceived appropriateness. Even when an LLM employs promotional or restrictive language, users may still trust the underlying information, particularly if their baseline confidence in AI is high. These results emphasize that trust is a complex, multidimensional construct influenced as much by stable individual predispositions as by the generated text itself. The study contributes to the ongoing discourse on LLM trust calibration by providing empirical, user-centered evidence on cue sensitivity, highlighting the necessity of designing systems that prioritize neutral presentation to mitigate the insidious effects of framing biases.

sted, utgiver, år, opplag, sider
2026. , s. 17
Emneord [en]
large language models, bias cues, trust, calibration, survey experiment, cog- nitive bias, framing
HSV kategori
Identifikatorer
URN: urn:nbn:se:lnu:diva-149359OAI: oai:DiVA.org:lnu-149359DiVA, id: diva2:2097167
Fag / kurs
Computer Science
Utdanningsprogram
Software Technology Programme, 180 credits
Presentation
2026-08-10, D3373V, Vejdes plats 7, Växjö, 09:00 (engelsk)
Veileder
Examiner
Tilgjengelig fra: 2026-09-01 Laget: 2026-08-31 Sist oppdatert: 2026-09-01bibliografisk kontrollert

Open Access i DiVA

fulltext(737 kB)19 nedlastinger
Filinformasjon
Fil FULLTEXT01.pdfFilstørrelse 737 kBChecksum SHA-512
510085f72add897666bf726f0b5bb6c18de676f0d84edfe8ee09ba75d7aa8c298fd9b67355af3a5000760ee8ab18fa24a862d916662a26fbd347eee08e6c3a73
Type fulltextMimetype application/pdf

Søk i DiVA

Av forfatter/redaktør
Brückner, JoonaNazeri, Mohammad
Av organisasjonen

Søk utenfor DiVA

GoogleGoogle Scholar
Antall nedlastinger er summen av alle nedlastinger av alle fulltekster. Det kan for eksempel være tidligere versjoner som er ikke lenger tilgjengelige

urn-nbn

Altmetric

urn-nbn
Totalt: 89 treff
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf