lnu.sePublications
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
How Bias Cues Affect Trust in Large Language Model Responses: An Empirical Study Design and Evaluation
Linnaeus University, Faculty of Technology, Department of computer science and media technology.
Linnaeus University, Faculty of Technology, Department of computer science and media technology.
2026 (English)Independent thesis Basic level (degree of Bachelor), 10 credits / 15 HE creditsStudent thesisAlternative title
Hur biassignaler påverkar tilliten till svar från stora språkmodeller : En empirisk studiedesign och utvärdering (Swedish)
Abstract [en]

Large language models (LLMs) are increasingly integrated into critical decision-making processes, serving as assistants for education, writing, and professional support. This widespread adoption places questions of bias and trust at the center of their responsible deployment. However, much of the existing research on LLM bias focuses on evaluating the internal mechanics and probabilities of models, rather than exploring how users perceive and interact with generated text in real-world scenarios. The core research problem addressed in this thesis is the misalignment between true model capabilities and human perception, often termed the calibration gap. Prior research establishes that state-of-the-art LLMs exhibit persistent social identity and cognitive biases despite rigorous value alignment [1, 2, 3]. Furthermore, users often systematically overestimate LLM accuracy when presented with confident or detailed explanations [4]. Parallel work demonstrates that superficial presentation cues, such as attribution or subtle framing, can systematically shift how information is evaluated [5]. Building upon this theoretical foundation, this thesis investigates how observable bias cues embedded in LLM responses, specifically operationalized as subtle positive or negative framing relative to a neutral baseline, influence readers’ perceived trustworthiness and related judgments. To examine this, we conducted an online survey experiment (N = 42) wherein participants evaluated five realistic AI assistant scenarios. For each scenario, participants were randomized to read one of three response variants and subsequently rated the output on four multidimensional Likert items: trustworthiness, objectivity, decision comfort, and appropriateness of tone. Our quantitative and qualitative analysis reveals that framing cues primarily affected perceptions of objectivity and tone, with both positively and negatively framed variants being rated as less objective and less appropriate than their neutral counterparts. Interestingly, the direct effects on overall factual trustworthiness were highlyscenario-dependent, yielding statistically significant differences in only two out of the five scenarios. Additionally, we found that a user’s baseline trust in AI correlated strongly and positively with their trustworthiness ratings across all scenarios(r = 0.51, p = 0.001). Taken together, the findings suggest that users often decouple factual trustworthiness from perceived appropriateness. Even when an LLM employs promotional or restrictive language, users may still trust the underlying information, particularly if their baseline confidence in AI is high. These results emphasize that trust is a complex, multidimensional construct influenced as much by stable individual predispositions as by the generated text itself. The study contributes to the ongoing discourse on LLM trust calibration by providing empirical, user-centered evidence on cue sensitivity, highlighting the necessity of designing systems that prioritize neutral presentation to mitigate the insidious effects of framing biases.

Place, publisher, year, edition, pages
2026. , p. 17
Keywords [en]
large language models, bias cues, trust, calibration, survey experiment, cog- nitive bias, framing
National Category
Other Engineering and Technologies
Identifiers
URN: urn:nbn:se:lnu:diva-149359OAI: oai:DiVA.org:lnu-149359DiVA, id: diva2:2097167
Subject / course
Computer Science
Educational program
Software Technology Programme, 180 credits
Presentation
2026-08-10, D3373V, Vejdes plats 7, Växjö, 09:00 (English)
Supervisors
Examiners
Available from: 2026-09-01 Created: 2026-08-31 Last updated: 2026-09-01Bibliographically approved

Open Access in DiVA

fulltext(737 kB)8 downloads
File information
File name FULLTEXT01.pdfFile size 737 kBChecksum SHA-512
510085f72add897666bf726f0b5bb6c18de676f0d84edfe8ee09ba75d7aa8c298fd9b67355af3a5000760ee8ab18fa24a862d916662a26fbd347eee08e6c3a73
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Brückner, JoonaNazeri, Mohammad
By organisation
Department of computer science and media technology
Other Engineering and Technologies

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 38 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf