lnu.sePublikasjoner
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Evaluating trade-offs on different KV cache quantization levels and RAM constraints on edge AI
Linnéuniversitetet, Fakulteten för teknik (FTK), Institutionen för datavetenskap och medieteknik (DM).
Linnéuniversitetet, Fakulteten för teknik (FTK), Institutionen för datavetenskap och medieteknik (DM).
2026 (engelsk)Independent thesis Basic level (degree of Bachelor), 10 poäng / 15 hpOppgaveAlternativ tittel
Utvärdering av avvägningar mellan olika nivåer av KV-cachekvantisering och RAM-begränsningar inom Edge AI (svensk)
Abstract [en]

Artificial Intelligence (AI) is rising in popularity day by day. More people are looking for a more local alternative instead of the traditional cloud-based AI models. A lot of these cloud-based AI services are not hosted locally; more often than not, they are hosted in a completely different country. This thesis looked at one of the best locally hosted alternatives, this being edge AI models. Edge AI runs locally on all sorts of Internet of Things (IoT) but with the recent rise in hardware components pricing, could outdated devices be capable of running these models? This thesis investigates how different KV-cache formats interact with RAM constraints in a CPU-based local inference environment. Controlled experiments were conducted in which different quantization levels were tested with different RAM constraints. The results show that the effect of KV-cache format on latency depended on the available RAM. They also show that the calculated storage requirements of the cache are not directly reflected in total runtime memory behaviour. There was no difference in answer correctness with all runs getting 11/12 answers correct while failing the same question. These results indicate that the performance effects of KV-cache quantization depend on the available RAM and runtime environment.

sted, utgiver, år, opplag, sider
2026.
Emneord [en]
Edge-AI, KV cache, AI (Artificial Intelligence), Quantization, RAM (Random Access Memory)
HSV kategori
Identifikatorer
URN: urn:nbn:se:lnu:diva-149770OAI: oai:DiVA.org:lnu-149770DiVA, id: diva2:2105278
Utdanningsprogram
Software Technology Programme, 180 credits
Presentation
2026-08-10, D2273V, Vejdes plats 6, 352 52, Växjö, 09:00 (engelsk)
Veileder
Examiner
Tilgjengelig fra: 2026-09-30 Laget: 2026-09-25 Sist oppdatert: 2026-09-30bibliografisk kontrollert

Open Access i DiVA

fulltext(902 kB)85 nedlastinger
Filinformasjon
Fil FULLTEXT01.pdfFilstørrelse 902 kBChecksum SHA-512
a8941b51520251f0a7ae90fa1b51cf3428bff854a54c28fa1a5aec5cc6d5f583431e84a196e6dc4a927784595b416a400badd0967a589deb176a641751f7923e
Type fulltextMimetype application/pdf

Søk i DiVA

Av forfatter/redaktør
Johansson, GustavKarlsson, Alfred
Av organisasjonen

Søk utenfor DiVA

GoogleGoogle Scholar
Totalt: 85 nedlastinger
Antall nedlastinger er summen av alle nedlastinger av alle fulltekster. Det kan for eksempel være tidligere versjoner som er ikke lenger tilgjengelige

urn-nbn

Altmetric

urn-nbn
Totalt: 7777 treff
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf