In this pilot study I used Whisper, the LLM Gemini by Google and the thesaurus developed by SAMLA and ISOF to automatically transcribe and index 14 interviews in Swedish mixed with Sámi from ISOF’s Sámi collections. The results were very accurate, although language mixing caused some difficulties. This raises both intriguing possibilities and concerns for digitized oral history collections.

In the archival media trinity of text, image and sound, the latter presents particular challenges for users’ searching and browsing. While a user can manually or digitally browse through texts and images to locate items of interest, sound needs to be played more or less in real time to do the same. Audio files can naturally be described and transcribed – digital or digitized audio files of speech automatically so – thus facilitating full-text search. However, the familiar Achilles’ heel of full-text search, namely the ambiguity of natural language, remains.

Enter the hallmark trade of librarianship: subject indexing. When subject index terms have been assigned to individual sections of an audio file, a user searching for these or similar terms can locate precisely where in the retrieved audio files the subject is discussed. With state-of-the-art AI systems, even this can be done automatically, thereby decimating the amount of time needed to index audio files, from real time to as fast as the system can process them. This heralds a brave new future for the accessibility and searchability of oral history archives.

This poster presents a pilot study on automatically transcribing interviews in Swedish from oral history archives using OpenAI’s Whisper, describing the content and assigning subject index terms to sections using Claude from Anthropic, and visualizing the results. The accuracy of the results depends on many factors, including sound quality, accents of the speakers, the amount of language mixing etc. The results are very promising, however, suggesting automatic subject indexing of interviews to be a worthwhile research direction going forward.

1 Aims

This pilot project investigates the possibility of using the free-to-use large language

model (LLM) Gemini 2.5 Pro by Google with a thesaurus jointy developed

by the Norwegian SAMLA project and the Swedish Institute for Language and

Folklore [1] for automatic subject indexing of oral history interviews. Considering

the difficulty of finding sections of interest in interviews from oral history

databases, subject indexing for audio files has been implemented in many institutions

[3]. These indexes are very time-consuming to construct manually,

suggesting a need for automatic or semi-automatic approaches that implement

controlled vocabularies to facilitate information retrieval. This study gauges the

feasibility of a fully automated workflow using automatic transcription, thematic

segmentation and subject indexing using modern AI tools.

2 Background

It is notoriously difficult to find segments of interest in an un-indexed oral history

recording, given that one needs to listen through the whole interview in moreor-

less real time to determine its content. Transcripts are naturally immensely

helpful in this regard, while time-consuming to create [3]. To overcome the issues

presented by searching for topics of interest in natural language documents,

transcripts can be combined with subject indices that make use of controlled

vocabulary terms [2].

While modern speech-to-text tools have been used to generate transcripts for

oral history interviews, the opportunity of using LLMs to generate the subject

indices has not to the author’s knowledge been leveraged to date.

3 Methods

A Google Colab notebook was designed to process a collection of 14 digitized

recordings in Swedish mixed with South Sámi from the Swedish Institute of

Language and Folklore. The notebook transcribes the interview using a version

of OpenAI’s Whisper finetuned for Swedish (KB-Whisper), whereafter the

transcript is passed to Google’s free-to-use Gemini API with a prompt with

instructions for subject indexing. The LLM is asked to perform the following

tasks:

1. Split the interview into thematic segments

2. Specify the start time of each segment

3. Mark segments which are unclear or seem to be in a language other than

Swedish

4. Give each segment a brief title

5. Summarize each segment

6. Describe each segment with free keywords

7. Subject index each segment using terms from the controlled vocabulary

The output is formatted as a csv-file, which can be opened in e.g. Google

Sheets. Due to the lack of a gold standard, the accuracy of the LLM output

for each individual task and the assigned index terms, as well as the overall

usefulness of the pipeline, was analyzed manually by the author.

4 Main findings

For the 14 interviews processed in this pilot study Gemini 2.5 Pro very successfully

divided them into thematic segments. The start time for each segment

was off only by a few seconds at most. Out of a total of 41 interview segments

in Sámi language, 14 were correctly marked by Gemini as [unclear/other language],

whereas 27 were not. The titles given to segments where descriptive and

accurate, except for misclassified segments in Sámi. Summarizations were very

good, except where subject expertise was needed. For all the segments in the

14 interviews, only one of the free keywords assigned was probably inaccurate.

The subject terms assigned by Gemini were only analyzed for their relevance

to the topic of the interview segment; possible alternative subject terms were

not identified by the author due to time constraints. Excluding the misclassified

segments which were in fact in Sámi language, the proportion of relevant subject

terms was 97%; including the segments in Sámi the proportion was 89%.

5 Relevance to workshop themes

This contribution connects to topic 10 of the workshop: Use of KOS in Artificial

Intelligence (AI) applications - methods and practical experience.

References

[1] Hans-Jakob ˚Agotnes. “Norske folkeminnesamlingar tilgjengelege i eit felles

digitalt arkiv: SAMLA”. In: Heimen 61.3 (Sept. 30, 2024), pp. 257–261.

issn: 0017-9841, 1894-3195. doi: 10.18261/heimen.61.3.7. url: https:

//www.scup.com/doi/10.18261/heimen.61.3.7 (visited on 06/11/2025).

[2] Koraljka Golub. “Automated subject classification of textual web documents”.

In: Journal of Documentation 62.3 (May 1, 2006), pp. 350–371.

issn: 0022-0418. doi: 10.1108/00220410610666501. url: https://www.

emerald . com / insight / content / doi / 10 . 1108 / 00220410610666501 /

full/html (visited on 06/11/2025).

[3] Douglas Lambert. “Oral History Indexing”. In: The Oral History Review

50.2 (2023), pp. 169–192. doi: https://doi.org/10.1080/00940798.

2023.2235000.

 The opportunities offered by artificial intelligence (AI) and machine learning (ML) for the archive sector have been addressed in several scholarly articles in recent years. Some of the articles describe projects (e.g. Carter et al., 2022; Han et al., 2022) featuring a collaboration between archivists, generally defined for this paper as persons working in the archival sector, and technologists, defined as persons with a professional technical background. Some other archival projects have been undertaken by computer scientists or digital humanities scholars without the involvement of archivists (e.g. Luthra et al., 2022). Providing a qualitative angle, some articles have looked at digital archivists’ opinions on the implementations of AI techniques in archives (e.g. Cushing & Osti, 2023). Finally, more general articles have been published, both by archivists and technologists on how AI may be implemented in the archival sector (e.g. Colavizza et al., 2022; Hutchinson, 2020; Sabharwal, 2017). 

This paper presents a qualitative analysis of how the perspectives of archivists and technologists on AI in archives are presented in a selection of recent articles. The selection includes both articles describing case studies, such as project descriptions and interview studies, as well as literature reviews. The articles are categorized according to the type of project described or suggested: articles written from an archival perspective, articles written from a technologist perspective, and articles that describe or propose joint projects run by archivists and technologists in cooperation. Differences can be observed in these research articles regarding 1) how archival expertise is valued, 2) the proposed importance of archival theory for successful AI implementation and 3) the degree of influence ascribed to the archivists in collaborations.

The results indicate that viewpoints clearly differ depending on the professional role and background of the contributors to the articles. Articles written from a technologist perspective are more likely to criticize archivist work, and in some cases even blame them for obstructing access and “perpetuating silences” in archives (Luthra et al., 2022). Randby & Marciano (2020) describe the goals of AIC (Advanced Information Collaboratory), a project in which Randby is involved, as aiming towards information professionals learning “to think computationally and rapidly adapt new technologies”; the article goes through a computational workflow without further mentioning archivists on a larger scale. 

Articles written by archivists, on the other hand, emphasize the importance of incorporating archival principles and taking advantage of archivists’ knowledge in the AI implementation process (Hutchinson, 2020). Cushing & Osti (2023) highlight the expertise of archivists and stress how their professional background is a requisite to control AI decision-making. An important distinction is also that articles published in archivist journals such as Murphy et al.’s (2015) highlight the perspective of archivists by portraying them as subjects (“archivists”) which is contrasted with a passive portrayal of the technological aspects (“technology” instead of “technologists” or “machine learning experts”). Other articles point out the challenges of using AI in archives; for example, Jaillant & Caputo (2022) mention “ethical challenges” and problems with bias in AI. Cushing & Osti (2023) describe how participants in their study (archival experts) are confident about the possibilities of AI, but also skeptical about practical integration into their work. Lee (2018) argues that certain AI tools are not supportive of the “holistic view” that archivists have of their work.

Those articles that present a mutual collaboration between archivists and technologists highlight the importance of combining the expertise of both fields for successful AI implementation (e.g. Murphy et al., 2015; Carter et al., 2022; Han et al., 2022). Poole & Garwood (2018), an information scientist and computer and informatics researcher respectively, call for the involvement of archivists and librarians in Digital Humanities projects. Tsabedze (2023) offers an important perspective on archival professionals in Eswatini, highlighting the need for digital education; Tsabedze argues that the interview participants in Eswatini are afraid to lose their jobs without proper training. Nevertheless, Marciano et al. (2018) are positive about the disciplines achieving more together than each would have on their own. As a contrasting addition, Jo & Gebru (2020) argue not for implementing AI technologies in archives but for implementing archival expertise in AI development. They propose that archival document collection practices can inform data collection in sociocultural ML, because archivists possess the language and procedures to address issues of consent, transparency, inclusivity etc.

Finally, Marciano et al. (2018) believe that archival expertise of the future will involve knowledge of digital systems. Sabharwal (2017) suggests that in the future, the distinction between archivists and technologists will not be as clear anymore. Jaillant & Caputo (2022) also highly encourage collaboration between archivists and technologists in order to address future challenges. Cushing & Osti (2023) suggest that AI technology in archives can even change how archivists describe “digital archival expertise”. In conclusion, our findings suggest that there is a great diversity of opinions; as Poole & Garwood (2018) suggest, more research is necessary on how the collaboration between professions, as well as implementation of AI in archives, can be improved.

Facilitating access to archives via metadata creation and enrichment can be a monumental task for large archival collections. The past decade has witnessed an increasing use of artificial intelligence (AI) and machine learning (ML) to assist in these tasks in automatic or semi-automatic workflows (Colavizza et al., 2022). While technologies such as named entity recognition and topic modeling are useful in many different archival contexts, they have found special relevance for colonial archives and archives pertaining to underrepresented communities. Recent projects have for example explored the possibilities of using AI and ML to optimize information discovery in under‑utilized, Holocaust‑related records (Carter et al., 2022), extract mentions of underrepresented people in Dutch colonial records (Luthra et al., 2023), and transform Indigenous and Spanish colonial archives originating from Mexico into Linked Open Data repositories (Candela et al., 2023).

This paper presents, firstly, an overview of current state-of-the-art approaches for AI in archives, and secondly, a project in progress intended to align with the three first goals of the ongoing InterPARES Trust AI project (Duranti et al., 2021): to identify specific AI technologies that can address critical records and archives challenges; determine the benefits and risks of using AI technologies on records and archives; and ensure that archival concepts and principles inform the development of responsible AI. The opportunities offered by these technologies are contrasted with the risks of using automated approaches in general and AI in particular for improving access to archives. The paper also discusses a related approach for increasing discoverability in collections, i.e. semantic search, and compares the pros and cons of these approaches. Furthermore, the potential uses of generative pre-trained transformers (GPTs) for both indexing and retrieval, and the risks associated with these, are addressed.

Bringing the overview to a Swedish context, the paper describes an ongoing project aiming to explore the risks and possibilities of using AI and ML to provide up-to-date, enriched metadata for Swedish archives pertaining to the Sámi, an Indigenous population of the Nordic countries. Managing Indigenous heritage material demands distinct sensitivities that acknowledge the colonial past and the voice of the community in creating and maintaining the cultural record. While AI technologies can be a means for promoting cultural heritage from a Sámi perspective, using them without properly addressing colonial aspects runs the risk of reifying and perpetuating the colonial dynamics of past history writing. When properly applied, however, the hope is that these technologies may be of assistance in the remediation of the digital cultural record to counter the colonial dynamics of the analog cultural record (see Risam, 2019).

The project aims to gauge the potential of these technologies for improving the searchability and usability of records pertaining to the Sámi, of both colonial and Indigenous origin. The research is intended to follow an iterative approach, with continuous evaluation and feedback from experts and end users ensuring the suitability of the metadata generated by the technologies and its usefulness in facilitating search. The expected outcome of the project as a whole is a framework for assessing how to safely and effectively implement selected AI techniques in archival and related institutions while maintaining authenticity.