Extreme weather events, such as heat waves, are occurring more frequently and intensively, imposing new climate-adaptation demands on municipal planning. We conducted a design study across the domains of urban planning and urban climate research, and identified challenges regarding a lack of heat-related information in current planning processes, and the high complexity of effective climate data representation. To address these challenges, and so enhance the information flow between these domains, we developed Urban Climate InteracTable, an immersive interface that supports exploratory analysis of spatio-temporal climate simulation data integrated with an urban environment representation. We describe several use cases in which this interface can be utilized to assist with planning-related decision processes and to communicate heat-related phenomena. We present the feedback obtained from our collaborating domain experts and relevant external experts, and reflect on our experiences throughout the design study. From this, we offer insights for future research.

The design of intuitive three-dimensional user interfaces is vital for interaction in virtual reality, allowing to effectively close the loop between a human user and the virtual environment. The utilization of 3D gestural input allows for useful hand interaction with virtual content by directly grasping visible objects, or through invisible gestural commands that are associated with corresponding features in the immersive 3D space. The design of such interfaces remains complex and challenging. In this article, we present a design approach for a three-dimensional user interface using 3D gestural input with the aim to facilitate user interaction within the context of Immersive Analytics. Based on a scenario of exploring time-oriented data in immersive virtual reality using 3D Radar Charts, we implemented a rich set of features that is closely aligned with relevant 3D interaction techniques, data analysis tasks, and aspects of hand posture comfort. We conducted an empirical evaluation (n=12), featuring a series of representative tasks to evaluate the developed user interface design prototype. The results, based on questionnaires, observations, and interviews, indicate good usability and an engaging user experience. We are able to reflect on the implemented hand-based grasping and gestural command techniques, identifying aspects for improvement in regard to hand detection and precision as well as emphasizing a prototype’s ability to infer user intent for better prevention of unintentional gestures.

Collaboration is an essential part of data analysis, allowing multiple users to combine their expertise and to debate about the interpretation of data discoveries using their contextual knowledge. The design of collaborative interfaces within the context of Immersive Analytics remains challenging, particularly due to the various user-centered characteristics of immersive technologies. In this article, we present the use case of a system that enables multiple users to synchronously explore the same data in a collaborative scenario that combines immersive and non-immersive interfaces in an asymmetric role setup. Such a setup allows for bridging the gap when applying heterogeneous display and interaction technologies, enabling each analyst to have an independent and different view of the data, while maintaining important collaborative aspects during the joint data exploration. We developed an immersive VR environment (head- mounted display, 3D gestural input) and a non-immersive desktop terminal (monitor, keyboard and mouse) centered around spatio-temporal data exploration. Supported through a real-time communication interface, synchronous collaborative features are integrated in both interfaces, facilitating the users in their ability to establish a shared context and to make spatio-temporal references. We conducted an empirical evaluation with five participant pairs (within-subject design) to investigate aspects of usability, user engagement, and collaboration during a confirmative analysis task. Synthesis of questionnaire results in combination with additional log file analysis, audio activity analysis, and observations, revealed good usability scores, high user engagement, as well as overall close and balanced collaboration of enthusiastic pairs during the task completion independent of their interface type, validating our system approach in general. Further supported through the self-constructed Spatio-Temporal Collaboration Questionnaire, we are able to contribute with discussion and considerations of the presented scenario and the synchronous collaborative features for the design of similar applications.

Immersive display and interaction technologies have rapidly evolved in recent years, offering advanced techniques compared to traditional Human-Computer Interaction. Computer-generated Virtual Environments viewed with stereoscopic depth perception and explored using 3D spatial interaction can represent more accurately how humans naturally interact in the real world. Data analysis is a promising area of application for such technologies, holding potential to promote intuitive interaction, user engagement, collaboration, and data curiosity, as well as to foster appropriate contextual visualization. Even when techniques such as Machine Learning and Data Mining assist with the analysis of data, human interpretation, contextualization, and meaning making are still needed. The design of immersive data visualization and interaction is challenging due to the complexity of the involved technologies and human factors, which calls for an interdisciplinary research effort.

The focus of this thesis is to investigate means of exploration, interaction, and collaboration using Virtual Reality and 3D gestural input in immersive environments within the context of spatio-temporal data analysis. Based on existing literature as well as following an applied and interdisciplinary research approach, a design space for this type of Immersive Analytics is defined. The emphasis on spatio-temporal data is relevant across various real-world contexts and scenarios, such as sociolinguistics and climate analysis, given that data collected nowadays commonly feature descriptors of where and when they were captured. An immersive data analysis system has been implemented and evaluated across three virtual environment iterations. Two core themes from a user-centered perspective are interaction and collaboration. The design of useful and engaging 3D gestural interaction techniques support the conduction of typical analytical tasks that aid the data exploration and thus the discovery of insights. Furthermore, data analysis is seldom a solitary activity, but can be conducted in collaboration with multiple analysts, who combine their knowledge to interpret and discuss the discoveries. For this purpose, the concept of Hybrid Asymmetric Collaboration is defined, aiming to facilitate an envisioned broader analytical workflow that assumes a mixture of immersive and non-immersive interfaces (hybrid) as well as distinct user roles (asymmetric). To bridge data analysis across heterogeneous interface types, the design of visual information cues is investigated to support foundational aspects of collaboration, such as awareness, common ground, reference, and deixis.

The conducted research has been empirically evaluated using a combination of standardized and custom methods in a total of six main studies. The outcomes of these studies allow for reflections and the proposal of design guidelines for collaborative data interaction in immersive spaces.

De senaste åren har så kallade immersiva skärm- och interaktionsteknologier utvecklas i snabb takt. Sådana teknologier erbjuder mer avancerade tekniska lösningar jämfört med mer traditionell människa-datorinteraktion. Immersiva inslag som datorgenererade virtuella miljöer med stereoskopisk djupuppfattning och 3D-interaktion kan på ett mer exakt och naturligt sätt representera de interaktioner vi utför i den verkliga världen.

Dataanalys är ett lovande användningsområde för immersiva teknologier, med potential att underlätta intuitiv interaktion, skapa och uppmuntra engagemang, samarbete och nyfikenhet för data, samt främja relevant kontextuell visualisering. Även när teknologier som maskininlärning och datautvinning används är det fortfarande nödvändigt att komplettera resultatet med en överordnad mänsklig nivå av tolkning, kontextualisering och meningsskapande för att göra analyserna kompletta och användbara. Designen av immersiv visualisering och interaktion är en krävande utmaning på grund av de teknologiska och mänskliga aspekternas komplexitet och fordrar därför en tvärvetenskaplig forskningsmetod.

Avhandlingens syfte är att utforska metoder som kan stödja undersökning, interaktion och samarbete med hjälp av Virtual Reality och 3D-handinteraktioner i immersiva miljöer inom ramen för analys av datamängder som innehåller rumsliga och tidsmässiga data. Denna typ av Immersive Analytics baseras på teoretiska utgångspunkter från tidigare forskning, samt en tillämpad, tvärvetenskaplig forskningsmetod. Fokuset på datamängder som innehåller rumsliga och tidsmässiga data är relevant för flera olika ämnesområden och sammanhang, som till exempel sociolingvistik och klimatanalys, eftersom data som samlas in i nu för tiden ofta innehåller beskrivningar av var och när de mättes eller observerades. Ett immersivt datanalys-system har för detta arbete implementerats och utvärderats i tre iterationer. Två grundläggande områden för ett användarcentrerat perspektiv är interaktion och samarbete. Designen av användbara och engagerande interaktionstekniker via 3D-handinteraktioner stödjer utförandet av typiska analytiska uppgifter och underlättar undersökningen av data och därmed upptäckten av nya insikter. Dataanalys är sällan en aktivitet som utförs ensam, utan sker snarare i ett samarbete där flera analytiker med sin kombinerade kunskap tolkar och diskuterar upptäckter tillsammans. För detta ändamål har begreppet Hybrid Asymmetric Collaboration definierats, vilket syftar till att beskriva ett betydligt bredare analytiskt arbetsflöde som förutsätter och omfattar en blandning av nya immersiva och nuvarande icke-immersiva gränssnitt (hybrid) såväl som distinkta användarroller (asymmetric). För att kombinera och integrera dataanalys mellan heterogena typer av användargränssnitt utforskas designen av visuella informationssignaler med syfte att stödja grundläggande samarbetsaspekter såsom medvetenhet, gemensam grund, referens och kontext.

Den genomförda forskningen har utvärderats empiriskt med en kombination av standardiserade och anpassade metoder i totalt sex större studier. Resultatet av dessa studier möjliggör reflektioner kring samt förslag på olika designriktlinjer för interaktion och samarbete i immersiva miljöer.

In this paper, we present an approach to interact with time-oriented data in Virtual Reality within the context of Immersive Analytics. We implemented a Virtual Reality application that enables its user to explore data in an immersive environment (head-mounted display, 3D gestural input), utilizing potential advantages of immersive technologies, for instance, depth cues for better spatial understanding, natural interaction, and user engagement. The visualization design is inspired by the overall concept of a radar chart, and using the third dimension to represent time-series related data. We conducted a user study with 15 participants, encouraging them to examine a representative dataset within an explorative analysis scenario with no time constraints. Based on the results of usability and user engagement scores, task completion analysis, observations, and interviews, we were able to empirically validate the approach in general, and gain insights in the users’ interaction and data analysis strategies.

We present a hybrid Immersive Analytics system to support asymmetrical collaboration between a pair of users during synchronous data exploration. The system consists of an immersive Virtual Reality application, a non-immersive web application, and a real-time communication interface connecting both applications to provide features to facilitate the collaborators’ mutual understanding and their ability to make (spatial) references. We conducted a real world case study with pairs of language students, encouraging them to use the developed system to investigate a large multivariate Twitter dataset from a sociolinguistic perspective within an explorative analysis scenario. Based on the results of usability scores, log file analyses, observations, and interviews, we were able to validate the approach in general, and gain insights into the users’ collaboration with respect to awareness, deixis, and group dynamics.

In this article, we compare three different input technologies (gamepad, vision-based motion controls, room-scale) for an interactive virtual reality (VR) environment. The overall system is able to visualize (open) data from multiple online sources in a unified interface, enabling the user to browse and explore displayed information in an immersive VR setting. We conducted a user interaction study (n=24; n=8 per input technology, between-group design) to investigate experienced workload and perceived flow of interaction. Log files and observations allowed further insights and comparison of each condition. We have identified trends that indicate user preference of a visual (virtual) representation, but no clear trends regarding the application of physical controllers (over vision-based controls), in a scenario that encouraged exploration with no time limitations.

In recent years, large textual data sets, comprising many data points and rich metadata, have become a common object of investigation and analysis. Information Visualization and Visual Analytics provide practical tools for visual data analysis, most commonly as interactive two-dimensional (2D) visualizations that are displayed through normal computer monitors. At the same time, display technologies have evolved rapidly over the past decade. In particular, emerging technologies such as virtual reality (VR), augmented reality (AR), or mixed reality (MR) have become affordable and more user-friendly (LaValle 2016). Under the banner of “Immersive Analytics”, researchers started to explore the novel application of such immersive technologies for the purpose of data analysis (Marriott et al. 2018).

By using immersive technologies, researchers hope to increase motivation and user engagement for the overall data analysis activity as well as providing different perspectives on the data. This can be particularly helpful in the case of exploratory data analysis, when the researcher attempts to identify interesting points or anomalies in the data without prior knowledge of what exactly they are searching for. Furthermore, the data analysis process often involves the collaborative sharing of information and knowledge between multiple users for the goal of interpreting and making sense of the explored data together (Isenberg et al. 2011). However, immersive technologies such as VR are often rather single user-centric experiences, where one user is wearing a head-mounted display (HMD) device and is thus visually isolated from the real-world surroundings. Consequently, new tools and approaches for co-located, synchronous collaboration in such immersive data analysis scenarios are needed.

In this software demonstration, we present our developed VR system that enables two users to explore data at the same time, one inside an immersive VR environment, and one outside VR using a non-immersive companion application. The context of this demonstrated data analysis activity is centered around the exploration of the language variability in tweets from the perspectives of multilingualism and sociolinguistics (see, e.g. Coats 2017 and Grieve et al. 2017). Our primary data come from the the Nordic Tweet Stream (NTS) corpus (Laitinen et al. 2018, Tyrkkö 2018), and the immersive VR application visualizes in three dimensions (3D) the clustered Twitter traffic within the Nordic region as stacked cuboids according to their geospatial position, where each stack represents a color-coded language share (Alissandrakis et al. 2018). Through the utilization of 3D gestural input, the VR user can interact with the data using hand postures and gestures in order to move through the virtual 3D space, select clusters and display more detailed information, and to navigate through time (Reski and Alissandrakis 2019) ( https://vrxar.lnu.se/apps/odxvrxnts-360/ ). A non-immersive companion application, running in a normal web browser, presents an overview map of the Nordic region as well as other supplemental information about the data that are more suitable to be displayed using non-immersive technologies.

We will present two complementary applications, each with a different objective within the collaborative data analysis framework. The design and implementation of certain connectivity and collaboration features within these applications facilitate the co-located, synchronous exploration and sensemaking. For instance, the VR user’s position and orientation are displayed and updated in real-time within the overview map of the non-immersive application. The other way around, the selected cluster of the non-immersive user is also highlighted for the user in VR. Initial tests with pairs of language students validated the proof-of-concept of the developed collaborative system and encourage the conduction of further future investigations in this direction.

We demonstrate an approach that utilizes immersive virtual reality (VR) to explore and interact with corpus linguistics data. Our case study focuses on the language identification parameter in the Nordic Tweet Stream corpus, a dynamic corpus of Twitter data where each tweet originated within the Nordic countries. We demonstrate how VR can provide previously unexplored perspectives into the use of English and other non-indigenous languages in the Nordic countries alongside the native languages of the region and showcase its geospatial variation. We utilize a head-mounted display (HMD) for a room-scale VR scenario that allows 3D interaction by using hand gestures. In addition to spatial movement through the Nordic areas, the interface enables exploration of the Twitter data based on time (days, weeks, months, or time of predefined special events), making it particularly useful for diachronic investigations.

In addition to demonstrating how the VR methods aid data visualization and exploration, we briefly discuss the pedagogical implications of using VR to showcase linguistic diversity. Our empirical results detail students’ reactions to working in this environment. The discussion part examines the benefits, prospects and limitations of using VR in visualizing corpus data.

In this paper we describe our work-in-progress to create an interface that enables users to browse and select data within an Augmented Reality environment, using a virtual cube object that can be interacted with through 3D gestural input. We present the prototype design (including the graphical elements), describe the interaction possibilities of touching the cube with the hand/finger, and put the prototype into the context of our Augmented Reality for Public Engagement (PEAR) framework. An interactive prototype was implemented and runs on a typical off-the-shelf smart-phone device.