Purpose

With more and more digital collections of various information resources becoming available, also increasing is the challenge of assigning subject index terms and classes from quality knowledge organization systems. While the ultimate purpose is to understand the value of automatically produced Dewey Decimal Classification (DDC) classes for Swedish digital collections, the paper aims to evaluate the performance of six machine learning algorithms as well as a string-matching algorithm based on characteristics of DDC.

Design/methodology/approach

State-of-the-art machine learning algorithms require at least 1,000 training examples per class. The complete data set at the time of research involved 143,838 records which had to be reduced to top three hierarchical levels of DDC in order to provide sufficient training data (totaling 802 classes in the training and testing sample, out of 14,413 classes at all levels).

Findings

Evaluation shows that Support Vector Machine with linear kernel outperforms other machine learning algorithms as well as the string-matching algorithm on average; the string-matching algorithm outperforms machine learning for specific classes when characteristics of DDC are most suitable for the task. Word embeddings combined with different types of neural networks (simple linear network, standard neural network, 1D convolutional neural network, and recurrent neural network) produced worse results than Support Vector Machine, but reach close results, with the benefit of a smaller representation size. Impact of features in machine learning shows that using keywords or combining titles and keywords gives better results than using only titles as input. Stemming only marginally improves the results. Removed stop-words reduced accuracy in most cases, while removing less frequent words increased it marginally. The greatest impact is produced by the number of training examples: 81.90% accuracy on the training set is achieved when at least 1,000 records per class are available in the training set, and 66.13% when too few records (often less than 100 per class) on which to train are available—and these hold only for top 3 hierarchical levels (803 instead of 14,413 classes).

Research limitations

Having to reduce the number of hierarchical levels to top three levels of DDC because of the lack of training data for all classes, skews the results so that they work in experimental conditions but barely for end users in operational retrieval systems.

Practical implications

In conclusion, for operative information retrieval systems applying purely automatic DDC does not work, either using machine learning (because of the lack of training data for the large number of DDC classes) or using string-matching algorithm (because DDC characteristics perform well for automatic classification only in a small number of classes). Over time, more training examples may become available, and DDC may be enriched with synonyms in order to enhance accuracy of automatic classification which may also benefit information retrieval performance based on DDC. In order for quality information services to reach the objective of highest possible precision and recall, automatic classification should never be implemented on its own; instead, machine-aided indexing that combines the efficiency of automatic suggestions with quality of human decisions at the final stage should be the way for the future.

Originality/value

The study explored machine learning on a large classification system of over 14,000 classes which is used in operational information retrieval systems. Due to lack of sufficient training data across the entire set of classes, an approach complementing machine learning, that of string matching, was applied. This combination should be explored further since it provides the potential for real-life applications with large target classification systems.

Elicited physiological affect in humans collaborating with their robot partners was investigated to determine its influence on decision-making performance in serious games. A turn-taking version of the Tower of Hanoi game was used, where physiological arousal and valence underlying such human-robot proximate collaboration were investigated. A comparable decision performance in the serious game was found between human and non-humanoid robot arm collaborator conditions, while higher physiological affect was found in humans collaborating with such robot collaborators. It is suggested that serious games which are carefully designed to take into consideration the elicited physiological arousal might witness a better decision-making performance and more positive valence using non-humanoid robot partners instead of human ones.

The presentation builds on the NKOS 2018 presentation of automatically produced Dewey Decimal Classification (DDC) classes for Swedish union catalogue (LIBRIS). Based on a dataset of 143,838 records, Support Vector Machine with linear kernel outperforms Multinomial Naïve Bayes algorithm. Impact of features shows that using keywords or combining titles and keywords gives better results than using only titles as input. Stemming only marginally improves the results. Removed stop-words reduced accuracy in most cases, while removing less frequent words increased it marginally. Word embeddings combined with different types of neural networks (Simple linear network, Standard neural network, 1D convolutional neural network, Recurrent neural network) produced worse results than Naïve Bayes /Support Vector Machine, but reach close results. The greatest impact is produced by the number of training examples: 81.37% accuracy on the training set is achieved when at least 1,000 records per class are available, and 66.13% when few records on which to train are available.

The advent of commodity 3D sensor technol- ogy has, amongst other things, enabled the efficient and effective assessment of human movements. Machine learning approaches do not rely manual definitions of gold standards for each new movement. However, to train models for the automated assessments of a new movement they still need a lot of data that map recorded movements to expert judg- ments. As camera technology changes, this training needs to be repeated if a new camera does not agree with the old one. The present paper presents an inexpensive method to check the agreement of cameras, which, in turn, would allow for a safe reuse of trained models regardless of the cameras. We apply the method to the Kinect, Astra Mini, and Real Sense cameras. The results show that these cameras do not agree and that the models cannot be reused without an unacceptable decay in accuracy. However, the suggested method works independent of movements and cameras and could potentially save effort when integrating new cameras in an existing assessment environment.

The advent of Massive Open Online Courses (MOOCs) and Small Private Online Courses (SPOCs) has brought opportunities to higher education institutions. Despite this, one of the main drawbacks of MOOCs and SPOCs has been relatively low retention rate of the registered students. Having this in mind in this paper we report our research efforts with a SPOC on Applied Machine Learning specifically tailored for professional students. More concretely, we report our findings with regard to the effects of the flipped classroom approach on the students' perceptions and attitudes. The initial results show that flipping the class had direct effects on students' knowledge and skills compared to a fully online class setting. These findings have offered complementary explanations of the survey regression analysis which revealed that course structure/instructional approach followed by course content are the main drivers in accounting for the variance in students' overall perceptions of the course.

The advent of commodity 3D sensor technology enabled, amongst other things, the efficient and effective assessment of human movements. Statistical and machine learning approaches map recorded movement instances to expert scores to train models for the automated assessment of new movements. However, there are many variations in selecting the approaches and setting the parameters for achieving high performance, i.e., high accuracy and low response time. The present paper researches the design space and the impact of approaches of statistical and machine learning on accuracy and response time in human movement assessment. Results show that a random forest regression approach outperforms linear regression, support vector regression and neuronal network approaches. Since the results do not rely on the movement specifics, they can help improving the performance of automated human movement assessment, in general.

The advent of commodity 3D sensor technology enabled, amongst other things, the efficient and effective assessment of human movements. Statistical and machine learning approaches map recorded movement instances to expert scores to train models for the automated assessment of new movements. However, there are many variations in selecting the approaches and setting the parameters for achieving good performance, i.e., high scoring accuracy and low response time. The present paper researches the design space and the impact of sequence alignment on accuracy and response time. More specifically, we introduce variants of Dynamic Time Warping (DTW) for aligning the phases of slow and fast movement instances and assess their effect on the scoring accuracy and response time. Results show that an automated stripping of leading and trailing frames not belonging to the movement (using one DTW variant) followed by an alignment of selected frames in the movements (based on another DTW variant) outperforms the original DTW and other suggested variants thereof. Since these results are independent of the selected learning approach and do not rely on the movement specifics, the results can help improving the performance of automated human movement assessment, in general.

With more and more digital collections of various information re- sources becoming available, also increasing is the challenge of assigning subject index terms and classes from quality knowledge organization systems. While the ultimate purpose is to understand the value of automatically produced Dewey Decimal Classification (DDC) classes for Swedish digital collections, the paper aims to evaluate the performance of two machine learning algorithms for Swe- dish catalogue records from the Swedish union catalogue (LIBRIS). The algo- rithms are tested on the top three hierarchical levels of the DDC. Based on a data set of 143,838 records, evaluation shows that Support Vector Machine with linear kernel outperforms Multinomial Naïve Bayes algorithm. Also, using keywords or combining titles and keywords gives better results than using only titles as input. The class imbalance where many DDC classes only have few records greatly affects classification performance: 81.37% accuracy on the training set is achieved when at least 1,000 records per class are available, and 66.13% when few records on which to train are available. Proposed future research involves an exploration of the intellectual effort put into creating the DDC to further improve the algorithm performance as commonly applied in string matching, and to test the best approach on new digital collections that do not have DDC assigned.

With the advancement in robot technology, smart human-robot interaction is of increasing importance for allowing the more excellent use of robots integrated into human environments and activities. If a robot can identify emotions and intentions of a human interacting with it, interactions with humans can potentially become more natural and effective. However, mechanisms of perception and empathy used by humans to achieve this understanding may not be suitable or adequate for use within robots. Electroencephalography (EEG) can be used for recording signals revealing emotions and motivations from a human brain. This study aimed to evaluate different machine learning techniques to classify EEG data associated with specific affective/emotional states. For experimental purposes, we used visual (IAPS) and physical (Tower of Hanoi) tasks to record human emotional states in the form of EEG data. The obtained EEG data processed, formatted and evaluated using various machine learning techniques to find out which method can most accurately classify EEG data according to associated affective/emotional states. The experiment confirms the choice of a method for improving the accuracy of results. According to the results, Support Vector Machine was the first, and Regression Tree was the second best method for classifying EEG data associated with specific affective/emotional states with accuracies up to 70.00% and 60.00%, respectively. In both tasks, SVM was better in performance than RT.