We have designed and implemented a Computer Aided Personal Interview (CAPI) system that learns from expert interviews and can support less experienced interviewers by for example suggesting questions to ask or skip. We were particularly interested to streamline the due diligence process when estimating the value for software startups. For our design we evaluated some machine learning algorithms and their trade-offs, and in a small case study we evaluates their implementation and performance. We find that while there is room for improvement, the system can learn and recommend questions. The CAPI system can in principle be applied to any domain in which long interview sessions should be shortened without sacrificing the quality of the assessment.

We investigate how demand-driven documentation is used by open source projects. Our study relies on questions and answers (i.e., demand-driven documentation) posted to Stack Overflow and focuses on two projects: Apache POI and Fennec (Firefox for Android). We find that the demand-driven documentation provided by respective community is of good quality and contributions are made within an acceptable amount of time. We conclude that demand-driven documentation can be suitable as a primary information source if other resources accompany it, e.g., websites. However, the suitability is closely related to size and activity of the project's community. We also investigate whether the type of project (e.g., API, application, product, etc.) matters, and find that given the nature of a forum, projects where the intended users are developers have larger and more active communities and are thus better suited for demand-driven documentation.

In this case study we investigate how demand-driven documentation can be used in open source projects. The study relied on questions and answers (demand-driven documenta- tion) posted on Stack Overflow and studied two open source projects, Apache POI and Firefox for Android, in depth. We found that the demand driven documentation provided by the respec- tive community is in general of good quality and contributed within an acceptable amount of time. We conclude that demand driven documentation can be suitable as a primary information source if other resources accompany it. However, the suitability is closely related to the size and activity of the project’s community. 

The current generation of undergraduate students are enthusiastic adopters of mobile technologies, and some of these technologies offers significant opportunities for broadening the times, places and manner that students study and learn. One of these is podcasting, which greatly eases distribution of content to portable devices, giving the students opportunity for anytime, anywhere learning. However, most educational podcasts do not take advantage of the potential of podcasting. In this paper we discuss the issues surrounding the use of podcasting for higher education. We report our experiences using podcasts to supplement two undergraduate Computer Science courses, and we provide tips for others considering the use of podcasts in education.

Indirect metrics in quality models define weighted integrations of direct metrics to provide higher-level quality indicators. This paper presents a case study that investigates to what degree quality models depend on statistical assumptions about the distribution of direct metrics values when these are integrated and aggregated. We vary the normalization used by the quality assessment efforts of three companies, while keeping quality models, metrics, metrics implementation and, hence, metrics values constant. We find that normalization has a considerable impact on the ranking of an artifact (such as a class). We also investigate how normalization affects the quality trend and find that normalizations have a considerable effect on quality trends. Based on these findings, we find it questionable to continue to aggregate different metrics in a quality model as we do today.

We need to aggregate metric values to make quality assessment actionable. However, due to properties of metric values, i.e., unknown distributions and different measurement scales, they are difficult to aggregate. We present and evaluate a method to aggregate metric values based on observed numerical distributions that are converted into cumulative density functions. We use these to determine the probability of each metric and file, and aggregate these. Our limited study suggests that the method improves correctness, communication, and the ability to take action. However, more evaluation is required.

We mine job ads to discover what skills are required from an employers' perspective. Some obvious trends appear, such as skills related to web and mobile technology. We aim to uncover more detailed information as the study continues to allow course content to better match the expressed needs.

The term technical debt is easy to understand as a metaphor, but can quickly grow complex in practice. We contribute with a dataset, TDMentions, that enables researchers to study how developers and end users use the term technical debt in online posts and discussions. The dataset consists of posts from news aggregators and Q&A-sites, blog posts, and issues and commits on GitHub.

Information quality assessment of technical documentation is nowadays an integral part of quality management of products and services. These are usually assessed using questionnaires, checklists, and reviews and consequently work that is cumbersome, costly and prone to errors. Acknowledging the fact that only humans can assess certain quality aspects, we suggest complementing these with automatic quality assessment using a software infrastructure that (i) reads information from documentations, (ii) performs analyses on this information, and (iii) visualizes the results to help stakeholders understand quality issues. We introduce the software infrastructure’s architecture and implementation, its adaptation to different formats of documentations and types of analyses, along with a number of real world cases exemplifying feasibility and benefit of our approach. Altogether, our approach contributes to more efficient and automatic information quality assessments. 

Information quality assessment of technical documentation is an integral part of quality management of products and services. Technical documentation is usually assessed using questionnaires, checklists, and reviews. This is cumbersome, costly and prone to errors. Acknowledging the fact that only people can assess certain quality aspects, we suggest complementing these with software-supported automatic quality assessment. The many different encodings and representations of documentation, e.g., various XML dialects and XML Schemas/DTDs, is one problem. We present a system, a software infrastructure, where abstraction and meta modelling are used to define reusable analyses and visualisations that are independent of specific encodings and representations. We show how this system is implemented and how it: 1) reads information from documentations; 2) performs analyses on this information; 3) visualises the results to help stakeholders understand quality issues. We introduce the system, the architecture and implementation, its adaptation to different formats of documentations and types of analyses, along with a number of real world cases exemplifying the feasibility and benefits of our approach. Altogether, our approach contributes to more efficient information quality assessments.

An initial study of how text clones can be detected and visualized in technical documentation, i.e., semi-structured text that describe a product, software, or service. The goal of the visualizations is to support human experts to assess and prioritize the clones, since certain clones can be either intentional or harmless. We study some existing visualizations designed for source code, and provide initial and limited adaption of these. A major difficulty in this adaptation is to manage the semi-structured technical documentation compared to structured source code. 

The productivity of a (team of) developer(s) can be expressed as a ratio between effort and delivered functionality. Several different estimation models have been proposed. These are based on statistical analysis of real development projects; their accuracy depends on the number and the precision of data points. We propose a data-driven method to automate the generation of precise data points. Functionality is proportional to the code size and Lines of Code (LoC) is a fundamental metric of code size. However, code size and LoC are not well defined as they could include or exclude lines that do not affect the delivered functionality. We present a new approach to measure the density of code in software repositories. We demonstrate how the accuracy of development time spent in relation to delivered code can be improved when basing it on net-instead of the gross-size measurements. We validated our tool by studying ca. 1,650 open-source software projects.

The challenge of having to deal with dependent variables in classification and regression using techniques based on Bayes' theorem is often avoided by assuming a strong independence between them, hence such techniques are said to be naive. While analytical solutions supporting classification on arbitrary amounts of discrete and continuous random variables exist, practical solutions are scarce. We are evaluating a few Bayesian models empirically and consider their computational complexity. To overcome the often assumed independence, those models attempt to resolve the dependencies using empirical joint conditional probabilities and joint conditional probability densities. These are obtained by posterior probabilities of the dependent variable after segmenting the dataset for each random variable's value. We demonstrate the advantages of these models, such as their nature being deterministic (no randomization or weights required), that no training is required, that each random variable may have any kind of probability distribution, how robustness is upheld without having to impute missing data, and that online learning is effortlessly possible. We compare such Bayesian models against well-established classifiers and regression models, using some well-known datasets. We conclude that our evaluated models can outperform other models in certain settings, using classification. The regression models deliver respectable performance, without leading the field.

Software quality models aggregate metrics to indicate quality. Most metrics reflect counts derived from events or attributes that cannot directly be associated with quality. Worse, what constitutes a desirable value for a metric may vary across contexts. We demonstrate an approach to transforming arbitrary metrics into absolute quality scores by leveraging metrics captured from similar contexts. In contrast to metrics, scores represent freestanding quality properties that are also comparable. We provide a web-based tool for obtaining contextualized scores for metrics as obtained from one’s software. Our results indicate that significant differences among various metrics and contexts exist. The suggested approach works with arbitrary contexts. Given sufficient contextual information, it allows for answering the question of whether a metric value is good/bad or common/extreme.

Commit classification, the automatic classification of the purpose of changes to software, can support the understanding and quality improvement of software and its development process. We introduce code density of a commit, a measure of the net size of a commit, as a novel feature and study how well it is suited to determine the purpose of a change. We also compare the accuracy of code-density-based classifications with existing size-based classifications. By applying standard classification models, we demonstrate the significance of code density for the accuracy of commit classification. We achieve up to 89% accuracy and a Kappa of 0.82 for the cross-project commit classification where the model is trained on one project and applied to other projects. Such highly accurate classification of the purpose of software changes helps to improve the confidence in software (process) quality analyses exploiting this classification information.

Metrics As Scores can be thought of as an interactive, multiple analysis of variance (abbr. "ANOVA," Chambers et al., 2017). An ANOVA might be used to estimate the goodness-of-fit of a statistical model. Beyond ANOVA, which is used to analyze the differences among hypothesized group means for a single quantity (feature), Metrics As Scores seeks to answer the question of whether a sample of a certain feature is more or less common across groups. This approach to data visualization and -exploration has been used previously (e.g., Jiang etal., 2022). Beyond this, Metrics As Scores can determine what might constitute a good/bad, acceptable/alarming, or common/extreme value, and how distant the sample is from that value, for each group. This is expressed in terms of a percentile (a standardized scale of [0, 1]), which we call score. Considering all available features among the existing groups furthermore allows the user to assess how different the groups are from each other, or whether they are indistinguishable from one another. The name Metrics As Scores was derived from its initial application: examining differences of software metrics across application domains (Hönel et al., 2022). A software metric is an aggregation of one or more raw features according to some well-defined standard, method, or calculation. In software processes, such aggregations are often counts of events or certain properties (Florac & Carleton, 1999). However, without the aggregation that is done in a quality model, raw data (samples) and software metrics are rarely of great value to analysts and decision-makers. This is because quality models are conceived to establish a connection between software metrics and certain quality goals (Kaner & Bond, 2004). It is, therefore, difficult to answer the question "is my metric value good?". With Metrics As Scores we present an approach that, given some ideal value, can transform any sample into a score, given a sample of sufficiently many relevant values. While such ideal values for software metrics were previously attempted to be derived from, e.g., experience or surveys (Benlarbi et al., 2000), benchmarks (Alves et al., 2010), or by setting practical values (Grady, 1992), with Metrics As Scores we suggest deriving ideal values additionally in non-parametric, statistical ways. To do so, data first needs to be captured in a relevant context (group). A feature value might be good in one context, while it is less so in another. Therefore, we suggest generalizing and contextualizing the approach taken by Ulan et al. (2021), in which a score is defined to always have a range of [0, 1] and linear behavior. This means that scores can now also be compared and that a fixed increment in any score is equally valuable among scores. This is not the case for raw features, otherwise. Metrics As Scores consists of a tool- and analysis suite and an interactive application that allows researchers to explore and understand differences in scores across groups. The operationalization of features as scores lies in gathering values that are context-specific (group-typical), determining an ideal value non-parametrically or by user preference, and then transforming the observed values into distances. Metrics As Scores enables this procedure by unifying the way of obtaining probability densities/masses and conducting appropriate statistical tests. More than 120 different parametric distributions (approx. 20 of which are discrete) are fitted through a common interface. Those distributions are part of the scipy package for the Python programming language, which Metrics As Scores makes extensive use of (Virtanen et al., 2020). While fitting continuous distributions is straightforward using maximum likelihood estimation, many discrete distributions have integral parameters. For these, Metrics As Scores solves a mixed-variable global optimization problem using a genetic algorithm in pymoo (Blank& Deb, 2020). Additionally to that, empirical distributions (continuous and discrete) and smooth approximate kernel density estimates are available. Applicable statistical tests for assessing the goodness-of-fit are automatically performed. These tests are used to select some best-fitting random variable in the interactive web application. As an application written in Python, Metrics As Scores is made available as a package that is installable using the PythonPackage Index (PyPI): pip install metrics-as-scores. As such, the application can be used in a stand-alone manner and does not require additional packages, such as a web server or third-party libraries.

Source code is changed for a reason, e.g., to adapt, correct, or adapt it. This reason can provide valuable insight into the development process but is rarely explicitly documented when the change is committed to a source code repository. Automatic commit classification uses features extracted from commits to estimate this reason.

We introduce source code density, a measure of the net size of a commit, and show how it improves the accuracy of automatic commit classification compared to previous size-based classifications. We also investigate how preceding generations of commits affect the class of a commit, and whether taking the code density of previous commits into account can improve the accuracy further.

We achieve up to 89% accuracy and a Kappa of 0.82 for the cross-project commit classification where the model is trained on one project and applied to other projects. Models trained on single projects yield accuracies of up to 93% with a Kappa approaching 0.90. The accuracy of the automatic commit classification has a direct impact on software (process) quality analyses that exploit the classification, so our improvements to the accuracy will also improve the confidence in such analyses.

Background: Nowadays, expensive, error-prone, expert-based evaluations are needed to identify and assess software process anti-patterns. Process artifacts cannot be automatically used to quantitatively analyze and train prediction models without exact ground truth. Aim: Develop a replicable methodology for organizational learning from process (anti-)patterns, demonstrating the mining of reliable ground truth and exploitation of process artifacts. Method: We conduct an embedded case study to find manifestations of the Fire Drill anti-pattern in n = 15 projects. To ensure quality, three human experts agree. Their evaluation and the process’ artifacts are utilized to establish a quantitative understanding and train a prediction model. Results: Qualitative review shows many project issues. (i) Expert assessments consistently provide credible ground truth. (ii) Fire Drill phenomenological descriptions match project activity time (for example, development). (iii) Regression models trained on ≈ 12–25 examples are sufficiently stable. Conclusion: The approach is data source-independent (source code or issue-tracking). It allows leveraging process artifacts for establishing additional phenomenon knowledge and training robust predictive models. The results indicate the aptness of the methodology for the identification of the Fire Drill and similar anti-pattern instances modeled using activities. Such identification could be used in post mortem process analysis supporting organizational learning for improving processes.

In software engineering, estimations are frequently used to determine expected but yet unknown properties of software development processes or the developed systems, such as costs, time, number of developers, efforts, sizes, and complexities. Plenty of estimation models exist, but it is hard to compare and improve them as software technologies evolve quickly. We suggest an approach to estimation model design and automated optimization allowing for model comparison and improvement based on commonly collected data points. This way, the approach simplifies model optimization and selection. It contributes to a convergence of existing estimation models to meet contemporary software technology practices and provide a possibility for selecting the most appropriate ones.

To take simple decisions comes naturally and does not re- quire additional considerations but when there are multiple alternatives and criteria to be considered, a decision-making technique is required. The most studied and developed technique is the Analytic Hierarchy Process (AHP). We focus on the practical implementation of AHP and study the set of features that are necessary when the process involves several experts together with a set of non-functional requirements, such as portability and usability. Since no existing tool satisfy all our require- ments, we design and implement a new AHP tool called Quick Decide. We perform a user experiment to evaluate our tool, and find that it fulfils all our requirements. To support our usability requirements, we extend AHP with an external consistency check, which measures the distance between expert opinions to avoid results that are mathematically correct but provide no semantic value (e.g., two opposite extreme opinions). 

In an increasingly networked world, the availability of high quality translations is critical for success, especially in the context of international competition. International companies need to provide well translated, high quality technical documentation not only to be successful in the market but also to meet legal regulations. We seek to evaluate translation quality, specifically concerning technical documentation, and formulate a method to evaluate the translation quality of technical documents both when we do have access to the original documents and when we do not. We rely on state-of-the-art machine learning algorithms and translation evaluation metrics in the context of a knowledge discovery process. Our evaluation is performed on a sentence level where each sentence is classified as either professionally translated or machine translated. The results for each sentence is then combined to evaluate the full document. The research is based on a database that contains 22,327 sentences and 32 translation evaluation attributes, which are used to optimize Decision Trees that are used to evaluate translation quality. Our method achieves an accuracy of 70.48% on sentence level for texts in the database and can accurately classify documents with at least 100 sentences.

We implement an automatic mapper that can find the corresponding architectural module for a source code file. The mapperis based on multinomial naive Bayes, and it is trained using custom keywords for each architectural module. For prediction,the mapper uses the path and file name of source code elements. We find that the needed keywords often match the modulenames, but also that ambiguities and discrepancies exist. We evaluate the mapper using nine open-source systems and findthat the mapper can successfully create a mapping with perfect precision, but in most cases, it cannot cover all source codeelements. Other techniques can, however, use the mapping as a foothold and create further mappings.

Reflexion modeling can be used to validate that source code conforms to an intended architecture. However, it requires a mapping of source code modules (e.g., classes) to (software) architecture elements. We have developed a tool suite that allows for evaluation and exploration of automatic techniques to map source code modules to architecture elements. The suite includes a reusable core component and tools to define the architecture, define and run experiments with mapping strategies, and explore the results of these experiments. The experiments can be executed locally or in a remote high-performance computing environment.

The automatic mapping of source code entities to architectural modules is a challenging problem that is necessary to solve if we want to increase the use of Static Architecture Conformance Checking in the industry. We apply the state-of-the-art automatic mapping technique to eight open-source systems and find that there are systematic problems in the automatically created mappings. All of these eight systems have small modules that are very hard to map correctly since only a few source code entities are mapped to these. All systems seem to use some naming strategy, mapping source code to modules; however, naming is often ambiguous. We also find differences in ground truth mappings performed by experts, which affect mappings based on these, and that architectural refactoring also affects the mapping performance. 

We implement an automatic mapper that can find the corresponding architectural module for a source code file. The mapper is based on multinomial naive Bayes, and it is trained using custom keywords for each architectural module. The mapper uses the path and file name of source code elements for prediction. We find that the needed keywords often match the module names; however, ambiguities and discrepancies exist. We evaluate the mapper using ten open-source systems with a mapping to an intended architecture and find that the mapper can successfully create a mapping with perfect precision. Still, it cannot cover all source code elements in most cases. However, other techniques can use the mapping as a foothold and automatically create further mappings. We also apply the approach to two cases where the architecture has been recovered from the implementation and find that the approach currently has limitations of applicability in such architectures. 

We motivate and describe why erosion modelingfrom a static architecture conformance checking point of view isimportant as well as the impact we think it can have on industry.Our research goal is to get a better understanding of erosionand develop smarter and simpler methods to assess erosion. Wedescribe preliminary results from a retrospective, longitudinalcase study of an open source software. The software system westudy, JabRef, was created in 2003, and refactored to a layeredstructure to “transition from a spaghetti to a more structuredarchitecture” in 2015. We construct a model from this new,layered architecture, and assess 30 versions of JabRef to findarchitectural conformance violations. Our preliminary resultsshow a significant difference in productivity after refactoring andsuggest that there is a connection between our model of erosionand productivity where more erosion gives lower productivity.

Some methods use the dependencies between source code entities to perform clustering to, e.g., automatically map to an intended modular architecture or reconstruct the implemented software architecture. However, there are many different ways that source code entities can depend on each other in an object-oriented system, and it is not likely that all dependencies are equally useful. We investigate how well an optimized set of weights for 14 different types of dependencies perform when automatically mapping source code to modules using an established mapping technique. The optimized weights were found using genetic optimization. We compare the F1 score of precision and recall to uniform weights and weights computed by module relation ratio in eight open-source systems to evaluate performance. Our experiments show that optimized weights significantly outperform the others, especially in systems that seem not to have been designed using the low coupling, high cohesion principle. We also find that dependencies based on method calls are not useful for automatic mapping in any of the eight systems.

Automatically mapping source code to architectural modules is an interesting and difficult problem. Mapping can be considered a classification problem, and machine learning approaches have been used to automatically generate mappings. Feature engineering is an essential element of machine learning. We study which source code features are important for an algorithm to function effectively. Additionally, we examine stemming and data cleaning. We systematically evaluate various combinations of features on five datasets created from JabRef, TeamMates, ProM, and two Hadoop subsystems. The systems are open-source with well-established mappings. We find that no single set of features consistently provides the highest performance, and even the subsystems of Hadoop have varied optimal feature combinations. Stemming provided minimal benefit, and cleaning the data is not worth the effort, as it also provided minimal benefit.

Architectural drift and erosion, where the implementation starts to deviate from the intended software architecture or the rules set by it, are common problems in long-lived software systems. This can be avoided by using techniques, such as Reflexion modeling, to validate that the implementation conforms to the indented architecture. Unfortunately, such techniques require a mapping from source code modules (e.g., classes) to elements of the architecture, something that is not always available or up to date. This is a known problem when, e.g., companies want to adopt static architecture conformance checking; the effort to manually create or bring this mapping up to date is just too time-consuming and error-prone. The s4rdm3x tool suite is designed for researchers and practitioners to study and evaluate algorithms that perform part of the mapping automatically, such as orphan-adoption clustering or information retrieval techniques. It includes a graphical user interface to define and run experiments with mapping algorithms and their parameters, and visualize and explore the results of these experiments. The experiments can be executed locally or in a remote high-performance computing environment. The tool suite includes reference implementations of state of the art mapping algorithms and a set of Java systems with validated mappings between classes and architecture elements. The tool suite is extensible, so it is easy to add new mapping algorithms and visualizations to explore their performance. 

The software industry has not adopted continuous use of static architecture conformance checking. One hindrance is the needed mapping from source code elements to elements of the architecture. We present a novel approach of generating and combining dependency and semantic information extracted from an initial set of mapped source code files. We use this to train a Naive Bayes classifier that is then used to map the remainder of the source code files. We compare this approach with the HuGMe technique on six open source projects with known mappings. We find that our approach provides an average performance improvement of 0.22 and an average precision and recall F1-score improvement of 0.26 in comparison to HuGMe.

We study the mapping of software source code to architectural modules. Background: To evaluate techniques for performing automatic mapping of code-to-architecture, a ground truth mapping, often provided by an expert, is needed. From this ground truth, techniques use an initial set of mapped source code as a starting point. The size and composition of this set affect the techniques’ performance, and to make comparisons, random sizes and compositions are used. However, while randomness will give a baseline for comparison, it is not likely that a human expert would compose an initial set on random to map source code. We are interested in letting an expert create an initial set based on their experience with the system and study how this affects how a technique performs. Also, previous research has shown that when comparing an automatic mapping with the ground truth mappings, human experts often accept the automated mappings and, if not, point to the need for refactoring the source code. We want to study this phenomenon further. Audience: Researchers and developers of tools in the area of architecture conformance. The system expert can gain valuable insights into where the source code needs to be refactored. Aim: We hypothesize that an initial set assigned by an expert performs better than a random initial set of similar size and that an expert will agree upon or find opportunities for refactoring in a majority of cases where the automatic mapping and expert mapping disagrees. Method: The initial set will be extracted from an interview with the expert. Then the performance (precision and recall f1 score) will be compared to mappings starting from random initial sets and using an automatic technique. We will also use our tool to find the cases where the automatic and human mapping disagrees and then let the expert review these cases. Results: We expect to find a difference when performance is compared. We expect the expert review to reveal source code that should be remapped, source code that needs refactoring (e.g., possible architectural violations), and points where the automatic technique needs to be improved. Limitations: The study will only focus on only a single system, which limits the external validity significantly. The protocol for the interaction with the human expert can also introduce validity problems; for example, a mapping presented by an algorithm could be perceived as more objective and thus more acceptable for a software engineer. Conclusions: We seek to improve our understanding of how a human creates an initial set for automatic mapping and its effect on how well an automated mapping technique performs. By improving the ground truth mappings, we can improve our techniques, tools, and methods for architecture conformance checking.

The open source application JabRef has existed since 2003. In 2015, the developers decided to make an architectural refactoring as continued development was deemed too demanding. The developers also introduced Static Architecture Conformance Checking (SACC) to prevent violations to the intended architecture. Measurements mined from source code repositories such as code churn and code ownership has been linked to several problems, for example fault proneness, security vulnerabilities, code smells, and degraded maintainability. The root cause of such problems can be architectural. To determine the impact of the refactoring of JabRef, we measure the code churn and code ownership before and after the refactoring and find that large files with violations had a significantly higher code churn than large files without violations before the refactoring. After the refactoring, the files that had violations show a more normal code churn. We find no such effect on code ownership. We conclude that files that contain violations detectable by SACC methods are connected to higher than normal code churn.

Background: The process of mapping a source code entity onto an architectural module is to a large degree a manual task. Automating this process could increase the use of static architecture conformance checking methods, such as reflexion modeling, in industry. Current techniques rely on user parameterization and a highly cohesive design. A machine learning approach would potentially require fewer parameters and better use of the available information to aid in automatic mapping.

Aim: We investigate how a classifier can be trained to map from source code to architecture modules automatically. This classifier is trained with semantic and syntactic dependency information extracted from the source code and from architecture descriptions. The classifier is implemented using multinomial naive Bayes and evaluated.

Method: We perform experiments and compare the classifier with three state-of-the-art mapping functions in eight open-source Java systems with known ground-truth-mappings.

Results: We find that the classifier outperforms the state-of-the-art in all cases and that it provides a useful baseline for further research in the area of semi-automatic incremental clustering.

Conclusions: We conclude that machine learning is a useful approach that performs better and with less need for parameterization compared to other approaches. Future work includes investigating problematic mappings and a more diverse set of subject systems.

An important step in Static Architecture Conformance Checking (SACC) is the mapping of source code entities to entities in the intended architecture. This step is currently relying on manual work, which is one hindrance for more widespread adoption of SACC in industry. Semi-automatic clustering is a promising approach to improve this, and the HuGMe clustering algorithm is an example of such a technique for use in SACC. But HuGMe relies on an initial set of clustered source code elements and algorithm parameters. We investigate the automatic mapping performance of HuGMe in two experiments to gain insight into what influence the starting set has in a medium-sized open source system, JabRef, which contain a relatively large number of architectural violations. Our results show that the highest automatic mapping performance can be achieved with a low number of elements within the initial set. However, the variability of the performance is high. We find a benefit in favoring source code elements with a high fan-out in the initial set.

We are in the pursuit of establishing a method for continuous data driven software architecture. We describe the problem with current methods for measuring the impact of refactoring long lived systems at the architectural level and architecture compliance checking. We summarize our studies of code churn, productivity and an automatic tool for compliance checking. We conclude that architecture violations seem to play an important role, but current methods are infeasible for industry practice. Finally we propose to use repository data mining to improve current methods for architecture compliance checking.

Static architectural conformance checking can be used to find architectural violations, cases where the implementation does not adhere to the architecture, and prevent architectural erosion. We implement a software service for automated conformance checking and investigate the effect this has on the number of architectural violations in software projects. The service is implemented using our heuristic-based approach to static architecture conformance checking of the Model-View-Controller pattern. The service is integrated in the source code management system of each project, so a report is generated every time the source code is modified. The service was evaluated in a field experiment that consisted of eight student projects. We found that the four projects that used the service produced significantly fewer violations compared to those that did not.

We present an evaluation of a simple method to find architectural problems in a product line of computer games. The method uses dependencies (direct, indirect, or no) to automatically classify types in the implementation to high-level components in the product line architecture. We use a commercially available tool to analyse dependencies in the source code. The automatic classification of types is compared to a manual classification by the developer, and all mismatches are reported. To evaluate the method, we inspect the source code and look for a pre-defined set of architectural problems in all types. We compare the set of types that contained problems to the set of types where the manual and automatic classification disagreed to determine precision and recall. We also investigate what changes are needed to correct the found mismatches by either designing and implementing changes in the source code or refining the automatic classification. Our evaluation shows that the simple method is effective at detecting architectural problems in a product line of four games. The method is lightweight, customisable and easy to implement early in the development cycle.

In game software it is important to separate game play code from rendering code to ease transitions to new technologies or different platforms. The architectural pattern Model-View-Controller (MVC) is commonly used to achieve such separation. We investigate how the MVC architectural pattern is implemented in five game projects from a small development studio. We define a metrics-based quality model to assess software quality goals such as portability and rendering engine independence and perform an architectural analysis. The analysis reveals three different evolutions of the pattern. We also assess the quality and find that 1. The evolutions of the architecture differ in quality and 2. An architectural refactoring to a newer version of the architecture increases the software quality.

Application Programming Interfaces (APIs) define how Web services, middle-wares, frameworks, and libraries communicate with their clients. An API that conforms to REpresentational State Transfer (REST) design principles is known as REST API. At present, it is an industry-standard for interaction among Web services. There exist mainly three categories of APIs: public, partner, and private. Public APIs are designed for external consumers, whereas partner APIs are designed aiming at organizational partners. In contrast, private APIs are designed solely for internal use. The API quality matters regardless of their category and intended consumers. To assess the (linguistic) design of APIs, researchers defined linguistic patterns (i.e., best API design practices) and linguistic antipatterns (i.e., poor API design practices.) APIs that follow linguistic patterns are easy to understand, use, and maintain. In this study, we analyze and compare the design quality of public, partner, and private APIs. More specifically, we made a large survey by analyzing and performing the detection of nine linguistic patterns and their corresponding antipatterns on more than 2,500 end-points from 37 APIs. Our results suggest that (1) public, partner, and private APIs lack quality linguistic design, (2) among the three API categories, private APIs lack linguistic design the most, and (3) end-points are amorphous, contextless, and non-descriptive in partner APIs. End-points have contextless design and poor documentation regardless of the API categories. 

Internet of Things (IoT) is a growing technology that relies on connected 'things' that gather data from peer devices and send data to servers via APIs (Application Programming Interfaces). The design quality of those APIs has a direct impact on their understandability and reusability. This study focuses on the linguistic design quality of REST APIs for IoT applications and assesses their linguistic quality by performing the detection of linguistic patterns and antipatterns in REST APIs for IoT applications. Linguistic antipatterns are considered poor practices in the naming, documentation, and choice of identifiers. In contrast, linguistic patterns represent best practices to APIs design. The linguistic patterns and their corresponding antipatterns are hence contrasting pairs. We propose the SARAv2 (Semantic Analysis of REST APIs version two) approach to perform syntactic and semantic analyses of REST APIs for IoT applications. Based on the SARAv2 approach, we develop the REST-Ling tool and empirically validate the detection results of nine linguistic antipatterns. We analyse 19 REST APIs for IoT applications. Our detection results show that the linguistic antipatterns are prevalent and the REST-Ling tool can detect linguistic patterns and antipatterns in REST APIs for IoT applications with an average accuracy of over 80%. Moreover, the tool performs the detection of linguistic antipatterns on average in the order of seconds, i.e., 8.396 s. We found that APIs generally follow good linguistic practices, although the prevalence of poor practices exists. (C) 2022 The Author(s). Published by Elsevier Inc.

Anti-patterns are harmful phenomena repeatedly occurring, e.g., in software development projects. Though widely recognized and well-known, their descriptions are traditionally not fit for automated detection. The detection is usually performed by manual audits, or on business process models. Both options are time-, effort- and expertise-heavy, prone to biases, and/or omissions. Meanwhile, collaborative software projects produce much data as a natural side product, capturing their status and day-to-day history. Long-term, our research aims at deriving models for the automated detection of process and project management anti-patterns, applicable to project data. Here, we present a general approach for studies investigating occurrences of these types of anti-patterns in projects and discuss the entire process of such studies in detail, starting from the anti-pattern descriptions in literature. We demonstrate and verify our approach with the Fire Drill anti-pattern detection as a case study, applying it to data from 15 student projects. The results of our study suggest that reliable detection of at least some process and project management anti-patterns in project data is possible, with 13 projects assessed accurately for Fire Drill presence by our automated detection when compared to the ground truth gathered from independent data. The overall approach can be similarly applied to detecting patterns and other phenomena with manifestations in Application Lifecycle Management data.

In software engineering, testing is one of the corner-stones of quality assurance. The idea of software testing can be applied to information quality as well. Technical documentation has a set of intended uses that correspond to use cases in a software system. Documentation is, in many cases, presented via software systems, e.g., web servers and browsers, and contains software, e.g., Javascript for user interaction, animation, and customization, etc. This makes it difficult to find a clear-cut distinction between a software system and technical documentation. However, we can assume that each use case of a technical documentation involves retrieval of some sort of information that helps a user answer a specific questions. To assess information testing as a method, we implemented QAnalytics, a tool to assess the information quality of documentation that is provided by a website. The tool is web-based and allows test managers and site owners to define test cases and success criteria, disseminate the test cases to testers, and to analyze the test results. This way, information testing is easily manageable even for non-technical stakeholders. We applied our testing method and tool in a case study. According to common perception, the website of Linnaeus University needs to be re-engineered. Our method and tool helped the stakeholders identify what information is presented well and which parts of the website that need to change. The test results allowed the design and development effort to prioritize actual quality issues and potentially save time and resources.

Observation is important when we teach programming. It can help identify students that struggle, concepts that are not clearly presented during lectures, poor assignments, etc. However, as development tools become more widely available or courses move off-campus and online, we lose our ability to naturally observe students. Online programming environments provide an opportunity to record how students solve assignments and the data recorded allows for in-depth analysis. For example, file activities, mouse movements, text-selections, and text caret movements provide a lot of information on when a programmer collects information and what task is currently worked on. We developed CSQUIZ to allow us to observe students on our online courses through data analysis. Based on our experience with the tool in a course, we find recorded sessions a sufficient replacement for natural observations.

To have direct observation of students during an online programming course is impossible. This makes it harder for teachers to help struggling students. By using an online programming environment we have the opportunity to record what the students actually do to solve an assignment. We can analyse the recordings and provide teachers with valuable information. We developed and used an online programming toolwith fine-grained event logging to observe how our students solve problems. Our tool provides descriptive statistics and accurate replays of a student’s programming sessions, including mouse movements. We used the tool in a course and collected 1028 detailed recordings. We compare fine-grained logging with existing coarsegrained logging solutions to estimate assignment-solving time. We find that time aggregations are improved by including time for active reading and navigation enabled by the increased granularity. We also divide the time users spent into editing (on average 14.8%), active use (on average 37.8%), passive use (on average 29.0%), and also estimate time used for breaks (on average 18.2%).Finally wesee a correlation between early student submission results and students that hand in later, but also see an example where the results differ significantly.

It is not possible to directly observe how students work in an online programming course. This makes it harder for teachers to help struggling students. By using an online programming environment, we have the opportunity to record what the students actually do to solve an assignment. These recordings can be analyzed to provide teachers with valuable information. We developed such an online programming tool with fine-grained event logging and used it to observe how our students solve problems. Our tool provides descriptive statistics and accurate replays of a student's programming sessions, including mouse movements. We used the tool in a course and collected 1028 detailed recordings. In this article, we compare fine-grained logging to existing coarse-grained logging solutions to estimate assignment-solving time. We find that time aggregations are improved by including time for active reading and navigation, both enabled by the increased granularity. We also divide the time users spent into editing (on average 14.8%), active use (on average 37.8%), passive use (on average 29.0%), and estimate time used for breaks (on average 18.2%). There is a correlation between assignment solving time for students who pass assignments early and students that pass later but also a case where the times differ significantly. Our tool can help improve computer engineering education by providing insights into how students solve programming assignments and thus enable teachers to target their teaching and/or improve instructions and assignments.

We investigate the benefits of our programming assignments in correlation to what the students learn and show in their programming solutions. The assignments are supposed to teach the students to use best practices related to program comprehension, but do the programming assignments clearly show the benefits of best practices? We performed an ex- periment that showed no significant result which suggests that the assignments did not emphasise the value of best practices. As lecturers, we understand that constructing as- signments that match the sought after outcome in students learning is a complex task. The experiment provided valu- able insights that we will use to improve the assignments to better mirror best practices. 

We wanted to introduce peer reviews for the final report in a course on Software Testing. The students had however experienced issues with peer reviews in a previous course which made this a challenge. To get a better understanding of the situation, we distributed a pre-questionnaire to the students. 48 of the 83 students provided their expectations on peer reviews. To deal with some of the perceived issues, we developed a peer review tool where we introduce anonymity, grading of reviews, teacher interventions, as well as let students score and comment on the reviews they receive. In total, 67 reports were submitted by 83 students and 325 reviews were completed. The post-questionnaire was answered by 48 students (not necessarily the same respondents as for the pre-questionnaire as both were collected anonymously). While 27 of the students expected incorrect feedback only 13 students agreed to have got incorrect feedback in the post-questionnaire. The students reported that they found the feedback from their peers more valuable (+15%) than expected, and 88% of the students reported that they learned from doing peer reviews. Overall, we find that the students' attitudes towards peer reviews have improved.

Learning programming with tutor tools has grown in popularity. These tools present programming assignments and provide feedback in the form of test-cases and compilation errors. Our timeline visualization of data from one such tool allows us to tell a story about what files were accessed and for how long, in what order files were edited, grown or shrunk, what errors the student ran into, and how those errors were addressed. This can be done without a need to read and replay the entire programming session. In sum, the tool has been used to visualize logs from students that tried to solve programming assignments and we find interesting stories that can help us improve how we address new assignments.

The core of this work-in-progress is that the best way to learn how to code is to practice by solving problems. However, if students have trouble with this, they can get frustrated and give up. Automated Tutoring Systems (ATS) aim to provide hints to help them solve the problems they encounter. Many of the existing systems offer general hints, e.g., “check the conditional statement” or help the student interpret the compiler or test-case errors. While this can be useful, we think that an ATS should provide interactive and specialized feedback for each program. We snowballed through publications on promising ATS and found that there are several such systems (in 27 publications), but we could also identify many challenges and that our requirements were not met by any existing system. For example, few of them work on general-purpose programming languages, e.g., Java, or scale to realistic problems consisting of multiple methods and classes. From the search, we find ATS based on Automated Program Repair (APR) shows the most promise. However, while program repair has the potential to generate specialized hints to help guide the student to a working state, studies that looked into these have identified further challenges. For example, many APR ATS tools only show the repaired program to the students, who then have to compare and modify their program accordingly. Another issue is that APR generally only modifies a few lines, so if the student solution is far from correct, the repair might fail. This can be solved by partial repair, i.e., the program is repaired so at least one additional test-case passes. While this increases the repair rate, it might make hints more difficult or point the students in a non-obvious or even “wrong” direction. The APR can take several minutes, which also makes it unsuitable for interactive ATS. We take a design science approach to define an ATS based on APR that attempts to address the identified challenges. We give a review of the state-of-the-art for the required components, e.g., APR, how to generate hints from differences between two programs. From this, we suggest a threestep roadmap; 1. identify suitable APR-tools, 2. construct an oversized test-suite, and 3. adopt APR to the tutoring context.