Birch has long suffered from a lack of active forest management, leading many researchers to use material without a detailed management history. Data collected from three birch (Betula pendula Roth, B. pubescens Ehrh.) sites in southern Sweden were analyzed using regression analysis to detect any trends or differences in wood properties that could be explained by stand history, tree age and stem form. All sites were genetics trials established in the same way. Estimates of acoustic velocity (AV) from non-destructive testing (NDT) and predicted AV had a higher correlation if data was pooled across sites and other stem form factors were considered. A subsample of stems had radial profiles of X-ray wood density and ring width by year created, and wood density was related to ring number from the pith and ring width. It seemed likely that wood density was negatively related to ring width for both birch species. Linear models had slight improvements if site and species were included, but only the youngest site with trees at age 15 had both birch species. This paper indicated that NDT values need to be considered separately, and any predictive models will likely be improved if they are specific to the site and birch species measured.

Ranking alternatives is fundamental to effective decision making. However, creating an overall ranking is difficult if there are multiple criteria, and no single alternative performs best across all criteria. Software engineering is no exception.

Software quality is usually decomposed hierarchically into characteristics, and their quality can be assessed by various direct and indirect metrics. Although such quality models provide a basic understanding of what data to collect and which metrics to use, it is not clear how the metrics should be combined to assess the overall quality. Due to different approaches for aggregation of metrics, the same quality model and the same metrics for assessing the same software artifact could still lead to different assessment results and even to different interpretations.

The proposed aggregation approach in this thesis is well-defined, interpretable, and applicable under realistic conditions. This approach can turn the quality- model- and metric-based assessment of (software) quality into a reliable and reproducible process. We express quality as the probability of detecting something with equal or worse quality, based on all software artifacts observed; good and bad quality is expressed in terms of lower and higher probabilities. 

We validated our approach theoretically and empirically. We conducted empirical studies on Bug prediction, Maintainability assessment, and Information Quality.

We used Software Visualization to analyze the usability of aggregation for analyzing multivariate data in general and the effect of different alternative aggregation approaches, i.e., we designed and implemented an exploratory multivariate data visualization tool.

Finally, we applied our approach to Multi-criteria Ranking to evaluate its transferability to other domains. We evaluated it on a real-world decision-making problem for assessment and ranking of alternatives. Moreover, we applied our approach to the context of Machine Learning. We created a benchmark from a collection of regression problems, and evaluated how well the aggregation output agrees with a ground truth, and how well it represents the properties of the input variables.

The results showed that our approach is not only theoretically sound, it is also accurate, sensitive, identifies anomalies, scales in performance, and can support multi-criteria decision making. Furthermore, our approach is transferable to other domains that require aggregation in hierarchically structured models, and it can be used as an agnostic unsupervised predictor in the absence of a ground truth.

A quality model is a conceptual decomposition of an abstract notion of quality into relevant, possibly conflicting characteristics and further into measurable metrics. For quality assessment and decision making, metrics values are aggregated to characteristics and ultimately to quality scores. Aggregation has often been problematic as quality models do not provide the semantics of aggregation. This makes it hard to formally reason about metrics, characteristics, and quality. We argue that aggregation needs to be interpretable and mathematically well defined in order to assess, to compare, and to improve quality. To address this challenge, we propose a probabilistic approach to aggregation and define quality scores based on joint distributions of absolute metrics values. To evaluate the proposed approach and its implementation under realistic conditions, we conduct empirical studies on bug prediction of ca. 5000 software classes, maintainability of ca. 15000 open-source software systems, and on the information quality of ca. 100000 real-world technical documents. We found that our approach is feasible, accurate, and scalable in performance.

It is a well-known practice in software engineering to aggregate software metrics to assess software artifacts for various purposes, such as their maintainability or their proneness to contain bugs. For different purposes, different metrics might be relevant. However, weighting these software metrics according to their contribution to the respective purpose is a challenging task. Manual approaches based on experts do not scale with the number of metrics. Also, experts get confused if the metrics are not independent, which is rarely the case. Automated approaches based on supervised learning require reliable and generalizable training data, a ground truth, which is rarely available. We propose an automated approach to weighted metrics aggregation that is based on unsupervised learning. It sets metrics scores and their weights based on probability theory and aggregates them. To evaluate the effectiveness, we conducted two empirical studies on defect prediction, one on ca. 200 000 code changes, and another ca. 5 000 software classes. The results show that our approach can be used as an agnostic unsupervised predictor in the absence of a ground truth.

Aggregation of software metrics is a challenging task, it is even more complex when it comes to considering weights to indicate the relative importance of software metrics. These weights are mostly determined manually, it results in subjective quality models, which are hard to interpret. To address this challenge, we propose an automated aggregation approach based on the joint distribution of software metrics. To evaluate the effectiveness of our approach, we conduct an empirical study on maintainability assessment for around 5000 classes from open source software systems written in Java and compare our approach with a classical weighted linear combination approach in the context of maintainability scoring and anomaly detection. The results show that approaches assign similar scores, while our approach is more interpretable, sensitive, and actionable.

Software quality becomes more critical as our dependence on software increases. We need better quality assessment than ever. Comparison and ranking of software artifacts, detection of bad or good quality are important tasks for quality assessment.

Software quality models are widely used to support quality assessment. In general, they have a hierarchical structure and defines quality in terms of sub-qualities and metrics in a tree-like structure. Different metrics evaluate different quality criteria, and several metrics often needs to be assessed and aggregated to obtain a total quality score. The quality models standards of today do not enable numerical metrics aggregation. They leave aggregation to decision makers, and different methods of aggregation lead to different assessment results and interpretations. Hence, there is a need to define metrics aggregation formally based on well-known theories.

We propose to consider the probabilistic nature of quality as a solution. We consider metrics as random variables and define quality scores based on joint probabilities. The aggregation, and the quality model in extension, express quality as the probability of detecting something with equal or worse quality, based on all software projects observed; good and bad quality is expressed in terms of lower and higher probabilities. We analyze metrics dependencies using Bayesian networks and define quality models as directed acyclic graphs. Nodes correspond to metrics, and edges indicate dependencies. We propose an implementation using multi-threading to improve the efficiency of joint probabilities computations.

We validate our approach theoretically and in an empirical study on software quality assessment of approximately 100\,000 real-world software artifacts with approximately 4\,000\,000 measurements in total. The results show that our approach gives likely results and scales in performance to large projects.

We also applied our approach to a multi-criteria decision-making task to propose a ranking method to aid evaluation processes. We use a real-world funding allocation problem for a call that attracted approximately 600 applications to evaluate our approach. We compared our approach with the traditional weighted sum aggregation model and found that ranks are similar between the two methods, but our approach provides a more sound basis for a fair assessment.

Further, we implemented an exploratory multivariate data visualization tool, which visualizes the similarities between software artifacts based on joint distributions. We illustrate the usability of our tool with two case studies of real-world examples: a set of technical documents and an open source project written in Java.

Our overall results show that our approach for multi-criteria quality scoring is well-defined, has a clear interpretation, and is applicable under realistic conditions, generalizable, and transferable to other domains.

Sound assessment and ranking of alternatives are fundamental to effective decision making. Creating an overall ranking is not trivial if there are multiple criteria, and none of the alternatives is the best according to all criteria. To address this challenge, we propose an approach that aggregates criteria scores based on their joint (probability) distribution and obtains the ranking as a weighted product of these scores. We evaluate our approach in a real-world use case based on a funding allocation problem and compare it with the traditional weighted sum aggregation model. The results show that the approaches assign similar ranks, while our approach is more interpretable and sensitive.

Assessing software quality, in general, is hard; each metric has a different interpretation, scale, range of values, or measurement method. Combining these metrics automatically is especially difficult, because they measure different aspects of software quality, and creating a single global final quality score limits the evaluation of the specific quality aspects and trade-offs that exist when looking at different metrics. We present a way to visualize multiple aspects of software quality. In general, software quality can be decomposed hierarchically into characteristics, which can be assessed by various direct and indirect metrics. These characteristics are then combined and aggregated to assess the quality of the software system as a whole. We introduce an approach for quality assessment based on joint distributions of metrics values. Visualizations of these distributions allow users to explore and compare the quality metrics of software systems and their artifacts, and to detect patterns, correlations, and anomalies. Furthermore, it is possible to identify common properties and flaws, as our visualization approach provides rich interactions for visual queries to the quality models’ multivariate data. We evaluate our approach in two use cases based on: 30 real-world technical documentation projects with 20,000 XML documents, and an open source project written in Java with 1000 classes. Our results show that the proposed approach allows an analyst to detect possible causes of bad or good quality.

Multi-dimensional goals can be formalized in so-called quality models. Often, each dimension is assessed with a set of metrics that are not comparable; they come with different units, scale types, and distributions of values. Aggregating the metrics to a single quality score in an ad-hoc manner cannot be expected to provide a reliable basis for decision making. Therefore, aggregation needs to be mathematically well-defined and interpretable. We present such a way of defining quality models based on joint probabilities. We exemplify our approach using a quality model with 30 standard metrics assessing technical documentation quality and study ca. 20,000 real-world files. We study the effect of several tests on the independence and results show that metrics are, in general, not independent. Finally, we exemplify our suggested definition of quality models in this domain.

Assessing software quality, in general, is hard; each metric has a different interpretation, scale, range of values, or measurement method. Combining these metrics automatically is especially difficult, because they measure different aspects of software quality, and creating a single global final quality score limits the evaluation of the specific quality aspects and trade-offs that exist when looking at different metrics. We present a way to visualize multiple aspects of software quality. In general, software quality can be decomposed hierarchically into characteristics, which can be assessed by various direct and indirect metrics. These characteristics are then combined and aggregated to assess the quality of the software system as a whole. We introduce an approach for quality assessment based on joint distributions of metrics values. Visualizations of these distributions allow users to explore and compare the quality metrics of software systems and their artifacts, and to detect patterns, correlations, and anomalies. Furthermore, it is possible to identify common properties and flaws, as our visualization approach provides rich interactions for visual queries to the quality models’ multivariate data. We evaluate our approach in two use cases based on: 30 real-world technical documentation projects with 20,000 XML documents, and an open source project written in Java with 1000 classes. Our results show that the proposed approach allows an analyst to detect possible causes of bad or good quality.