Overview
TD Benchmarker toolbox is an interactive web-based application implemented using Shiny framework taking advantage of the R statistical language in an easy-to-use frontend. The toolbox supports a fully automated investigation of the degree of agreement between three leading TD assessment tools. Moreover, TD Benchmarker provides a general framework to capture the diversity of the examined tools with the aim of identifying few 'reference assessments' representing different profiles of classes with respect to their TD levels. After the characterization of the derived reference assessments, the goal is to extract a set of classes presenting high similarity to the Max-Ruler reference assessment. The Max Ruler reference assessment represents the profile of classes accumulating high amount of TD based on the results of all applied tools.
The core methodology of the toolbox is Archetypal Analysis, which is a statistical methodology that explores the multidimensional space with the aim of identifying certain points, namely the archetypes, located on the boundaries of a swarm of given points, known as their convex hull.
TD Benchmarker is ready to report the results derived from the analysis on TD assessments provided by the application of three leading TD assessement tools (Sonar, Squore, Cast) through a benchmark experiment of ten Java OSS project (approx. 12 K classes) of the Apache Foundation with a fully automated manner offering:
- Report on the degree of agreement of the applied tools with respect to the measured TD of classes.
- Identification of few reference TD assessments.
- Extraction of a benchmark of high-TD classes based on the agreement of all TD tools.
Development Team
Developer: Dr. Nikolaos Mittas
Maintainer: MSc. Maria Papoutsoglou
Maintainer: MSc. Theodoros Amanatidis
Contact Info
Dr. Nikolaos Mittas
e-mail: nmittas@chem.ihu.gr
MSc. Maria Papoutsoglou
e-mail: mpapouts@csd.auth.gr
MSc. Theodoros Amanatidis
e-mail: tamanatidis@uom.edu.gr
This component of the TD Benchmarker Platform is demoed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. In case the component of this site is used it should be explicited mentioned and cited in relevant works.
Info
Description
The Summary tabpanel summarizes the results of the analysis conducted on the selected Project.
- Number of Classes: Displays the total number of the examined classes.
- Kendall's Coefficient of Concordance: Displays the Kendall's W coefficient of concordance coefficient and the p-value regarding the agreement among TD tools.
- RSS Plot: Diplays the presentage reduction of the Residual Sum of Squares (RSS) after consecutive executions of the algorithm with different number of archetypes.
- Archetypal Profiles Plot: Displays the TD reference assessments based on the optimal archetypal solution.
Actions
Select Project
RSS Plot
Archetypal Profiles
Info
Description
The Benchmark Classes tabpanel summarizes the benchmark classes that are neighbors to the Max-Ruler archetype.
- Number of Classes close to Max-Ruler: Displays the total number (and the percentage) of benchmark classes that are close to the Max-Ruler archetype.
- Benchmark Classes: Displays the name (col. Class), the TD rankings (cols. Sonar, Squore, Cast), the degree of resemblance to the the Max-Ruler archetype (col. The Max-Ruler) for benchmark classes.
Actions
- Threshold for Benchmark Classes: Select the threshold value for the identification of classes that are close to the Max-Ruler archetype.