Prabhat Ram
Silverio Martínez-Fernández
Alessandra Bagnato
ABSTRACT
Background: Practitioners would like to take action based on software metrics, as long as they find them reliable. Existing literature explores how metrics can be made reliable, but remains unclear if there are other conditions necessary for a metric to be actionable. Context & Method: In the context of a European H2020 Project, we conducted a multiple case study to study metrics' use in four companies, and identified instances where these metrics influenced actions. We used an online questionnaire to enquire about the project participants' views on actionable metrics. Next, we invited one participant from each company to elaborate on the identified metrics' use for taking actions and the questionnaire responses (N=17). Result: We learned that a metric that is practical, contextual, and exhibits high data quality characteristics is actionable. Even a non-actionable metric can be useful, but an actionable metric mostly requires interpretation. However, the more these metrics are simple and reflect the software development context accurately, the less interpretation required to infer actionable information from the metric. Company size and project characteristics can also influence the type of metric that can be actionable. Conclusion: This exploration of industry's views on actionable metrics help characterize actionable metrics in practical terms. This awareness of what characteristics constitute an actionable metric can facilitate their definition and development right from the start of a software metrics program.
- Raymond P.L. Buse and Thomas Zimmermann 2011. Information Needs for Software Development Analytics - Microsoft Research. MSR Tech Report 2011--8. (2011), 1--16.Google Scholar
- Alistair Croll and Benjamin Yoskovitz 2013. Lean Analytics: Use Data to Build a Better Startup Faster.Google Scholar
- Xavier Franch, Claudia Ayala, Lidia López, Silverio Martínez-Fernández, Pilar Rodríguez, Cristina Gómez, Andreas Jedlitschka, Markku Oivo, Jari Partanen, Timo Raty, and Veikko Rytivaara 2017. Data-driven requirements engineering in agile projects: the Q-rapids approach. Proceedings - 2017 IEEE 25th International Requirements Engineering Conference Workshops, REW 2017. (2017), 411--114. DOI:https://doi.org/10.1109/REW.2017.85.Google ScholarCross Ref
- Tracy Hall and Norman Fenton 1997. Implementing effective software metrics programs. IEEE Software. 14, 2 (1997), 55--64. DOI:https://doi.org/10.1109/52.582975.Google ScholarDigital Library
- Hennie Huijgens, Davide Spadini, Dick Stevens, Niels Visser, and Arie Van Deursen 2018. Software analytics in continuous delivery: A case study on success factors. International Symposium on Empirical Software Engineering and Measurement. (2018). DOI:https://doi.org/10.1145/3239235.3240505.Google ScholarDigital Library
- ISO 2008. ISO/IEC 25012:2008, Software Engineering -- Software Product Quality Requirements and Evaluation (SQuaRE) -- Data Quality Model.Google Scholar
- Tapani Kilpi 2001. Implementing a Software Metrics Program at Nokia. IEEE Software. 18, 6 (2001), 72--77. DOI:https://doi.org/10.1109/52.965808.Google ScholarDigital Library
- Eetu Kupiainen, Mika V. Mäntylä, and Juha Itkonen 2015. Using metrics in Agile and Lean software development - A systematic literature review of industrial studies. Information and Software Technology. 62, 1 (2015), 143--163. DOI:https://doi.org/10.1016/j.infsof.2015.02.005.Google ScholarDigital Library
- Frank Van Latum, Rini Van Solingen, Markku Oivo, Barbara Hoisi, Dieter Rombach, and Gunther Ruhe 1998. Adopting GQM-Based Measurement in an Industrial Environment. IEEE Software. 15, February 1998 (1998), 78--86. DOI:https://doi.org/10.1109/52.646887.Google ScholarDigital Library
- Yang W. Lee, Diane M. Strong, Beverly K. Kahn, and Richard Y. Wang 2002. AIMQ: A methodology for information quality assessment. Information and Management. 40, 2 (2002), 133--146. DOI:https://doi.org/10.1016/S0378-7206(02)00043--5.Google ScholarCross Ref
- Manoel G. Mendonça and Victor R. Basili 2000. Validation of an approach for improving existing measurement frameworks. IEEE Transactions on Software Engineering. 26, 6 (2000), 484--499. DOI:https://doi.org/10.1109/32.852739.Google ScholarDigital Library
- Andrew Meneely 2016. Actionable metrics are better metrics. Perspectives on Data Science for Software Engineering. Elsevier. 283--287.Google Scholar
- Tim Menzies and Thomas Zimmermann 2013. Software analytics: So what? IEEE Software. 30, 4 (2013), 31--37. DOI:https://doi.org/10.1109/MS.2013.86.Google ScholarDigital Library
- Marc Oriol, Pertti Seppänen, Woubshet Behutiye, Carles Farré, Rafal Kozik, Silverio Martínez-Fernández, Pilar Rodríguez, Xavier Franch, Sanja Aaramaa, Antonin Abhervé, Michal Choras, and Jari Partanen 2019. Data-Driven Elicitation of Quality Requirements in Agile Companies. Communications in Computer and Information Science (2019), 49--63.Google Scholar
- K Petersen and C. Wohlin 2011. Measuring the flow in lean software development. Software - Practice and Experience. 41, 9 (2011), 975--996. DOI:https://doi.org/10.1002/spe.Google ScholarCross Ref
- Dan Port and Bill Taber 2017. Actionable Analytics for Strategic Maintenance of Critical Software: An Industry Experience Report. IEEE Software. 35, 1 (2017), 58--63. DOI:https://doi.org/10.1109/MS.2017.4541055.Google ScholarCross Ref
- Prabhat Ram, Pilar Rodriguez, and Markku Oivo 2019. Success Factors for Effective Process Metrics Operationalization in Agile Software Development: A Multiple Case Study. Proceedings of the 2019 International Conference on Software and System Process (2019).Google ScholarDigital Library
- Per Runeson and Martin Höst 2009. Guidelines for conducting and reporting case study research in software engineering. Empirical Software Engineering. 14, 2 (2009), 131--164. DOI:https://doi.org/10.1007/s10664-008-9102--8.Google ScholarDigital Library
- Vibhu Saujanya Sharma and Vikrant Kaulgud 2013. Adoption and use of new metrics in a large organization: A case study. International Workshop on Emerging Trends in Software Metrics, WETSoM. (2013), 21 -27. DOI:https://doi.org/10.1109/WETSoM.2013.6619332.Google ScholarCross Ref
- Miroslaw Staron and Wilhelm Meding 2009. Ensuring reliability of information provided by measurement systems. Software Process and Product Measurement, International Conferences IWSM 2009 and Mensura 2009 (Berlin, Heidelberg, 2009), 1--16.Google Scholar
- Miroslaw Staron and Wilhelm Meding 2012. Factors determining long-term success of a measurement program: An industrial case study. e-Informatica Software Engineering Journal. 1, 1 (2012), 7--23. DOI:https://doi.org/10.2478/v10233-011-0027-z.Google Scholar
- Miroslaw Staron and Wilhelm Meding 2015. Transparent measures: cost-efficient measurement processes in SE. Software Technology Transfer Workshop (Kista, Sweden., 2015), 1--4.Google Scholar
- Daniel Vacanti and Bennet Vallet 2014. Actionable Metrics at Siemens Health Services. September 2011 (2014).Google Scholar
- Ye Yang, Davide Falessi, Tim Menzies, and Jairus Hihn 2018. Actionable analytics for you. IEEE Software. 35, 1 (2018), 51--53.Google ScholarCross Ref
- Dongmei Zhang, Shi Han, Yingnong Dang, Jian Guang Lou, Haidong Zhang, and Tao Xie 2013. Software Analytics in Practice. IEEE Software. 30, 5 (2013), 30--37. DOI:https://doi.org/10.1109/MS.2013.94.Google ScholarDigital Library
Index Terms
- Actionable Software Metrics: An Industrial Perspective
Recommendations
An Empirical Investigation into Industrial Use of Software Metrics Programs
Product-Focused Software Process ImprovementAbstractPractitioners adopt software metrics programs to support their software development from the perspective of either overall quality, performance, or both. Current literature details and justifies the role of a metrics program in a software ...
Understanding the value of considering client usage context in package cohesion for fault-proneness prediction
By far, many package cohesion metrics have been proposed from internal structure view and external usage view. Based on whether client usage context (i.e., the way packages are used by their clients) is exploited, we group these metrics into two ...
Software fault prediction metrics
ContextSoftware metrics may be used in fault prediction models to improve software quality by predicting fault location. ObjectiveThis paper aims to identify software metrics and to assess their applicability in software fault prediction. We ...
Comments