Practitioners perceptions of the goals and visual explanations of Defect Prediction Models

Project Abstract

Background: Software defect prediction models have been proposed to help developers optimize the limited Software Quality Assurance (SQA) resources and help managers develop SQA plans. Prior studies have different and use different techniques for generating visual. It is unclear what are the practitioners' perceptions of these defect prediction model goals and techniques used to visualize these models.

Method: We conducted a qualitative survey to investigate practitioners' perceptions of the goals of defect prediction models and the model-agnostic techniques used to generate visual explanations of defect prediction models.

Conclusion: We found that -  (1) 82%-84% of the respondents perceived that the three goals of defect prediction models are useful;  (2) LIME is the most preferred technique for understanding the most important characteristics that contributed to a prediction of a file, while ANOVA/VarImp is the second most preferred technique for understanding the characteristics that are associated with software defects in the past.

Implications: Our findings highlight the significance of investigating how to improve the understanding of defect prediction models and their predictions. Hence, model-agnostic techniques from explainable AI domain may help practitioners to understand defect prediction models and their predictions.

Further reading

• Jiarpakdee, J., Tantithamthavorn, C. & Grundy, J. (2021). Practitioners' Perceptions of the Goals and Visual Explanations of Defect Prediction Models. arXiv preprint arXiv:2102.12007 .
• Jiarpakdee, J., Tantithamthavorn, C., Dam, H. K., & Grundy, J. (2020). An Empirical Study of Model-Agnostic Techniques for Defect Prediction Models. IEEE Transactions on Software Engineering.
• Jiarpakdee, J., Tantithamthavorn, C., & Hassan, A. E. (2019). The Impact of Correlated Metrics on the Interpretation of Defect Models. IEEE Transactions on Software Engineering.
• Jiarpakdee, J. (2019, May). Towards A More Reliable Interpretation of Defect Models. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) (pp. 210-213). IEEE.
• Tantithamthavorn, C., Jiarpakdee, J., & Grundy, J. (2020). Explainable AI for Software Engineering. arXiv preprint arXiv:2012.01614.

Project Lead

• Jirayus Jiarpakdee