Improving Human-Centric Software Defect Evaluation, Reporting, and Fixing

We are exploring ways to make defect reporting more "human-centric". This includes better support for reporting human aspect-related software defects;  supporting diverse users in reporting such defects; and supporting software engineers to better appreciate, understand, diagnose and fix such defects.

Defect reporting customarily exists in most applications and web sites to support issue reporting by end users and for developers to receive actionable feedback. However the impact of “human-centric” issues - such as age, gender, language, culture, physical and mental challenges, and socio-economic status - is often overlooked in the development process and during product inception and defect reporting. Most defect reporting tools lack necessary human-centric features to enable a challenged user to adequately navigate and report defects i.e. do not take into account the human differences between end users that cause defects for them in their software and make reporting of such defects difficult for them. Most defect reporting tools also lack sufficient defect report structuring, reporting guidance, and do not emphasize the possible perceived severity of the defect to developers from end users who are very different to them. If users are unable to report defects due to usability issues, this makes those same defect reports difficult to understand by the developer. In this paper, we aim to improve human-centric defect reporting by employing cognitive walkthrough with diverse end user personas, develop a prototype of an improved defect reporting tool, and evaluate the prototype’s defect reporting process from end user and developer perspectives. Our findings offer a foundation for improved human-centric defect evaluation, reporting and fixing.

Method

Based on review of existing literature on software and usability defect reporting, as well as reviewing the Web Content Accessibility Guidelines, we set out to improve the human-centric defect reporting process. To do this we developed a set of personas to represent a range of end users with quite different human aspects (age, gender, physical and mental challenges, language etc). We then designed and prototyped a new set of defect reporting tools for web sites and mobile phones that try to take into account (i) the range of end user challenges encountered when using apps/web sites i.e. human-centric defects to report, and (ii) the range of end user challenges encountered when trying to report these human-centric defects. We also developed personas of developers that are quite different to the end users reporting these human-centric defects. Then we investigated how to present the reported defects to the developers in order to help them better understand the defect and to fix them.

Findings

From this study we developed a set of guidelines that can improve defect reporting for people with usability issues, such as vision, hearing, motor and reading impairments. These guidelines help to overcome possible interface and interaction issues which currently hinder these users ability to generate useful defect reports. Another significant outcome of this study is the identification of factors that can assist developers in human-centric defect evaluation and resolution. The first factor was that educating users about defect reporting as well as educating developers about personas of different possible users and their diverse challenges was useful. Second was that features such as capturing the frequency of the use of application/encountering a defect doesn’t affect the developers perceived severity of the issue. The last key factor was that increasing the amount of extra information collected about a defect, while taking appropriate steps to prevent over complication of defect reports, was effective.

Conclusions

Our research involved creation of personas to represent end users with specific disabilities and differences that manifest human-centric issues when using apps and websites. We conducted a cognitive walk-through on a mix of mobile and web applications based on our created personas. We identified a number of design decisions to assist diverse end users report different challenges they have – human-centric defects – when using these apps. We then developed a prototype human-centric defect reporting app and carried out a second cognitive walk-through with our prototype to evaluate its performance. Our research questions and methods consisted of finding ways to assist both bug reporters and developers in reporting and understanding human-centric defects in apps and web sites. Our evaluation helped us to develop some guidelines to increase the usability of defect reporting tools for disabled end users and increase useful information about these defects to be provided to developers.

Publication

  • Huynh, K., Benarivo, J., Xuan, C.D., Sharma, G.G., Kang, J., Grundy, J.C., Madugalla, A., Improving Human-Centric Software Defect Evaluation, Reporting, and Fixing, 2021 IEEE International Conference on Computers, Software, and Applications Conference (COMPSAC2021), July 12-16 2021, online. --  Author pre-published version PDF

Presentation


Project Lead

Investigators

Students

  • Kenny Huynh
  • Juvent Benarivo
  • Chew Da Xuan
  • Giridhar Gopal Sharma
  • Jeffrey Kang

Human-centric defect reporting examples