An Empirical Study of Release Note Production and Usage in Practice

Abstract

Background: The release note is one of the most important software artifacts that serves as a communication bridge between development teams and users. Release notes contain a set of crucial information, such as descriptions of enhancements, improvements, potential issues, development, evolution, testing, and maintenance of projects throughout the whole development life cycle.

Method: We then conducted interviews with 15 professionals and an online survey with 314 respondents to investigate their opinions on release notes in practice.

Conclusion: We summarised 27 statements about release notes grouped into eight topics based on participants' opinions. Our study uncovers significant discrepancies between release note producers and users in perceiving release notes.

Background / Aims

A large scale empirical study: we analyzed and characterized 32,425 release notes and relevant data of 1,000 GitHub projects.

Interviews and one online survey with professionals: we gathered professionals' opinions on release notes.

Associated release notes with software activities: we investigated how the information in release notes impacts various software activities (e.g., software evolution).

Methods

Three phases of our study

  1. Empirical study
  2. Interview
  3. Survey

Release note study process

Summary of findings

Who: There are many release note producers (i.e., contributors) who document release notes in GitHub projects. There are usually main release note producers who play critical roles and are more influential in project development.

When: The time intervals of release note production for major and minor releases in different domains are significantly different.

What: Release notes contain rich and important information, which spans the whole development life cycle.

Release notes are related to a set of software activities that can shed light on ways to produce and use release notes more effectively.

Several significant discrepancies exist between Release Note Producers and Release Note Users in perceiving release notes.

Conclusions

27 statements concluded from release note producers and users' considerations of perceiving release notes, and the discrepancies between them were identified.

Release note producers and users confirmed that the release note is a vital artifact of software development.

Reference

  • Bi, T., Xia, X., Lo, D., Grundy, J.C., Zimmerman, T., An Empirical Study of Release Note Production and Usage in Practice, to appear in IEEE Transactions on Software Engineering.