August 2007 - Software Testing 3.0: Delivering on the Promise of Software Testing

By Hung Q. Nguyen and Rob Pirozzi

Introduction

This first article of this two article series (The Early Evolution of Software Testing), discussed the early phases of software testing (Software Testing 1.0 and 2.0). It highlighted how both of these phases have largely underperformed expectations and discussed the shortcomings of these early phases of software testing.

Software Testing is now on the threshold of the next major evolutionary development in software testing - Software Testing 3.0. Many organizations are starting to realize the strategic importance of Software Testing 3.0, and others are starting to adopt Software Testing 3.0 principals in their Software Testing programs that are now finally delivering better software faster and meeting management expectations.

This article discusses Software Testing 3.0 and how it delivers on the promise of software testing. It also discusses why senior management will find this evolutionary step strategically important.

Software Testing 3.0

The foundation for Software Testing 3.0 is a strategic end-to-end framework for change based upon a strategy to drive testing activities, tool selection, and people development. Fundamental to its success is the realization that testing is its own function, and an integral part of the development process. Testing must have its own budget to build the necessary infrastructure, as well as clear leadership and ownership. It must provide executive management with visibility into the status of the software under test so that executives can make intelligent and informed decisions about projects and have confidence that they are developing and delivering a quality product that will meet customer needs.

Moving from Software Testing 2.0 to Software Testing 3.0 requires a strategic commitment on the part of the organization, and elevation of software testing from a task or tactical effort to a "strategic effort". An organization must form a team or task force to address the needs of people, practice, and process (SP3T). This task force will develop a team-based methodology to drive tool selection (tools that support the methodology) as well as people development (training in the methodology, tools, and software testing techniques that support the methodology), and determination of necessary metrics to provide management with visibility into the status and quality of the software under test.

This transformation of culture, methodology, beliefs, and techniques will yield a team-based software testing program that is test design focused with an emphasis on testing as a whole rather than in parts. It will address feature reliability; compatibility, usability; and interoperability; performance acceptability, availability, and scalability; security; as well as accessibility. More importantly, it will yield a software testing program that:

• Improves quality by increasing testing coverage and effectiveness
• Decreases testing time
• Decreases costs
• Provides the necessary information for informed decision making
• Provides several assets, including:
  • Tested, high quality software
  • Reusable software tests that codify organization knowledge about the software under test and provide the basis for the testing of future generations of that software
• Focuses on test design and agile test development practices, and
• Extends test automation to multiple platforms including non-GUI platforms

What Software Testing 3.0 Means for Senior Management

Software Testing 3.0, treating software testing as a strategic effort, enables an organization to turn quality into a strategic advantage. To be convinced of the value of this proposition, it is important for senior management to understand the "whys" of Software Testing 3.0.

• Why Treat Software Testing as a Strategic Initiative?

  Software development represents 40% of the product release budget for any company that develops software for resale (LogiGear internal studies). Software testing typically represents 40% of the development budget. Any company task that represents that much money should be treated as a strategic initiative. Strategically "doing-the-right-thing" coupled with "doing it right" can have a profound impact both on the outcome as well as the cost. Conversely, treating it as a tactical initiative can (and will) yield sub-optimal results and lead to unnecessary costs. Put simply, Software Testing 3.0 will save time and money, and result in higher quality software.

• Why Should Software Testing be its Own Organization with a Separate Budget?

  There are several reasons why software testing should be a separate organization from development, with its own budget. First, it is important to realize that software testing performs functions that are very different from software development. Development is a creative process of providing new functionality and features. Testing is a destructive process where testers actually try to "break" the software so that customers do not.

  Testers are not developers. The skills and experience needed differ from those of a developer. Having testing as its own organization with its own budget allows testing-centric management to nurture and develop testing-specific skills in their testers (skills in methodology, tools, metric reporting, etc.).

  Having testing as its own organization also gives it the independence necessary to "stand up" to development when there are problems with the software under test. It is much less likely that development will be able to ram sub-standard code through an independent testing organization to meet delivery promises without a red flag being waved to senior management. While this may potentially cause delays in delivery, these are delays that can prevent significant down-stream costs as well as lost revenue and customer good-will that results when bad software makes it to market. Having software testing and its budget separate prevents testing getting short changed or circumvented and will help to deliver higher quality products.

  Software Testing's budget should also be separate because there are large amounts of money involved. Having the budget separate gives the organization the ability to exert more direct fiscal control and oversight. With an independent budget, software testing becomes much easier to measure the return and the costs savings that may be realized from initiatives such as automation or offshoring of testing efforts.

• Why is Visibility so Important?

  Software Testing performs several key functions for the benefit of the company, product development, and company management. Testing:

  • Is a service to the company that helps the organization produce and release higher quality software
  • Is also a service to the development team to help it produce higher quality code
  • Is an information service for senior management providing information about product quality
  • Organizational improvement
  • Better confidence in the quality (consistency and dependability) of the software
  • Better decision making, and better confidence in the decision making because it is based on data
  • Fewer "surprises"
  • Better allocation and utilization of resources and budget
  • Ability to spend more time and money on development and less on maintenance
  • Delivering higher quality software

Testing performs these services by providing key metrics or data, including test cases run, testing coverage statistics, defect reports and more. This data provides visibility into the status and quality of the software under test. This data can be used by the testing group itself to further refine and enhance testing efforts. Development can use this data to gain an understanding of what parts of the product are causing problems, which may lead to new processes or development tactics to deliver better code to testing. Senior management uses this data to make critical decisions about product schedules and delivery, as well as resource allocations.

With visibility comes:

Conclusion

Software testing has undergone an evolutionary process. The early phases emphasized tactics and tools and yielded sub-optimal results. As with any evolutionary process, it is imperative for an organization to continue to evolve, or perish. Software Testing 3.0 is the next phase in the evolutionary development of software testing.

Organizations that make the transition to viewing testing as a strategic initiative - organizations that put testing on a even footing with development by making testing its own organization with its own budget, place methodology and strategy first and then move on to tool selection - will be able to turn quality into a competitive advantage.

• They will have happier customers.
• They will have a happy sales force with less turnover.
• They will meet or exceed their marketing and promotional claims.
• They will deliver quality software to channels and customers in a timely manner.
• They will minimize potentially damaging product failures.

All of this will help to improve sales and generate revenue, while reducing costs and speeding product to market. Software Testing 3.0 is an evolutionary imperative for any organization that develops software.