The Challenge

Halliburton’s product enhancement (PE) software technology organization develops critical internal enterprise software. The business unit’s two major software products are developed on a Windows platform using the .NET framework. This software is extremely complex, featuring a rich user interface, vast source code, and N-Tier architecture.

The PE software technology organization has employed Agile practices for several years and recently implemented Continuous Integration and Deployment. Major updates demanded running the full regression suite, which took approximately 8-10 weeks to complete.

Multiple process management platforms between functional roles were hampering collaboration among Agile teams. The development team was using Microsoft’s Visual Studio 2012 for source control. Scrum Masters and product managers used TFS for user stories and bug management. In contrast, the test team used Hewlett Packard’s Quality Center (QC) for test management, execution, and status reporting. The separate platforms made it difficult for teams to keep track of the latest changes, often resulting in duplicate tests that had been run in unit testing or failing to run needed tests. The lack of a single product traceability matrix for requirements, code tests, and bugs also impacted the teams’ effectiveness. However, the most crucial issue was that automation coverage decreased with each new build after initial success due to the amount of maintenance work required to keep the automation scripts up-to-date.

The vision for Halliburton was to implement a completely integrated development and test environment, including:

• Implementing Test Automation without impacting Project Schedules
• Training the test team globally on Action Based Testing (ABT) method, Microsoft Test Manager (MTM), and TestArchitect
• Migrating manual tests from QC to MTM by recreating them using ABT
• Creating detailed status reports that include images and log files on failure
• Automating and certifying manual tests before moving into production
• Clearly defining a workflow between the manual and automation teams
• Running tests unattended by auto-deploying the execution as a part of the build definition

The Solution

The initial task was to identify tests that would be the most beneficial to automate. All 3,000 test cases were audited, and 600 were selected for the initial automation effort. The automation work, which included designing and creating tests implementing the ABT method, was outsourced to LogiGear.

LogiGear began the automation effort by creating custom actions in the TestArchitect action library that were unique to PE software. Simultaneously, the PE team worked to customize MTM and TestArchitect to ensure a seamless workflow between the two tools.

Automation engineers created and stored tests in TestArchitect’s repository from business-level actions. Testers could then select individual tests for manual testing or create a full automation suite to run using the association feature—a TestArchitect plug-in that installs in MTM to connect the test repositories.

The final stage of the integration project involved migrating all remaining manual tests from QC to MTM. Ultimately, they were redesigned employing the ABT method, and the test migration was outsourced to LogiGear after considering the cost and resource requirements.

The Result

Halliburton’s PE team set out to improve Continuous Integration while achieving maximum test coverage within highly compressed release cycles consisting of two-week sprints and multiple feature branches. After automating over 2,000 tests, the regression suite, which previously required 8 weeks, could now be completed in 5 hours using a combination of 17 virtual and physical machines.

Additionally, by transitioning to the ABT method, the average time to develop and execute a single verification test was reduced from 3 hours to less than 10 minutes. The project successfully achieved the objective of reducing the test team’s cycle to match the velocity of testing with development.