04/03/2018 Chapter 5 - Stabilizing Phase

Chapter 5 - Stabilizing Phase

On This Page
Goals for the Stabilizing Phase
Team Focus during the Stabilizing Phase
Testing the Solution
Test Tracking and Reporting
Conducting the Pilot
Closing the Stabilizing Phase
Key Milestone: Release Readiness Approved

Goals for the Stabilizing Phase

Figure 5.1: The Stabilizing Phase in the MSF Process Model

The primary goal of the Stabilizing Phase is to improve solution quality to meet the acceptance criteria for release to
production. The team conducts testing on a UNIX migration solution whose features are complete. During this phase, the
team completes tasks and creates deliverables that transition the feature-complete build into a state where the defined
quality level is reached and the solution is ready for full production deployment. Testing during this phase extends testing
that was conducted during development by emphasizing usage and operation under realistic environment conditions. The
team focuses on resolving and triaging (prioritizing) bugs and preparing the solution for release.

During the early stages of this phase it is common for testing to report bugs at a rate faster than developers can fix them.
There is no way to tell how many bugs there will be or how long it will take to fix them. However, two statistical signposts
known as bug convergence and zero bug bounce can help the team estimate when the solution will reach stability.

Note: The terms alpha and beta are widely used to describe the state of IT projects. These terms can lead to confusion
because they are interpreted in many different ways. If you use them, be sure to define them clearly and make sure that the
team, customer, and stakeholders all understand the given definitions.

Once a build has been deemed stable enough to be a release candidate, the solution is deployed to a pilot group. This phase
culminates in the Release Readiness Approved Milestone, indicating team and customer agreement that all outstanding
issues have been addressed.

The tasks summarized in Table 5.1 should be completed during the Stabilizing Phase. This project guide describes the
processes and roles required to accomplish them. Detailed information specific to each migration project about each task,
especially technical information, is provided in the migration guide for that project.

Table 5.1 Major Stabilizing Phase Tasks and Owners

Major Tasks Owners

https://technet.microsoft.com/en-us/library/bb497042(d=printer).aspx 1/7
04/03/2018 Chapter 5 - Stabilizing Phase

Testing the solution Test

The team executes the test plans created during the Planning Phase, which were enhanced and tested
during the Developing Phase.

Resolving defects Development,

The team corrects defects identified through testing and from other sources. New tests are developed to Test
reproduce issues reported from other sources and are integrated into the test suite.

Conducting the pilot Release

The team moves a solution pilot from development to a staging area in order to test the solution with Management
actual users and real scenarios. The pilot is conducted prior to starting the Deploying Phase.

Closing the Stabilizing Phase Project team

The team documents the results of completing the tasks performed in this phase and seeks management
approval at the Release Readiness Approved Milestone meeting.

Top of page
Team Focus during the Stabilizing Phase
Table 5.2 addresses the tasks described previously, but considers them from the perspective of the team roles.

The primary team roles driving the Stabilizing Phase are Test and Release Management.

Table 5.2 Role Cluster Focuses and Responsibilities in Stabilizing Phase

Role Cluster Focus and Responsibility

Product Management Communications plan execution; launch planning

Program Management Project tracking; bug triage

Development Bug resolution; code optimization; hardware or service reconfiguration

User Experience Stabilization of user documentation materials; training materials

Test Testing; bug reporting and status; configuration testing

Release Management Pilot setup and support, deployment planning; operations and support training

Top of page
Testing the Solution
In the Stabilizing Phase, testing is performed not only on individual components of the solution, but on the solution as a
whole, because all features and functions of the solution are now complete, and all solution elements have been built.

The testing that was begun during Developing, according to the test plan created during Planning, continues along with the
tracking, documentation, and reporting activities.

User Acceptance Testing

Although user testing and usability studies began during the Developing Phase, they receive more emphasis during
Stabilizing. These are conducted to ensure that the new system is able to successfully meet user and business needs. This is
not to be confused with customer acceptance, which occurs at the end of the project.

User acceptance testing is performed on a collection of business functions in a production environment after the completion
of functional testing. This is the final stage in the testing process before the system is accepted for operational use. It
https://technet.microsoft.com/en-us/library/bb497042(d=printer).aspx 2/7
04/03/2018 Chapter 5 - Stabilizing Phase

involves testing the system with data supplied by the actual user or customer rather than the simulated data developed as
part of the testing process.

User acceptance testing often reveals errors and omissions in the system requirements definition. The requirements may not
reflect the actual facilities and performance required by the user. User acceptance testing may demonstrate that the system
does not exhibit the anticipated performance and functionality.

The result of this test is an answer to the question of whether or not the solution conforms to the overall user requirements,
which determines if the system is ready for production.

Running a pilot for a select set of users helps in user acceptance testing. Surveys and their results from these users about
different aspects of the solution (user friendliness, convenience, visual appeal, relevance, and responsiveness) are keys to
reaching final user acceptance criteria.

User acceptance testing also gives support personnel and users the opportunity to understand and practice the new
technology through hands-on training. The process helps to identify areas where users have trouble understanding, learning,
and using the solution. Release testing also gives Release Management the opportunity to identify issues that could prevent
successful implementation.

Regression Testing
Regression testing refers to retesting previously tested components and functionality of the system to ensure that they
function properly even after a change has been made to parts of the system. For migration projects, this is the most
important class of tests.

As defects are discovered in a component, modifications should be made to correct them. This may require retesting of other
components in the testing process.

Component system errors can present themselves later in the testing process. The process is iterative because information is
fed back from later stages to earlier parts of the process. Repairing program defects may introduce new defects. Therefore,
the testing process should be repeated after the system is modified.

Here are some guidelines for regression testing:

Test any modifications to the solution to ensure that no new problems are introduced and that the operational
performance is not degraded due to the modifications.

Any changes to the solution after the completion of any phase of testing or after the final testing of the system must
be subjected to a thorough regression test. This is to ensure that the effects of the changes are transparent to other
areas of the system and other systems that interface with the system.

Modifications to the solution components may also require modifications to the testing cases. The project team must
create test data based on predefined specifications. The original test data should come from other levels of testing
and then be modified along with the test cases.

Top of page
Test Tracking and Reporting
Test tracking and reporting occurs at frequent intervals during the Developing and Stabilizing phases. During the Stabilizing
Phase, this reporting is driven by the bug count. Regular communication of test status to the team and other key
stakeholders ensures a well-informed project.

Bug Convergence
Bug convergence is the point at which the team makes visible progress against the active bug count. At bug convergence,
the rate of bugs resolved exceeds the rate of bugs found; thus the actual number of active bugs decreases. Figure 5.2
illustrates bug convergence.

Because the bug count will still go up and down, even after it starts its overall decline, bug convergence usually manifests
itself as a trend rather than a fixed point in time. After bug convergence, the number of bugs should continue to decrease
until zero bug bounce.
https://technet.microsoft.com/en-us/library/bb497042(d=printer).aspx 3/7
04/03/2018 Chapter 5 - Stabilizing Phase

Figure 5.2: Bug convergence is the point where the new bug rate drops below the bug resolution rate
Interim Milestone: Bug Convergence
Bug convergence tells the team that the end is actually within reach.

Zero Bug Bounce

Zero bug bounce is the point in the project when development finally catches up to testing and there are no active bugs for
the moment. Figure 5.3 illustrates a zero bug bounce.

After zero bug bounce, the bug peaks should become noticeably smaller and should continue to decrease until the product
is stable enough for the team to build the first release candidate.

Figure 5.3: Zero bug bounce

Careful bug triaging is vital because every bug that is fixed creates the risk of creating a new bug or regression issue.
Achieving zero bug bounce is a clear sign that the team is nearing a stable release candidate.

Note that new bugs will certainly be found after this milestone is reached. But it does mark the first time when the team can
honestly report that that there are no active bugs even if it is only for the moment and it focuses the team on working to stay
at that point.

Interim Milestone: Zero Bug Bounce

After zero bug bounce, the bug peaks should become smaller and should continue to decrease until the solution remains
stable enough for the team to build the first release candidate.

Release Candidates
After the first achievement of zero bug bounce, a series of release candidates are prepared for release to the pilot group.
Each release is marked as an interim milestone.

Guidelines for declaring a build a release candidate include the following:

https://technet.microsoft.com/en-us/library/bb497042(d=printer).aspx 4/7
04/03/2018 Chapter 5 - Stabilizing Phase

Each release candidate has all the required elements to qualify for release to production.

The test period that follows determines whether a release candidate is ready to release to production or the team
must generate a new release candidate with the appropriate fixes.

Testing the release candidates, done internally by the team, requires highly focused and intensive efforts, and focuses
heavily on flushing out critical bugs.

Testing requires a triage process for resolving any newly discovered bugs.

It is unlikely that the first release candidate will be the one that is actually released, because, typically, critical bugs will
be found during its intensive testing.

Interim Milestone: Release Candidates

As each new release candidate is built, there should be fewer bugs reported, triaged, and resolved. Each release candidate
marks significant progress in the team's approach toward deployment. With each new candidate, the team must focus on
maintaining tight control over quality.

Interim Milestone: Pre-Production Testing Complete

Eventually, a release candidate is built that contains no defects whose resolution requires a change. Once that has occurred,
no more defects will be found within an isolated staging environment; all testing that can be done before putting the
migrated component into production has been completed.

Top of page
Conducting the Pilot
During the pilot, the team tests as much of the entire solution, in a true production environment, as possible. A pilot release
is a deployment to a subset of the live production environment or user group. Depending on the context of the project, the
pilot can take various forms:

In an enterprise, a pilot can be a group of users or a set of servers in a data center.

For migration projects, a pilot might involve testing the most demanding application or database that is being
migrated with a sophisticated group of users that can provide helpful feedback.

Commercial software vendors, such as Microsoft, often release products to a special group of early adopters prior to
final release.

The common element in all these forms of piloting is testing under live conditions. The pilot is not complete until the team
ensures that the solution is viable in the production environment and every component is ready for deployment.

Following Best Practices

Prior to beginning a pilot, the team and the pilot participants must clearly identify and agree upon the success criteria
of the pilot. These should map back to the success criteria for the development effort.

Any issues identified during the pilot must be resolved either by further development, by documenting resolutions
and workarounds for the installation teams and production support staff, or by incorporating them as supplemental
material in training or help materials.

Before the pilot is started, a support structure and issue-resolution process must be in place. This may require that
support staff be trained. The procedures used for issue resolution during a pilot may vary significantly from those
used during deployment and when in full production.

In order to determine any issues and confirm that the deployment process will work, it is necessary to implement a
trial run or a rehearsal of all the elements of the deployment prior to the actual deployment.

Deciding Next Steps

https://technet.microsoft.com/en-us/library/bb497042(d=printer).aspx 5/7
04/03/2018 Chapter 5 - Stabilizing Phase

Once enough pilot data has been collected and evaluated, the team is at a point of decision. One of several strategies must
be selected:

Stagger forward. Deploy a new release to the pilot group.

Roll back. Execute the rollback plan and revert the pilot group back to the previous configuration state they had
before the pilot (as closely as feasible). Then try again with a more stable release.

Suspend. Suspend the entire pilot.

Fix and continue. Issue a fix to the existing code to the pilot group.

Proceed. Advance to the Deploying Phase.

Creating Pilot Test Reports

As cycles of the pilot tests are completed, the team prepares reports detailing each lesson learned and how new information
is incorporated and issues are resolved. The following may result from performing the pilot testing:

The identification of additional risks.

The identification of frequently asked questions for training purposes.

The identification of user errors.

The ability to secure buy-in and support from pilot users.

Documentation of concerns and issue resolutions.

Updates to documentation, particularly help files and deployment plans.

Determination of whether or not all success criteria were met.

Interim Milestone: Pilot Complete

This milestone signifies that the pilot (or pilots) has been successfully completed and that the team is ready to proceed to
Deploying.

Top of page
Closing the Stabilizing Phase
Closing the Stabilizing Phase requires completing a milestone approval process. The team needs to document the results of
the different tasks it has performed in this phase in order to submit the project to management for approval.

Key Deliverables from the Stabilizing Phase

A deliverables checklist for the Stabilizing Phase includes:

Final release

Release notes

End-user help and training materials

Testing and bug reports

Test tools

Traceability audit

Source documentation and executables

https://technet.microsoft.com/en-us/library/bb497042(d=printer).aspx 6/7
04/03/2018 Chapter 5 - Stabilizing Phase

Project documents

Updated risk management document

Milestone review report

Team member project progress report

Team lead project progress report

Top of page
Key Milestone: Release Readiness Approved
The Stabilizing Phase culminates in the Release Readiness Approved Milestone. This milestone occurs at the point when the
team has addressed all outstanding issues and has released the solution and made it available for full deployment. It is the
opportunity for customers and users, operations and support personnel, and key project stakeholders to evaluate the
solution and identify any remaining issues they need to address before release.

Project teams usually mark the completion of a milestone with a formal sign-off. Key stakeholders, typically representatives
of each team role and any important customer representatives who are not on the project team, signal their approval of the
milestone by signing or initialing a document stating that the milestone is complete. The sign-off document becomes a
project deliverable and is archived for future reference.

As the team transitions to the Deploying Phase, responsibility for ongoing management and support of the solution officially
transfers from the project team to the operations and support teams.

Top of page
Download

Get the UNIX Migration Project Guide

Update Notifications

Sign up to learn about updates and new releases

Feedback

Send us your comments or suggestions

Top of page

© 2018 Microsoft

https://technet.microsoft.com/en-us/library/bb497042(d=printer).aspx 7/7