Testing Facts

Testing Facts
By: Sarvani Ganesh IBM Global Services India This paper describes the typical problems associated with testing activities in a project. It also describes the reasons for ineffective testing. It explains few of the best practices that can be followed to improve the test effectiveness in any project. II I The problems due to improper testing are of Testing as an activity is performed in every project irrespective of whether it is development, enhancement or maintenance project and on all platforms. The concepts of testing remain similar irrespective of technology/platform. They remain similar whether the entity being tested is an application or a product or a package. It is one of the main activities in the life cycle of the project, which involves around 30% to 35% effort of the total project effort. Most of the projects rely on unearthing the defects in the application by testing rather than any other activity. Everyone in the project, the development teams, testers, analysts, customers, use testing as a tool to map the application functionality to the user specified requirements. In spite of all these facts it is the most ignored or unplanned activity in most of the projects. This leads to delivery of the applications with lesser quality, which in turn leads to customer dissatisfaction. This paper tries to explain the most frequent mistakes done in testing activities and also the tries to bring out few of the guidelines/best practices that can be followed in projects to improve the testing efficiency. Following these practices does not guarantee a better quality, but increases the probability of delivering a better quality. 2. 1. Lack of independent test teams in projects team Most of the small projects with size 10 or lesser would many folds. The problems could be either in planning or execution. Few of the major problems, which are seen in most testing projects, are highlighted here.

generally not have an independent test team. The same team member would perform the role of a developer or a tester. This will not allow having an independent view at the testing, which would have been the case otherwise. The team member may not do all the testing activities like test environment setup, release expected. management, Mostly the etc unit as and

system testing would be done in a similar fashion. This will lead to incomplete and ineffective testing. Lack leads. Most often in smaller projects there is no separate test manager to define/track the testing activities. It is the project manager, who plays the of understanding of testing activities by project managers and

IBM Global Services India

1 of 5

Testing Facts
role of a test manager too. Most of the times in a project the project manager is not aware of the testing activities completely and hence will not be able to plan the test activities well. 3. Improper Test Planning Test planning is never taken up seriously in projects. This is done more like a ritual. Mostly because the person who would be assigned to do this would not know what is test planning and hence they tend to copy it from some other project with out realizing the importance of planning. Most of the problems start in testing because of this. 4. Lack of skilled resources Most of the times it is difficult to get a person with required skills and who is motivated to perform testing activities. Because of this people with less suitable skills or people who are not motivated are assigned the jobs. This will lead to improper or ineffective testing 5. Lack of focus on test data Most of the times the testers do not understand the importance of the test data. If the customer has provided the test data, they would use the same. But in many a projects due to customer confidentiality the test data would not be provided. The team does not really spend enough time to understand the requirements and test data. They cook up some sample data and use the same to test all scenarios. Hence not all scenarios would be tested because 7. 6. of lack of data. This is one of the main contributors for improper test coverage, because with one set of sample data all the user scenarios will not be covered in most of the cases. Lack of understanding of test environment setup Most of the testers/developers do not understand test environment and hence its setup. Generally the same development environment would be used for testing or the if there is a separate test environment available, not enough care is taken by team to ensure that it is properly deployed with a new build and configured properly and fresh set of data is loaded, etc. And also the developers may start uploading the defect fixes directly to test their defect fixes. Due to these reasons the test environment will not have the latest build deployed and hence the testing exercise becomes futile. Improper release mgmt Most of the projects do not follow proper release management for internal system testing releases. Though a release would be made to test teams, the developer may find defects and provide a patch instead of giving it through proper release. This kind of fixes will lead to lot of inconsistencies. The defect may have got fixed temporarily but may have not been under source control and hence may not be available in the next release. Or it may so happen that the patch provided may

IBM Global Services India

2 of 5

Testing Facts
break the whole system and hence the testing cannot proceed. 8. Improper defect tracking In many projects the defect reported by internal teams are not tracked or if tracked, the tracking may be done manually using excel sheets or word documents. This is very error prone and many a defects may not get tracked to closure. And also is a very time consuming process to track them manually. 9. No proper configuration management of testing artifacts Many a times the projects have proper configuration would application. This leads to incomplete testing in many a cases where there are new team members involved in testing of such applications 11. Incomplete regression testing The test case repositories are mostly not available or incomplete in most of the older applications. The application functionality also keeps changing with enhancements being made to the application. The test cases available up to which the need date is to to be the time is maintained functionality, applications

consuming. Hence in most of the repository incomplete or not up to date. The regression testing of the application would be done only for the impacted areas with the knowledge the team member has on the application and not for the entire application, because of lack of time. This would lead to lot of test escapes and hence higher cost of fixing the defects that are identified at a later stage in the lifecycle. 12. Lack of automation Many applications that is prevalent since years would have lot of tasks that are repetitive and manual. There is not enough focus given to automate the tasks. Few of such tasks could be build & deployment processes, regression testing, release management, etc 13. Low usage of tools Many projects are not aware of the tools available or do not have licenses available for the same. If the tools are not used especially for

management for the source code, but the test artifacts would be mostly under manual configuration control. This may be due the to lack of and understanding need

importance of configuration control and management of test artifacts using a similar tool. This will lead to inconsistent set of test artifacts 10. Lack of test case repository In many applications, which are there since years and undergoing enhancements, a repository of test cases would not be available. The test cases would be available only for the recent set of enhancements. The Most test of cases the it and for team would would the entire functionality will not be available. members know do the the maintaining functionality cases.

testing with out referring the test This would become a problem for new people joining the team to understand and test the

IBM Global Services India

3 of 5

Testing Facts
areas like configuration an inefficient way, which could have been avoided by sharing of the assets

management, defect management, etc, it would be very difficult to efficiently track 14. Unaware of methodologies Many teams are not aware of or in its a and relevance

III Here is an attempt to list the best practices that need to be followed in a project to make testing more efficient and effective 1. 2. Have a independent test team wherever feasible Plan for testers to participate early in the life cycle of a project. Right from requirements phase along with the requirements team 3. 4. Define a test strategy along with clients Involve architects/designers, project managers, developers and testers in test planning activities 5. Prepare test data along with test cases. And baseline the test data along with the test cases. 6. If the test data used is live production data, ensure that all the fields that need to be masked or changed are done. E.g., email id, address, etc need to be changed to avoid test mails being sent to the customers 7. Have a separate environment for testing and development. And let the test environment be similar to production environment as much as possible 8. 9. Focus on preparing test environments prior to every testing with out fail Usage of tools for defect configuration tracking, management,

methodologies/processes importance

project, leading to inefficient ways of managing a project 15. Measurements Many projects do capture the data, but would rarely metrics

attempt to analyze the data and identify the areas of improvements. The projects would not also track the progress to see if there was improvement year on year or quarter on quarter. Analyzing the data, would give a better chance of arriving at quality goals that are meet able. 16. Lack of re-use of Unit test scripts and tools and simulators developed The unit test scripts that are by developers/testers developed

mostly are not stored in the centrally repository. The developers/testers develop the unit test cases and have them in their local repository and do not share with anyone. If there is any one else testing the functionality they would have to develop a test script from scratch. This leads to duplication of effort spent. This holds good for even the tools or simulators developed by individuals. Lot of reusable assets lie with the individuals and others would end up spending effort in developing a similar one or performing testing in

requirements management, etc to ease the work and for improving efficiency 10. Build and maintain a test case repository

IBM Global Services India

4 of 5

Testing Facts
11. Maintain matrix, a which FunctionalityTestcase indicates which

functionality maps to what set of test cases. This will help identifying the functionality that need to be retested when there are new releases and there is not enough time to test 12. Maintain a repository of unit test cases, tools and simulators to help sharing across the team. 13. Capture measurements and analyze and share the results with the team. Use the data to talk about the effectiveness. IV Testing is a very important activity is every project and needs to be given enough attention and focus to make a project successful. Testing also needs to be well planned and implemented with skilled resources to make the results effective.

IBM Global Services India

5 of 5