SMOKE TESTING Lessons from a Report

Table of Contents
1 Introduction........................................................................................................................................................... 1.1 What is Smoke Testing..................................................................................................................................2

1 Introduction

1.1 What is Smoke Testing? Smoke testing is done by developers before the build is released to the testers or by testers before accepting a build for further testing. Microsoft claims [1] that after code reviews, smoke testing is the most cost effective method for identifying and fixing defects in software. In software engineering, a smoke test generally consists of a collection of tests that can be applied to a newly created or repaired computer program. Sometimes the tests are performed by the automated system that builds the final software. In this sense a smoke test is the process of validating code changes before the changes are checked into the larger products official source code collection or the main branch of source code. In software testing, a smoke test is a collection of written tests that are performed on a system prior to being accepted for further testing. This is also known as a build verification test. This is a "shallow and wide" approach to the application. The tester "touches" all areas of the application without getting too deep, looking for answers to basic questions like, "Can I launch the test item at all?", "Does it open to a window?", "Do the buttons on the window do things? The purpose is to determine whether or not the application is so badly broken that testing functionality in a more detailed way is unnecessary. These written tests can either be performed manually or using an automated tool. When automated tools are used, the tests are often initiated by the same process that generates the build itself. This is sometimes referred to as "rattle" testing.

2 Lessons Learnt
There are some established best practices followed by the RUN Team, when it comes to smoke tests. These practices recommend smoke tests to be broad in scope as opposed to depth, smoke tests should exercise the system from end-end, they should cover the most frequently used and basic operations, smoke tests need to be automated and made part of the regression testing suite, the number of smoke tests are to increase in size and coverage as the system evolves, a system test environment is required for smoke testing and smoke tests need to take advantage from historical database of fault-finding subset of test suites that have proved valuable over other projects. 2.1 Prerequisites There are certain prerequisites that are mandatory to be ready with, before the stipulated time of smoke Testing. The following are the details

Links of the portal. User IDs and Passwords for the roles, that is to be tested. Test Data of the scenarios that is to be tested. Test Cases and the Expected results of the functionality to be tested are chosen well ahead.

Prerequisites of Smoke test 3.0

Link http://*******.healthcare company in US.net/irj/portal. User IDs and Passwords First name Jeff Peter Vincent Last name Berkhouse Burke Abbatiello UserID berkhoj burkepe abbativ Password ******** ******** ********

Role Manager VP Rep

2.2 Important Tips: Check the functioning of the Application, for all the roles, before the actual testing slot starts. Open three different portals, for all the three roles and begin the test simultaneously, as it would save time. Do not perform Wild Char search during smoke testing, as it might take lots of time, and it might result in 500 Timed out error Make a list of the Test data that should be used for testing.

2.3 Measures

Before we assure functionality is failed, it should be tested with different test data.

2.4 Measuring adequacy of test data The prerequisites for an effective test data are a good data dictionary and detailed design documentation .While acquiring the test data, there are several concerns to address including depth, breadth and scope of test data, data integrity during test execution and conditions specific data . Test data could be defined as a predicate that defines what properties of a program must be exercised to constitute a thorough test.

3 Guidelines for Smoke Testing

3.1 Look for modifications Smoke testing basically focuses on the modifications done to the application. Testing team must work with developer who changed the code to understand

What are the exact changes in the application How the functionality is affected by the change. How the change affects the interdependencies of various components.

Emphasize breadth more than depth Smoke testing deals with verifying the functionality at a high level in order to determine if further testing is possible. The purpose of smoke testing is not to ensure that the application is 100% errorfree, rather it validates the application at a higher level. It is the initial level of testing effort to determine if the newly developed or changed code is performing well enough for its major level of testing. The Smoke test scenarios should emphasize breadth more than depth. Each and every major feature should be tested briefly to cover the entire functionality.

4 Process to setup Smoke/Regression Testing

Have a dedicated setup exclusively for smoke so that no other tests other than smoke will run on it. Smoke test may also involve in process like running JOBS /WORKFLOWS. These things should planned well ahead of the time slot. Create and maintain test cases. Consider all the initial conditions to run before testing any indepth functionality. Test case should have details on what is being tested, how the test should be performed and the expected result of the test. Track, monitor and analyze the changes in your application and the components of the system environment. Log the bugs in the defect-tracking database which is identified by the team. Deliverables would be a report of all tests runs with expected versus actual results, plus a summary of issues.

5 Demerits in Smoke test 3.0

Though we had proper guidelines to be followed for the smoke test the RUN team did faced some demerits

The application to be tested was too slow, where some times it lead to Time out errors. So it should have had a good speed to respond to the user. The results could not be properly documented, because there were no specific templates to serve this purpose.

6 Solutions Suggested
As it is very essential to document the smoke test results of each release I created this template and recommend the same which could be exclusively used to serve this purpose.

Recommended Template

Sm oke Testing-Results Tem plate.doc