Copyright IBM Corporation 2004.

http://www-106.ibm.com/developerworks/rational/library/4865.html

Search for:
Use + - ( ) " "
IBM home
developerWorks

within

All of dW

Search help

| Products & services | Support & downloads | My account

> Rational

Four lessons for software testers (and their managers)

Len DiMaggio Software QE Engineer and Manager, IBM 28 May 2004 from The Rational Edge: Written with the software tester in mind, but relevant to everyone on the software development team, this article offers sound advice on staying current with technology, understanding the difficulties of integration, and keeping your team members well-informed. It dawned on me recently that between my school and work life, I've spent nearly a quarter of a century in software engineering, and a big chunk of it in software testing. In that time, I've witnessed and taken part in a tremendous amount of technological change. Disk drives once the size of washing machines now fit on key-chains. Laptops have more computing power than systems that filled entire rooms. Computer languages that were worldwide standards have been completely forgotten. Once famous and successful corporations have been absorbed or put out of business. The means by which software is designed, built, and tested has evolved from an ad hoc, seat-of-the-pants, fire drill, to a well-organized, streamlined, repeatable process. (Well, at least many organizations agree this is the ideal.) Contents: Lesson #1 Don't fall behind by standing still Lesson #2 Always look under the hood Lesson #3 Mind the Gap Lesson #4 Ward Bond was right: Tell people why, not just what The value of teams Notes About the author Rate this article Subscriptions: dW newsletters dW Subscription (CDs and downloads)

I've been fortunate to learn many technical skills involving programming languages, operating systems, databases, and middleware such as application and Web servers. I've also learned some important lessons that have stood the test of time, regardless of changes in technology and trends in software development methodologies. These lessons are the subject of this essay. Note that I've intentionally excluded some standard principles such as understanding and mapping customer requirements to software functions, since those topics have been covered in detail in numerous books and articles. My goal here is to describe some useful, but often overlooked, principles for people just starting their careers in software test engineering.

Lesson #1 Don't fall behind by standing still

Software engineering is a technology-driven field in which the rate of change is likely to increase over time. If you're going to be successful, you'll have to work hard to keep up. The minute you think that you've reached a milestone in learning a skill, such as some aspect of a programming langauge or an architecture such as J2EE, you'll probably find that a new feature or capability is available for you to digest. One of my favorite lines from the movie Inherit the Wind is "All motion is relative. Perhaps it is you who have moved away by standing still."1 This notion is very true in software engineering. If you stop moving forward, you'll quickly be left behind. Not only is technology constantly changing; the Internet, the Web, and their associated technologies are rapidly increasing the rate at which technology is changing. You won't be able to make business or technical decisions unless you keep growing along with it. So, don't look for the pace to slow down. It won't. For most people, the older you get, the harder it is to learn. In the business world, the farther up the corporate food chain people rise, the further they become detached from the nuts and bolts of the technology they're using and building. This is partly caused by their need to focus on other tasks, such as managing and directing the actions of their team members. It's sometimes also caused by an unfortunate attitude that permits them to think performing hands-on technical tasks is beneath their station. By contrast, a former manager2 presented a wonderful attitude about new technology. He was in his late fifties when I worked for him, but he approached new technology with the enthusiasm of a freshman. He devoured Internet development information and was always tinkering with Web servers, firewalls, and new languages such as Java.3 He also had a special appreciation for the role that a QA or test organization performed as his background was primarily software development and development management. Because of his past experiences, he understood just how much easier

The Rational Edge--June 2004

Four lessons for software testers (and their managers)

Page 2 of 5

the life of a development manager could be when an independent test or QA group performed a thorough testing job. What can you do to help yourself stay up-to-date? While the following may read like a list of character traits, I think you can actually train yourself to behave in the following ways: Be curious. Actively look for new tools and technologies as part of your everyday activities. You don't necessarily have to spend lots of time looking. I've found it very useful to set my browser's default page to IBM developerWorks.4 This is a great Web resource for learning what's new in a variety of IBM and non-IBM specific technologies such as Linux, Java, or XML. It's a cliche that you'll "earn what you learn," but in a field such as software engineering, it's very true! Be creative. Look for ways to integrate new tools and technologies into everyday tasks. For example, I recently had to document some department procedural information for a company-internal Website. Instead of using HTML or a word processor, I used XML and stylesheets. The page looks great and I now have a better understanding of stylesheets. (I already knew about XML.) Be rebellious. If the reason behind a technical decision is "because we always do it that way," ask "why?" There may be a better way, based on more current technologies. At the very least, your asking the question may make people consider why they "always do it that way." For example, maybe a CGI architecture made sense for your Web application a long time ago, but maybe Web services is a better way to do it today.

Lesson #2 Always look under the hood

Just what do I mean when I say that software test engineers should look under the hood?5 For our international readers, this is an American metaphor based on raising the "hood" (or the "bonnet," in Great Britain) of an automobile you're thinking about purchasing to ensure that the engine appears in good condition. In terms of software development, I mean that you should not only look at a program in terms of its outward appearance (i.e., its visible inputs and outputs), but you should also make the extra effort to understand its internal design, data flows, and history of bugs and bug fixes. The knowledge that you gain from this effort will enable you to build much more robust and complete tests. In practical terms, looking under the hood means that you should: Be a detective. A great deal of informal documentation is written in the course of developing and testing a software product. This informal documentation may come in many forms, such as email messages, memos, or minutes from meetings. Reviewing the informal documents from previous products can be a good source of information on the design and functional constraints (e.g., not just on what the design decisions were, but why the decisions were made). But don't stop with written information! You need to get up from your desk, leave your office, and talk to your fellow team members. You should exercise some detective skills and mine the brains of your fellow team members, beyond the folks who actually write the code: Marketing personnel. Engineering departments generally make fun of marketing departments, but without them, nothing gets sold and no one gets paid! Marketing personnel quantify the customer's needs and expectations. Reading the plans they devise to meet those needs and expectations will help test engineers design tests to verify that the needs are met. Customer support personnel. These are the people who staff the telephone hotlines. In the course of helping customers with problems with the product, the Customer Support people acquire a detailed understanding of how a product works and how customers use it. Because they deal with bugs reported by customers, they encounter the limitations in the product's design on a regular basis. They also deal with the differences between customers' expectations for a product and the product's actual performance. Taking the time to talk to these folks about their experiences resolving customers' problems can give you valuable insight into building tests that better simulate real-world usage of the software you're testing. Be an archeologist. Given the schedule pressures that govern software development and testing, it's unlikely that anyone will have the time to write a history of your project. The closest thing you're likely to encounter is the product's bug database. What can you learn about a program by reading through old bug reports? You can learn about where you're likely to find bugs in the future. The probability of errors in any part of a program is directly proportional to the number of errors already found in that part. In other words, where there is one bug, there is likely to be another. For example, program sections that are frequently patched may lose the integrity of their original design.6 You can also learn a great deal about what code is being changed by reading source code control system check-in messages. Reading those messages can give you a good sense of which parts of the code are being changed frequently, and can help you understand what subsystems the product is divided into. Be a student. When you start on a new software project, wouldn't it be helpful if you could visit the library, take out a book, and learn exactly how the software should be designed and how it should function? Well, guess what? You can! Admittedly, you won't be able to find reference books on how to construct tests for the newest release of Product XYZ; but you may be able to find reference information on the types of programs and systems used to build that product. I encountered a situation like this a few years ago when I encountered a misbehaving daemon (a Unix daemon, that is). The daemon supported the operation of an Internet firewall product. But the daemon was badly written and broke some of the basic rules for daemons. For example, it didn't write status messages to a log file; rather, it displayed them on the system console. (This is a bad idea for a program that is

The Rational Edge--June 2004

Four lessons for software testers (and their managers)

Page 3 of 5

supposed to run unattended. If no one was sitting at the console, then no one would see its logging messages!) I turned the testing of this daemon into a small research program and ended up learning a good deal about Unix system programming7 in general and daemons in particular.8

Lesson #3 Mind the Gap

While there's a great deal of literature available on functional, system, stress, and other types of software testing, one type of testing is often overlooked: integration testing.9 Why is integration testing important? I'll answer that question with a question: Does anyone actually write all the software for their products any more? The answer is no. More and more products are built with components supplied, at least in part, by a third party. Sometimes this means open source or freeware, and sometimes it means partnering with another company. Either way, testing the integration of components, systems, servers, and so on is increasingly important. When you integrate multiple systems, you are in effect building a common language to bind the systems together. The elements of this language include software dependencies; configuration data; internal and external process coordination and data flows; event reporting; and security. Integration testing (maybe "aggregation testing" is a better term) is performed to verify the operation of these language elements and of the entire integrated system, and to identify the points of conflict between them. What aspects of software integration should you verify in order to perform integration testing successfully? Here's my list: Installation and Configuration. You're not going to get very far if you can't move the various software systems to be integrated from their distribution media to your target system. And you won't get much further if the systems install conflicting libraries or define conflicting environment variables. Data Compatibility. You can perform multiple levels of data parsing and translation in any program, but if you have to do it all the time, then you'll suffer performance degradation somewhere. And, if you miss a subtle data pattern, then you'll leave a time bomb lurking in your system for your users to discover. Interprogram Coordination. Any one system that you're integrating may be well-behaved, operating without any impact on other software running on its server. But put two systems together, and you may find that either one can only operate properly if it starts before the other. Monitoring and Logging. If a program falls in the forest and no one hears it, does it bring down the entire system? No matter how well designed and built a system is, it's going to encounter problem situations someday. And you're going to need all the information you can get to debug it. This task gets more complicated when you integrate multiple systems because you have to be able to access debugging information from different places, and in different forms. Security. First, last, and always. The system that you chain together is only as strong as its weakest link. And that one weak link can compromise the entire system. Performance and Throughput. Slow performance can kill your integrated system. You'll have to identify and resolve the bottlenecks before your users become frustrated by waiting for the system to respond. Reliability. If (i.e., when) problems occur, can the integrated systems avoid outages? If not, can affected subsystems recover? Maintainability and Expandability. Sooner or later, every system will have to receive a patch, service pack, hot fix, and so on. Can these software updates install in a consistent manner, and can the system continue to run while maintenance is done? Must the support staff learn multiple installation procedures? Also, if you're hoping that your business will grow over time, you'd better make sure that your integrated system will be able to support that expansion.

Lesson #4 Ward Bond was right: Tell people why, not just what
The first three of these lessons are directed primarily at software testers. This last lesson is aimed squarely at the folks who make decisions about the direction a software testing takes: in other words, software test team leads and managers. A former manager of mine was both an accomplished equestrian, and a huge fan of the 1960's television western show "Wagon Train." She often drew analogies between the situations in which the main character in the show, Ward Bond, found himself and problems that she or members of her software engineering team were dealing with. In one episode, as Ward Bond and his wagon train are crossing a desert, one of the wagon masters arranges to have all the water barrels consolidated in his wagon, so that the water can be guarded and rationed. When he does this, the other wagon train people immediately accuse him of hoarding water for his own personal use and threaten to lynch him. (This is set in the American wild west, after all.) But, just in the nick of time, Ward Bond diffuses the situation by explaining that the water rationing plan is actually a very sane approach that everyone should all follow in crossing the desert.

The Rational Edge--June 2004

Four lessons for software testers (and their managers)

Page 4 of 5

What was the moral of her telling (and retelling) this story? That when you make an important decision, especially when it alters something familiar, in which people have considerable emotional and intellectual investment, it's not enough to tell people the "what" and the "how" about the decision. If you want them to "buy into" the decision, you also have to tell them "why." This approach applies to software in that it's very common for dramatic changes to be made to the design of a software product during its development. For a software development or testing team to be able to adjust to these changes, they have to be able to understand and accept not just what will be changed and how it will change, but also the reasons behind why it's being changed: Take time to understand the reasons for change. Why are so many design changes made to a software product during it's development? Changes to the production environment is one reason, since new releases of software development tools often make once impossible things possible. If we can add features that make the client happy, it's human nature to do so. But these changes must be made within the limitations of the physical properties of the materials used. For example, it's difficult to redesign a suspension bridge after it's half completed. In software, however, changes are only limited by the imagination and skills of the programmer and the software tools available. To be successful, your software development or test team must be able to understand and adapt to these changes. It will be a lot easier for your team (and you!) to be able to fully implement these changes if everyone understands what is behind them. One important point about getting your team to buy in to a decision: You have to be honest with them about the reasons for the decision, and even more importantly, you have to be honest with yourself. If you and they don't believe in what you're doing, no amount of clarity is going to help. There's a great line attributed to John D. Rockefeller that drives this point home. Late in his life, he took up playing golf. He was a terrible golfer, but he always had an assistant keep scrupulously honest scores. When asked why, he gave some insight to his success in life by saying simply that that "we don't deceive ourselves." Stay focused. As changes occur, it can be confusing to keep track of them as a software product is being created. As I mention above, you and your team members will likely have to deal with shifting or evolving software designs. Your task in situations such as this is to keep your team focused on specific actions being taken. It's easy for some people to become overwhelmed by what may be chaotic conditions. As the leader, you have to bring as much clarity to your team's assignments as possible.

The value of teams

In closing, there's just one more point that I'd like to make. There's an old saying that one of the best things that you can do for your career is to always work for a good manager. In this regard, I've been lucky to always, except for one or two "interesting" characters, have worked for good managers. It's equally important, however, to have good co-workers and employees! No matter how competent you are, you won't be able to get very far totally on your own. I've also been very lucky in this regard. John Donne was right about no one being an island. So was Casey Stengal when he said, after being congratulated on managing the New York Yankees to an American baseball World Series championship, "I couldn't have done it without my players!"

Notes
1 Inherit the Wind, directed by Stanley Kramer, with Spencer Tracy, Frederic March, and Gene Kelly, United Artists, 1960. 2 When I last saw my former manager, he was attending night classes at a local college extension school. He's not studying computer science this time; he's learning German. Why German? Why not? 3 Java, J2EE and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

Other company, product and service names may be trademarks or service marks of others.
4 http://www-136.ibm.com/developerworks 5 L. DiMaggio, "Looking Under the Hood: An Investigative Approach to Software Testing," in STQE Magazine, January, 2000. 6 For a great description of conceptual integrity with regards to software design, take a look at Fredrick Brooks' classic book The Mythical

Man-Month.
7 The Unix programming books by the late Richard Stevens were very valuable resources for me. If you're doing any programming on

Unix systems, you owe it to yourself to invest in a set of these books.

8 I was also fortunate enough to publish the results of this research and testing in "Testing Unix Daemons," L. DiMaggio, Dr. Dobbs' Journal, March 2000. 9 DiMaggio, Len, "Testing at the Boundaries," in The Rational Edge (http://www-

The Rational Edge--June 2004

Four lessons for software testers (and their managers)

Page 5 of 5

106.ibm.com/developerworks/rational/library/content/RationalEdge/apr03/IntegrateSystems_TheRationalEdge_Apr2003.pdf) and DiMaggio, Len, "Orchestrating Integration Testing," in STQE Magazine, July/August, 2003.

About the author

Len DiMaggio is a software quality engineer and manager at IBM Rational. Prior to joining Rational, he led software testing teams at BBN, GTE, and Genuity. Len has published articles on software testing in STQE, Software Development, Dr. Dobbs Journal, and the QAI Journal.

What do you think of this document? Killer! (5) Comments? Good stuff (4) So-so; not bad (3) Needs work (2) Lame! (1)

Submit feedback
developerWorks

> Rational

About IBM

| Privacy | Terms of use | Contact

The Rational Edge--June 2004