Experimental Software Design

Soham Thaker
Masters-Automation & IT

FH-Kln Under the guidance of Prof. Bongards

Topics to be discussed:
What is Experimental Software Engineering (ESE)? Definition. History Experimental Paradigm Process of ESE Examples Conclusion References

What is Experimental Software Engineering?

Development of an experimental paradigm for evolving knowledge of the discipline & do research in order to achieve a scientific progress is known as Experimental Software Engineering. Needs to follow the model like that of other physical sciences. Example: Any Project development through Waterfall Model.

Analysis
Problem Model No. of User i/ps Type of Project User Requirements

Design
Solution Model Time of Project Team Size Planning the requirements

Implementation
Coding

Test
Check Integrity No.of Bugs Types of Bugs

Evaluation
Complete as a whole Number & types of Defects.

Coding Parameters No. of Lines of code Loops , etc

History of Experimental Software Engineering

Evolution done in early 1970s. Phase-wise Developments:
1974-1985: Isolated studies done for particular purpose. 1986-1999: Building S/w processes & technical knowledge in specefic domains & environments. 2000-2005: Expanding studies out across other domains & environments but limiting the technologies being studied. 2006 till now: Building knowledge about a specific domain using many technologies and then integrating these domain specific know ledges.

Experimental Paradigm
ESE- a research that helps establish a scientific & engineering basis for the s/w engineering field. ESE entails an Experimental Paradigm which involves Experimental Design, Observation, Data collection through measurement and Validation of the process.
Hypothesis Experiments Result Data Collection

Conclusions

Experimental Method is to understand the effects of a particular tool usage in some environment & to validate hypothesis about how best the s/w engineering can be accomplished. 1) The Scientific method: observe the world, propose a model or a theory of behavior, measure and analyze, validate hypotheses of the model or theory, and if possible repeat the procedure.

Experimental Paradigm-2
1.1) The Engineering method: observe existing solutions, propose better solutions, build/develop, measure and analyze, and repeat the process until no more improvements appear possible. 1.2) The Empirical method: propose a model, develop statistical/qualitative methods, apply to case studies, measure and analyze, validate the model and repeat the procedure.

2) The Mathematical method: propose a formal theory or set of axioms, develop a theory, derive results and if possible compare with empirical observations.

Process of ESE
1. Build Project Variables: i. Identify the Project variables that affect the project. E.g.: Project Time, Success, Reliability, customer satisfaction, etc. ii. Understand the variables iii.Build Models using the variables to understand & characterize the environment. 2. Identify where the methods might make a difference. 3. Collecting data from live projects through measurement. 4. Feedback on data collection. 5. Storage and analysis of large amounts of data.

Example of Technology Transfer at Motorola

Adopted Goal-Question-Metric Paradigm Adopted the Quality Improvement Paradigm, a three part processimprovement approach. (Planning, execution, and analysis and packaging) identify, tailor, and transfer software-engineering technology. process package which evolves over time Motorola set a 3-stage Formal-Software Review Process Preparation, Meeting & Follow-up Review Goals, Review Metrics, Analyze Metrics

Quality Improvement Paradigm @ Motorola

Results of Technology Transfer at Motorola-(i)

Results of Technology Transfer at Motorola-(ii)

Phase Containment Effectivness of above table at each phase.

Conclusion
Does the experimental approach make sense?

Many decisions were made in the past, with compelling arguments, but without real proof

Of course logical arguments should not be neglected, but instead validated by experiments

References:
The Past, Present, and Future of Experimental Software Engineering by Victor Robert Basili. (Springer) Experimental Software Engineering Issues: Critical Assessment and Future Directions -International Workshop Dagstuhl Castle, Germany, September 14-18, 1992 Proceedings (Springer-Verlag) Technology Transfer at Motorola Victor Basili, Daskalantonakis &Yacobellis. (IEEE JOURNAL) Experimental Software Engineering: A Report on the State of the Art Lawrence Votta, Adam Porter & Dewayne Perry. (Springer) Guide to Advanced Empirical Software Engineering Shull, F.; Singer, J.; Sjoeberg, D. The Experimental Paradigm in Software Engineering - William W. Agresti, Victor R. Basili.

Any Questions?

THANK YOU !!!