Professional Documents
Culture Documents
Code of Conduct
v Software Engineering is a collaborative activity. You are encouraged to work together, but ... v Some tasks may require individual work. v Always give credit to your sources and collaborators. Good professional practice: To make use of the expertise of others and to build on previous work, with proper attribution. Unethical and academic plagiarism: To use the efforts of others without attribution.
2
Projects
v Project teams, about 3 to 5 peoples. v Select your own project, any branch of software engineering v Real project for real client who intends to use the software in production. v Feasibility study and plan: during semester v Presentations: requirements design final
Project Selection
v Some suggested projects v Recitation section to suggest projects Contact potential clients: v Gain idea of their expectations v Estimate scope and complexity of the project v Discuss business decisions Assemble project team v Advertise on the web site
4
Previous Experience
Your background v v v v v v Biggest program that you have written? Biggest program that you have worked on? Biggest project team that you have been part of? Longest project that you have worked on? Most people who have used your work? Longest that your project has been in production?
My background
5
Course Themes
1. Leadership of large software projects v Software as a product Clients and their needs Quality v Requirements and specification Usability Evolution v Project management Personnel management Economic, legal, and social factors
6
Course Themes
2. Large and very large systems v Software design Software architecture Object-oriented design v Dependable systems Reliability Verification v Legacy systems
7
Software as a Product
Software is expensive!! Every software project has a trade-off between: Functionality Resources (cost) Timeliness Example: Accounting Management System
v The client provides resources and expects some product in return. v Client satisfaction is the primary measurement of success. Question: Who is the client for Microsoft Excel?
10
Categories of Product
Categories of client and software product: v Generic (e.g., Microsoft Excel) v Bespoke (customized) (e.g., IRS internal system) Many systems are customized versions of generic packages (e.g., Cornell's payroll system)
12
Software products are very varied --> Client requirements are very different --> There is no standard process for software engineering --> There is no best language, operating system, platform, database system, development environment, etc. A skilled software developer knows about a wide variety of approaches, methods, tools. The craft of software engineering is to select appropriate methods for each project and apply them effectively.
13
Professional Responsibility
Organizations put trust in software developers: v Competence: Software that does not work effectively can destroy an organization. v Confidentiality: Software developers and systems administrators may have access to highly confidential information (e.g., trade secrets, personal data). v Legal environment: Software exists in a complex legal environment (e.g., intellectual property, obscenity). v Acceptable use and misuse: Computer abuse can paralyze an organization (e.g., the Internet worm). 14
Software Engineering
Books
v Frederick P. Brooks, Jr. The Mythical Man Month. Addison-Wesley, 1972. v Ian Sommerville, Software Engineering, 6th edition. Addison-Wesley, 2000. v Grady Booch, James Rumbach, Ivar Jacobson, The Unified Modeling Language. Addison-Wesley 1999.
16
Software Process
Fundamental Assumption: Good processes lead to good software Good processes reduce risk
17
Risk Management
What can go wrong in a software project? How can the risk be reduced?
18
Requirements
Design
Implementation
System design: Partition the requirements to hardware or software systems. Establishes an overall system architecture Software design: Represent the software system functions in a form that can be transformed into one or more executable programs v Unified Modeling Language (UML)
22
The software design is realized as a set of programs or program units. (Written specifically, acquired from elsewhere, or modified.) Individual components are tested against specifications.
23
The individual program units are: v integrated and tested as a complete system v tested against the requirements as specified v delivered to the client
24
25
Disadvantages: Each stage in the process reveals new understanding of the previous stages, that requires the earlier stages to be revised.
26
Iterative Refinement
Evaluation Requirements
Implementation (prototype)
Design
29
Iterative Refinement
Concurrent Activities Requirements Outline Description Design Initial Version Intermediate Versions Final Version
Implementation
30
Implementation
Final Version
31
Iterative Refinement
When is iterative refinement appropriate?
32
Outline Description: Add vector graphics to Dartmouth Basic. Phase 1: Extend current language with a preprocessor and run-time support package. (1976/77) Phase 2: Write new compiler and run-time system incorporating graphics elements. (1978/80)
33
Design Issues: v Pictorial subprograms: coordinate systems, window/viewport v User specification of perspective Design Strategy: (Iterative Refinement) v Write a series of prototypes with various proposed semantics v Evaluate with a set of programming tasks
34
Completed projects should look like the Waterfall Model but ... the development process is always partly evolutionary. Risk is lowered by: v Prototyping key components v Dividing into phases v Following a visible software process v Making use of reusable components
36
Software Engineering
Lecture 3
(a) Feasibility Study (b) Requirements Definition
Feasibility Study
Before beginning a project, a short, low-cost study to identify Client Scope Potential benefits Resources needed: staff, time, equipment, etc. Potential obstacles
38
Feasibility Study
A feasibility study leads to a decision: go ahead do not go ahead think again In production projects, the feasibility study often leads to a budget request. In research, a feasibility study is often in the form of a proposal.
39
CS 501: Client
In CS 501, you have two clients: The client for the project The professor for the course Can you satisfy them both?
40
Scope
What are the boundaries of the project? CS 501 Examples: Static web pages with open access on the Web [Web Profiler] Used by the general public [Digital Collections] Varying data formats [Legal Information]
Thousands of sensors [Data mining] Support for Windows, Mac, Unix [SALSA]
41
Potential Benefits
Why are you doing this project? Examples Create a marketable product Improve the efficiency of an organization Control a system that is too complex to control manually New or improved service Safety or security
Resources
Examples: CS 501 Staff: 5 to 7 students, with some help. How many hours per week? What skills do people have? Time: Must be completed by end of semester, including operational system, documentation, presentation Equipment and software: What special needs are there? Client: Will the client be sufficiently available and helpful?
43
Obstacles
CS 501 projects Start-up time. Creating a team, scheduling meetings, acquiring software, learning new systems, ... Business considerations. Licenses, trade-secrets, ... Too ambitious. Nothing to show at the end of the semester. Changing circumstances. Client leaves the university, ... What else?
44
Feasibility Report
A written document For a general audience: client, financial management, technical management, etc. Short enough that everybody reads it Long enough that no important topics are skipped In CS 501, I am looking for a well written, well presented document.
46
Outline Description The Library of Congress requires a repository system to store and make accessible very large amounts of highly varied material over long periods of time.
48
Chronology
1993-94 CNRI carries out research on architectures for digital libraries 1995-97 CNRI implements prototype repository for Library of Congress 1998 CNRI and Library of Congress carry out requirements definition
49
The Repository
Repository
Users
Identification System
Search System
50
Structuring complex information in digital libraries Data driven digital library interfaces
Comparison of object-oriented, relational, and file based storage systems Naming and identification of library objects
Resistance to change within Library of Congress Technical weakness of Library of Congress Gaps in CNRI architecture
54
Mistakes
Confusion of objectives (research and implementation) Failure to involve all stakeholders Over-ambitious (no proper feasibility study)
55
Requirements Document
Specification of Requirements 56
Requirements Definition
High-level abstract description of requirements: Specifies external system behavior Comprehensible by customer, management and users Should reflect accurately what the customer wants: Services that the system will provide Constraints under which it will operate
57
Team (all experienced): Librarian, Software Engineer (CNRI), Computing Project Leader (Library of Congress), + 2 others Advisors: Mailing list of about 20 knowledgeable stakeholders. Timetable: Preliminary report (2 months). Final report (1 month).
58
Functional Requirements
Example: Library of Congress repository Support for complex digital objects Access management Identification Information hiding Open protocols and formats Integration with other systems (scope)
59
DRAFT OVERVIEW OF ITS SUPPORT FOR NDLP PRODUCTION AND DELIVERY OF AMERICAN MEMORY
NDLP collections already released Coolidge collection (for repository test) NDLP collections in conversion Future NDLP collections Other applications and materials
60
NOW
FUTURE
Non-functional Requirements
Environment: Estimates of sizes, numbers of users, etc. Reliability and performance measures and targets Preferred: Example: Library of Congress repository Hardware and software systems (e.g., IBM/Unix) Database systems (e.g., Oracle) Programming languages (e.g., C and C++)
61
Evolution of Requirements
If the requirements definition is wrong, the system will be a failure. With complex systems, understanding of requirements always continues to improve.
Therefore... The requirements definition must evolve. Its documentation must be kept current (but clearly identify versions).
62
Software Engineering
Lecture 4
Management I: Project Management
To complete a project: On time On budget With required functionality To the satisfaction of the client Without exhausting the team
64
Start activities before previous activity complete Sub-contract activities Renegotiate deliverables Keep senior management informed
65
Deliverables: 16 8 8 4 1 4 Written texts (bound in pairs) Television programs Radio programs Computer programs Home experimental kit (scientific calculator) Assignments and sample solutions
67
Flexibility
Schedule: Dates for broadcasting TV and radio programs are fixed. Printing and mailings can be accelerated if overtime is used. Functionality: The course team can decide what goes into the components of the course. Resources: The size of the course team can be increased slightly.
68
An activity
A dummy activity
An event
A milestone
69
other activities
START
Revise Unit 3
Edit Unit 3
Print Unit 3
Mail Unit 3
END
70
other activities
START
Revise Unit 3
Edit Unit 3
Typeset Unit 3
other activities
Revise Unit 4
Edit Unit 4
Typeset Unit 4
71
Script TV 2
START
Document Computer 1
Program Computer 1
72
Prototype Computer 1
4 6 1 12 12 4 4
73
1 3 1 3 1 2
1 1 12 12 0 4 4 12 12 3
4 15 2 17 3 3 17 4 2 19 3
26 6 1 22 1 2 17
74
23
25
11 1 12 12 0 12 14 4 13 3
4 15 2 17 3 3 17 4 2 20 3
26 6 1 23 1 2 17
75
24
25
Critical Path
1/11 12/12 0/0 12/14 4/13 17/17 19/20 17/17
76
Slack
1/11 10 10 12/12 0/0 0 0 15/15 0 0 19/20 1 0 9 17/17
77
Key Personnel
In computing, not all people are equal: The best are at least 5 times more productive Some tasks are too difficult for everybody Adding more people adds communications complexity Some activities need a single mind Sometimes, the elapsed time for an activity can not be shortened. What happens to the project if a key person is sick or quits?
78
Start-up Time
On a big project, the start-up time is typically three to six months: Personnel have to complete previous projects (fatigue) or recruited. Hardware and software has to be acquired and installed. Staff have to learn new domain areas and software (slow while learning) Clients may not be ready.
80
81
Lecture 5
(a) Documentation (b) Requirements Analysis
Assignments
Feasibility and plan Requirements Midterm exam Design Project presentations Final examination
Assignment 1
Wednesday, September 13: Project plan due -- report. Title of project Client/customer Team members Outline description Current status (e.g., previous work) Plan (e.g., major stages, assignment to tasks, technical environment, schedule, etc.) Any other relevant information
84
Documentation
Reasons for documentation: visibility (e.g., project plan, interim report) user support (e.g., user manual) team communication (e.g., interface specifications) maintenance and evolution (e.g., requirements) Characteristics of documentation: accurate and kept current appropriate for audience maintained online (usually) simple but professional in style and appearance Documentation is expensive --> Quality not volume
85
Form of Documentation
External Printed Web site Internal Program documentation Program context (e.g., copyright notices)
86
Requirements Document
Specification of Requirements 88
Requirements Analysis
1. Understand the requirements in depth: Domain understanding Examples: science research, application Stakeholders Example: companies, ministries, Danang City
89
Viewpoint Analysis
Example: University Admissions System Applicants University administration Admissions office Financial aid office Special offices (e.g., athletics, development) Computing staff Operations Software development and maintenance Academic departments
90
Requirements Analysis
2. Organize the requirements: Classification into coherent clusters (e.g., legal requirements) Recognize and resolve conflicts (e.g., functionality v. cost v. timeliness) Example: Dartmouth general ledger system
92
Requirements Analysis
3. Model the requirements: Informal Prose Systematic Procedural models Data-centric models Object models Formal models
93
Form received
Update database
Evaluate
95
Data-Flow Models
Rejection Application Completed form Receive application Evaluate application Applicant Offer
98
Acknowledgment
AND
Receive
Completed application
Initiate evaluation
Evaluation request
AND
Acceptance
Financial aid
Offer
100
Dilemma. Requirements analysis should make minimal assumptions about the system design. But the requirements definition must be consistent with computing technology and the resources available. In practice, analysis and design are interwoven. However, do not to allow the analysis tools to prejudge the system design.
101
Lecture 6
(a) Requirements Analysis (continued) (b) Requirements Specification
Requirements Document
Requirements Analysis
Methods for data modeling and design Data flow diagrams Entity-relation diagrams Data dictionaries Object models Many of these methods blur the distinction between analysis and design.
104
Entity-Relation Model
A Design Methodology for Relational Databases A database of entities and relations Tools for displaying and manipulating entityrelation diagrams Tools for manipulating the database (e.g., as input to database design) Warning: There is much confusion about definitions and notation
105
Entity-Relation Diagram
108
1:n
Is about
Subject heading
Short title
Control numb
109
Data Dictionaries
A data dictionary is a list of names used by the system Brief definition (e.g., what is "date") What is it (e.g., number, relation) Where is it used (e.g., source, used by, etc.) May be combined with a glossary As the system is implemented, the data dictionary in the requirements is input to the system data dictionary, which is a formal part of the system specification.
110
This course teaches object models as a tool for design. Some people recommend object models for requirements analysis, but it is difficult to use them without constraining the system design.
111
Non-Functional Requirements
Product requirements performance, reliability, portability, etc... Organizational requirements delivery, training, standards, etc... External requirements legal, interoperability, etc...
112
Unspoken Requirements
114
Requirements Specification
115
1. It describes the requirements to the stakeholders Expressed in the terms that the stakeholders understand Comprehensible from many viewpoints Reviewed by stakeholders so that they understand implications Must be clear about assumptions (things left out)
116
2. It describes the requirements to the implementers As precise and specific as possible Expressed in terms that they understand Comprehensible to new team members
117
3. It records the requirements for the future An essential part of system evolution 4. If may be a contractual document See you in court!
118
Natural language Structured natural language Design description language Requirements specification language Graphical notation Formal specification See Sommerville, Chapter 7.
119
Lecture 7
Management II Business and Legal Aspects of Software Engineering
Legal Environment
Software is developed in a complex legal and economic framework. Changes in laws follow changes in technical world. Jurisdictions: Vietnamese laws International treaties Federal and state statues Precedents Supreme Court Cost of establishing precedent
121
Legal Topics
International Intellectual property (copyright, patent, contract) Tort (e.g., liability of Internet service provider) Privacy Free speech and its limitations (government secrets, obscenity, blasphemy, hate) Legal Information Institute: http://www.law.cornell.edu/
122
Copyright
A copyright gives the owner the exclusive right to: reproduce distribute perform display license Gradually extended to cover text, music, photographs, designs, software, ...
123
Copyright
Copyright at creation Works for hire Contracts and licenses First sale Fair use Infringement (contamination)
124
Software Patents
Should be: non-obvious, novel, useful 17 years from award (20 years from application) Poor quality of examining can lead to broad patents for routine computing concepts International differences Copyright applies to the expression of ideas, patents to the ideas themselves.
125
Derivative Works
When software is derived from other software: New code is owned by new developer
Conditions that apply to old code apply to derived work If you write S, which is derived from A, B, C and D, you can not distribute or licenses S unless you have right to distribute each of A, B, C and D. To create a software product, you must have documented rights to use every component.
127
Privacy
Invasions of privacy: intrusion appropriation of name or likeness unreasonable publicity false light
Be very careful about collecting personal data without the knowledge of the individual
128
129
Employment contract may restrict your next job (not working for competitors, etc.) Trade-secret information (non-disclosure agreement) Ask when you are interviewed!
130
You and a few friends create a company to develop software. How much should you charge per hour? You plan to work 40 hours a week for 50 weeks of the year and want to earn $50,000. Hourly rate = $50,000 / (40 x 50) = $25 But ...
133
$10M
$5M
2,500
5,000
7,500
Community Development
Shareware Open source (e.g., Linux, Apache, Perl, etc.) -> Shared development -> Market penetration Example: TCP/IP for Vax/VMS Software may be open source, but packaging and services can be profitable businesses
137
Open Source
Free redistribution Source code Derived works Integrity of the author's source code No discrimination against persons or groups
138
Open Source
No discrimination against fields of endeavor Distribution of license License must not be specific to a product License must not contaminate other software http://www.opensource.org/osd.html
139
Practical Advice
Be aware of the law, but do not pretend to be a lawyer. Use a professional for: Contracts and licenses Troubles (complaints, injunctions, subpoenas, etc.) Personnel issues When in doubt, ask help!
140
Source Code Management Or Configuration Management: How I learned to Stop Worrying and Hate My Co-workers Less
141
142
143
144
145
146
Version Control
Companies ship several products from the same source base (i.e. Win NT and Windows 2000 versions of MS Office) When tracking down bugs you want to examine the code as it was when the product shipped
147
Code Sharing
Multiple people can work on the same source base without colliding (1) Locks individual files so only one person at a time can modify it *OR* (2) Allows multiple people to modify a source file and the system will automatically merge the changes (usually)
148
Locking
Only one person can work on a file at once Works fairly well if developers work on different areas of the project and dont conflict often Problem 1: People forget to unlock files when they are done Problem 2: People work around locking by editing a private copy and checking in when the file is finally unlocked - easy to goof and lose changes
149
Merging
Several people can work on a file at once Before committing changes, each user merges their copy with the latest copy in the database This is normally done automatically by the system and usually works, but you should not blindly accept the result of the merge
150
Labeling
Label all the files in the source base that make up a product at each milestone Just before and just after a major change (e.g.. changing several interfaces) When a new version ships
151
Version Trees
Each file in the database has a version tree Can branch off the version tree to allow separate development paths Typically a main path (trunk) for the next major version and branches off of shipped versions for maintenance
152
Branching
When a new version ships, typically create a branch in the version tree for maintenance Double update: fix a defect in the latest version and then merge the changes (often by hand) into the maintenance version Also create personal versions so you can make a change against a stable source base and then merge in the latest version later
153
Examples
RCS Solaris: man rcsintro CVS Solaris: man cvs www.cyclic.com/cvs/info.html Visual SourceSafe msdn.microsoft.com/SSAFE ClearCase www.rational.com
154
RCS
File management only Transaction model check out and lock edit check in and unlock Little support for binaries
155
CVS
Built on top of RCS Therefore little support for binaries Database can be remote No locking: merge before commit Fast Integrates with emacs
156
SourceSafe
Microsofts entry into the field Project-based Checkout-edit-checkin model Built-in web site creation tools Integrates with MSDEV
157
Clearcase
Clearcase is configuration management on steroids You create a view of the database with a config spec, which describes how to select files from the database. When you set a view, Clearcase creates a virtual filesystem containing only those versions of the files selected by the config spec
158
Clearcase Features
Distributed System Several groups at different locations can work on the same database Can install triggers Example: e-mail the author of a file when some one makes a change to it Uses merging model like CVS, but can also lock files
159
160
161
Software Engineering
Lecture 10
Formal Specification
Formal Specification
Why? Precise standard to define and validate software Why not? May be time consuming Methods not suitable for all applications
163
Formal Specification
Ben Potter, Jane Sinclair, David Till, An Introduction to Formal Specification and Z (Prentice Hall) 1991 Jonathan Jacky The Way of Z (Cambridge University Press) 1997
164
Mathematical Specification
Example of specification B1, B2, ... Bk is a sequence of m x m matrices
Two Rules
Formal specification does not guarantee correctness Formal specification does not prescribe the implementation
168
intrt: N a : N
Example: Algorithm
1 + 3 + 5 + ... (2n - 1) = n2
170
Example: Program
int intrt (int a) /* Calculate integer square root */ { int i, term, sum; term = 1; sum = 1; for (i = 0; sum <= a; i++) { term = term + 2; sum = sum + term; } return i; }
171
A broadly used method of formal specification: Event driven systems (e.g., games) User interfaces Protocol specification etc., etc., ...
172
173
Start Beam on
Select Select Enter Patient Field Patients Fields Patients Setup Patients Fields Ready Patients Fields Beam on Fields Setup
ok
Start
Stop interlock
Z Specification
STATE ::= patients | fields | setup | ready | beam_on EVENT ::= select_patient | select_field | enter | start | stop | ok | interlock FSM == (STATE X EVENT) STATE
Z Specification (continued)
control = no_change transitions s}
no_change = { s : STATE; e : EVENT (s, e) transitions = { (patients, enter) (fields, select_patient) fields,
setup, fields,
(ready, select_patient) patients, (ready, select_field) fields, (ready, start) beam_on, (ready, interlock) setup, (beam_on, stop) ready, (beam_on, interlock) setup }
177
Schemas
Schema: The basic unit of formal specification. Describes admissible states and operations of a system.
178
LibSys: An Example of Z
Library system: Stock of books Registered users. Each copy of a book has a unique identifier. Some books on loan; other books on shelves available for loan. Maximum number of books that any user may have on loan.
179
LibSys: Operations
Issue a copy of a book to a reader. Reader return a book. Add a copy to the stock. Remove a copy from the stock. Inquire which books are on loan to a reader. Inquire which readers has a particular copy of a book. Register a new reader. Cancel a reader's registration.
180
LibSys
Level of Detail: Assume given sets: Copy, Book, Reader Global constant: maxloans
181
Naming conventions for objects: Before: plain variables, e.g., r After: with appended dash, e.g., r' Input: with appended ?, e.g., r? Output: with appended !, e.g., r!
182
183
dom m x
ran m y
m:X
Y y} y}
184
dom m = { x X : y Y x ran m = { y Y : x X x
Schema Inclusion
LibDB stock : Copy Book readers: F Reader
LibLoans issued : Copy Reader shelved : F Copy r : Reader #(issued > {r}) < maxloans shelved dom issued =
188
189
Schema Decoration
Issue Library Library' c? : Copy; r? : Reader c? shelved; r? readers #(issued > {r?}) < maxloans issued' = issued {c? r?} stock' = stock; readers' = readers
190
Schema Decoration
Issue Library c? : Copy; r? : Reader c? shelved; r? readers #(issued > {r?}) < maxloans issued' = issued {c? r?} stock' = stock; readers' = readers
191
Software Engineering
Lecture 11
Object-Oriented Design I
Example (Web Butler and Web Site Profiler) Run web data collection in real time or batch mode How are jobs started? Job parameters How are the parameters set up (interactive, edit file, ...)? What are the parameters (specify)? Can job parameters be stored and used again? If so, how? Job monitoring What feedback is given while job is running? Can the user pause or break a job? If so, are the results retained? 194
Remember The requirements document specifies the functionality that you plan to deliver to the client It must be comprehensive and detailed. Everything must be written out -- no hand waving! The requirements document is likely to be several times as long as Assignment 1.
195
Useful Texts
Grady Booch, James Rumbaugh, Ivar Jacobson, The Unified Modeling Language. Addison-Wesley 1999. Grady Booch, Object-Oriented Analysis and Design with Applications, second edition. Benjamin/Cummings 1994. Rob Pooley, Perdita Stevens, Using UML Software Engineering with Objects and Components. Addison-Wesley 1999.
198
BRJ
199
Principles of Modeling
The choice of what models to create has a profound influence on how a problem is attacked and how a solution is shaped. Every model can be expressed at different levels of precision. The best models are connected to reality. No single model is sufficient. Every nontrivial system is best approached through a small set of nearly independent models. BRJ
200
Notation: Classes
Window origin size open() close() move() display() name attributes
operations
A class is a description of a set of objects that share the same attributes, operations, relationships and semantics.
202
Notation: Interface
ISpelling An interface is a collection of operations that specify a service of a class or component, i.e., the externally visible behavior of that element.
203
Chain of responsibility A collaboration defines an interaction, i.e., a society of roles and other elements that work together to provide some cooperative behavior. Place order A use case is a description of a set of sequence of actions that a system performs that yields an observable result.
204
EventManager eventlist suspend() flush() An active class is a class whose objects own one or more processes or threads and therefore can initiate control activity.
205
A component is a physical and replaceable part of a system that conforms to and provides the realization of a set of interfaces. Server
A node is a physical element that exists at run time and represents a computational resource.
206
207
Business rules A package is a general-purpose mechanism for organizing elements into groups. return copy of self A note is a symbol for rendering constraints and comments attached to an element or a collection of elements.
208
Notation: Relationships
A dependency is a semantic relationship between two things in which a change to one may effect the semantics of the other.
0..1 employer
* employee
An association is a structural relationship that describes a set of links, a link being a connection among objects.
209
A generalization is a specialization/generalization relationship is which objects of the specialized element (child) are substitutable for objects of the generalized element (parent).
A realization is a semantic relationship between classifiers, wherein one classifier specifies a contract that another classifier guarantees to carry out.
210
Diagrams in UML
A diagram is the graphical representation of a set of elements, usually rendered as a connected graph of vertices (things) and arcs (relationships). Class diagram shows a set of classes, interfaces, and collaborations with their relationships. Object diagram shows a set of objects and their relationships. Use case diagram shows a set of use cases and actors (a special kind of class) and their relationships.
211
Interaction diagram shows an interaction, consisting of a set of objects and the relationships, including the messages that may be dispatched among them. => A sequence diagram emphasizes the time ordering. => A collaboration diagram emphasizes the structural organization of the objects that send and receive messages.
212
Statechart diagram shows a state machine consisting of states, transitions, events, and activities. Activity diagram is a statechart diagram that shows the flow from activity to activity within a system. Component diagram shows the organization and dependencies among a set of components. Deployment diagram shows the configuration of processing nodes and the components that live on them.
213
operations paint()
214
215
import java.awt.Graphics; class HelloWorld extends java.applet.Applet { public void paint (Graphics g) { g.drawString ("Hello, World!", 10, 10); } } Example from: BJR
216
Class Diagram
Note that the Applet and Graphics classes are shown elided.
paint()
dependency Graphics
217
Packaging Classes
Graphics
awt
lang
219
Software Engineering
Lecture 12
Object-Oriented Design II
222
223
Modeling Classes
Given a real-life system, how do you decide what classes to use? What terms do the users and implementers use to describe the system? They are candidates for classes. Is each candidate class crisply defined? For each class, what is its set of responsibilities? Are the responsibilities evenly balanced among the classes? What attributes and operations does each class need to carry out its responsibilities?
224
Candidate Classes
Library Book Journal Copy ShortTermLoan LibraryMember Week MemberOfLibrary Item Time MemberOfStaff System Rule the name of the system
event measure repeat book or journal abstract term general term general term
227
is an is an is a copy of a
is a
LibraryMember
228
Operations
229
Class Diagram
MemberOfStaff LibraryMember
Book
231
Shipment
233
RetailStore
association 1 * Transaction
Payment Invoice
234
Modeling Invoice
Shipment
???
RetailStore invoiceRecord
goodsShipped
Lessons Learned
Design is empirical. There is no single correct design. During the design process: Eliding: Elements are hidden to simplify the diagram Incomplete: Elements may be missing. Inconsistency: The model may not be consistent The diagram is not the whole design. Diagrams must be backed up with specifications.
236
Levels of Abstraction
The complexity of a model depends on its level of abstraction: High-levels of abstraction show the overall system.
Low-levels of abstraction are needed for implementation. Two approaches: Model entire system at same level of abstraction, but present diagrams with different levels of detail. Model parts of system at different levels of abstraction.
237
Component Diagram
executable component hello.hml HelloWorld.class
hello.java
hello.jpg
238
BookBorrower
An actor is a user of a system in a particular role. An actor can be human or an external system.
Borrow book
A use case is a a task that an actor needs to perform with the help of the system.
239
240
Borrow copy of book BookBorrower Return copy of book Reserve book Extend loan
241
242
BookBorrower
Refuse loan
243
Intuitive -- easy to discuss with clients Use cases are often hard to translate into class models
244
Software Engineering
Lecture 13
Object-Oriented Design III
Comments on Presentations
Presentation Standard of graphics has been high Some text too small (diagrams, screen dumps) Content Level of detail Requirements v. design The client defines the requirements Well done, but time is short. What is your critical path?
246
Interaction diagrams: set of objects and their relationships including messages that may be dispatched among them Sequence diagrams: time ordering of messages Collaboration diagrams: structural organization of objects that send and receive messages Activity diagram: flow chart showing flow of control from activity to activity Statechart diagram: models a state machine
247
248
Client
Servers
Actions on Objects
returnCopy(c) call okToBorrow() return send create destroy status notifyReturn(b) <<create>> <<destroy>> stereotypes
249
local
asynchronous signal
Links
LibraryMember 1 Copy
on loan association
0..*
libMem:LibraryMember object
251
252
:Toolkit
:ComponentPeer
target:HelloWorld
callbackLoop
handleExpose paint
254
guard expression
Stream audio
256
State Diagram
returned()
not borrowable
guard expression
Implementation Modeling
Subsystem A grouping of elements that specifies what a part of a system should do. Component (UML definition) "A distributable piece of implementation of a system, including software code (source, binary, or executable) but also including business documents, etc., in a human system." A component can be thought of as an implementation of a subsystem.
258
Component Diagram
executable component hello.hml HelloWorld.class
hello.java
hello.jpg
259
agent.dll
AgentAction Policy
PatternSearch
260
Classes represent logical abstractions. Components represent physical things. Components may live on nodes. Classes have attributes and operations directly. Components have operations that are reachable only through interfaces.
262
Interfaces
render.java
realization
263
IModels
ILighting
264
Components allow system to be assembled from binary replaceable elements. A component is physical -- bits not concepts A component can be replaced by any other component(s) that conforms to the interfaces. A component is part of a system. A component provides the realization of a set of interfaces.
265
Software Engineering
Lecture 14
System Architecture I Data Intensive Systems
System Architecture
The overall design of a system: Computers and networks (e.g., monolithic, distributed) Interfaces and protocols (e.g., http, CORBA) Databases (e.g., relational, distributed) Security (e.g., smart card authentication, SSL) Operations (e.g., backup, archiving, audit trails) Software environments (e.g., languages, source control tools)
267
Examples Electricity utility customer billing Telephone company call recording and billing Car rental reservations (e.g., Hertz) Stock market brokerage (e.g., Charles Schwab) Web sales (e.g., Amazon.com)
268
Transaction
Data input
Master file
Bill
270
Transaction Types
Create account / close account Meter reading Payment received Other credits / debits Check cleared / check bounced Account query Correction of error etc., etc., etc.,
271
Typical Requirements
All payments to be credited on day received Customers must be able to query account by telephone Cutting off service for non-payment requires management authorization Data input staff should process n transactions per day per person Error rate must be below 0.01% System available 99.9% of business hours
272
errors Incoming transactions Data input read only Edit & validation Validated transactions
Master file
273
Reports
Bills
Instructions 274
All transactions for an account are processed together Backup and recovery have fixed checkpoints Better management control of operations Efficient use of staff and hardware
275
Online Inquiry
Customer service read only
Transactions
Data input
Master file
Bills
276
Transactions Received by mail or over telephone For immediate or later action Complex customer inquiries Highly competitive market
277
A Database Architecture
Database(s): Customer and account database Financial products (e.g., account types, pension plans, savings schemes) External databases (e.g., stock markets, mutual funds, insurance companies)
278
Database Architecture
External services
279
Real-time Transaction
Real-time transactions
External services
280
External services
281
Architectural considerations
Real-time service during scheduled hours + batch processing overnight Combine information from several databases Database consistency after any type of failure two-phase commit reload from checkpoint + log detailed audit trail How will transaction errors be avoided? How will transaction errors be corrected?
282
Each bank has a database with its customer accounts. The databases are used by staff at many branches and for back-office processing. The requirement is to integrate the two banks so that they appear to the customers to be a single organization and to provide integrated service from all branches.
283
???
???
284
Protocols point-to-point, multicast, broadcast message passing, RPC, distributed objects stateful or stateless Quality of service
286
Network Choices
Public Internet: Ubiquitous -- worldwide Low cost Private network: Security Predictable performance Choice of protocols (e.g., IBM's SNA)
287
Performance Maximum throughput Variations in throughput Real-time media (e.g., audio) Business Suppliers Trouble shooting and maintenance Upgrades
288
Firewall
Public network Firewall A firewall is a computer at the junction of two network segments that: Inspects every packet that attempts to cross the boundary Rejects any packet that does not satisfy certain criteria, e.g., an incoming request to open a TCP connection an unknown packet type
289
Private network
Software Engineering
Lecture 15
System Architecture II Distributed and Real Time Systems
Content Level of detail -- will be used to validate the implementation Requirements, not design Precise, but not legalistic
291
theBook:Book theCopy:Copy
294
User
296
Example: UseNet
297
Stateless protocol Example: http Open connection Send message Return reply Close connection State in http must be sent with every message (e.g., as parameter string or in a cookie)
298
Stateful (session) protocol Example: Z39.50 Open connection Begin session Interactive session End session Close connection Server remembers the results of previous transactions (e.g., authentication, partial results) until session is closed.
299
Firewall
Public network Firewall A firewall is a computer at the junction of two network segments that: Inspects every packet that attempts to cross the boundary Rejects any packet that does not satisfy certain criteria, e.g., an incoming request to open a TCP connection an unknown packet type
300
Private network
.edu server
302
A real time system is a software system whose correct functioning depends upon the results produced and the time at which they are produced. A soft real time system is degraded if the results are not produced within required time constraints A hard real time system fails if the results are not produced within required time constraints
304
The daemon listens at port 80. When a message arrives it: spawns a processes to handle the message returns to listening at port 80
305
Embedded Systems
Software and hardware are combined to provide an integrated unit, usually dedicated to a specific task: Digital telephone Automobile engine control GPS Scientific instruments
The software may be embedded in the device in a manner that can not be altered after manufacture.
306
GPS Steer Sonar Model Laser Sensors Signal processing Control signals Throttle Controls
307
Other Applications
Response critical Network router Telephone switch Seat bag controller
Techniques
Special purpose hardware Multi-threading and multi-tasking Parallel processing => digital signal processing
309
Multi-Threading
Several similar threads operating concurrently: Re-entrant code -- separation of pure code from data for each thread Testing -- single thread and multi thread
310
Schedules and dispatches tasks in a real time system Real time clock Interrupt handler Scheduler Resource manager Dispatcher
Timing
Timing mechanisms Synchronous (clocked) -- periodic stimuli Asynchronous -- wait for next signal
312
Hardware v. Software
Design of embedded systems requires close understanding of hardware characteristics Special purpose hardware requires special tools and expertise. Some functions may be implemented in either hardware of software (e.g., floating point unit) Design requires separation of functions
313
I/O Mulitplexor
Software Considerations
Resource considerations may dictate software design and implementation: Low level language (e.g., C) where programmer has close link to machine Inter-process communication may be too slow (e.g., C fork). May implement special buffering, etc., to control timings
315
Example: CD Controller
4 Input block 7 6 5
3 2
1 Output block
Circular buffer
316
Continuous Operation
Many systems must operate continuously Software update while operating Hardware monitoring and repair Alternative power supplies, networks, etc. Remote operation
Interoperation with third party devices Support for several versions of protocols Restart after total failure Defensive programming -- must survive => erroneous or malicious messages => extreme loads Time outs, dropped packets, etc. Evolution of network systems
318
Software Engineering
Lecture 15
System Architecture II Distributed and Real Time Systems
Administration
Assignment 2: Requirements Grades -- presentation, report, individual Comments at presentation Comments from teaching assistant
Assignment 3: Design
320
Content Level of detail -- will be used to validate the implementation Requirements, not design Precise, but not legalistic
321
theBook:Book theCopy:Copy
324
User
326
Example: UseNet
327
Stateless protocol Example: http Open connection Send message Return reply Close connection State in http must be sent with every message (e.g., as parameter string or in a cookie)
328
Stateful (session) protocol Example: Z39.50 Open connection Begin session Interactive session End session Close connection Server remembers the results of previous transactions (e.g., authentication, partial results) until session is closed.
329
Firewall
Public network Firewall A firewall is a computer at the junction of two network segments that: Inspects every packet that attempts to cross the boundary Rejects any packet that does not satisfy certain criteria, e.g., an incoming request to open a TCP connection an unknown packet type
330
Private network
.edu server
332
A real time system is a software system whose correct functioning depends upon the results produced and the time at which they are produced. A soft real time system is degraded if the results are not produced within required time constraints A hard real time system fails if the results are not produced within required time constraints
334
The daemon listens at port 80. When a message arrives it: spawns a processes to handle the message returns to listening at port 80
335
Embedded Systems
Software and hardware are combined to provide an integrated unit, usually dedicated to a specific task: Digital telephone Automobile engine control GPS Scientific instruments
The software may be embedded in the device in a manner that can not be altered after manufacture.
336
GPS Steer Sonar Model Laser Sensors Signal processing Control signals Throttle Controls
337
Other Applications
Response critical Network router Telephone switch Seat bag controller
Techniques
Special purpose hardware Multi-threading and multi-tasking Parallel processing => digital signal processing
339
Multi-Threading
Several similar threads operating concurrently: Re-entrant code -- separation of pure code from data for each thread Testing -- single thread and multi thread
340
Schedules and dispatches tasks in a real time system Real time clock Interrupt handler Scheduler Resource manager Dispatcher
Timing
Timing mechanisms Synchronous (clocked) -- periodic stimuli Asynchronous -- wait for next signal
342
Hardware v. Software
Design of embedded systems requires close understanding of hardware characteristics Special purpose hardware requires special tools and expertise. Some functions may be implemented in either hardware of software (e.g., floating point unit) Design requires separation of functions
343
I/O Mulitplexor
Software Considerations
Resource considerations may dictate software design and implementation: Low level language (e.g., C) where programmer has close link to machine Inter-process communication may be too slow (e.g., C fork). May implement special buffering, etc., to control timings
345
Example: CD Controller
4 Input block 7 6 5
3 2
1 Output block
Circular buffer
346
Continuous Operation
Many systems must operate continuously Software update while operating Hardware monitoring and repair Alternative power supplies, networks, etc. Remote operation
Interoperation with third party devices Support for several versions of protocols Restart after total failure Defensive programming -- must survive => erroneous or malicious messages => extreme loads Time outs, dropped packets, etc. Evolution of network systems
348
Software Engineering
Lecture 16
System Architecture III Distributed Objects
Resource considerations may dictate software design and implementation: Low level language (e.g., C) where programmer has close link to machine Inter-process communication may be too slow (e.g., C fork). May implement special buffering, etc., to control timings
350
4 Input block 7 6 5
3 2
1 Output block
Circular buffer
351
Continuous Operation
Many systems must operate continuously Software update while operating Hardware monitoring and repair Alternative power supplies, networks, etc. Remote operation
messages
Transaction monitor
processes
A transaction monitor: monitors transactions, routes them across services, balances the load, restarts transactions after failure.
354
Package supports a standard application (e.g., payroll, user interface to Internet information, mathematical algorithms) Functionality can be enhanced by: => configuration parameters (e.g., table driven) => extensibility at defined interfaces => custom written source code extensions
355
An object is a piece of code that owns attributes and provides services through methods. The methods operate on instance data owned by the object. A class is a collection of like objects.
357
358
Objects on separate computers interact through method calls and instance data. Major systems: CORBA (Common Object Request Broker Architecture) Microsoft family: OLE, COM, DCOM, Active X ...
360
361
A research project to explore extensibility: -- very simple Interface Definition Language -- powerful tools for extensions -- interoperability, Cornell and CNRI http://www.cs.cornell.edu/cdlrg/fedora.html
362
C IDL
Cobol Objects
Java
Other IDL
Server
Define a class
Define a class
364
[<op_type] <identifier>(<parameters>) Define a method [raises exception] [context]; .... [<op_type] <identifier>(<parameters>) Define a method [raises exception] [context]; .... }
365
Self-describing system Local/remote transparency Inter-ORB protocols Internet Inter-ORB Protocol (IIOP)
367
Interface repository
Client
CORBA Services
Naming service Event service Concurrency control service Transaction service Relationship service Externalization service Query service Life cycle service Persistence service Licensing service Properties service Security service Time service
369
Development environments change with time. Language bindings and IIOP permit changes. Production environments changes with time. Code can be reused in different environments.
370
Software Engineering
Lecture 17
Design for Usability I
Q2: Finite State Machine The cruise control system on an automobile is controlled by a master switch and three buttons. Initially, it is turned on by the master switch. The master switch can be turned off at any time. When first turned on, the system enters stand-by mode. When the system is in stand-by mode, the driver of the automobile can press Button A to engage the cruise control at the current speed of the automobile. When the cruise control is engaged, if the driver presses the brake or presses Button B the system will be disengaged and return to stand-by mode. After returning to standby mode, the driver can press Button C to engage the cruise control at the speed that it was set at previously. (After the system is first turned on, Button C has no effect.) When the cruise control is engaged, the driver can press Button A to increase speed by one mile per hour or Button C to decrease 372 speed by one mile per hour.
Standby1
MS off
B Brake
Question 4
When software is written, who owns the copyright?
375
Question 4
When software is written, who owns the copyright? The person who writes the software Except work for hire -- the employer How can somebody else be permitted to use the software? By permission from the copyright owner (usually a license) How can copyright be transferred to somebody else? Copyright is property that can be sold or given away (usually a contract)
376
Question 4
You are employed for company X writing software. When you leave, who owns your work?
377
Question 4
You are employed for company X writing software. When you leave, who owns your work? The company (work for hire) What use can you make of the work? None without permission of the copyright owner
378
Question 4
You work free-lance for company X. When you finish, who owns your work?
379
Question 4
You work free-lance for company X. When you finish, who owns your work? It depends on the circumstances Have a written contract What use can you make of the work? If you hold the copyright -- unrestricted Otherwise -- none without agreement
380
Development environments change with time. Language bindings and IIOP permit changes. Production environments changes with time. Code can be reused in different environments.
381
Iterative Design
Evaluation Requirements
Implementation (prototype)
Design
383
Levels of Usability
interface design conceptual model functional design data and metadata computer systems and networks
385
The conceptual model is the user's internal model of what the system provides: The desk top metaphor -- files and folders The web model -- click on hyperlinks The library model -- search and retrieve The form filling model -- fill form, submit Example: The Mercury page turner
386
Interface Design
The interface design is the appearance on the screen and the actual manipulation by the user Fonts, colors, logos, key board controls, menus, buttons Mouse control or keyboard control? Conventions (e.g., "back", "help") Example: Screen space utilization in the Mercury page turner
387
Disabilities
What if the user: is visually impaired or color blind? does not speak English? is a poor typist? There is a tradition of blind programmers Navigation of web sites need not be only visual You may have a legal requirement to support people with disabilities
389
Functional Design
The functional design, determines the functions that are offered to the user Selection of parts of a digital object Searching a list or sorting the results Help information Manipulation of objects on a screen Pan or zoom
390
Example: The desk top metaphor Mouse -- 1 button (Macintosh), 2 button (Windows) or 3 button (Unix) Close button -- left of window (Macintosh) right of window (Windows)
391
Client computers and network connections vary greatly in capacity Client software may run on various operating systems; it may be current or an earlier version System designers wish to control clients; users wish to configure their own environments
394
Software Engineering
Lecture 18
Design for Usability II
A system generates weather maps using data collected from unattended weather stations. Each weather station collects meteorological data and produces summaries of the data. On request, it sends the summary information to an area computer. The area computer uses a database of digitized maps to generate a set of local weather maps.
398
A system generates weather maps using data collected from unattended weather stations. Each weather station collects meteorological data and produces summaries of the data. On request, it sends the summary information to an area computer. The area computer uses a database of digitized maps to generate a set of local weather maps.
400
Candidate Classes
Library Book Journal Copy ShortTermLoan LibraryMember Week MemberOfLibrary Item Time MemberOfStaff System Rule the name of the system
event measure repeat book or journal abstract term general term general term
401
general term
404
Class Diagram
MemberOfStaff LibraryMember
Book
1...*
Direct Interaction
User interacts with computer by manipulating objects on screen Can be intuitive and easy to learn Users get immediate feedback Not suitable for complex interactions Does not require typing skills Straightforward for casual users, slow for skilled users Icons can be language-independent Difficult to build scripts Only suitable for human users
408
Conceptual models, metaphors, icons => there may not be an intuitive model Navigation around large space
Conventions are growing over the years => scroll bars, buttons, help systems, sliders => good for users, good for designers
409
Menus
Easy for users to learn and use Certain categories of error are avoided Enables context-sensitive help Major difficulty is structure of large menus Scrolling menus (e.g., states of USA) Hierarchical Associated control panels Menus plus command line Users prefer broad and shallow to deep menu systems
410
Information Presentation
Information to be displayed Presentation software
Display
411
Information Presentation
Text precise, unambiguous fast to compute and transmit Graphics simple to comprehend uses of color shows variations
412
Shared systems have unpredictable performance Data validation often requires access to shared data
414
Costs are multiplied if a user interface has to be used on different computers or migrate to different versions of systems Web browsers provide a general purpose user interface that others maintain
415
Web servers
Scripts can configure pages Scripts can validate information All interaction requires communication with server
418
JavaScripts can validate information as typed Some interactions are local Server interaction constrained by web protocols
419
Any executable code can run on client Client can connect to any server
420
Software Engineering
Lecture 19
Performance of Computer Systems
Moore's Law
Original version: The density of transistors in an integrated circuit will double every year. (Gordon Moore, Intel, 1965) Current version: Cost/performance of silicon chips doubles every 18 months.
424
Design system: Production use: Withdrawn from production: Processor speeds: Memory sizes: Disk capacity: System cost:
425
426
Parkinson's Law
Original: Work expands to fill the time available. (C. Northcote Parkinson) Planning assumptions: (a) Demand will expand to use all the hardware available. (b) Low prices will create new demands. (c) Your software will be used on equipment that you have not envisioned.
427
False Assumptions
Unix file system will never exceed 2 Gbytes (232 bytes). AppleTalk networks will never have more than 256 hosts (28 bits). GPS software will not last 1024 weeks. Nobody at Dartmouth will ever earn more than $10,000 per month. etc., etc., .....
428
1965
2000?
When?
429
Mathematical models Simulation Direct measurement All require detailed understanding of the interaction between software and systems.
430
Queues
arrive
wait in line
service
depart
431
Queues
service arrive wait in line depart
Multi-server queue
432
Mathematical Models
Queueing theory Good estimates of congestion can be made for singleserver queues with: arrivals that are independent, random events (Poisson process) service times that follow families of distributions (e.g., negative exponential, gamma) Many of the results can be extended to multi-server queues.
433
mean service time utilization = mean inter-arrival time When the utilization of any system component exceeds 30%, be prepared for congestion.
434
mean delay
utilization 0 1
435
Simulation
Model the system as set of states and events advance simulated time determine which events occurred update state and event list repeat Discrete time simulation: Time is advanced in fixed steps (e.g., 1 millisecond) Next event simulation: Time is advanced to next event Events can be simulated by random variables (e.g., arrival of next customer, completion of disk latency) 436
Timescale
Operations per second CPU instruction: Disk latency: read: Network LAN: dial-up modem: 400,000,000 60 25,000,000 bytes 10,000,000 bytes 6,000 bytes
437
Benchmarks: Run system on standard problem sets, sample inputs, or a simulated load on the system. Instrumentation: Clock specific events.
438
Single thread v. multi-thread e.g., Unix fork Granularity of locks on data e.g., record locking Network congestion e.g., back-off algorithms
439
Each transaction must: wait for specific disk platter wait for I/O channel signal to move heads on disk platter wait for I/O channel pause for disk rotation read data Close agreement between: results from queueing theory, simulation, and direct measurement (within 15%).
440
Software Engineering
Coding Standards
Or How to Pound all of your odd-shaped programmers into a one size fits all hole
444
445
446
447
448
Prime Directive
Document every time you violate a standard. No standard is perfect for every application, but failure to comply with your standards requires a comment
449
450
451
Accessors
use getVar() and setVar() functions on all class variable unless class is being used solely as a data structure (OOP) Member Functions Documentation What and why member function does what it does Parameters / return value How function modifies object Preconditions /Postconditions Concurrency issues Restrictions Internal Documentation Control Structures Why as well as what the code does Difficult or complex code Processing order
452
Three Rules
Coding standards neednt be onerous - find a standard that works for your team. Standardize early - the effort to bring your old work into the standard will be too great otherwise. Encourage a culture where standards are followed.
453
454
Software Engineering
Lecture 21
Dependable Systems I Reliability
Software Reliability
Failure: Software does not deliver the service expected by the user (e.g., mistake in requirements) Fault: Programming or design error whereby the delivered system does not conform to specification Reliability: Probability of a failure occurring in operational use. Perceived reliability: Depends upon: user behavior set of inputs pain of failure
456
Reliability Metrics
Probability of failure on demand Rate of failure occurrence (failure intensity) Mean time between failures Availability (up time) Mean time to repair Distribution of failures
Hypothetical example: Cars are safer than airplane in accidents (failures) per hour, but less safe in failures per mile.
457
Traditional metrics are hard to apply in multi-component systems: In a big network, at a given moment something will be giving trouble, but very few users will see it. A system that has excellent average reliability may give terrible service to certain users.
There are so many components that system administrators rely on automatic reporting systems to identify problem areas.
458
1. A personal computer that crashes frequently v. a machine that is out of service for two days. 2. A database system that crashes frequently but comes back quickly with no loss of data v. a system that fails once in three years but data has to be restored from backup. 3. A system that does not fail but has unpredictable periods when it runs very slowly.
459
Permanent System fails to operate non-corrupting with any card -- reboot Transient System can not read non-corrupting an undamaged card Corrupting A pattern of transactions corrupts database
461
The human mind can encompass only limited complexity: => Comprehensibility => Simplicity => Partitioning of complexity
462
High-quality has to be built-in => Each stage of development must be done well => Testing and correction does not lead to quality => Changes should be incorporated into the structure
463
Assumption: Good processes lead to good software The importance of routine: Standard terminology (requirements, specification, design, etc.) Software standards (naming conventions, etc.) Internal and external documentation Reporting procedures
464
Change management: Source code management and version control Tracking of change requests and bug reports Procedures for changing requirements specifications, designs and other documentation Release control
465
Colleagues review each other's work: can be applied to any stage of software development can be formal or informal The developer provides colleagues with: documentation (e.g., specification or design), or code listing talks through the work while answering questions Most effective when developer and reviewers prepare well
466
Statistical Testing
Select or generate a profile of test data Apply test data to system, record failure patterns
469
Statistical Testing
Advantages: Can test with very large numbers of transactions Can test with extreme cases (high loads, restarts, disruptions) Can repeat after system modifications Disadvantages: Uncertainty in operational profile (unlikely inputs) Expensive Can never prove high reliability
470
Step 3. Invest resources where benefit will be maximum, e.g., Orderly shut down after power failure
Priority order for software improvements Changed procedures for operators Replacement hardware
473
Approach to software design and implementation that hides complexity (e.g., structured design, object-oriented programming) Use of software tools that restrict or detect errors (e.g., strongly typed languages, source control systems, debuggers) Programming style that emphasizes simplicity, readability, and avoidance of dangerous constructs Incremental validation
474
Error Avoidance
Risky programming constructs Pointers Dynamic memory allocation Floating-point numbers Parallelism Recursion Interrupts
All are valuable in certain circumstances, but should be used with discretion
475
Defensive Programming
Murphy's Law: If anything can go wrong, it will.
Defensive Programming: Redundant code is incorporated to check system state after modifications Implicit assumptions are tested explicitly
476
Build debugging code into program with a switch to display values at interfaces Error checking codes in data, e.g., checksum or hash
477
Software Engineering
Lecture 22
Dependable Systems II Validation and Verification
Defensive Programming
Murphy's Law: If anything can go wrong, it will.
Defensive Programming: Redundant code is incorporated to check system state after modifications Implicit assumptions are tested explicitly
480
Build debugging code into program with a switch to display values at interfaces Error checking codes in data, e.g., checksum or hash
481
Terminology
Fault avoidance Build systems with the objective of creating faultfree systems Fault tolerance Build systems that continue to operate when faults occur Fault detection (testing and validation) Detect faults before the system is put into operation.
482
Fault Tolerance
Basic Techniques: After error continue with next transaction Timers and timeout in networked systems Error correcting codes in data Bad block tables on disk drives Forward and backward pointers Report all errors for quality control
483
Fault Tolerance
Backward Recovery: Record system state at specific events (checkpoints). After failure, recreate state at last checkpoint. Combine checkpoints with system log that allows transactions from last checkpoint to be repeated automatically.
484
Fault Tolerance
Fault recovery Fault repair N-version programming -- Execute independent implementation in parallel, compare results, accept the most probable.
485
Validation: Are we building the right product? Verification: Are we building the product right? In practice, it is sometimes difficult to distinguish between the two. That's not a bug. That's a feature!
486
It is always better to prevent defects than to remove them later. Example: The four color problem.
487
Static verification: Techniques of verification that do not include execution of the software. May be manual or use computer tools. Dynamic verification Testing the software with trial data. Debugging to remove errors.
488
Carried out throughout the software development process. Validation & verification
Requirements specification
Design
Program
489
Requires team commitment, e.g., trained leaders So effective that it can replace unit testing
490
Cross-reference table: Shows every use of a variable, procedure, object, etc. Information flow analysis: Identifies input variables on which an output depends. Path analysis: Identifies all possible paths through the program.
493
Test Design
Testing can never prove that a system is correct. It can only show that (a) a system is correct in a special case, or (b) that it has a fault. The objective of testing is to find faults. Testing is never comprehensive. Testing is expensive.
494
System and sub-system testing uses trial data emphasis is on integration and interfaces Acceptance testing uses real data in realistic situations emphasis is on meeting requirements
495
Acceptance Testing
Alpha Testing: Clients operate the system in a realistic but non-production environment Beta Testing: Clients operate the system in a carefully monitored production environment Parallel Testing: Clients operate new system alongside old production system with same data and compare results
496
Management and client reports are important parts of testing. What is the definition of "done"?
497
Testing Strategies
Bottom-up testing. Each unit is tested with its own test environment. Top-down testing. Large components are tested with dummy stubs. user interfaces work-flow client and management demonstrations Stress testing. Tests the system at and beyond its limits. real-time systems transaction processing
498
Test Cases
Test cases are specific tests that are chosen because they are likely to find faults. Test cases are chosen to balance expense against chance of finding serious faults. Cases chosen by the development team are effective in testing known vulnerable areas. Cases chosen by experienced outsiders and clients will be effective in finding gaps left by the developers. Cases chosen by inexperienced users will find other faults.
499
500
if-then-else
loop-while
502
Fixing Bugs
Isolate the bug Intermittent --> repeatable Complex example --> simple example Understand the bug Root cause Dependencies Structural interactions
503
Bug fixes need static and dynamic testing Repeat all tests that have the slightest relevance (regression testing) Bugs have a habit of returning! When a bug is fixed, add the failure case to the test suite for the future.
504
Regression Testing
Applied to modified software to provide confidence that modifications behave as intended and do not adversely affect the behavior of unmodified code. Basic technique is to repeat entire testing process after every change, however small.
505
506
Lecture Caveats
I am not a lawyer and do not have any formal legal training This lecture is made up of my observations of the legal system to make you aware of important issues concentrating on an information technology workplace Cardinal Rule: Be aware of the law, but always consult an attorney if/when you become involved with it.
510
Law Caveats
Some people read the text of the law and think they know it. Things are never so easy. If you have questions ask a lawyer. Others ignore the law relying on corporate lawyers in case something goes wrong. This is not a good idea. As in any other system, catching problems in the design phase is always better than in the debugging phase.
511
Talk Overview
Life for Lawyers Vs. Life for Engineers Patents, copyright, trademarks, trade secrets reviewed Defamation ISP Liability Privacy Jurisdiction Issues
512
Problem solutions either work or not. Little room for gray areas.
Physical and mathematical laws ultimate authority when disputes arise Guiding Philosophy - Tell me what you need and I will create a system with appropriate trade-offs at least cost to solve your problem.
513
514
515
Patents
Embodiment of a specific methodology Competing products must use different method for achieving same task to avoid payments Definite lifespan beyond which patent information freely available for use by the public
516
Copyright
Specific work Automatically held when work is created, but easier to defend if it is registered Definite lifetime beyond which the work is freely available to the public
517
Trademark
Specific name or phrase Generic terms cannot be trademarked Trademarks can be lost if they are not defended
518
Trade Secrets
Does not expire - as long as it is kept secret Competitors may not use secrets obtained through extraordinary means Example: Walled chemical plant layout learned through helicopter use
519
Defamation
Publishing damaging statements you cannot prove about others The publisher and author are both liable Slander is a less serious, but similar, crime where damaging statements that cannot be proven are made in a public arena
520
521
522
ISP Liability
What is an Internet Service Provider Like?
Phone Company: Route information flows between individuals Newspaper: Package content for distribution in a public forum
Answer determines ISPs legal liability The rules have been in a constant state of flux in recent years
523
Prodigy a large ISP Claimed to be family friendly. Prodigy advertised that internal newsgroups monitored for bad/inappropriate language Role of a publisher - hence, Prodigy like a newspaper CompuServe did not monitor users activity - like a telephone company (Cubby Inc. Vs. CompuServe Inc. in 1991)
524
Area for complaints must be available Complaint response must happen in a timely fashion
525
DMCA
Digital Millennium Copyright Act If a copyright infringement is claimed a web site must be taken down (however tenuous the claim may be) Web site can only be reinstated after an appeals process.
526
Near Future? . . .
European Computer Crime Treaty may be created by the end of this year ISPs may be required to monitor user traffic with a 40 day data-log. ISPs not explicitly exempt from liability Hacker/Security Tools Illegal Citizens must provide passwords for data seized by police
527
528
Privacy in E-mail
Legally, e-mail is like a postal letter
Expectation of privacy in transit Mail loses its special protected status once it leaves the letter carrier's grasp
For e-mail,
Expectation of privacy while signal travels over Internet E-mail loses its protected status at the mail server whether you have read it or not
529
530
Business E-mail
Electronic Communications Privacy Act (1986) says all business communication belongs to that business Deleting e-mail can be ruled spoliation (intentionally destroying company records) Archive worthless if it cannot be indexed effectively (in effect, saving everything can be equivalent to saving nothing)
531
532
Data Collection
Data collection has few boundaries in U.S. Check privacy policy (can change!!) EU Safe Harbor agreement may change things in the future (TRUSTe web site privacy seal program)
533
Jurisdiction
The Internet has no boundaries Is that really true? If you break a law in Finland, but you were on the Internet in the United States, what happens to you? What if you are in California and you break a law in Minnesota?
534
535
Obscene or Offensive?
Indecent speech and offensive speech protected under the 1st Amendment Obscene speech is not But what is obscene speech?
536
537
538
US V. Thomas (1994)
Mr. And Mrs. Thomas ran a pornographic BBS in California State officer paid a membership fee and downloaded pornography in Tennessee Couple tried in Federal court in Tennessee and lost their case
539
International Jurisdiction
Extradition over civil suits unlikely Big Question #1: Do you have assets in the country in question? Big Question #2: Will you ever try to enter country X?
540
541
542
543
Conclusions . . .
The law is constantly changing and never as simple as it seems You should try to be familiar with the law to protect yourself (corporate lawyers are like a fire department, not like a seeing eye dog) Even so, you DO need the help of someone with formal training when dealing with legal issues
544
Software Engineering
Lecture 25
Management III Managing People
Administration
Return of laptops and wireless cards -> Dates for return will be announced on "Notices" -> All equipment must be returned before the examination. Bring the receipt to the exam. -> If an extension granted, (e.g., independent research) must return and be issued again If any repairs needed, please swap for replacement since warranty runs out on December 15.
546
Administration
Early examination December 7, 10:00 to 11:30, Upson 5130 Send email to rosemary@cs.cornell.edu if you plan to take the early examination, by December 5 All laptops and wireless cards must be returned before the examination
547
Managing People
Theoretical: Organizational behavior Industrial psychology Group behavior Cognitive fundamentals Economic motivation
548
Self-realization needs Esteem needs Social needs Safety needs Physiological needs
549
551
Group Working
552
Communication
Informal Kitchen, smokers' doorway, after work, etc. Walkabout (tours) Ad hoc meetings Staff meetings (non-technical) Example: Tektronics Technical meetings Facilitation Record of decisions
553
554
Hiring Criteria
Productivity is a combination of: Analytic ability Verbal ability and communication skills Education Application domain knowledge
Adaptability and inquisitiveness Personality and attitude Platform experience Programming language experience
555
Staff Retention
Technically interesting work up to date hardware and software opportunities to learn and experiment Feeling of appreciation management recognition money and promotion Working conditions space, light, noise, parking flexibility Organizational dynamics
556
Firmness
Managers must be firm when needed: Assignment of tasks must be equitable and open; everybody will have to tackle some of the dreary tasks Carrots are better than sticks, but poor performance must be addressed. Nobody is indispensable; nobody should be allowed to think that they are indispensable
557
Technical Challenges
Canceling projects Example: the Andrew window manager Changes of environment Example: the World Wide Web Technical tinkering v. needed re-engineering
558
Resignations and terminations Respect people who try, yet refuse to accept problem areas Brutal and abrupt rarely equals persistent and firm
559
How to be Led
As a junior member of a team, what can you do to make it productive?
560
Software Engineering
Lecture 26
Risk in Software Engineering
Managing Risk
Manage projects to avoid risk: Open and visible software process => Avoid surprises Continual review of requirements Willingness to modify or cancel projects
563
Canceling a Project
Example: Andrew Window Manager (wm) Technically superior to X (MIT's Athena project) in 1986 but ... Digital Equipment Corporation turning X into a product with massive support nobody ready to support wm Therefore wm cancelled in 1986, Andrew user interface and applications ported to X
564
567
569
A Sense of Urgency
Example: A not-for-profit corporation is developing a system for a government organization. By 1996 all research had been completed and the system demonstrated successfully with real users. In 2000, the system is still not in full production Reasons: => Incremental improvements to the software => Repeated requests for more functionality => Reluctance to reorganize clerical staff Nobody had a sense of urgency
570
Overtaken by Events
Example: University C has a project to develop a digital library system, with funds from Company Z , private foundations and the government. By 1993 an extensive system is running at the university and Z is marketing the technology to its customers. By 1994 it is clear that web browsers and web formats (though technically weak) are becoming widely adopted. => What should the university do? => What should the company do?
571
Design decisions made in 1994 had to be changed. Software was rewritten and greatly improved in 1998/9. If a job's worth doing, it's worth doing twice!
572
Changes of Leadership
Many projects are wasted because of management changes Example: In 1988, Company W gave University D $1,000,000 to port a new operating system to its personal computers. The work was well done, on time. Company W changed its president and senior technical staff during the project. The work was wasted. A decade later and several presidents later, Company W is releasing a modern version of the same operating system. A graduate student from University D is now Senior Vice President of Company W!
573
Client Oversight
When work is out-sourced, the client must be vigilant. Example: Company G was the world's leader in software for optimization (e.g., linear and integer programming). G had implemented several packages for various manufacturers. An operating system Company H contracted with G to develop an optimization package for its new operating system. The package was late, performed badly and disliked by customers. What went wrong? What can we learn?
574
Too Difficult!
Example: A development team at University E was given government funds to build a high-performance gateway from protocol x to protocol y. A promising young developer was hired and assigned to this task The project was too difficult for him, but he hid his problems for many months. The project produced nothing of value. What can we learn from this experience?
575
Sobering Thoughts
Major computing projects are very complex. Inevitably there are delays and failures. Few organizations know how to manage risk & uncertainty. The best CIO's => Manage to minimize risk => Have the confidence of their staff who keep them truthfully informed => Have the self-confidence to keep their seniors truthfully informed
579
Software Engineering
Lecture 27
Software Engineering as Engineering
In 1967 memory cost $1 per byte The Air Force used single digit dates If 2-digit dates saved 1% of memory... savings over 20 years $16 to $24 million per gigabyte
581
582
584
Computers fail everyday. What's special about this bug? What if they all fail at the same time? What if we lose telephone, electricity, radio, etc.?
585
Social Consequences
Worry creates its own problems: Wal-Mart forecast lower profits in Q1 2000
Legislation to limit law suits Opportunities for computer fraud and sabotage
Trading partners
586
Organizational Procedures
Ostrich => do nothing => buy insurance
Bureaucratic => fill in forms that programs are compliant Subcontract => hire Y2K specialists Do it yourself => in-house computing department
587
Y2K Validation
Request from Library of Congress to confirm that our code is Y2K compliant: Our code is fine .... but it depends on ... which depends on ... Yes. Our code is fine. Request from DARPA to confirm that our code is Y2K compliant: It's been validated by another part of the US government Thank you!
588
Technical Strategies
Replace noncompliant applications with compliant ones (e.g., new versions of packages) Repair noncompliant applications (e.g., in-house applications) Terminate noncompliant programs on an as-needed basis Mask the data exchange between applications Object code interception
589
New Bugs
If it's not broke don't fix it. 10 billion lines of code checked (often automatically) 10 million new bugs introduced accidentally ?? security holes, errors, etc. introduced accidentally or deliberately
590
Replace the old package Sell something to your customers What boss will turn turn a request for Y2K funds? What systems administrators will not install Y2K upgrades?
591
Profiteering
Buy gold, wood stoves, bottled water Y2K specialists Pundits, consultants, writers Religious cranks
592
We create computer systems that are more complex than our understanding of them: We over estimate our ability to validate systems We under estimate our ability to adapt and respond Software engineering usually thinks of systems as independent. Will the long-term benefit of the Y2K problem be a greater understanding of how software systems interact with each other and with our social systems?
593
Software as a product: => Awkward to use => Full of errors => No chance to try it out => No guarantees Not much of a product
594
What is Engineering?
595
What is Engineering?
The profession of: ... creating cost-effective solutions ... ... to practical problems ... ... by applying scientific knowledge ... ... and established practices ... ... building things ... and taking responsibility for them!
596
597
Craft
598
Part craft -- part engineering Embryonic scientific basis Evolving body of knowledge Too much flux for the apparatus of a profession (e.g., accreditation) Example: Texas and the ACM
599
The End
Good process leads to good software: the limits of heroic efforts Minimize risks: visible process function v. time v. cost The importance of people Requirements, requirements, requirements!
600