The IEEE has two definitions for “Quality”: ‫ בודקים‬, ‫קבוצת העבודה בסקראם מכילה מפתחים‬ Structured programming-Programming
ים‬ Structured programming-Programming paradigm aimed on
The degree to which a system, component, or process
meets specified requirements
The degree to which a system, component, or process meets
customer or user needs or expectations
The ISO (ISO 8402) defines “Quality” as:
The totality of features and characteristics of a product or
service that bear on its ability to meet stated or implied
needs
Philip B. Crosby defined “Quality as:
We define quality as conformance to requirements.
Requirements must be clearly stated. Measurements
determine conformance….. Nonconformance detected is the
absence of quality
Product view: the quality of a product is measurable in
objective manner;
User view: quality is fitness for use;
Manufacturing view: quality is the result of the right
development of the product;
Value-based view (Economic): quality is a function of costs
and benefits
Verification of a product shows proof of compliance with
requirements—that the product can meet each “shall”
statement as proven though performance of a test, analysis,
inspection, or demonstration (or combination of these).
Validation of a product shows that the product
accomplishes the intended purpose in the intended
environment—that it meets the expectations of the
customer and other stakeholders as shown through
performance of a test, analysis, inspection, or
demonstration.
Software quality (Definitions)
(1) The degree to which a system, component, or process
meets specified requirements. [IEEE_Std_610.12-1990]
(2) The degree to which a system, component, or process
meets customer or user needs or expectations.
[IEEE_Std_610.12-1990]
(3) Conformance to explicitly stated functional and
performance requirements, explicitly documented
development standards, and implicit characteristics that are
expected of all professionally developed software.
[Roger Pressman. Software Engineering: A Practitioner's
Approach. McGraw-Hill, 6 ed., 2004]
(4) Set of systematic activities providing evidence of the
ability of the software process to produce a software
product that is fit to use
Constraints of the iron triangle
Scope is the work to be done – such as features and
functionalities – to deliver a working product.
Resources include budget and team members working to
deliver and execute.
Time is when teams will deliver to the market such as
releases and milestones.
How to use it
Assess all changes, risks and issues against the triangle and
weigh up your course of action against the impact on your
critical objective. For example if the key project constraint is
cost, only the most business critical change requests are
likely to be approved. However, if quality is the biggest goal
time and cost might move to accommodate enhancement
requests
Application lifecycle management
‫ההיבט הניהולי‬Governance - ‫ מטרת ההיבט היא להבטיח‬-
‫ ההיבט‬.‫שהתוכנה תספק את צרכי הארגון שאליו נועדה התוכנה‬
‫הניהולי הוא ההיבט היחיד שמתפרש לאורך כל מחזור החיים של‬
‫ הוא ההיבט החשוב ביותר במחזור החיים‬,‫ במובנים רבים‬.‫התוכנה‬
.‫של תוכנה‬
- ‫ההיבט הפיתוחי‬Development ‫ההיבט אחראי על פיתוח‬
‫ זה מתחיל קצת אחרי הגאית הרעיון וממשיך בתחזוקה‬,‫התוכנה‬
‫ גם אחרי שלב ה‬.‫של התוכנה גם אחרי כניסתה לשימוש‬
Deployment, ‫ גרסאות‬,‫יש צורך בהמשך פיתוח התוכנית‬
‫ עדכון גרסה יכול לעלות יותר‬,‫ בתוכנות מסוימות‬.‫ עדכונים‬,‫חדשות‬
‫ לכן יש לקחת בחשבון את העלות לעומת‬.‫מתועלת התוכנה לארגון‬
.‫התועלת‬
- ‫ההיבט התפקודי‬Operations ‫ל יישום פעיל חייב להיות נתון‬
‫ היבט זה מתחיל קצת לפני שלב ה‬.‫לניטור ולניהול‬Deployment
.‫ומתפרש עד שלב המוות‬
Software development life cycle - ‫עקרונות‬
,‫ משתמ שים‬,‫ סיכונים‬,‫פרק הקונספט – מגבלות‬
‫רשימת מכולת‬
‫פרק הדרישות – פרק המה מסמך דרישות‬
‫ פרק האיך (כולל‬- ‫עיצוב‬/‫(פרק תכנון‬DFD ‫נעשה‬
‫בד"כ בשני שלבים או יותר – חלק גדול מהקורס‬
.‫יוקדש לנושא‬
‫פרק הקידוד – נעסוק רק באיך עושים את זה יותר‬
‫נקי משגיאות‬/‫טוב‬
‫ או ביידיש‬- ‫ בדיקות בפיתוח‬Unit test ‫מי‬
.‫שמתחמק אומר יעשו זאת המפתחים‬
‫אינטגרציה – הפרק הנעלם שלוקח חלק ענק‬
‫מהמאמצים‬
‫בדיקות מערכת – מה שהארגון עושה על מנת‬
‫שיוכל למכור את התוכנה‬
‫בדיקות קבלה – מה שהלקוח עושה על‬
‫מנת לקנות את התוכנה‬
‫התקנה אצל לקוח – מה שהארגון והלקוח‬
‫עושים בשביל לממש‬
‫תחזוקה ועדכונים – על זה עושים את‬
.‫הכסף‬
‫מחזור חיי הפיתוח של תוכנה‬
1.the specification, design, development and testing a
software application. It covers the entire lifecycle from idea
conception through the development, testing, deployment,
support and ultimately retirement of systems.
2. An umbrella term that covers several different disciplines
that traditionaly were considered separately, including ….
The Linear Sequential model It is the oldest process models
in SE history Sometimes called the classic life cycle or the
waterfall model, the linear sequential model suggests a
systematic, sequential approach to software development
Prototypes development models The Prototyping Model is a
systems development method (SDM) in which a prototype
(an early approximation of a final system or product) is built,
tested, and then reworked as necessary until an acceptable
prototype is finally achieved from which the complete
system or product can now be developed. This model works
best in scenarios where not all of the project requirements
are known in detail ahead of time. It is an iterative, trial-and-
error process that takes place between the developers and
the users. Step1 – determine objectives, step 2 – Identify
Risks, Step 3 – development and testing, Step 4 – Plan next
iteration.
‫הצגות אלטרנטיביות למודל העקרוני‬
The basic idea in Prototype model is that instead of freezing
the requirements before a design or coding can proceed, a
throwaway prototype is built to understand the
requirements. This prototype is developed based on the
currently known requirements. By using this prototype, the
client can get an “actual feel” of the system, since the
interactions with prototype can enable the client to better
understand the requirements of the desired system.
Prototyping is an attractive idea for complicated and large
systems for which there is no manual process or existing
system to help determining the requirements.
The prototype are usually not complete systems and many
of the details are not built in the prototype. The goal is to
provide a system with overall functionality.
Start: requirement gathering-quick design-building
prototype-customer evaluation-refining prototype- (loop to
quick design) – Engineer product – stop.
Prototypes development models (spiral)
Evolutionary prototyping also called as breadboard
prototyping is based on building actual functional prototypes
with minimal functionality in the beginning. The prototype
development forms the heart of the future prototypes on
top of wich the entire system is build.
Agile development principles
‫סקראם הוא תהליך אג'ילי שמאפשר לנו להתמקד‬
‫ערך עסקי גבוה בזמן הקצר ביותר בהפקה של‬
‫סקראם מאפשר לנו בזריזות ובמחזוריות לבחון את‬
)‫התוכנה במצב עובד (כל שבועיים עד חודש‬
‫ הצוותים מנהלים‬.‫הגוף העסקי מספק סדרי עדיפויות‬
‫את עצמם על מנת להבין כיצד לספק בצורה הטובה‬
‫ביותר את הדרישות בעלות העדיפות הגבוהה ביותר‬
Component, System, User Acceptance Test (UAT),
‫ובעלי מקצוע אחרים ומנוהלת עצמאית באחריות‬
‫משותפת בכל שבועיים עד חודש כל אחד יכול לראות‬
‫תוכנה עובדת ולהחליט לשחרר אותה או להמשיך‬
‫ התקדמות של פרוייקט‬.‫לשפר אותה ספרינט נוסף‬
‫ אורך‬.‫סקראם מתבצעת ע"י סדרה של ספרינטים‬
‫ אורך קבוע מסייע לקבלת‬.‫ שבועות‬2 -4 ‫מקובל נע בין‬
‫ במהלך הספרינט המוצר עובר את כל‬.‫קצב יותר טוב‬
.‫ קידוד ובדיקות‬,‫ עיצוב‬:‫שלבי הפיתוח‬
Work is structured in cicles of work called Sprints, iterations
of work – 2 to 4 weeks, during each sprint, teams pull from a
prioritized list of customer requirements – user stories, so
that the features that are developed first are of the highest
value for the customer. At the end a potentially shippable
product is delivered.
KanBan development principles
Kanban is a method for managing the creation of products
with an emphasis on continual delivery while not
overburdening the development team. Like Scrum, Kanban
is a process designed to help teams work together more
effectively.
Visualize what you do today (workflow): seeing all the items
in context of each other can be very informative
Limit the amount of work in progress (WIP): this helps
balance the flow-based approach so teams done start and
commit to too much work at once
Enhance flow: when something is finished, the next highest
thing from the backlog is pulled into play
Scrum: Pre-defined roles of Scrum master, Product owner
and team member - Timeboxed sprints - Work is pulled
through the system in batches (the sprint backlog)- No
changes allowed mid-sprint- Velocity- More appropriate in
situations where work can be prioritized in batches that
can be left alone
KanBan: No prescribed roles- Timeboxed sprints- Work is
pulled through the system (single piece flow)- Changes can
be made at any time - Cycle time - More appropriate in
operational environments with a high degree of
variability in priority
RUP - Rational Unified Process
a waterfall of iterations – each phase has a list of objectives.
Iterative development: business value is delivered
incrementally in timeboxed cross-discipline iterations.
Reliable software:
‫תוכנה אמינה – אפשר לסמוך עליה תמיד – תבצע את‬
.‫מה שהיא צריכה (ורק את זה) ותמיד‬
‫הבדיקות של אמינות התוכנה הן‬NFT – ‫מובהקות‬
‫ מה היא עושה ? אלא איך‬:‫זאת אומרת איננו שואלים‬
‫היא מצליחה לעשות תמיד את מה שהתחייבנו‬
? ‫שתעשה‬
– ‫חלק מנושא האמינות הוא גם התאוששות מכשל‬
)‫(מבחני הקפה‬
Measures, Metrics, And Indicators
Measure Provides a quantitative indication of the extent,
amount, dimension, capacity, or size of some attribute of a
product or process
Measurement The act of determining a measure
Metric(IEEE) A quantitative measure of the degree to which
a system, component, or process possesses a given attribute
Indicator A metric or combination of metrics that provides
insight into the software process, a software project, or the
product itself
Activities of a Measurement Process
Formulation The derivation (i.e., identification) of software
measures and metrics appropriate for the representation of
the software that is being considered
Collection The mechanism used to accumulate data required
to derive the formulated metrics
Analysis The computation of metrics and the application of
mathematical tools
Interpretation The evaluation of metrics in an effort to gain
insight into the quality of the representation
Feedback Recommendations derived from the
interpretation of product metrics and passed on to the
software development team
Attributes of Effective Software Metrics
Simple and computable It should be relatively easy to learn
how to derive the metric, and its computation should not
demand inordinate effort or time
Empirically and intuitively persuasive The metric should
satisfy the engineer’s intuitive notions about the product
attribute under consideration
Consistent and objective The metric should always yield
results that are unambiguous
Consistent in the use of units and dimensions The
mathematical computation of the metric hould use
measures that do not lead to bizarre combinations of units
Programming language independent Metrics should be
based on the analysis model, the design model, or the
structure of the program itself
An effective mechanism for high-quality feedback The
metric should lead to a higher-quality end product
Opportunities for measurement during the software life
cycle
Project initiation – measures from past projects are used to
estimate cost and duration of new project. Requirements
specification – requirements specified quantitatively, “the
mean time to failure <180 days”. Project planning –
concrete plan specify performance goals in a measurable
way. Coding&UnitTesting – Measures of code complexity
used to establish testing and maintenance plane. System
test – measures of product quality used to decide when the
software is stable enough to ship. Operation – measure
customer satisfaction.
Testing Techniques
Specification-Based: Equivalence Partitioning,Classification
Tree Method,Boundary Value Analysis,State Transition
Testing,Decision Table Testing,Cause-Effect Graphing,Syntax
Testing,Combinatorial Test,Scenario Testing (including Use
Case Testing),Random Testing
Structure-Based: Statement Testing, Branch Testing,
Decision Testing, Condition Testing, Data Flow Testing
Experience-Based: Error Guessing, Exploratory, Checklist
based ,Attacks.
Static and Dynamic analytics: Static analysis is performed in
a non-runtime environment. Typically a static analysis tool
will inspect program code for all possible run-time behaviors
and seek out coding flaws, back doors, and potentially
malicious code. Dynamic analysis adopts the opposite
approach and is executed while a program is in operation. A
dynamic test will monitor system memory, functional
behavior, response time, and overall performance of the
system. This method is not wholly dissimilar to the manner
in which a malicious third party may interact with an
application. Having originated and evolved separately, static
and dynamic analysis have, at times, been mistakenly
viewed in opposition.
Strengths and Weaknesses of Static and Dynamic Analyses:
Static analysis, with its white box visibility, is certainly the
more thorough approach and may also prove more cost-
efficient with the ability to detect bugs at an early phase of
the software development life cycle. For example, if an error
is spotted at a review meeting or a desk-check. Had the
error become lodged in the system, costs would multiply.
Static analysis can also unearth future errors that would not
emerge in a dynamic test. Dynamic analysis, on the other
hand, is capable of exposing a subtle flaw or vulnerability
too complicated for static analysis alone to reveal and can
also be the more expedient method of testing. A dynamic
test, however, will only find defects in the part of the code
that is actually executed.
Procedural Programing is a programming paradigm, derived
from structured programming, based upon the concept of
the procedure call. Procedures, also known as routines,
subroutines, or functions (not to be confused with
mathematical functions, but similar to those used in
functional programming), simply contain a series of
computational steps to be carried out. Any given procedure
might be called at any point during a program's execution,
including by other procedures or itself. Traditional
procedural programming is far less dogmatic: you have data.
You have functions. You apply the functions to the data. If
you want to organize your program somehow, that's your
problem, and the language really isn't going to help you.
determine its sub-ranges and transition points *if the
improving the clarity, quality, and development time of
a computer program by making extensive use
of subroutines, block structures and for and while loops—in
contrast to using simple tests and jumps such as
the goto statement which could lead to "spaghetti code"
which is both difficult to follow and to maintain.
( ‫מובניות‬Structural) ‫של התוכנה‬
( ‫בדרך כלל מכילה הסתכלות ותכנון מלמעלה למטה‬Top down )
‫כאשר המפתח מגדיר סט של רוטינות ופונקציות המקודדות בנפרד‬
‫שמשמעותו שהקוד יכול להיות מועמס לתוך הזיכרון בצורה יעילה‬
.‫והשימוש החוזר ברוטינות אלו הוא חלק חשוב בעיצוב התוכנה‬
( ‫עיצוב שכזה מקל על בדיקת הקוד כל רוטינה בנפרד‬Unit Test )
‫והאינטגרציה ביניהם כתהליך בדיקה עצמאי‬
In a procedural program, modules interact by reading and
writing state that is stored in shared data structures.
In an object oriented program, modules in the form of
objects interact by sending messages to other objects
In a procedural program, the code is king and the data is
subordinate. In other words, you have programs which act
on data and they're not usually tightly bound.
In the OO world, objects are the primary thing of interest.
An object consists of data and the code that is allowed to act
on that data, and they are very tightly bound. It is the
concept of encapsulation, the hiding of information.
Naming convention is a set of rules for choosing the
character sequence to be used for identifiers which
denote variables, types, functions, and other entities
in source code and documentation.
CamelCase is the practice of writing compound words or
phrases in which the words are joined without spaces and
are capitalized within the compound like BreakFast,
myName etc. CamelCase is so named because each word in
the identifier is put together without spaces, but with the
first letter of each word captilised, looking like humps of a
camel. There are two varieties in CamelCase
- UpperCamelCase and lowerCamelCase. In java variables,
references, method names are declared in lowerCamelCase
Object Oriented Metrics – Motivation
Object-oriented design and development are popular
concepts in today's software development environment.
Object oriented development requires not only a different
approach to design and implementation, it requires a
different approach to software metrics.
Thus, the approach to software metrics for object oriented
programs must be different from the standard metrics set.
Coupling Metrics are the counts of interactions between
classes
Main metrics: CBO (coupling between object classes) For a
class, is defined as the number of other classes to which it is
coupled. RFC (response set for class( The number of
methods in the set of all methods that can be invoked in
response to a message sent to an object of a class
Cohesion Metrics measure how well the methods of a class
are related to each other. Main metrics: LCOM (Lack of
Cohesion) Measures the dissimilarity of methods in a class
by instance variable or attributes. TCC (tight class cohesion(
Ratio of number of similar method pairs to total number of
method pairs in the class. NP = maximum number of
possible connections. NDC = number of direct
connections.TCC = NDC/NP. TCC<0.5 considered non-
cohesive classes.
Inheritance Related Measures Main metrics: DIT (depth of
inheritance tree) Maximum inheritance path from the class
to the root class.The deeper a class is in the hierarchy, the
more methods and variables it is likely to inherit, making it
more complex.NOC (number of children)Number of
immediate sub-classes of a class.NOC measures the breadth
of a class hierarchy, where DIT measures the depth. Depth is
generally better than breadth, since it promotes reuse of
methods through inheritance. High NOC has been found to
indicate fewer faults. This may be due to high reuse, which is
desirable.
‫מה ניתן להבין כאשר מסתכלים על הקוד‬
‫ אובייקטים כלולים בקלאס מוצהר‬,‫ מספר התפצלויות‬:‫כמויות‬
LOC- line of code, #of object declared in source file, # of
class declared in source file tested, data members in a
class,complexity: cyclomatic complexity, reachability,
looping depth. .‫ תעוד פנימי‬,‫ מכסה‬,‫ מובנה‬,‫מתאים לסטנדרט‬
( ‫מאיירס‬Glenford J. Myers ‫) מציע לנו לשאול עצמנו לסיכום‬
:‫הבדיקה‬
Was the program easy to understand?
Was the high-level design visible and reasonable?
Was the low-level design visible and reasonable?
Would it be easy for you to modify this program?
Would you be proud to have written this program?
‫להשתמש בטכניקות הבאות לשם התמודדות עם הקוד‬
Code inspections using checklists
Group walkthroughs
Desk checking
Peer reviews
I say – find software to do the job!
Cyclomatic Complexity reflects the decision-making
structure of the program. It is recommended that for any
given module the metric should not exceed ten. This value
should be used as an indicator of modules which may
benefit from redesign. It is a measure of the size of the
directed graph,
Definition: The cyclomatic number V(G) of a graph G with n
vertices, e edges, and p connected components is: v(G) = e -
n +p
Fan-in = number of ingoing dependencies
Fan-out = number of outgoing dependencies
Heuristic: a high fan-in/fan-out indicates a high complexity
Microservice principles
Many smaller (fine grained), clearly scoped services, Single
Responsibility Principle, Domain Driven Development,
Bounded Context, Independently Managed, Clear ownership
for each service.
Control Flow Graphs (CFG) A CFG models all executions of a
method by describing control structures, Nodes : Statements
or sequences of statements (basic blocks), Edges : Transfers
of control, Basic Block : A sequence of statements such that
if the first statement is executed, all statements will be (no
branches), CFGs are sometimes annotated with extra
information, branch predicates, defs, uses, Rules for
translating statements into graphs …
Testability is the quality that allows a component to be
easily tested in isolation. Testability is a quality factor; it is a
measurement or evaluation to predict the amount of effort
required for testing and help in allocating required
resources. There is no clear definition of what aspects of
software are actually related to testability . However,
testability has always been an elusive concept and its
correct measurement or evaluation is a difficult exercise.
Reducing effort in measuring testability of Microservice and
IOT is a must, in order to deliver quality software within time
and budget
Test oracle in general means The mechanism that
determines success or failure of a test.Testing is a sequence
of stimuli and response observations. Ground truth- is a total
test oracle that always gives the “right answer”.
Deterministic approach.Oracle tests: Derived, implicit,
specified.
Automated software testing :Able to run two or more
specified test cases,Able to run a subset of the automated
test cases, No intervention needed after launching tests,
Automatically set-up and/or record relevant, Test
environment,Run test cases,Capture relevant
results,Compare actual with expected results,Report analysis
of pass/fail.
Test oracle challenges: Completeness of information: e.g.,
inputs, specified results are usually quite abstract*Different
context may affect the expected behavior*Human Oracle
Cost: writing test cases, evaluate test outcomes,
execution*Accuracy of information: e.g., similarity to SUT,
arithmetic accuracy* Usability of the oracle: e.g., form of
results, data set size, compotators
availability*Maintainability: COTS or Custom*Complexity:
e.g., domains, tools, test data
Using Oracles in Test Automation Automated verification
assumes that *we know what to check to know whether the
software did what it was supposed to do.*We know the
correct answer
Testing levels &Types: Integrate test oracle with testing
activities*Testing types : Functional testing (FT), Non
Functional (NFT)*Testing levels: Unit, Integration,
Regression
Unit test *Defined and written to test the smallest unit of
code (function, class, method)*Is done during the
implementation (or even before TDD)*By the developer
*Small and independent, quick and simple*Stabilize the
code*Test a specific portion of the code- sends a specific
input to a method and verifies the results*Oracle is used to
design unit test*Usually is automated*Gives the confidence
to do long-term development, support the refactoring of the
code.* Saves time, help prevent regressions from being
introduced and released.* Provides excellent implicit
documentaion because they show exactly how the code is
designed to be used.
What to automate Not all the code should be automated*
Framework helps* Unit tests are recommended* Repetitive
tests (part of the regression)*Activities that are prone to
human errors (checking detailed data)* Can’t be done
manually (e.g., simulating 100 concurrent users )*Time
consuming tests
Asserts types assertEquals- Very common*assertNotNull -
Asserts that an object isn't null*assertNull - Asserts that an
object is null*assertSame - Asserts that two objects refer to
the same object*assertNotSame - Asserts that two objects
do not refer to the same object*assertTrue - Asserts that a
condition is true*assertFalse - Asserts that a condition is
false
Unit test coverage: 1. Code coverage, using dedicated
environment: Statement – 100% coverage, every statement-
line of executable code *Branches- logical branches n the
code (e.g., if, while)*Path- testing all possible aths in the
code. 2.Data coverage: Boundary condition*Typical data
values*Pre and post conditions* Illegal data 3. Model Driven
Test Design Coverage (MDTD),Find values, automate, run,
evaluate and report, considering test requirements (TR) and
test criterion (TC): Graphs*Logic expressions (based on e.g.,
state charts, machine state charts)*Input domains
Test data coverage: Check for functions with arguments/
parameters*Did we cover all the data types*Valid and
invalid data*Test boundary condition – maximum value,
minimum value, 0, positive value, and negative value as
applies for that data type*Test typical data values – most
common data based on domain and customer scenarios
(often get from customer)*Test pre- and post- conditions
– looking at code check assumptions made such as some
data being greater than 0 (division by zero)*Test bad
data:*Illegal data values*No data*Too little or too much
data*Uninitialized variables
Test Driven Design (TDD): Performed Before
implementation start*Similarly to UT, is written to test the
smallest unit of code usually a function, method or
class*Code is written to Pass the code*Iterative process.
Why TDD? Programmers dislike testing: *They will test
reasonably thoroughly the first time*The second time
however, testing is usually less thorough*The third time,
well..*Testing is considered a “boring” task*Testing might be
the job of another department / person*TDD encourages
programmers to maintain an exhaustive set of repeatable
tests*Tests live alongside the Class/Code Under Test
(CUT)*With tool support, tests can be run selectively*The
tests can be run after every single change
What are the symptoms of code rot
Rigid*Fragile*Unpredictable*Inseparability (for
modules)*Not reliable (hard to estimate changes)
What are the reasons Stupid managers?Impatient
customers?Requirements changes?Stressed schedules?Go
fast BUT go clean!Don’t blame-- > programmer writes the
code
What is clean code? Like making Sushi*Elegant efficient-say
a lot by only few words*Efficient –runs quickly*Can be read
as a good prose*Written by somebody who cares*Each
routine to read is like you expected
Test Driven Design Clean code gives confident, not afraid to
clean it *“Before you write code, think about what it will do.
Write a test that will use the methods you haven’t even
written yet.”*A test is not something you “do”, it is
something you “write” and run once, twice, three times,
etc.*It is a piece of code*Testing is therefore
“automated”*Repeatedly executed, even after small
changes*Maintain short list of defects*TDD is a low level
design*TDD creates testable code*TDD eliminated the fear,
creates the courage to change
How to implement TDD? 3 laws 1.Write NO production
code xcept to pass a failing test 2.Write only ENOUGH of a
test to demonstrate a failure 3.Write only ENOUGH
production code to pass the test TDD Stages: write a test-
compile-fix compile errors-run test, watch it fail-write code-
run test, watch it pass-refactor code – run test, watch it
pass, loop…
Mocking objects object -‫ מממש את אותו ה‬interface ‫של‬
.‫ניתן לדמות את התנהגות גמישה של האובייקט אותו הוא מדמה‬
‫האובייקט המדומה (זה לא‬stub*)"‫קביעת ערך "טיפש‬
.‫**החזרה‬.‫זריקת *השמת ערכים לפרמטרים‬exception.
Mocks and stubs are both dummy implementations of
objects the code under test interacts with.*Stubs can be
thought of as inputs
to the code under test.*Mocks can be thought of as outputs
from the code under test. Stub-Class-Test. Class-Mock-Test.
Test Types: Specification, Unit, Risk based, scenarious,
regression, stress, user, model and state, load.
‫ לפעמים‬.‫ פונקציה היא יכולת של המוצר‬.‫בדיקות פונקציונליות‬
‫ מדרישות‬:‫ זיהוי כל פונקציה‬.‫נקרא לפונקציה גם תכונה או פקודה‬
‫או ממדריך משתמש (אפילו חלקי)*ממעבר לאורך כל ממשקי‬
‫המשתמש*מניסוי פקודות בכל שורות הפקודה* חיפוש בתוך‬
.‫ הקוד או קבצי המקור של פקודות או שמות של פקודות‬,‫התוכנית‬
‫ קבע באיזה אופן תדע‬:‫פעילויות מפתח לבדיקת פונקציה‬
‫שהפונקציה פועלת (שאלת אורקל)*זהה משתנים המשמשים את‬
‫הפונקציה ובדוק את קצותיהם* זהה משתני סביבה שעלולים‬
‫להגביל את הפונקציה בתוך הבדיקה* בדוק שכל הפו נקציה‬
‫מבצעת את מה שהיא אמורה לבצע ואינה עושה את מה שהיא לא‬
‫ מוכוון‬:‫ שימושים עיקריים בבדיקת פונקציונליות‬.‫אמורה לעשות‬
‫ ביכולתך להשוות את רשימת‬:‫להבטיח את כל התפקודים‬
‫הפונקציות לרשימות בתכנית הבדיקות כולל רשימת תסריטים‬
.‫ תת רכיב של תכונה‬,‫ תפקוד‬,‫ומקרי בדיקה ע"מ לכסות כל רכיב‬
‫ הכירות סימפטים עם‬:‫שימושי עבור בדיקות ראשונות של המוצר‬
‫היכולות של המוצר*סריקה מהירה לזיהוי בעיות רציניות שצריך‬
‫ יופי‬: ‫ הסיכונים של בדיקות פונקציונאליות‬.‫לטפל בהן בהתחלה‬
‫של דרך להתחיל איתה אבל מספר אנשים עשויים להסתמך על כך‬
‫ כשיטה עיקרית לבדיקה הבעיה היא‬.‫כמהלך בדיקת מרכזי‬
‫ חסרה‬:‫בהדגשת בדיקות יחידה תוך בידוד יחידה מסויימת‬
,‫אינטראקציה בין רכיבים* חסרים ה בטים העוסקים בעומס‬
‫ ולא נבחן נושא הפסקות‬,‫אינטרקציה עם מה שקורה ברקע‬
‫ *לעתים קרובות מתמקד בתפקודים‬.)‫ועצירות (מתוכננות או לא‬
‫ללא התחשבות בגבולות או פרטי מידע אחרים * אינו משיב על‬
‫מטלות המשתמשים – כאשר הלקוח יכול להשיג את היתרון‬
.‫המובטח ע"י תוכנה‬
Test Case is a finite and tree-structured labeled transition
system. A set of test inputs, execution conditions, and
expected results developed for a particular objective, such
as to exercise a particular program path or to verify
compliance with a specific requirement.* A test case is the
combination of test data and oracle information to
determine the validity of the test. *A form of contract
between a service provider and a service user .Design Test
Cases: 1. Creating a MODEL of the requirements, using
requirements tree*Model can be table, flow graph, and
state diagram 2.Cover the model, using base TCs (no data in
this stage)*Design the scenario (set of TCs) 3.Define the TEST
ORACLE 4.Supplementing With TEST DATA 5.TCs contains set
of steps and expected results & conditions 6.Inspection &
Analysis 7.Run & evaluate the results. Elements of test case:
title, priority, status, initial configuration, software
configuration, steps, expected behavior.Test technique is:
Selection tool- what to test, sampling tool*Design tool- how
to test, what to include in the test. Domain testing: Say two
test cases A and B are test – equivalent If we believe
software will fail on A if it fail on B. Test – equivalent should
be an equivalent relation.Equivalent classes (EC ) for test
cases; set S of test case such as: If A ad B are in S the A and B
are test- equivalent. If A is in S and A is a test-equivalent to B
then B is in S. Equivalent partition - for test cases: Set P of
equivalent classes such that every test case is in same class S
in P. Domain – set of values. Subdomain- equivalence
classes. Representatives of each sub-domain. Equivalence
analysis- 2 values belong to the same class if the program
treats them similarly. Select the RISKY candidates. A schema
for domain testing: Characterize the variables: * Identify
the potentially interesting variables* Identify the variable(s)
you can analyze now, this is the variable (s) of interest
*Determine the primary dimensions of the variable of
interest *Determine the type and scale of variable’s primary
dimension and what values it can take *Determine whether
you can order variable’s values (from the smallest to the
largest) *Determine whether this is an input variable or
result *Determine how the program uses this variable
*Determine whether other variables are related to this one.
Analyze the variable and create tests: *Partition the
variable (its primary dimension) *if the dimension is ordered,
dimension is not ordered, base the partitioning on similarity
*Lay out the analysis in classical boundary/equivalent table.
Identify best representatives *Create test for the
consequences of the data entered, not just input filter
*Identify secondary dimensions. Analyze them in classical
way *Summarize your analysis with a risk/equivalence table .
Generalize to multidimensional variables *Analyze
independent variables that should be tested together.
*Analyze variables the hold results *Analyze non-
independent variables. Deal with relationships and
constrains. Prepare for additional testing *Identify and list
unanalyzed variables. Gather information for late analysis
*Imagine and document risks that don’t necessarily map to
an obvious dimension. For even small programs, the input
domain is so large that it might as well be infinite. Testing is
fundamentally about choosing finite sets of values from the
input domain. Input parameters define the scope of the
input domain: *Parameters to a method *Data read from a
file *Global variables *User level inputs. Domain for each
input parameter is partitioned into regions. At least one
value is chosen from each region. If the partitions are not
complete or disjoint, that means the partitions have not
been considered carefully enough.
Modeling the Input Domain: Step1: Identify testable
functions: * Individual methods have one testable function
*In a class, each method often has the same characteristics
*Programs have more complicated characteristics—
modeling documents such as UML use cases can be used to
design characteristics *Systems of integrated hardware and
software components can use devices, operating systems,
hardware platforms, browsers, etc. Step 2 : Find all the
parameters *Often fairly straightforward, even mechanical
*Important to be complete *Methods : Parameters and
state (non-local) variables used *Components : Parameters
to methods and state variables *System : All inputs,
including files and databases Step 3 : Model the input
domain *The domain is scoped by the parameters *The
structure is defined in terms of characteristics *Each
characteristic is partitioned into sets of blocks *Each block
represents a set of values *This is the most creative design
step in using ISP Step 4 : Apply a test criterion to choose
combinations of values *A test input has a value for each
parameter *One block for each characteristic *Choosing all
combinations is usually infeasible *Coverage criteria allow
subsets to be chosen Step 5 : Refine combinations of blocks
into test inputs *Choose appropriate values from each
block. Two Approaches to Input Domain Modeling:
Interface-based approach: * Develops characteristics
directly from individual input parameters * Simplest
application *Can be partially automated in some situations
Functionality-based approach * Develops characteristics
from a behavioral view of the program under test *Harder to
develop—requires more design effort *May result in better
tests, or fewer tests that are as effective
Software Requirements specification A formal crucial
document that contains the following: *Purpose of the
software to be specified *Product/service description *
Architecture *Users characteristics *Assumptions *Business
high level requirements *Requirements for development
*Priority, estimation *Limitations *Functional/Non
functional requirements. A good requirement *Correct
*Unambiguous (all statements have exactly one
interpretation) *Complete (where TBDs are absolutely
necessary, document why the information is unknown, who
is responsible for resolution, and the deadline) *Consistent
*Ranked for importance and/or stability *Verifiable (avoid
soft descriptions like “works well”) -testable *Modifiable
(evolve the Requirements Specification only via a formal
change process, preserving a complete audit trail of
changes) *Does not specify any particular design *Traceable
(cross-reference with source documents and spawned
documents).
Re(f)= P(f)*C(f)
Risk Based Testing Re(f) - Risk
Exposure of function f
*P(f) - Probability of a fault in function f *C(f) - Cost related
to a fault in function f.
Model-based testing is an application of model-based
design for designing and optionally also executing artifacts
to perform software testing or system testing. Models can
be used to represent the desired behavior of a system under
test (SUT), or to represent testing strategies and a test
environment.
Importance: *Unit testing wont be sufficient to check the
functionalities *To ensure that the system is behaving in the
same sequence of actions.*Model-based testing technique
has been adopted as an integrated part of the testing
process.*Commercial tools are developed to support model-
based testing.
Advantages: *Higher level of Automation is achieved.
*Exhaustive testing is possible. *Changes to the model can
be easily tested. Disadvantages: *equires a formal
specification or model to carry out testing. *Changes to the
model might result in a different set of tests altogether.
*Test Cases are tightly coupled to the model.
Regression testing for agile development For effective
regression testing in agile development, it is important that
a testing team builds a regression suite right from the initial
stages of software development and then keeps building on
it as sprints add up. A few things to determine before a
regression test plan is built are: *Identifying what
improvements must be implemented in the test-cases.
*Identify the time to execute regression testing. *Outline
what needs to be automated in the regression test plan and
how. *Analyze the outcome of the regression testing.
Regression testing is retesting changed segments of
application system. It is performed frequently to ensure the
validity of the altered software. In most of the cases, time
and cost constraint is prominent; hence the whole test suite
cannot be run. Thus, prioritization of the test cases becomes
essential. The priority criteria can be set accordingly e.g. to
increase rate of fault detection, to achieve maximum code
coverage and so on. Sprint level Regression testing: This
regression test is focused on testing the new functionalities
that are implemented since the last release. End to End
Regression testing: This test incorporates end-to-end testing
of ‘all’ the core functionalities of the product.
Pairwise and combinatorial approach All-Pairs technique is
very helpful for designing tests for applications involving
multiple parameters. Tests are designed such that for each
pair of input parameters to a system, there are all possible
discrete combinations of those parameters. The test suite
covers all combinations; therefore it is not exhaustive yet
very effective in finding bugs.
Optimization methods minimizing test volume Regression
Test Selection (RTS) technique selects a subset of test cases
from the original test suite and uses it to revalidate the
unmodified portions of a program: *Data-flow Analysis
Approach *Graph-Walk Approach *Modification-based
Technique *Slicing-based test Selection *Coverage-based
Prioritization *Risk based prioritization *Cost-Aware Test
Case Prioritization * Coverage calculation Techniques Code
Coverage is a technique to measure how much the test
covers the software and how much part of the software is
not covered under the test. The tester is able to find out
what features of the software are exercised by the code.
Using the code coverage technique and number of bugs in
the application we can build confidence upon the system on
its quality and functioning.
Coverage criteria are method to measure how much code
has been exercised by the test suite. The test suite must
satisfy the rule of the coverage criteria. The various
Coverage criteria are as follows: * Statement Coverage *
Decision Coverage * Condition Coverage * Path Coverage
Generating Test Sequences We can use the machine-
readable model to create test sequences: *Random walk
*All transitions *Shortest paths first *Most likely paths first
Testing artifacts The products developed into different
phases of software testing life cycle and shared with the
stake holders are known as Test Artifacts. Generally the
software test team should prepare these artifacts and they
are supposed to take sign off on those artifacts from the
stake holders to make sure that there is no communication
gap between customer and test team. Any change
requirement during the testing process can also be tracked
easily through these test artifacts. Some of the deliverable
test artifacts are presented below: Test strategy, test plan,
test scenario, Test case, test reports.
Verification Calls (VC) Definition: An external investigator
entity that observes and documents selective states and
occurrences represented by data items during, and at the
end of, the TC execution, and signals their validity at a
specific time purpose: Anticipates, controls and documents
certain behaviors of a TC execution Requirements: * Access
to all types of storages and data items * Ability to deal with
Zipped/Encrypted data * Application of runtime/dynamic
parameters * External to the tested application *
Operational as API call * Ability to interface with reporting
system * Fast / Efficient * Can deal with complex logic and
data structures
Traditional Testing levels unit < component< subsystem<
system *
System = ∑ 𝑆𝑢𝑏𝑠𝑦𝑠𝑡𝑒𝑚𝑠 {∑ 𝑐𝑜𝑚𝑝𝑜𝑛𝑒𝑛𝑡𝑒𝑠 [∑ 𝑢𝑛𝑖𝑡𝑠]}
Differentiate testing types into dimension levels
1.Technical / Code levels – signify the actual code
development & implementation during the SDLC progress
2.Functional / Business Levels signify the usage of the
application from Separate microservices conglomerates into
services that must “talk” to one another and should be
applied within the usage context 3.Nonfunctional levels are
all the extra’s which can be described with the question of
how? and how well? it is being executed
DEFECT Anomaly: Any condition that deviates from
expectation based on requirements specifications, design
documents, user documents, standards, etc. or from
someone’s perception or experience. Anomalies may be
found during, but not limited to, reviewing, testing, analysis,
compilation, or use of software products or applicable
documentation. [IEEE 1044] See also bug, defect, deviation,
error, fault, failure, incident, problem.Defect: A flaw in a
component or system that can cause the component or
system to fail to perform its required function, e.g. an
incorrect statement or data definition. A defect, if
encountered during execution, may cause a failure of the
component or system. Other related definitions: defect
based technique:, defect density: Defect Detection
Percentage (DDP): defect management tool: defect masking:
defect report: defect taxonomy: defect tracking tool:
Software Errors : What Kind of Error? Coding Error: The
program doesn’t do what the programmer would expect it
to do. Design Issue: It’s doing what the programmer
intended, but a reasonable customer would be confused or
unhappy with it. Requirements Issue: The program is well
designed and well implemented, but it won’t meet one of
the customer’s requirements. Documentation / Code
Mismatch: Report this to the programmer (via a bug report)
and to the writer (usually via a memo or a comment on the
manuscript). Specification / Code Mismatch: Sometimes
the spec is right; sometimes the code is right and the spec
should be changed.
Defects measurement dimension A software bug arises
when the expected result don't match with the actual
results. It can also be error, flaw, failure, or fault in a
computer program. Most bugs arise from bad requirements.
Following are the common types of defects that occur
during development: Arithmetic Defects, Logical Defects,
Syntax Defects, Multithreading Defects, Interface Defects,
Performance Defects
Defects Root cause and timing Root Cause Analysis is like a
chain of events which go backward, right from the last
possible action to the previous and so on, till you reach the
start of the problem and the exact point at which it was
introduced as a defect. Possible Factors responsible for
defect occurrence: Inadequate Requirements, Lack of Unit
testing, Communication gap, Negligence, Incorrect
Assumption, Design Gaps, Deployment issue, Inappropriate
Environment, Ineffective Test case coverage, Ineffective Test
data
A Bug Workflow *You find a bug, investigate it, and report
it. *A programmer looks into it and fixes it, or decides that a
fix will take so long - it needs management approval,
recommends that it be deferred (fix it later), or determines
(argues) it is not a bug. *The project manager prioritizes the
unfixed bugs and may reassign them. *The project team
(with representatives of the key stakeholder groups) reviews
deferred bugs and may reprioritize unfixed bugs. *The test
group retests bugs marked as fixed, deferred, or
irreproducible and closes them, or adds new information to
them and ask that they be reworked or reconsidered. It
turns some of these into appeals to the project team. *This
is a simplified description. Workflows vary significantly
:‫ טיפול בשגויים‬:‫ דרגות חומרה‬Show stopper, Critical,
Medium, low. Urgency levels: Urgent, immediate, Soon, No
rush. Status: ne- new, in-investigate, as – assign, on – on
hold, fi – fixed, re – retest, can – canceled, d – defer, cl –
closed. Permissions: tester: test, open; team leader: test,
open, status change, close; developer - fix, change status;
manager – test, open, fix, close, change status; customer
presentor – test, open; appellation commission – close,
change status;
UTS – Design And Perspective Entity Model – Best practice
• ‫תמיכה מוחלטת של ההנהלה‬
• ‫תשתיות אוטומציה מוטמעות בארגון‬
• ‫צוותי עבודה בין תחומיים‬
• ‫מערכת דיווח כלל אירגונית‬
• ‫מעבדה להרצה ממכונת של התסריטים‬
• ‫שנוי כוח אדם ומקצועי‬
• ‫מעורבות הלקוח (פנימי וחיצוני‬
• ‫תהליך הטמעה מסודר לכל הארגון‬
Test Plan The purpose of the Test Plan is to outline and
communicate the intent of the testing effort for a given
schedule. Primarily, as with other planning documents, the
main objective is to gain the acceptance and approval of the
stakeholders in the test effort. As such, the document
should avoid detail that would not be understood, or would
be considered irrelevant by the stakeholders in the test
effort. *Secondly, the test plan forms the framework within
which the test roles work for the given schedule. It directs,
guides and constrains the test effort, focusing the work on
the useful and necessary deliverables
Who should be involved in planning? (stakeholders)
Principals: These are the persons who buy our software End
Users: The group of people who work with the software
system Partners: These are the group of people who will
make the software work in production environment
Insiders: These are people inside the organisation knowing
how the team is working. The test plan can consist of any
and all of the following: Test assignment, Test strategy,
Test approach, Test organization, Test estimates, Test
planning, Test resourcing, Test environment,
requirements/design, Test tools...and more Manage and
control the test plan: Test monitoring activity includes:
Providing feed back to the team and the other required
stakeholders about the progress of the testing efforts.
*Broadcasting the results of testing to associated
members.*Finding and tracking the test metrics. *Planning
and estimation and deciding the future course of action
based on the metrics calculated.
Test Automation Artifacts: Tools & Infrastructure
Repository Management *Automation Tools / Engines /
Vehicles*Impact on Defect Management Tool *Monitoring &
Control the process and outcomes *Reporting & Analyzing
Tools *…ALM – how it can fit here? *What is missing? *In
your Org *How to connect all this to the world…?
Challenges in testing* Debugging: *Multithreads *Long
cyclic procedure to reach critical point *Debugging of OS (if
crashes, so does the debugger) *Creating identical test
environments for many testers *Complete restoration of
machine state time travelling Ability to record the
program’s run, Travel back and forth the lines of
code,Inspect variables values throughout time, Inspect
variables history values before certain point So what do
we get Ability to duplicate complicated system
environments to many testers *Ability to save a machine (or
many) state *Ability to restore to a stable starting point
*Ability to debug OS *Ability to inspect dynamic changes in
multithreads environments *Ability to time travel
throughout the program’s run without the need reconstruct
our steps *Reach the point of failure, and start travelling
back!
Some of the most common challenges of Test Data The
teams may not have adequate test data generator tools
knowledge and skills *Test data coverage is often
incomplete *Less clarity in data requirements covering
volume specifications during the gathering phase *Testing
teams do not have access to the data sources *Delay in
giving production data access to the testers by developers
*Production environment data may be not fully usable for
testing based on the developed business scenarios *Large
volumes of data may need in short period of given time
*Data dependencies/combinations to test some of the
business scenarios *The testers spend more time than
required for communicating with architects, database
administrators and BAs for gathering data *Mostly the data
is created or prepared during the execution of the test
*Multiple applications and data versions *Continuous
release cycles across several applications *Legislation to look
after Personal Identification Information (PII)
Strategies for Test Data Preparation creation of flat files
based on the mapping rules-> data from production
environment ->retrieving sql queries that extract data from
client’s existing data bases -> automated test data
generation tools ->loop. Test Data Generation activities:
data analysis, data matching, test data update, sensitive
data, automation, warehouse.
basic needs for test management requirement management
tool, configuration management tool, test execution tool,
incident management tool – all of these are test
management tool.
continuous innovation (CI) continuous integration
Continuous Integration is a software development practice
where members of a team integrate their work frequently,
usually each person integrates at least daily - leading to
multiple integrations per day. Each integration is verified by
an automated build (including test) to detect integration
errors as quickly as possible. Many teams find that this
approach leads to significantly reduced integration problems
and allows a team to develop cohesive software more
rapidly
continuous deployment (CD) means that every change goes
through the pipeline and automatically gets put into
production, resulting in many production deployments every
day. Continuous Delivery just means that you are able to do
frequent deployments but may choose not to do it, usually
due to businesses preferring a slower rate of deployment.
benefits of continuous delivery are: * Reduced Deployment
Risk: *Believable Progress *User Feedback. Develop-
>Automated build->deploy to dev server ->automated
testing -> deploy to QA -> Manual/performance testing ->
deploy to Prod.
Traditional to Agile Transfer: Step 1: accept agile thinking.
Enterprise Stakeholders: Product Manager, Program
Manager, CEO, CTO, IT Director/VP. Agile Product
Stakeholders: Product Owner. Product backlog -> sprint
backlog->(scrum master, scrum team) ->deployable feature.
Ops, PS&M, infra team - ??? Step 2: adapt continues
pipeline.
Generalization and trends The move to CI Project is a large
organizational project which requires management support
*Automation is a must - with out it no way to perform *Unit
testing level are playing important part for achieving the
desired quality.* Even thou the desired formation of the
needed infrastructure could be specify. Hardly any single
tool provides the full solution. Each company assemble its
own tool selection and integrate all of them. * The
responsibility for quality transferring to development teams
* Testing is becoming an activity done daily by programmer
* The measurement are not yet matures * The economic
benefit of the different testing levels is yet to be formalized.
* The industry galloping forward without the academic
support. *It is hard to see a theoretic justification and
support for the new trends in academic literature.
Conclusion and next step: * The CI / CD trends are affecting
testing levels. Eliminating the boundaries between the
deferent levels and force a single unit to perform all of them
*Unit test is becoming one of the milestones for achieving it
*key success factor is the ability to automate all previously
manual activities *Agile development teams are required to
perform new activities that were done by dedicated teams
in the past. * New testing consideration should be addressed
by the teams such as – performance, security, deploy ability,
regression
The story of Atomation Atomation offers an IoT platform for
making any object intelligent and connected. The platform
consists of small, connectable hardware along with big data
aggregation and analytics via the Cloud. *The hardware, an
"atom," contains a circuit board with embedded code, which
has its own power source, Bluetooth connectivity with other
atoms and smart devices, and connectivity to the Cloud.
Atoms connect via a plug. Atomation also gathers big data
from atoms, performs analysis, and provides feedback to
customers through the Atomation dashboard. *Atomation's
platform has important agricultural applications, including
recommendations for optimal growth, atering and
fertilization guidelines, future predictions, and farm
automation.
Internet Of Things simplification Internet of Things can be
realized in three paradigms – internet-oriented
(middleware), things oriented (sensors) semantic-oriented
(knowledge). Although this type of delineation is required
due to the interdisciplinary nature of the subject, the
usefulness of IoT can be unleashed only in an application
domain where the three paradigms intersect The promise –
will it be? Abstract away the specificities of physical devices
from applications and/or users *Key elements to promote
interoperability and seamless integration of physical devices
*Contribute to make the development of IoT applications
easier *Recent research field that has drawn attention from
industry and academia - I doubt that
Future based around microservices Same say the future will
be based around microservices. These are discrete pockets
of functionality, but carry out single functions, rather than
trying to take on too much. As a process is broken down into
its constituent parts, you are left with a set of tasks – and it
is surprising how often these tasks appear in other
processes. *Functions, Also, should a function need
updating, this can be done in one place, and all dependent
processes get the update automatically. *As a process is
broken down into its constituent parts, you are left with a
set of tasks – and it is surprising how often these tasks
appear in other processes *Dynamic creation of an overall
microservice-based approach to process requires complex
orchestration
Testing microservices Testing in microservices architecture
can be more challenging than in traditional, monolithic
architecture. In combination with continuous integration
and deployment, it’s even more complex. It’s important to
understand layers of tests and how they differ from each
other. Putting effort on automation aspect of your tests,
doesn’t mean you shouldn’t drop on manual testing. Only
combination of various test approaches gives you
confidence in product quality ***The Netflix development
team established several best practices for designing and
implementing a microservices architecture. * Create a
Separate Data Store for Each Microservice *Keep Code at a
Similar Level of Maturity *Do a Separate Build for Each
Microservice *Deploy in Containers *Treat Servers as
Stateless Docker Containers Ver. Virtual machines
Containers and virtual machines have similar resource
isolation and allocation benefits, but function differently
because containers virtualize the operating system instead
of hardware, containers are more portable and efficient.
Containers include the application and all of its
dependencies --but share the kernel with other containers,
running as isolated processes in user space on the host
operating system Virtual machines include the application,
the necessary binaries and libraries, and an entire guest
operating system -- all of which can amount to tens of GBs.
Hierarchical Organization Often also called centralized
organization. Examples: Military, church, traditional
businesses. Key property: The organization has a tree
structure. Decisions are made at the root and communicated
to the leaf nodes. The decision association is also used for
reporting and communication. Advantages: *Centralized
control over roject selection *One set of management and
reporting rocedures for all project participants across all
projects *Established working relationships among people
*Clearly established lines of authority to set priorities and
resolved conflicts *Clearly defined career path
‫מבנה ארגוני פונקציונאלי המבנה הקלאסי שבו כל יחידה‬
‫ארגונית שייכת לתחום מקצועי ואחראית לביצוע כל הפעולות‬
‫ הנדסה יתרונות פיתוח‬,‫ כספים‬,‫ שיווק‬, ‫ ייצור‬:‫ לדוגמא‬.‫בתחום זה‬
, ‫ תקשורת בהירה‬, ‫ שימוש יעיל במשאבים‬,‫מומחיות מקצועית‬
‫ חסרונות‬.‫ ריכוז נושאים אדמיניסטרטיביים‬,‫מסלולי קידום ברורים‬
‫ לכל יח' פונקציונאלית יש לו"ז‬,‫אין סמכות מרכזית לפרויקט‬
‫ קשה לתקשר ולשלב בין בעלי התפקידים‬, ‫ותקציב נפרדים‬
‫מבנה ארגוני פרויקטלי מבנה ארגוני שבו כל יחידה עוסקת‬
‫ קיימות יחידות‬,‫ בנוסף‬.‫ מוצר או קבוצת מוצרים מוגדרת‬,‫בפרויקט‬
‫ יתרונות מנהל פרויקט אחראי‬.‫מטה המשותפות לכל הארגון‬
,‫ התמקדות על השגת מטרות הפרויקט‬, ‫לתכנון וביצוע הפ רויקט‬
‫ ההתקשרות עם הלקוח מתבצעת‬, ‫צוות הפרויקט מחויב לפרויקט‬
. ‫ תקשורת טובה בתוך צוות הפרויקט‬,‫רק על ידי מנהל הפרויקט‬
‫ אין ניצול מלא של‬,‫חסרונות קשה לפתח מומחיות מקצועית‬
‫ חוסר‬, ‫ אין אופק תכנון מעבר לסיום הפרויקט‬,‫משאבים מקצועיים‬
.‫יציבות בארגון עקב מעבר אנשים בין פרויקטים‬
‫מבנה ארגוני מטריציוני מבנה ארגוני המשלב בין המבנה‬
‫ כל פרויקט מנוהל על ידי מנהל‬.‫הפונקציונאלי למבנה הפרויקטאלי‬
‫ המקבל את המשאבים המקצועיים ממנהלי היחידות‬,‫פרויקט‬
,‫ ניצול יעיל של משאבי הארגון‬,‫ יתרונות מקצועיות‬.‫המקצועיות‬
‫התאמה לסביבה דינאמית שבה צוות מצומצם בכל תחום משרת‬
‫ דורש תיאום בין‬,‫ חסרונות מורכב לניהול‬.‫את כל הפרויקטים‬
‫ תקשורת‬, ‫ אנשי צוות הפרויקט כפופים לשני מנהלים‬,‫גורמים רבים‬
‫ למנהל הפרויקט קשה לפקח‬, ‫מורכבת בין גורמים מתחומים שונים‬
‫על האיכות המקצועית של הביצוע‬
Agile - > CI Acceptance test driven development process.
Tight collaboration between business and delivery teams.
Cross-functional teams include QA and operations.
Automated build, testing, DB migration and deployment.
Incremental development on mainline with continuous
integration. Software always production ready. Releases tied
to business needs, not operational constraints.
‫ להבין מה מצפים‬:‫העצמת עובדים – עקרונות מה צריך עובד‬
‫ לכל‬,‫ממנו בכל שלב* יעדים ברורים להשגה (בתקופת זמן מוגדרת‬
‫ *מה האפשרויות להתקדמות ומה צריך לעשות בשביל זה‬.)‫תפקיד‬
‫ מה רוצה‬.)‫*איפה הוא עומד היום (איך רואים אותו מנהלים‬
‫המעביד‬
‫להכיר את עובדיו *לברור את הטובים מהם על מנת לקדמם‬
‫(עתודות העתיד של הארגון) *לברור את הפחות טובים על מנת‬
‫ יחסי‬:‫לשחררם *להשיג מהם תפוקות ברורות מה כולם רוצים‬
.‫ לקדם את החברה‬,‫ תוצרת ויעילות‬,‫עבודה טובים‬
‫ מאוד קשה‬:‫מה עושה המעבר למודל אג'ילי למסלולי קידום‬
‫לממש עבודה באג'ייל ללא מבנה מטריציוני *יש קושי בקידום‬
‫מקצועי (רמת הידע והמקצועיות של העובד היחיד) * מאוד קשה‬
‫לזהות כישורי ניהול אמתיים בתוך קבוצת העבודה – ניגוד עניינים‬
‫* מודל הניהול החדש עובד בעיקר בקבוצות קטנות ואחידות‬
‫*למרות האחריות הכוללת של הצוות בזמן הפרויקט – האחריות‬
‫על קידום העובד מוטלת בעיקר על עצמו‬
‫מודל ניהול מודרני – בואו נשטח את ההיררכיה *להתאים את‬
‫המבנה האירגוני והמסלולים האישיים לעבודה בשיטות אג'יליות‬
‫*הקשיחות המבנית של הדרגות בארגון פוגעת ומסרסת‬
‫**לאפשר שכל אחד יוכל לבטא עצמו גם ניהולית וגם מקצועית‬
,‫ תפקידים‬:‫*לשמר ולטפח עובדים מצטיינים * מעבר גמיש בין‬
‫ מיקום גאוגרפי‬,‫ אופי עבודה‬,‫מקצועות‬
Cognitive processes Cognition is an interdisciplinary topic
and its study is generally found within “cognitive science” *A
mental process that is applied to build and use knowledge
*Software testing is a cognitive process of analysis, research
and decision making *Developers claim to write unit tests
systematically and to measure code coverage *Human
decisions –oracle definition, design test ware, tools,
equivalent testing, parameters… *Developing testing
competence with cognitively complex skills requires mastery
of routine tasks *Selection testing tool- what to test,
sampling tool *Design tool- how to test, what to include in
the test
Human factors and testing A hunting metaphor for human
factors in testing (Prado, 2015) *The goal is to reveal failures
by transforming the testing problem into feasible automatic
(or semi-automatic) solution *The difficulty in writing new
tests correspond to (Daka and Fraser, 2014) *the
identification of which code to test and *the isolation of the
unit under test *Difficulty in the evaluation of the unit
testing (Runeson, 2006) *functional approaches rely on
specifications to derive and evaluate the testing
requirements *functional testing is more prone to
interpretation and subjectivity than other techniques *Was
the hunter left out…??
Cognitive framework Tools Interaction usability Problems --
> usability and applicability of testing tools *Clarity,
documentation, intuitiveness, simplicity, compatibility,
feedback, GUI auxiliary or aesthetic (Norman, 1994)
*Cognitive framework- three dimensions to support unit
testing (Prado, 2015) *It is possible to classify cognition into
experiential and reflective: 1) Experiential cognition is the
type of mental effort spent during automatic tasks such as
talking, riding a bicycle, cooking the every-day meal etc.
2)Reflective cognition requires much more concentration
and is related to decision making, creativity, and reasoning
Knowledge management in software testing The aim of a
KM program is to enable individuals to solve problems more
efficiently (Andrade et al., 2013) *transfer knowledge
existing at the individual level to the organizational level *to
explicit the knowledge gained from experiences *Lessons
learned (LL) are KN enablers.
Human skills in testing Creativity *Intelligence *The ability
to learn and adapt to new situations *The ability to
efficiently recognize problems *Most new bugs are found by
humans testing the software *Exploration can bring direct
utilization of knowledge and learning (Itkonen,2016)
Exploration testing (ET) Exploratory testing is neither black
nor white, but rather a continuum of exploration exists
*Based on the personal freedom, and the experience and
skills of the tester, there is a lack of test charter design for ET
*Simultaneous learning, test design and test execution *Five
levels exist: 1.Fully scripted: the tester is provided with the
test steps, but also with the test data, which does not
provide room for exploration steps. 2.Low degree of
exploration: Besides the information for medium degree of
exploration the tester is also required to follow certain test
steps, which farther may bias the tester and reduce the
exploration space. The tester is encouraged to choose the
test data to be used in the test steps. 3.Medium degree of
exploration the tester is provided with one ore more high
goals for the test session. At the same time additional
restrictions are required, that may bias and thus limit the
tester in his/her testing session. Biasing aspects can be too
detailed goals, priorities, risks that the tester is required to
focus on, tools used, the functionality that needs to be
covered, or the test method to be used. 4.High degree of
exploration The tester provided with one ore more high
goals for test session, also knowing the test object. Besides
that, the tester can freely explore the system. 5.Freestyle
Only the test object is provided to the tester. The tester can
freely explore the system.
Session-based test management (SBTM) Is an enhancement
of Exploratory testing *Reflects the concept of time boxing
*Test results should be reported in a consistent and
accountable way *No standard test charter framework is
being used by practitioners in the testing field
Test charted design The charter is a test plan which is
usually generated from a test strategy *Identifying the major
test planning challenges, including the risks *Clarifying the
mission of the test *Product analysis *Products risks *Test
strategy design points to the approach used *Requirements
*Implicit Expectation: *Existing artifacts (e.g., source code,
defects database, design documents)
Test charter quality criteria Usefulness: To check if the test
plan meets the intended functions Accuracy: Checks for
accuracy with respect to any factual statements Efficiency:
Checks if the available resources are used efficiently
Adaptability: Checks for the immunity to change and
unpredictability in the project Clarity: Checks for the self-
consistency and unambiguity of the test plan Usability:
Checks if the test plan document is concise, easily
maintainable and well organized Compliance: Checks if it
meets the requirements which are externally demanded
Foundation: Checks if it is an output of an effective test
planning process Feasibility: Checks if it is within the reach
of the organization which performs it
Mobile App Testing issues Ensuring Multiple mobile apps
on different platforms *Common bugs issues *Sensors *E2E
testing *In the lab or on the Wild *Manual , Automation or
both *Test Types – Functionality , UX and Performance and
Exploratory, Accessibility, Maintainability *Check
Workflows, Details and the Navigation *Security Testing
PRODUCT MANAGEMENT / PRODUCT OWNER
PRELIMANARY PHASE Who is my user base? How old is the
average user? How many men or women are in my target
group? Which platform is used most among that user base?
Which device is used most? Which software version is
installed on most of the phones? What kind of sensors does
my app use? How does the app communicate with the
outside world? What is my app’s main use case? Monitor the relevant application. * Lots of issues will be reported. *
the mobile device and software market. Know when new Crowd testing is best suited to user-centric software
phones will be rolled out. Find out about the new features applications. Con’s *Crowd testers are not generally testing
of the operating systems. Keep an eye on your target group experts. *The identity of the testers is sometimes unknown
to see if new devices are showing up in your statistics. Think *Bug reports may be of low quality. *Access to staging
twice before updating a phone to the latest operating systems can be very difficult due to legal hurdles, data
system version. privacy and security. * Crowd testing can take a long time to
Consideration while developing and Testing Mobile apps prepare.
Gather information about your possible target group. *Ask
the customers about their needs. What are the app needs to
:‫טבלת החלטות‬
solve a problem for the user. Usability is really important.
The app needs to be reliable and robust. App performance is
crucial. Apps need to be beautiful. Mobility and Networks
Mobile Apps Types
Native Apps Developed for use on a particular platform or
device. Coded in a specific programming language, such as
Objective C for OS and Java for Android. Provide fast
performance and a high degree of reliability. Have access to
a phone’s various devices, such as its camera and address
book. Users can use some apps without an Internet
connection. Web Apps Stored on a remote server and
delivered over the internet through o browser. Are not real
apps; they are really websites that, in any ways, look and
feel like native apps. They are run by a browser Hybrid Apps
Are like native apps, run on the device, and are written with
web technologies. Hybrid apps run inside a native container,
and leverage the device’s browser engine (but not the
browser) to render the HTML and process the JavaScript
locally. A web-to-native abstraction layer enables access to
device capabilities that are not accessible in Mobile. Web
applications, such as the accelerometer camera and local
storage.
Mobile Apps Types NATIVE – PROS & CONS
Pro’s Due to all of the app’s elements being included in a
single native package, native applications tend to have fast
graphics with fluid animations built in. Native applications
can access exclusive ative APIs in the phone’s operating
DevOps: Developers vs Operation: Developers: Need to
system such as push notifications, camera, and in-app
wait weeks in order for there work to be placed in
purchases, which would otherwise be prohibited, or
provided in a cumbersome manner on a mobile web production. *Hard to manage the code that are pending in
application. If you’re developing a native application for production and developing new features. Operations: Code
iPhone, they have many resources, development tools, and that's works on the dev env sometimes not working on the
reading material to help you out. Con’s If you intend to also production env * Responsible for checking that the code
publish your app to a different app store, your application that been deploy doesn’t cause any errors to the production
system.
will need to be rewritten in order to be a native app on
another mobile OS. This usually delays features for the next
platform in development. It is a time consuming process to ‫ שבה משולבים בתהליכי‬,‫ זוהי גישה לפיתוח תוכנה‬:‫הגדרה‬
create a native app for both iOS and Android, as well as ‫ בודקי האיכות וכן האנשים‬,‫ המפתחים‬,‫ מזמיני הפרויקט‬,‫הפיתוח‬
money consuming. Native platforms define their own rules ‫ שילוב פעולה הדוק‬.‫שיטמיעו את התוכנה לאחר מכן בארגון‬
and frameworks and inherit little from other disciplines, ‫ מאפשר לקצר באופן משמעותי‬,‫ומתמשך לאורך כל מהלך הפיתו ח‬
requiring more investment. Native applications typically ‫ לוודא כי התוצר אכן עונה במדויק‬,‫את זמן הפיתוח של היישומים‬
require you to define phones and tablets separately, or ‫ להבטיח שמדובר בתוצר אמין וכן כזה שיהיה‬,‫לדרישות המזמין‬
‫ מאחר ומדובר בגישה שמתבססת‬.‫קל להטמיעו ולשלבו במהירות‬
define individual layouts. While this step is available and
repeated for web apps ‫ היא מאפשרת גמישות רבה וכן‬,‫גם על אוטומציה ופתרונות בענן‬
Mobile Apps Types hybrid: Pro’s: Hybrid mobile apps don’t . ‫חיסכון כספי‬
have that “mobile web” browser look because they can
include native hardware features.*The content of a hybrid Agile: Addresses the gap between customer requirements
app is portable and just requires a native harness to run it. and development team. Devops: Address the gap between
*Developers have the option to package the app locally or development and operations
through a server, which provides access both online and
offline. Con’s: Automatic generation may not work on all
devices, which can get especially complicated when trying to ‫ שיטת פיתוח‬-‫אופס הוא חלק חשוב במתודולוגית אג'ייל‬-‫דב‬
accommodate to different Android phones. *Several ‫ שינוי המקנים יתרון‬-‫מואץ שמאפשרת תהליכים מהירים ברי‬
vendors have started offering build-platforms for hybrid ‫ עם יכולת לשינויים בדרישות על פי‬,‫משמעותי בתחרות הלקוח‬
,‫דחיפות בזמן הפיתוח ואינטגרציות מהירות ותמידיות על כל שינוי‬
frameworks, simplifying the build knowledge that was
‫כאשר המטרה המרכזית הינה אספקה של תוכנה עובדת באופן‬
previously required for multi-platform. Just be prepared to
‫ כדי לממש את השינויים‬.‫ עם שאיפה ללוחות זמנים קצרים‬,‫תדיר‬
pay for it. *If the App Store is able to recognize that
‫ פותחו‬,‫ תוך שמירה על שלמות המוצר בכל רגע‬,‫בקצב תדיר‬
application is not truly native, it may be denied from the App
‫שיטות עבודה וכלים חדשניים שתומכים בתהליכי‬Continues
Store. *If your app can’t be published on the App Store, then
that would reduce your monetization and distribution
potential since purchase price or in-app purchases are native ‫ניהול סיכונים – דוגמאות לסיכון בפרויקט המערכת לא עוברת‬
features. ‫את מבחני האינטגרציה * המתכנת הראשי עזב את הצוות * איחור‬
Mobile Apps Performance Testing definitions : Performance ‫בהספקת רכיב קריטי * שרפה באתר העבודה * תביעה משפטית‬
Testing is in a key factor in the to be or not be of the Mobile – ‫ ניהול סיכונים‬.‫על הצבת אנטנת שידור * המזמין איבד עניין‬
App *In software engineering, performance testing is in ‫ מרכיב חדש גדול * ההיקף של חלק‬: ‫רמזים לקיום סיכונים‬
general, a testing practice performed to determine how ‫מהעבודה (או כולה) לא לגמרי מוגדר * חלק מהדרישות יגיעו‬
a system performs in terms of responsiveness and stability ‫במהלך ההתקדמות בפרוייקט * צוות חסר ניסיון * ריבוי שינויים‬
‫ תוכנית העבודה לא‬:‫ סיבות עיקריות לכישלון פרוייקט‬.‫בתכנון‬
under a particular workload. It can also serve to investigate,
‫ חלק מהפעילויות פשוט נשכחו‬,‫כללה את כל הפעילויות הנדרשות‬
measure, validate or verify other quality attributes of the
‫* שינויים משמעותיים בתכולה נוספו במהלך הפרוייקט * לא בוצע‬
system, such as scalability , reliability and resource usage.
‫ניתוח סיכונים אשר היה משמש כלי לאיתור פערים בתכנית‬
*Performance testing can verify that a system meets the
‫העבודה * בפעילויות שנרשמו יש פער ממשי באמידת המשאבים‬
specifications claimed by its manufacturer or vendor. The ‫ שינוי‬:‫ הקטנת ההסתברות‬:‫והזמן לפעילויות טיפול בסיכון‬
process can compare two or more devices or programs in ‫בתכולת העבודה (לא כל הביצים בסל אחד) * שימוש בטכנולוגיה‬
terms of parameters such as speed, data transfer rate ‫ הכנת‬:‫ צמצום הנזק‬.‫חליפית * ביצוע ניסויים להקטנת אי וודאות‬
,bandwidth, throughput, efficiency or reliability ‫אמצעי מיגון לפני האירוע (תכנית מגירה) * הכנת עתודה‬
*Performance testing can also be used as a diagnostic aid in ‫) *העברת ה סיכון לגורם אחר‬...‫ כסף‬,‫ זמן עודף‬,‫(משאבים‬
locating communications bottlenecks. Often a system will ‫ *העברת המשימה לגורם אחר אשר אצלו הסיכון נמוך‬.)‫(ביטוח‬
work much better if a problem is resolved at a single point or ‫ פעולה מונעת – פעילות שיש לבצע‬:‫ דרכי התמודדות‬.‫יותר‬
in a single component ‫ מטרתה להפחית את‬,‫ בד"כ במועד מוקדם ככל האפשר‬,‫מראש‬
‫ פעולת‬.)‫הסיכוי להתממשות הסיכון (הקטנת סבירות הסיכון‬
‫שיכוך – פעולה שיש לבצע בעת התרחשות הסיכון ומטרתה‬
Mobile Apps Performance Testing challenges Prevalence of
‫להפחית את הנזק שיגרם בעת התרחשות הסיכון (הקטנת חומרת‬
customer protocols Rapid scalability *Lack of mobile
‫ פעולה מתקנת – פעילות שיש לבצע אחרי שהסיכון‬.)‫הנזק‬
monitoring solutions *Lack of diagnostic tools *Time to
– ‫ העברת סיכון‬.‫התממש במטרה להחזיר את המצב לקדמותו‬
market *Selection of load testing tool *Test environment
‫ קבלת‬.‫העברת אחריות על הסיכון ועל הטיפול בו לגורם אחר‬
*High concurrency
.‫ לקיחת "סיכון מחושב" ולא ל נקוט בשום פעולה‬- ‫סיכון‬
Mobile Apps Performance Testing strategies Test in
‫ נקבע ונתעד‬,‫ בתהליך זיהוי הסיכונים נאתר‬:‫זיהוי סיכונים‬/‫איתור‬
Production environment *Test for 2-3 times expected
‫ לצורך הזיהוי ניתן‬.‫אילו סיכונים עלולים להשפיע על הפרויקט‬
capacity *Test accurately for the mobile environment *Test ‫ טכניקות‬,‫ הנחות‬,‫ חוות דעת מומחים‬:‫להיעזר בכלים כגון‬
across geographies *Test with Real time results ‫ * התוצר של תהליך זה הוא רשימת‬.‫ ניתוח הנחות ועוד‬,‫דיאגרמות‬
Mobile Apps AUTOMATION Automate the business critical ‫ * יש לחפש סיכונים על בסיס תוכנית העבודה‬.‫סיכונים‬/ WBS /
parts.*Automate user workflows and scenarios. *Automate
: ‫ הלקוח‬.‫ מערכות ותהליכים עסקיים‬/ ‫גאנט ונתיב קריטי‬
only complex app scenarios. * Automate sequences that
‫ העם‬:‫פרמטרים המסעיים בזיהוי סיכונים גנריים הקשורים ללקוח‬
needs to be repeated several times. * Automate only the * ?‫עבדנו עם הלקוח בעבר? * האם הלקוח יודע מה הוא רוצה‬
acceptance criteria. * Support the development team in ‫האם הלקוח מוכן להקדיש זמן משמעותי לתהליך איסוף‬
providing fast feedback. * Build up a regression test suite. * ‫הדרישות? *האם הלקוחות מוכנים להקים קשרי תקשורת מהירים‬
Automate only if it is economic. ‫עם המפתח? * האם לקוח מוכן להשתתף בביקורת? *האם‬
Criteria FOR TEST AUTOMATION TOOLS Does the tool ‫הלקוח מיומן ומבין אילוצים טכניים? *האם הלקוח מבין את תהליך‬
support different mobile app types(native, hybrid, web ,‫ דרגת חומרה‬,‫ (כשל צפוי‬:‫פיתוח תוכנה? טבלת ניהול סיכונים‬
apps)? *Which mobile platforms are supported (Android, )‫ תאור‬,‫ גורם מטפל‬,‫ דרגת דחיפות‬,‫סיכוי להתרחשות‬
iOS, Windows Phone, BlackBerry)? *Which recognition
technology does the tool use (native, image, text,
coordinate)? *Does the tool change the app you want to test
(e.g.by adding a server, instrumentation)? *Is the tool able
to execute the tests on real devices as well as on emulators
and simulators?*Can the test suite be executed on several
devices at the same time? *Does the tool support all the UI
and control elements of the mobile platform *Mobile Test
Automation and Tools 189 *Is the tool able to awake the
device from the sleep or standby mode? * Are all gestures
supported like swipe, scroll, click, tapor pinch-to-zoom? *Is
the tool able to simulate native buttons like the back or
home button? *Does the tool use the device’s soft keyboard
to enter data?*Can the app be tested in several
languages?*Does the tool support a programming language
with which you are able to write test scripts?*Can the tool
be integrated into your development environment (IDE)?
*Can the tool be integrated into a continuous integration
system? *Can the tool be combined with other tools like a
defect management or test management tool? *Is the tool
able to connect to a test cloud provider in order to execute
the tests within in a cloud? *Is the tool well documented .
*Is the tool open source or closed source? *Is there a large
community/ support behind the tool?
crowd testing process An emerging trend in software
testing which exploits the benefits, effectiveness, and
efficiency of crowd sourcing and the cloud platform *It
differs from traditional testing methods in that the testing is
carried out by a number of different testers from different
places, and not by hired consultants and professionals. *The
software is put to test under diverse realistic platforms
which makes it more reliable, *In addition, crowd source
testing allows for remote usability testing because specific
target groups can be recruited through the crowd. *This
method of testing is considered when the software is more
user-centric:
crowd testing pro’s and CON’S: Pro’s: Different testers from
around the world, with differ-ent demographic backgrounds
and skill sets. *Lots of different mobile devices with different
hard-ware and software combinations can be used for
testing. *The mobile app is tested in real world conditions
with real users. *The crowd provides a fresh set of eyes for