Comprehensive Overview of Software Testing and Strategies

Testing Principles

Testing Objectives

•

T

e

s

t

i

n

g

i

s

•

t

h

e

p

r

o

c

e

s

s

o

f

e

x

e

r

c

i

s

i

n

g

a

p

r

o

g

r

a

m

•

w

i

t

h

t

h

e

s

p

e

c

i

f

i

c

i

n

t

e

n

t

o

f

f

i

n

d

i

n

g

e

r

r

o

r

s

•

p

r

i

o

r

t

o

d

e

l

i

v

e

r

y

t

o

t

h

e

e

n

d

u

s

e

r

.

What Testing Shows?

e

e

r

r

r

r

o

o

r

r

s

s

r

r

e

e

q

q

u

u

i

i

r

r

e

e

m

m

e

e

n

n

t

t

s

s

c

c

o

o

n

n

f

f

o

o

r

r

m

m

a

a

n

n

c

c

e

e

p

p

e

e

r

r

f

f

o

o

r

r

m

m

a

a

n

n

c

c

e

e

a

a

n

n

i

i

n

n

d

d

i

i

c

c

a

a

t

t

i

i

o

o

n

n

o

o

f

f

q

q

u

u

a

a

l

l

i

i

t

t

y

y

Testing Principles

1.

All the tests should be traceable to customer

requirements

2.

Tests should be planned

3.

80% of errors – uncovered, during testing

4.

Testing should begin “in the small” & progress toward

testing  “in the large”

5.

Testing should be conducted by – independent third party

A strategic Approach to Software Testing

•

Testing strategy provides a process,

•

that describes for the developer, quality

analysis & the customer

•

The steps conducted as a part of testing

•

The testing strategy includes -

The testing strategy includes

•

Test planning

•

Test case design

•

Test execution

•

Data collection

•

Effectiveness evaluation

The Strategic Approach for S/W Testing Can Be

•

Before starting testing process,

•

Formal technical reviews must be conducted

•

Various 

components

 of the system are tested

•

Then 

interface

 is tested

•

i.e. entire 

computer based system 

is tested

•

Different testing techniques can be applied at

different point of time

•

The 

developer

 of the software conducts testing

•

For the large projects, 

independent test groups

also assists the developers

•

Testing & debugging 

are different activities

•

That must be carried out  in software testing

•

Debugging also lies within any testing strategy

Testing Strategy

System engineering

Analysis modeling

Design modeling

Code generation

Unit test

Integration test

Validation test

System test

Testing Strategy

•

We begin by 

‘testing-in-the-small’

•

And

•

move toward 

‘testing-in-the-large’

Verification & Validation

verification

•

Set of activities

•

That ensure that

•

Software correctly implements a specific

function

Strategic Issues

•

Before the testing stars, specify the product requirements appropriately

•

Specify testing objective clearly

•

Identify categories of users for the software e & their role with reference

to software system

•

Develop a test plan that emphasizes rapid cycle testing

•

Build robust software which can b e useful for testing

•

Use effective formal reviews before actual testing process begins

•

Conduct formal technical reviews to assess the test strategy & test cases

•

Throughout the testing process adopt the continuous development

strategy

Testing Strategies for Conventional  Software

1.

Unit testing

2.

Integration testing

•

Non-incremental integration

-

big-bang approach

•

Incremental integration

-

top down testing

-

bottom up integration

-

regression testing

-

smoke testing

3.

Validation testing

-

acceptance testing

-

alpha testing

-

beta testing

4.

System testing

-

recovery testing

-

security testing

-

stress testing

-

performance testing

•

A unit is the smallest testable part of software.

•

It usually has one or a few inputs and usually a single output.

•

In procedural programming

-

a unit may be an individual program, function, procedure, etc.

•

In object-oriented programming,

-

the smallest unit is a method, which may belong to

-

a base/ super class, abstract class or derived/ child class.

•

Unit testing frameworks, drivers, stubs, and mock/ fake objects are

used to assist in unit testing.

1. Unit Testing

•

Individual components are tested independently

•

Driver 

and 

stub

 software are used in unit testing

•

Driver –

-

a program,

-

that 

accepts the test data 

& 

prints the relevant results

•

Stub –

-

a subprogram,

-

that 

uses – module interfaces

-

and

-

performs minimal 

data manipulation

-

If required

Unit Testing

m

o

d

u

l

e

t

o

b

e

t

e

s

t

e

d

t

t

e

e

s

s

t

t

c

c

a

a

s

s

e

e

s

s

r

r

e

e

s

s

u

u

l

l

t

t

s

s

s

s

o

o

f

f

t

t

w

w

a

a

r

r

e

e

e

e

n

n

g

g

i

i

n

n

e

e

e

e

r

r

Unit Testing

i

i

n

n

t

t

e

e

r

r

f

f

a

a

c

c

e

e

l

l

o

o

c

c

a

a

l

l

d

d

a

a

t

t

a

a

s

s

t

t

r

r

u

u

c

c

t

t

u

u

r

r

e

e

s

s

b

b

o

o

u

u

n

n

d

d

a

a

r

r

y

y

c

c

o

o

n

n

d

d

i

i

t

t

i

i

o

o

n

n

s

s

i

i

n

n

d

d

e

e

p

p

e

e

n

n

d

d

e

e

n

n

t

t

p

p

a

a

t

t

h

h

s

s

e

e

r

r

r

r

o

o

r

r

h

h

a

a

n

n

d

d

l

l

i

i

n

n

g

g

p

p

a

a

t

t

h

h

s

s

m

o

d

u

l

e

t

o

b

e

t

e

s

t

e

d

t

t

e

e

s

s

t

t

c

c

a

a

s

s

e

e

s

s

Unit Test Environment

M

M

o

o

d

d

u

u

l

l

e

e

s

s

t

t

u

u

b

b

s

s

t

t

u

u

b

b

d

d

r

r

i

i

v

v

e

e

r

r

R

R

E

E

S

S

U

U

L

L

T

T

S

S

i

i

n

n

t

t

e

e

r

r

f

f

a

a

c

c

e

e

l

l

o

o

c

c

a

a

l

l

d

d

a

a

t

t

a

a

s

s

t

t

r

r

u

u

c

c

t

t

u

u

r

r

e

e

s

s

b

b

o

o

u

u

n

n

d

d

a

a

r

r

y

y

c

c

o

o

n

n

d

d

i

i

t

t

i

i

o

o

n

n

s

s

i

i

n

n

d

d

e

e

p

p

e

e

n

n

d

d

e

e

n

n

t

t

p

p

a

a

t

t

h

h

s

s

e

e

r

r

r

r

o

o

r

r

h

h

a

a

n

n

d

d

l

l

i

i

n

n

g

g

p

p

a

a

t

t

h

h

s

s

t

t

e

e

s

s

t

t

c

c

a

a

s

s

e

e

s

s

BENEFITS (Advantages) of Unit Testing

•

Unit testing increases confidence in changing/ maintaining code.

•

If good unit tests are written

-

and if they are run every time any code is changed,

-

we will be able to promptly catch any defects

-

introduced due to the change.

•

Also, if codes are already made less interdependent

-

to make unit testing possible,

-

the unintended impact of changes to any code is less.

•

Codes are more reusable.

-

In order to make unit testing possible,

-

codes need to be modular.

•

This means that codes are easier to reuse.

•

The cost of fixing a defect

-

detected during unit testing is lesser

-

in comparison to that of defects

-

detected at higher levels.

•

Compare the cost (time, effort, destruction, humiliation)

-

of a defect detected

-

during acceptance testing or when the software is live.

•

Debugging is easy.

-

When a test fails, only the latest changes need to be

debugged.

-

With testing at higher levels,

-

changes made over the span of several days/ weeks/ months

need to be scanned.

Integration Testing

•

is a 

level of software testing

-

where individual units are combined and tested as

a group.

•

The purpose of this level of testing is

-

to expose

-

faults in the interaction

-

between integrated units.

•

Test drivers and test stubs are used

-

to assist in Integration Testing.

•

Definition by ISTQB

•

integration testing:

•

Testing performed to expose defects

•

in the interfaces and in the interactions

•

between integrated components or systems

•

ANALOGY

•

During the process of manufacturing a ballpoint

pen, the cap, the body, the tail and clip, the ink

cartridge and the ballpoint are produced

separately and unit tested separately.

•

When two or more units are ready, they are

assembled and Integration Testing is performed.

For example, whether the cap fits into the body

or not.

•

When is Integration Testing performed?

•

Integration Testing is performed after 

Unit

Testing

 and before 

System Testing

.

•

Who performs Integration Testing?

•

Either Developers themselves or independent

Testers perform Integration Testing.

•

APPROACHES

•

Big Bang

 is an approach to Integration Testing

where all or most of the units are combined

together and tested at one go. This approach is

taken when the testing team receives the entire

software in a bundle. So what is the difference

between Big Bang Integration Testing and System

Testing? Well, the former tests only the

interactions between the units while the latter

tests the entire system.

•

Top Down

 is an approach to Integration

Testing where top level units are tested first

and lower level units are tested step by step

after that. This approach is taken when top

down development approach is followed. Test

Stubs are needed to simulate lower level units

which may not be available during the initial

phases.

•

Bottom Up

 is an approach to Integration

Testing where bottom level units are tested

first and upper level units step by step after

that. This approach is taken when bottom up

development approach is followed. Test

Drivers are needed to simulate higher level

units which may not be available during the

initial phases.

•

Sandwich/Hybrid

 is an approach to Integration

Testing which is a combination of Top Down

and Bottom Up approaches.

2.Integration Testing

•

Group of dependent components are tested together

•

Uncovers errors in 

–

-

Design and construction of software architecture.

-

Integrated functions and operation at system level

-

Interfaces and interactions between them

-

Resource integration and environmental integration

Integration Testing Strategies

O

O

p

p

t

t

i

i

o

o

n

n

s

s

:

:

•

•

t

t

h

h

e

e

“

“

b

b

i

i

g

g

b

b

a

a

n

n

g

g

”

”

a

a

p

p

p

p

r

r

o

o

a

a

c

c

h

h

•

•

a

a

n

n

i

i

n

n

c

c

r

r

e

e

m

m

e

e

n

n

t

t

a

a

l

l

c

c

o

o

n

n

s

s

t

t

r

r

u

u

c

c

t

t

i

i

o

o

n

n

s

s

t

t

r

r

a

a

t

t

e

e

g

g

y

y

Approaches Of Integration Testing

•

Non incremental integration

-

big-bang approach

•

Incremental integration

-

top down testing

-

Bottom up testing

-

Regression testing

-

Smoke testing

Non Incremental Integration

(Big-bang Approach)

•

Steps –

-

All components are combined in advance

-

The entire program is tested as a whole

-

And  - 

chaos 

– usually results

-

set of errors

 are tested as a whole

-

once these errors are corrected, new ones

appear

-

this process continues infinitely

•

Advantages –

-

simple

•

Disadvantages –

-

Hard to debug

-

difficult to isolate errors while testing

Incremental Integration

-

top down integration testing

-

Bottom up integration  testing

-

Regression testing

-

Smoke testing

Top Down Integration

t

t

o

o

p

p

m

m

o

o

d

d

u

u

l

l

e

e

i

i

s

s

t

t

e

e

s

s

t

t

e

e

d

d

w

w

i

i

t

t

h

h

s

s

t

t

u

u

b

b

s

s

s

s

t

t

u

u

b

b

s

s

a

a

r

r

e

e

r

r

e

e

p

p

l

l

a

a

c

c

e

e

d

d

o

o

n

n

e

e

a

a

t

t

a

a

t

t

i

i

m

m

e

e

,

,

"

"

d

d

e

e

p

p

t

t

h

h

f

f

i

i

r

r

s

s

t

t

"

"

a

a

s

s

n

n

e

e

w

w

m

m

o

o

d

d

u

u

l

l

e

e

s

s

a

a

r

r

e

e

i

i

n

n

t

t

e

e

g

g

r

r

a

a

t

t

e

e

d

d

,

,

s

s

o

o

m

m

e

e

s

s

u

u

b

b

s

s

e

e

t

t

o

o

f

f

t

t

e

e

s

s

t

t

s

s

i

i

s

s

r

r

e

e

-

-

r

r

u

u

n

n

A

A

B

B

C

C

D

D

E

E

F

F

G

G

42

Bottom-Up Integration

d

d

r

r

i

i

v

v

e

e

r

r

s

s

a

a

r

r

e

e

r

r

e

e

p

p

l

l

a

a

c

c

e

e

d

d

o

o

n

n

e

e

a

a

t

t

a

a

t

t

i

i

m

m

e

e

,

,

"

"

d

d

e

e

p

p

t

t

h

h

f

f

i

i

r

r

s

s

t

t

"

"

w

w

o

o

r

r

k

k

e

e

r

r

m

m

o

o

d

d

u

u

l

l

e

e

s

s

a

a

r

r

e

e

g

g

r

r

o

o

u

u

p

p

e

e

d

d

i

i

n

n

t

t

o

o

b

b

u

u

i

i

l

l

d

d

s

s

a

a

n

n

d

d

i

i

n

n

t

t

e

e

g

g

r

r

a

a

t

t

e

e

d

d

A

A

B

B

C

C

D

D

E

E

F

F

G

G

c

c

l

l

u

u

s

s

t

t

e

e

r

r

Regression Testing

•

re-execution 

of some subset of tests that have already been conducted

-

to ensure that

-

changes have not propagated 

unintended side effects

•

Whenever software is corrected,

-

some aspect of the software configuration is changed.

•

Regression testing helps to ensure that

-

changes 

(due to testing or for other reasons)

-

do not introduce unintended behavior or additional errors

.

•

Regression testing may be conducted manually,

-

by re-executing a subset of all test cases

-

or using automated capture/playback tools.

Smoke Testing

•

A common approach for creating 

“daily builds”

 for product software

•

steps:

–

Software components that have been translated into code are

integrated into a “build.”

•

A build includes all data files, libraries, reusable modules,  etc.

–

The build is integrated with other builds

–

and the entire product is smoke tested daily.

3. Validation Testing

Acceptance testing

•

Conducted to ensure that –

•

The s/w works correctly

•

In the user work environment

•

Can be conducted over a period of weeks or

months

Types of Acceptance Testing

•

Alpha testing

-

complete s/w is tested by customer

-

under supervision of developer

-

at developer’s site

-

in controlled environment

•

Beta testing

-

complete s/w is tested by customer

-

without developer being present

-

at customer’s site

-

no control of developer

4. System Testing

•

DEFINITION

•

System Testing 

is a 

level of the software testing

-

where a complete and integrated software is tested.

•

The purpose of this test is

-

to evaluate the system’s compliance

-

with the specified requirements.

•

Definition by ISTQB

system testing:

-

The process of testing an integrated system

-

to verify that it meets specified requirements.

•

The system test is

-

a series of tests conducted

-

to fully test

-

the computer based system

-

(As a single unit)

•

When is it performed?

-

System Testing is performed

-

after 

Integration Testing

 and before

Acceptance Testing

.

•

Who performs it?

-

Normally,

-

independent Testers perform System Testing.

•

ANALOGY

•

During the process of manufacturing a

ballpoint pen, the cap, the body, the tail, the

ink cartridge and the ballpoint are produced

separately and unit tested separately. When

two or more units are ready, they are

assembled and Integration Testing is

performed. When the complete pen is

integrated, System Testing is performed

Types of System Tests

•

Recovery testing

-

System’s ability to recover from failures.

-

s/w is forced to fail & then it is verified

-

whether the system is properly recoverable or not

-

Reinitialization, checkpoint mechanisms, data

recovery & restart

-

are verified

•

Security testing

-

Verifies the protection mechanism – unauthorized

internal or external access

-

Or willful damage

•

Stress testing

-

determines breakpoint of a system, to 

establish

maximum 

service level

-

Software is tested at highest load

-

than normal working environment

-

i.e. system is executed for – resources in abnormal

quantity, frequency or volume

•

Performance testing

-

runtime performance of the software is evaluated

-

resource utilization is measured

-

such as – CPU load, throughput, response time,

memory usage

-

ex. Beta testing

Acceptance Testing

•

DEFINITION

•

Acceptance Testing 

is a 

level of the software testing

 where a system

is tested for acceptability.

•

The purpose of this test is

-

to evaluate the system’s compliance with the business

requirements

-

and assess whether it is acceptable for delivery.

•

Definition by ISTQB

•

acceptance testing:

-

Formal testing with respect to user needs, requirements, and

business processes

-

conducted to determine  whether or not a system satisfies the

acceptance criteria and

-

to enable the user, customers or other authorized entity

-

to determine whether or not to accept the system.

•

ANALOGY

•

During the process of manufacturing a ballpoint pen,

-

the cap, the body, the tail and clip, the ink 

cartridge and

the ballpoint are produced 

separately and unit tested

separately.

•

When two or more units are ready,

-

they are assembled and Integration Testing is performed.

•

When the complete pen is integrated,

-

System Testing is performed.

•

Once System Testing is complete, Acceptance Testing is

performed

-

so as to confirm that

-

the ballpoint pen is ready to be made

-

available to the end-users.

•

METHOD

•

Usually, 

Black Box Testing

 method is used in

Acceptance Testing.

•

Testing does not normally follow a strict

procedure

•

and is not scripted but is rather 

ad-hoc

.

•

When is it performed?

•

Acceptance Testing is performed

•

after 

System Testing

 and before making the

system available for actual use.

•

Who performs it?

•

Internal Acceptance Testing 

(Also known as Alpha

Testing)

•

is performed by members of the organization that

developed the software

•

but who are not directly involved in the project

(Development or Testing).

•

Usually, it is the members of Product Management,

Sales and/or Customer Support.

•

External Acceptance Testing

•

is performed by people who are not employees of the

organization that developed the software.

–

Customer Acceptance Testing 

is performed by the

customers

–

User Acceptance Testing 

(Also known as Beta Testing)

–

is performed by the end users of the software.

–

They can be the customers themselves or the customers’

customers.

Testing Strategies For

Object-Oriented Software

•

Basic unit of testing is  - a 

class

-

That contains attributes & operations

•

These classes 

integrate

-

To form object oriented architecture

•

These are called collaborating classes

•

Unit testing, in OO context

•

Integration testing, in OO context

Unit Testing

•

Classes and objects are tested

-

By considering

-

data attributes & corresponding

operations.

-

encapsulation – is tested.

-

derived classes (inherited) are tested.

-

polymorphism is tested.

Integration Testing

•

Thread based testing –

-

Integrates all classes that respond to one

input or event for the system.

-

Each thread is then tested individually

•

Use based testing –

-

Independent classes are tested first

-

and then, dependent classes are tested

Validation Testing

•

Conventional Black box testing

OO Testing Strategy

•

class testing is the 

equivalent of

unit testing

–

operations 

within the class are tested

–

the 

state behavior 

of the class is examined

•

integration applied three different strategies

–

thread-based testing

—

–

integrates the set of classes

–

 required to respond to one input or event

–

use-based testing

—

–

integrates the set of classes

–

required to respond to one use case

–

cluster testing

—

–

integrates the set of classes

–

required to demonstrate one collaboration

OO Testing Methods

•

Conventional Test Case Design Methods

-

White box testing –  basis path testing and loop testing 

-

Black box testing  -  use cases

•

Fault Based Testing

-

valid faults are identified

-

Integration testing detects – three types of faults -

-

wrong operation used

-

unexpected result

-

incorrect invocation

•

Scenario Based Testing

-

many interaction errors are uncovered by building the

scenarios

Testing strategies for - webapps

WebApp Testing I

•

The 

Content Model

 for the WebApp

-

contents are evaluated 

Syntactically & Semantically

-

syntactic  errors – like – 

spellings, grammar etc.

 are assessed.

-

semantic  errors – 

correctness, consistency & ambiguity are

assessed.

•

The Interface Model

-

is reviewed to ensure that all use cases can be accommodated.

•

The design model

-

for the WebApp is reviewed to uncover navigation errors.

•

The User Interface

-

is tested to uncover errors

-

in presentation and/or navigation 

mechanics.

•

Each functional component is unit tested.

Webapp Testing II

•

Navigation throughout the architecture is tested.

•

The WebApp is implemented

-

in a variety of different environmental configurations

-

and is tested for compatibility with each configuration.

•

Security tests are conducted

-

in an attempt to exploit vulnerabilities in the WebApp

-

or within its environment.

•

Performance tests are conducted.

•

The WebApp is tested

-

by a controlled and monitored population

-

of end-users.

-

The results of their interaction with the 

system

-

are evaluated

-

for  content and navigation errors,

-

usability concerns, compatibility concerns, and

WebApp reliability and performance.

High Order Testing

•

Validation testing - 

Focus is on software requirements

•

System testing - 

Focus is on system integration

•

Alpha/Beta testing - 

Focus is on customer usage

•

Recovery testing - 

forces the software to fail in a variety of ways and verifies that recovery

is properly performed

•

Security testing - 

verifies that protection mechanisms built into a system will, in fact,

protect it from improper penetration

•

Stress testing - 

 executes a system in a manner that demands resources in abnormal

quantity, frequency, or volume

•

Performance Testing - 

test the run-time performance of software within the context of an

integrated system

Debugging: A Diagnostic Process

Art Of Debugging

•

Debugging –

-

is a process of 

removal of a defect

-

It occurs as 

a consequence 

of successful

testing

Debugging Process

1.

Test cases are executed

2.

Actual results are compared with expected results

3.

Find – the lack of correspondence between actual and

expected results

4.

The suspected causes are identified

5.

And additional tests or regression tests are performed;

To make the system work as per the requirement

The Debugging Process

Debugging Approaches (Techniques)

•

Brute Force Method –

-

“Let Computer Find The Error” approach is used

-

Memory dumps & runtime traces are examined

-

Program with right statements is loaded

-

To obtain clues to errors

•

Backtracking Method 

–

-

The source code is examined by looking backwards

-

From  



symptom - To potential causes of errors

•

Cause Elimination method –

-

binary partitioning method is used

-

to reduce number of locations

-

where errors can exist

Testing Vs Debugging

Debugging Effort

t

t

i

i

m

m

e

e

r

r

e

e

q

q

u

u

i

i

r

r

e

e

d

d

t

t

o

o

d

d

i

i

a

a

g

g

n

n

o

o

s

s

e

e

t

t

h

h

e

e

s

s

y

y

m

m

p

p

t

t

o

o

m

m

a

a

n

n

d

d

d

d

e

e

t

t

e

e

r

r

m

m

i

i

n

n

e

e

t

t

h

h

e

e

c

c

a

a

u

u

s

s

e

e

t

t

i

i

m

m

e

e

r

r

e

e

q

q

u

u

i

i

r

r

e

e

d

d

t

t

o

o

c

c

o

o

r

r

r

r

e

e

c

c

t

t

t

t

h

h

e

e

e

e

r

r

r

r

o

o

r

r

a

a

n

n

d

d

c

c

o

o

n

n

d

d

u

u

c

c

t

t

r

r

e

e

g

g

r

r

e

e

s

s

s

s

i

i

o

o

n

n

t

t

e

e

s

s

t

t

s

s

Symptoms & Causes

s

s

y

y

m

m

p

p

t

t

o

o

m

m

c

c

a

a

u

u

s

s

e

e

s

s

y

y

m

m

p

p

t

t

o

o

m

m

m

m

a

a

y

y

d

d

i

i

s

s

a

a

p

p

p

p

e

e

a

a

r

r

w

w

h

h

e

e

n

n

a

a

n

n

o

o

t

t

h

h

e

e

r

r

p

p

r

r

o

o

b

b

l

l

e

e

m

m

i

i

s

s

f

f

i

i

x

x

e

e

d

d

c

c

a

a

u

u

s

s

e

e

m

m

a

a

y

y

b

b

e

e

d

d

u

u

e

e

t

t

o

o

a

a

c

c

o

o

m

m

b

b

i

i

n

n

a

a

t

t

i

i

o

o

n

n

o

o

f

f

n

n

o

o

n

n

-

-

e

e

r

r

r

r

o

o

r

r

s

s

c

c

a

a

u

u

s

s

e

e

m

m

a

a

y

y

b

b

e

e

d

d

u

u

e

e

t

t

o

o

a

a

s

s

y

y

s

s

t

t

e

e

m

m

o

o

r

r

c

c

o

o

m

m

p

p

i

i

l

l

e

e

r

r

e

e

r

r

r

r

o

o

r

r

c

c

a

a

u

u

s

s

e

e

m

m

a

a

y

y

b

b

e

e

d

d

u

u

e

e

t

t

o

o

a

a

s

s

s

s

u

u

m

m

p

p

t

t

i

i

o

o

n

n

s

s

t

t

h

h

a

a

t

t

e

e

v

v

e

e

r

r

y

y

o

o

n

n

e

e

b

b

e

e

l

l

i

i

e

e

v

v

e

e

s

s

s

s

y

y

m

m

p

p

t

t

o

o

m

m

m

m

a

a

y

y

b

b

e

e

i

i

n

n

t

t

e

e

r

r

m

m

i

i

t

t

t

t

e

e

n

n

t

t

Consequences of Bugs

d

a

m

a

g

e

m

i

l

d

a

n

n

o

y

i

n

g

d

i

s

t

u

r

b

i

n

g

s

e

r

i

o

u

s

e

x

t

r

e

m

e

c

a

t

a

s

t

r

o

p

h

i

c

i

n

f

e

c

t

i

o

u

s

B

u

g

T

y

p

e

B

B

u

u

g

g

C

C

a

a

t

t

e

e

g

g

o

o

r

r

i

i

e

e

s

s

:

:

f

f

u

u

n

n

c

c

t

t

i

i

o

o

n

n

-

-

r

r

e

e

l

l

a

a

t

t

e

e

d

d

b

b

u

u

g

g

s

s

,

,

s

s

y

y

s

s

t

t

e

e

m

m

-

-

r

r

e

e

l

l

a

a

t

t

e

e

d

d

b

b

u

u

g

g

s

s

,

,

d

d

a

a

t

t

a

a

b

b

u

u

g

g

s

s

,

,

c

c

o

o

d

d

i

i

n

n

g

g

b

b

u

u

g

g

s

s

,

,

d

d

e

e

s

s

i

i

g

g

n

n

b

b

u

u

g

g

s

s

,

,

d

d

o

o

c

c

u

u

m

m

e

e

n

n

t

t

a

a

t

t

i

i

o

o

n

n

b

b

u

u

g

g

s

s

,

,

s

s

t

t

a

a

n

n

d

d

a

a

r

r

d

d

s

s

v

v

i

i

o

o

l

l

a

a

t

t

i

i

o

o

n

n

s

s

,

,

e

e

t

t

c

c

.

.

Correcting The Error

•

Is the cause of the bug reproduced in another part of the program?

•

In many situations, a program defect is caused by an erroneous pattern of

logic that may be reproduced elsewhere.

•

What "next bug" might be introduced by the fix I'm about to make?

•

Before the correction is made,

•

the source code (or, better, the design) should be evaluated to assess

coupling of logic and data structures.

•

What could we have done to prevent this bug in the first place?

•

This question is the first step toward establishing a statistical software

quality assurance approach.

•

If you correct the process as well as the product, the bug will be removed

from the current program and may be eliminated from all future programs.

Final Thoughts

•

Think

 -- before you act to correct

•

Use tools to gain additional insight

•

If you’re at an impasse, get help from someone

else

•

Once you correct the bug, use regression testing

to uncover any side effects