Software Nightmare Analysis and Testing Strategies
Explore the identification, consequences, and testing of software nightmares, ranging from incomplete requirements to feature bugs. Learn how to prioritize nightmares based on cost and probability of bugs, and design effective tests to mitigate them.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
UNIT-1 Introduction And Flowgraphs And Path Testing
1.List your worst software nightmares. State them in terms of the symptoms they produce and see how your user will react to those symptoms. 2.Convert the consequences of each nightmare into a cost. Usually this is a labor cost for correcting the nightmare. 3.Order the list from the costliest to the cheapest and then discard the low-concern nightmares
4.Based on your experience, measured data and statistics postulate the bugs that are expressed by each nightmare. 5.For each nightmare you ve developed a list of possible causative bugs, order that list by decreasing probability. 6.Rank decreasing importance to you. likely to create the symptoms the bug types in order of
7.Design tests(based on your knowledge of test techniques) and inspection process using most effective against the most important bugs 8.If a test is passed, then some nightmares or parts of them go away. If test is failed, then nightmare is possible, but upon correcting bug, it too goes away 9.Stop testing when probability of all nightmares has been shown to be inconsequential design your the methods that are quality assurance
3 3. .1 1 General 3 3. .2 2 Requirements, 3 3. .2 2. .1 1 Requirements 1.Requirements incomplete, ambiguous or self-contradictory 2.They may be misunderstood or impossible to understand 3.The specification may assume, but not mention, other specification and prerequisite that are known to the specifier but not to the designer General Requirements, Features Requirements and Features and and Specifications and and Functionality Specifications specifications Functionality Bugs Bugs developed can be
4.Requirements are major source of expensive bugs 5.The range is from a few percent to 50% depending on application and environment 3.2.2 Feature Bugs 1. Specification Problems usually create corresponding feature problems 2. A feature can be wrong, missing, or superfluous. 3. A missing feature or case is the easiest to detect and correct 3.2.2 Feature Bugs
4.A wrong feature could have deep design implications 5.Removing features might complicate the software, consume more resources, and foster more bugs. 3.2.3 Feature Interaction 1.Providing clear, correct, implementable and testable feature specifications is not enough 3.2.3 Feature Interaction
2.Features features. 3.The features of each group and the interaction of features within group are usually well tested. 4.The between feature groups or even between individual features. 5.E.g your phone is provided with call holding and call forwarding. Call holding allows you to put a new incoming - call on hold while you continue with first caller usually come in groups of related problem is unpredictable interactions
Call forwarding allows you to redirect incoming calls to some other phone no. Here questions: a)How about holding a third call when there is already a call on hold? b)Forwarding forwarded calls? And so on Every application has its peculiar set of features and a much bigger set of unspecified feature interaction interaction bugs. are some simple feature interacting potentials and therefore feature
3 3. .2 2. .4 4 Specification 1.Most feature bugs are rooted in human- to-human communication problems. 2.One solution is the use of high-level, formal specification languages or systems. Short facilitate formalization of requirements and inconsistency & ambiguity analysis. With formal specifications, partially to fully automatic test case generation is possible Specification and and feature feature bug bug remedies remedies Short term term support support: : Specification languages
Long term support:Assume we have a great specification language and that it can be used to create complete test and test criteria. The long-term impact of formal specification languages and systems will probably be that they will programming languages so that more of current specification can be formalized. This approach specification bugs. unambiguous, complete specification, influence the design of ordinary will reduce, but not eliminate,
3 3. .2 2. .5 5 Testing Techniques which are based on a behavioural description flow(4), syntax testing(9),domain testing(6), logic testing(10) and state testing(11) are useful in testing functional bugs. 3 3. .3 3 Structural 3 3. .3 3. .1 1 Control 1. Control and Sequence Bugs include paths left out, unreachable code, improper nesting of loops, loop back or loop termination criteria incorrect Testing techniques techniques of software such as transaction Structural Bugs Control and Bugs and Sequence Sequence Bugs Bugs CONTD..
Missing process steps, duplicated processing, unnecessary processing,rampaging GOTO s etc., Most control-flow bugs are easily tested and caught in unit testing. Control and sequence bugs at all levels are caught by structural tessting, more specifically path testing(3), combined with a bottom-line functional test based on a specification
These bugs are partially prevented by language choice(e.g languagues that restrict control flow options) style, most important lots of memory 3 3. .3 3. .2 2 Logic 1.Bugs in logic, especially misunderstanding logic combinations, improper layout of cases,improper negation of boolean expression CONTD.. Logic Bugs Bugs those related statements singly inconsistent to how behave include case and in operators and cases,
For e.g., using greater than as the negation of less than , confusing exclusive OR with inlusive OR. 3 3. .3 3. .2 2 Processing 1.Processing algebraic, algorithm selection and general processing. 2.E.G., improper use of greater-than, greater than equal,assumption of equality to zero in floating point,improper formats as in ASCII to binary or integer to floating point Processing Bugs Bugs bugs mathematical include arithmetic function bugs, evaluation, or equal, less than, less-than or comparison between different
3 3. .3 3. .4 4 Initialization 1.These performance. 2.Typical bugs are as follows:forgetting to initialize work space, registers, a bug in first value of a loop- control parameter; accepting an initial value without a validation check; and initializing to the wrong format, data representation or type. 3 3. .3 3. .5 5 Data 1.A along which we expect to do expect to do Something unreasonable with data Initialization Bugs bugs Bugs are less harmful but can affect Data flow data flow anomaly occurs when there is a path flow Bugs Bugs and and Anomalies Anomalies
E.G: Using an uninitialized variable, attempting to use a variable before it exists, modifying data and then not storing or using the result, or initializing twice 3.4 Data Bugs 3.4.1.General Data bugs include all bugs that arise from the specification of data objects, their formats, no of such objects and their initial values 3.4 Data Bugs 3.4.1.General
3 3. .4 4. .2 2 Dynamic 1.Dynamic data are transitory, what ever their purpose, typically the processing time of one transaction. E.G.,Object used to store data of diff types 2.Static data are fixed in form and content, what ever their purpose, they appear in the source code or database, directly or indirectly E.G., a number, a string of characters Dynamic vs vs static static they have a relatively short lifetime,
3 3. .4 4. .3 3. . Information, 1.Static or dynamic data can serve in one of three roles 2.What constitutes control or inf perspective can shift from one processing level to other. 3 3. .4 4. .4 4 Contents, Contents:The actual bit pattern, character string, or number put into a data structure. Structure:The size and shape and nos that describe the data object, i., memory locations used to store the content Information, parameter parameter and and control control is a matter of Contents, Structure Structure and and Attributes Attributes
Attributes: Specifications of meaning & semantics associated with content e.g., integer 3.5 Coding Bugs. 1.Given good source-syntax checking, the most common pure coding errors are typographical Followed by errors caused by not understanding the operation of an instruction or statement. 3.6 Interface, Integration and System Bugs 3.6.1 The external interfaces communicate with the world. 3.5 Coding Bugs. 3.6 Interface, Integration and System Bugs 3.6.1 The external interfaces are the means used to
E.g., invalid timing or sequence assumptions related to external signals,misunderstanding,external input and output formats etc., 3.6.2 Internal interfaces 1.Internal Interfaces are more controlled. 2.E.G., input and output format bugs,inadequate protection against corrupted data, wrong subroutine call sequence. 3.6.3 Hardware Architecture 1.Software bugs related to hardware architecture originate mostly from misunderstanding how the hardware works. 3.6.2 Internal interfaces 3.6.3 Hardware Architecture
E.G., paging mechanism ignored, address generation error,I/O device operation or instruction error. 3.6.4 Operating System 1.Program bugs related to the operating system are a combination of hardware architecture and interface bugs, caused by a misunderstanding of what OS Does. 3.6.5 Software Architecture 1.Software architecture bugs are often the kind that are called interactive 3.6.4 Operating System 3.6.5 Software Architecture
E.G., failure to block or unblock interrupt, assume that code is reentrant or not reentrant, called routine is resident or not resident, assumption that calling program is resident or not resident and so on. 3 3. .6 6. .6 6 Control 3 3. .6 6. .7 7 Resource 1.Memory resources such as buffer blocks, queue blocks, task control blocks. 2.Similarly external mass storage units such as discs are subdivided into memory resource pools. assumption that Control and Resource Management is and sequence Management Problems subdivided sequence bugs bugs: : refer Problems: : into refer 3 3. .3 3. .1 1 dynamically allocated
E.G., Required resource not obtained, wrong resource used, resource already in use, race condition in getting a resource and so on 3 3. .6 6. .8 8 Integration 1.Integration bugs are having to do with the integration of, and with the interfaces between, presumably working and tested components. Integration Bugs Bugs 2.Most of these bugs result from inconsistencies or incompatibilities between components. 3.All methods used to transfer data directly or indirectly between components and all methods by which components share data can host integration bugs and are therefore proper targets for integration testing
3 3. .6 6. .9 9 System 1.System Bugs is a catch-all phrase covering all kinds of bugs that cannot be ascribed to components or to their simple interactions, but result from the totality of interactions between many components such as programs, data, hardware and OS. A)All test techniques can be useful at all levels, from unit to system B)There can be no meaningful system testing until there has been thorough component and integration testing. System Bugs Bugs
Path Testing: Path Testing is the name given to a family of test techniques based on judiciously selecting a set of test paths through the program. 2.1.1.If the set of paths are properly chosen then we thoroughness. For example, pick enough paths to assure executed at least once. 2.1.2 Motivation The statement and branch atleast once under some test comes to almost every person who examines software testing in depth have achieved every some measure statement of test been that source has common sense idea of executing every
2 2. .1 1. .3 3THE 1.The strategies is that something has gone wrong with the software that makes it take a different path than intended. o As an example "GOTO X" where "GOTO Y" had been intended. 2.2 CONTROL o representation of a program's control structure. It uses decisions, and junctions THE BUG bug BUG ASSUMPTION assumption ASSUMPTION for the path testing CONTROL FLOW The the FLOW GRAPHS control elements GRAPHS: : flow named graph is process a graphical blocks, Flow different types of elements. (1) Process Block (2) Decisions (3) Junctions (4) Case Statements Flow Graph Graph Elements Elements: : A A flow graph contains four
2 2. .2 2. .2 2Process A uninterrupted by either decisions or junctions. It is a sequence of statements such that if any one of statement of the block is executed, then all statement thereof is executed. Formally, a process block is a piece of straight line code of one statement or hundreds of statements. A process has one entry and one exit. It can consist of a single statement or instruction, a sequence of statements or instructions, a single entry/exit subroutine, a macro or function call, or a sequence of these. 2 2. .2 2. .3 3 Decisions 1.A decision is a program point at which the control flow can diverge Process Block process Block: : block is a sequence of program statements Decisions
2.Machine language conditional branch and conditional skip instructions are examples of decisions. 3.Most of the decisions are two-way but some are three way branches in control flow. 2 2. .2 2. .3 3 Case 1.A case statement is a multi-way branch or decisions. 2 2. .2 2. .4 4Junctions 1.A junction is a point in the program where the control flow can merge. Examples of junctions are: the target of a jump or skip instruction in ALP, a label that is a target of GOTO. Case Statements Statements Junctions: :
2.2.5CONTROL 1.A program's flow chart resembles a control flow graph. 2.In flow graphs, we don't show the details of what is in a process block. 3.In flow charts every part of the process block is drawn. 4.The flowchart focuses on process steps, where as the flow graph focuses on control flow of the program. 5.The act of drawing a control flow graph is a useful tool that can help us clarify the control flow and data flow issues. CONTROL FLOW FLOW GRAPHS GRAPHS Vs Vs FLOWCHARTS FLOWCHARTS: :
2.2.6NOTATIONAL EVOULTION: The control flow graph is simplified representation of the program's structure. NOTATIONAL EVOULTION: Figure 2.2: Program Example (PDL) Figure 2.2: Program Example (PDL)
PATH TESTING Path: a path through a program is a sequence of instructions or statements that starts at an entry, junction, or decision and ends at another, or possibly the same junction, decision, or exit. A path may go through several junctions, processes, or decisions, one or more times. Paths consist of segments. The segment is a link - a single process that lies between two nodes. A path segment is succession of consecutive links that belongs to some path. The length of path measured by the number of links in it and not by the number of the instructions or statements executed along that path. PATH TESTING - - PATHS, NODES AND LINKS: Path: a path through a program is a sequence of instructions or statements that starts at an entry, junction, or decision and ends at another, or possibly the same junction, decision, or exit. PATHS, NODES AND LINKS:
FUNDAMENTAL There are many paths between the entry and exit of a typical routine. Every decision doubles the number of potential paths. And every loop multiplies the number of potential paths by the number of different iteration values possible for the loop. Defining complete testing: 0. Exercise(=WORK OUT) every path from entry to exit 1. Exercise every statement or instruction at least once 2. Exercise every branch and case statement, in each direction at least once FUNDAMENTAL PATH PATH SELECTION SELECTION CRITERIA CRITERIA: :
Path Execute all possible control flow paths through the program: typically, this is restricted to all possible entry/exit paths through the program. Path Testing Testing : : Statement Execute all statements in the program at least once under some test. If we do enough tests to achieve this, we are said to have achieved 100% statement coverage. Statement Testing Testing (P (P1 1) ): : Branch Execute enough tests to assure that every branch alternative has been exercised at least once under some test. Branch Testing Testing (P (P2 2) ): :
Testing Blindness 1.Most prog languages permit compound predicates at decisions predicates of form A OR B or A AND B. and boolean expressions E.G X=17 (that is IF X=17 THEN ----) The opp branch is X NE 17, Which is equivalent to X>17 OR X<17. ---
E.G IF A GOTO BOB ELSE GOTO COB BOB:DO SOMETHING -------- GOTO SAM COB:DO SOMETHING ELSE ------- GOTO SAM ------- SAM:IF X DO ALPHA ELSE DO BETA
Testing blindness is a pathological situation In which the desire path is achieved for the wrong reason. a)Assignment blindness: Occurs when the buggy predicate appears to work correctly because the specific value chosen for an assignment statement works with both the correct and incorrect predicate Correct X:=7 x:=7 ----- IF Y>0 THEN IF X+Y>0 THEN Buggy ------
Testing blindness is a pathological situation In which the desire path is achieved for the wrong reason. a)Equality blindness: Occurs when the path selected by a prior predicate results in a value that works both for the correct and buggy predicate Correct Y:=2 THEN Y:=7 THEN ----- IF X+Y>3 THEN IF X>1 THEN Buggy ------
Testing blindness is a pathological situation In which the desire path is achieved for the wrong reason. a)Self blindness: Occurs when the buggy predicate is a multiple of the correct predicate and as a result is indistinguishable along that path Correct X:=A ----- IF X-1>0 THEN IF X+A-2>0 THEN Buggy ------ X:=A
REVIEW: ACHIEVABLE AND UNACHIEVABLE PATHS: 1. achieve a satisfactory notion of test completeness. 2.Extract the programs control flowgraph and select a set of tentative covering paths. 3. For any path in that set, interpret the predicates along the path as needed to express them in terms of the input vector. In general individual predicates are compound or may become compound as a result of interpretation. 4.Trace the path through, multiplying the individual compound predicates to achieve a boolean expression such as(A+BC) (D+E) (FGH) (IJ) (K) (l) (L). REVIEW: ACHIEVABLE AND UNACHIEVABLE PATHS: 1.We want to select and test enough paths to (A+BC) (D+E) (FGH) (IJ) (K) (l) (L).
5. Multiply out the expression to achieve a sum of products form: ADFGHIJKL+AEFGHIJKL+BCDFGHIJKL+BCEFGHIJKL 6 6. .Each product term denotes a set of inequalities that if solved will yield an input vector that will drive the routine along the designated path. 7 7. . Solve any one of the inequality sets for the chosen path and you have found a set of input values for the path. 8 8. . If you can find a solution, then the path is achievable. 9 9. . If you cant find a solution to any of the sets of inequalities, the path is un achievable. 10 expression is called PATH ADFGHIJKL+AEFGHIJKL+BCDFGHIJKL+BCEFGHIJKL 10. .The act of finding a set of solutions to the path predicate PATH SENSITIZATION SENSITIZATION.
HEURISTIC PROCEDURES FOR SENSITIZING PATHS: 1. the paths without considering how to sensitize, attempt to choose a covering path set that is easy to sensitize and pick hard to sensitize paths only as you must to achieve coverage. 2. Identify all variables that affect the decision. 3. Classify the predicates as dependent or independent. 4. Start the path selection with un correlated, independent predicates. HEURISTIC PROCEDURES FOR SENSITIZING PATHS: 1.This is a workable approach, instead of selecting
5. If coverage has not been achieved using independent uncorrelated predicates, extend the path set using correlated predicates. 6. If coverage has not been achieved extend the cases to those that involve dependent predicates. 7. Last, use correlated, dependent predicates.
PATH what we have to do to confirm that the outcome was achieved by the intended path. Co correctness stands for achieving the desired outcome for wrong reason. PATH INSTRUMENTATION: INSTRUMENTATION:Path instrumentation is Co- -incidental Correctness: incidental Correctness: The coincidental
The above figure is an example of a routine that, for the (unfortunately) chosen input value (X = 16), yields the same outcome (Y = 2) no matter which case we select. Therefore, the tests chosen this way will not tell us whether we have achieved coverage. For example, the five cases could be totally jumbled and still the outcome would be the same.
