Efficient Test Generation for Large Systems Using SLAM
undefined
SLAM: SLice And Merge –
Effective Test Generation
for Large Systems
ICCAD’13 Review
Reviewer: Chien-Yen Kuo
Introduction
More logic squeezed into a single chip
Verification by test cases
Evaluated by functionality coverage
Consideration of system-level verification
Interaction between components
Stress on common resources
Test templates
To produce test cases more efficiently, use
test
template
-based
generator
A template consists of statements written in high-
level language
Each statement represents a hardware transaction
E.g., memory access of CPU, interruption from
external I/O device to CPU
A test generator parses statements and generate
stimuli for corresponding components
Related works
Top-level scenario composed of several
components (scenarios)
Component independency is needed
Operating system
Additional booting and running time
Interleaving
Additional scenario requirements
Irritation
A special case of SLAM
SLAM: SLice And Merge
Fact: usually a test template is associated
with only a subset of full system
Assumption: exercising a stimulus(test case)
on a subset of model does not undermine its
effectiveness (prove in experiments)
Slicing
Pre-defined/ Random/ Constrained-random
slicing
Each slice is assigned with a different test
template
Merging
Integrate SLAM into test generator
Take more than one template/slice as input
Alternatively parse statements from each
template
Maintenance
History of generated statements(?)
Current stimuli being generated
Current state of components
Verification cycle
For each function (unit, I/O bridge, or feature),
write test templates for various scenarios
Target various scenarios in one template
Target various functionalities and stress the
common resources in one template
Benefits of SLAM
Reducing manual effort on template
generation
No complex template needed, use SLAM to
generate complex test cases
Cover unexpected scenarios
Reducing regression time
Since stimuli of templates exercise slices in
parallel
Experiments
Implemented in IBM System-level Test-
Generator
Use a subset of template library
CACHE
RESERVATION
PREFETCH
NEW FEATURE
Coverage model used by System p
verification team
400,000+ events divided into 8,000+ classes
Simulation on subsets of system
On a chip with 64 hardware threads
Vary the number of participating threads
Observations
Saturated event accumulation
Larger subset does not necessarily lead to larger
coverage
Coverage growth
On a chip with 32 hardware threads
Two combinations of templates
NEW FEATURE and PREFETCH
NEW FEATURE and RESERVATION
6 test cases per template vs. 12 SLAM test
cases
Unique event coverage
On a chip with 32 hardware threads
Average results of two combinations of
templates
NEW FEATURE and PREFETCH
NEW FEATURE and CACHE
Observation
A class of events are almost always hit by SLAM
tests but not hit by any test of individual template
Conclusions
SLAM, a method for test generation for large
systems
Divide systems into slices, each of which is the
bench of a scenario
Run test cases in parallel
Reaching additional coverage events
Used as part of the system level verification
methodology for IBM System p servers
Comment and suggestions
The benefits should be solid
Suggestions
Be more specific
Slicing methods
Related works
Explanations on experimental results (example?)
Put related work review in introduction
Cite more in introduction
This research discusses SLAM (SLice And Merge), a method for effective test generation in large systems. It focuses on using test templates to generate test cases efficiently, slicing the system into subsets with different templates assigned to each slice, and merging multiple templates for thorough testing. The verification cycle involves targeting various scenarios and functionalities within a single template, stressing common resources. SLAM aims to enhance system-level verification and interaction between components, addressing the challenges of testing complex systems.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
SLAM: SLice And Merge Effective Test Generation for Large Systems ICCAD 13 Review Reviewer: Chien-Yen Kuo
Introduction More logic squeezed into a single chip Verification by test cases Evaluated by functionality coverage Consideration of system-level verification Interaction between components Stress on common resources
Test templates To produce test cases more efficiently, use test template-based generator A template consists of statements written in high- level language Each statement represents a hardware transaction E.g., memory access of CPU, interruption from external I/O device to CPU A test generator parses statements and generate stimuli for corresponding components
Related works Top-level scenario composed of several components (scenarios) Component independency is needed Operating system Additional booting and running time Interleaving Additional scenario requirements Irritation A special case of SLAM
SLAM: SLice And Merge Fact: usually a test template is associated with only a subset of full system Assumption: exercising a stimulus(test case) on a subset of model does not undermine its effectiveness (prove in experiments)
Slicing Pre-defined/ Random/ Constrained-random slicing Each slice is assigned with a different test template
Merging Integrate SLAM into test generator Take more than one template/slice as input Alternatively parse statements from each template Maintenance History of generated statements(?) Current stimuli being generated Current state of components
Verification cycle For each function (unit, I/O bridge, or feature), write test templates for various scenarios Target various scenarios in one template Target various functionalities and stress the common resources in one template
Benefits of SLAM Reducing manual effort on template generation No complex template needed, use SLAM to generate complex test cases Cover unexpected scenarios Reducing regression time Since stimuli of templates exercise slices in parallel
Experiments Implemented in IBM System-level Test- Generator Use a subset of template library CACHE RESERVATION PREFETCH NEW FEATURE Coverage model used by System p verification team 400,000+ events divided into 8,000+ classes
Simulation on subsets of system On a chip with 64 hardware threads Vary the number of participating threads Observations Saturated event accumulation Larger subset does not necessarily lead to larger coverage
Coverage growth On a chip with 32 hardware threads Two combinations of templates NEW FEATURE and PREFETCH NEW FEATURE and RESERVATION 6 test cases per template vs. 12 SLAM test cases
Unique event coverage On a chip with 32 hardware threads Average results of two combinations of templates NEW FEATURE and PREFETCH NEW FEATURE and CACHE Observation A class of events are almost always hit by SLAM tests but not hit by any test of individual template
Conclusions SLAM, a method for test generation for large systems Divide systems into slices, each of which is the bench of a scenario Run test cases in parallel Reaching additional coverage events Used as part of the system level verification methodology for IBM System p servers
Comment and suggestions The benefits should be solid Suggestions Be more specific Slicing methods Related works Explanations on experimental results (example?) Put related work review in introduction Cite more in introduction
