Test Strategy and Delivery Squad Coverage Recap
Agenda
▷
Recap test strategy (1 chart)
▷
Delivery squad function test coverage for 1.2
▷
How to utilize the new E2E test framework
▷
How to manage and track in Jira
(E2E)
(SDK)
Test strategy – 2018
System test
Test squads
PTE, LTE, OTE,
etc
•
Tests that ensure unit
tested components
work together
•
Tested outside of the
production package
implementations
•
Using test friendly
configuration but real
implementations
•
Production code
structured for test
•
True unit tests
with high coverage
•
Tool generated
mocks for
dependencies
•
Orchestrated tests that use documented, user facing
assets
•
Each user consumable feature should extend this suite
•
These are validation tests, not tests that are expected to
stress the system or introduce errors
•
Use a small number of deployed assets (eg. chaincode,
peers, orgs) across multiple tests
•
Limit execution time to seconds per variation, not minutes
Function test
Integration Test
E2E
Unit test
V1.2 integration test strategy
▷
Lots of good progress has been made, but we can’t
swallow everything at once.
▷
Fabric is not well structured for integration test in
v1.2 timeframe
▷
How to cover integration test gap while we evolve
towards true integration test?
■
Expand Unit test coverage
■
Expand E2E coverage
Test strategy – V1.2 Phased approach
Test squads
PTE, LTE, OTE,
etc
•
Tests that ensure unit tested
components work together
•
Tested outside of the
production package
implementations
•
Using test friendly
configuration but real
implementations
•
Production code
structured for test
•
True unit tests with
high coverage
•
Tool generated mocks
for dependencies
•
Orchestrated tests that use documented, user facing assets
•
Each user consumable feature should extend this suite
•
These are validation tests, not tests that are expected to stress the
system or introduce errors
•
Use a small number of deployed assets (eg. chaincode, peers, orgs)
across multiple tests
•
Limit execution time to seconds per variation, not minutes
Strategy
V1.2
•
Production code structured for test (best effort)
•
Combination of true unit tests and “unit
integration tests”
•
Tool generated mocks for dependencies
•
Orchestrated tests that use documented, user facing assets
•
Each user consumable feature should extend this suite
•
These are validation tests, not tests that are expected to stress the system or introduce errors
•
Use a small number of deployed assets (eg. chaincode, peers, orgs) across multiple tests
•
Limit execution time to seconds per variation, not minutes
Not ready in v1.2 timeframe
System test
Function test
Integration Test
E2E
Unit test
E2E test tool demo (Latitia)
▷
….........
▷
............
Jira management
▷
For each v1.2 Epic add the following
○
1 task that describes test coverage for the EPIC, UT,
Integration test, E2E, SVT
■
We know integration test will not happen in v1.2 –
describe how UT and E2E coverage covers the gap
○
Add 1 task for each automated E2E test you plan to add.
■
Add Epic Link
■
Add label ‘E2ETest’
■
Add Fix Version/s to be 1.2
Jira management – Example
https://jira.hyperledger.org/secure/Dashboard.jspa?selectPageId=1070
Bottom left widget
Explore a comprehensive test strategy focusing on delivery squad coverage, E2E test framework utilization, Jira management, integration test gap solutions, and phased approach implementation for feature delivery squads. The strategy emphasizes validation tests, unit testing components, user-friendly configurations, and orchestrated tests using a small number of deployed assets. With a combination of true unit tests and unit integration tests, the phased approach ensures high coverage and effective management in the v1.2 timeframe.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Agenda Recap test strategy (1 chart) Delivery squad function test coverage for 1.2 How to utilize the new E2E test framework How to manage and track in Jira
(SDK) (E2E)
Test strategy 2018 Feature delivery squads Test squads go test go test PTE, LTE, OTE, etc ginkgo based BDD Unit test Function test System test Integration Test E2E Tests that ensure unit tested components work together Tested outside of the production package implementations Using test friendly configuration but real implementations Orchestrated tests that use documented, user facing assets Each user consumable feature should extend this suite These are validation tests, not tests that are expected to stress the system or introduce errors Use a small number of deployed assets (eg. chaincode, peers, orgs) across multiple tests Limit execution time to seconds per variation, not minutes Production code structured for test True unit tests with high coverage Tool generated mocks for dependencies
V1.2 integration test strategy Lots of good progress has been made, but we can t swallow everything at once. Fabric is not well structured for integration test in v1.2 timeframe How to cover integration test gap while we evolve towards true integration test? Expand Unit test coverage Expand E2E coverage
Test strategy V1.2 Phased approach Feature delivery squads Test squads go test go test PTE, LTE, OTE, etc ginkgo based BDD Unit test Function test System test Integration Test E2E Not ready in v1.2 timeframe Tests that ensure unit tested components work together Tested outside of the production package implementations Using test friendly configuration but real implementations Orchestrated tests that use documented, user facing assets Each user consumable feature should extend this suite These are validation tests, not tests that are expected to stress the system or introduce errors Use a small number of deployed assets (eg. chaincode, peers, orgs) across multiple tests Limit execution time to seconds per variation, not minutes Production code structured for test True unit tests with high coverage Tool generated mocks for dependencies Strategy Orchestrated tests that use documented, user facing assets Each user consumable feature should extend this suite These are validation tests, not tests that are expected to stress the system or introduce errors Use a small number of deployed assets (eg. chaincode, peers, orgs) across multiple tests Limit execution time to seconds per variation, not minutes Production code structured for test (best effort) Combination of true unit tests and unit integration tests Tool generated mocks for dependencies V1.2
E2E test tool demo (Latitia) ......... ............
Jira management For each v1.2 Epic add the following 1 task that describes test coverage for the EPIC, UT, Integration test, E2E, SVT We know integration test will not happen in v1.2 describe how UT and E2E coverage covers the gap Add 1 task for each automated E2E test you plan to add. Add Epic Link Add label E2ETest Add Fix Version/s to be 1.2
Jira management Example https://jira.hyperledger.org/secure/Dashboard.jspa?selectPageId=1070 Bottom left widget
