ISO/IEC/IEEE 29119 Software Testing is an internationally agreed set of standards for software testing that can be used within any software development life cycle or organisation. By implementing these standards, you will be adopting the only internationally-recognised and agreed standards for software testing, which will provide your organisation with a high-quality approach to testing that can be communicated throughout the world.
There are currently five standards in the series:
- ISO/IEC 29119-1: Concepts & Definitions
- ISO/IEC 29119-2: Test Processes
- ISO/IEC 29119-3: Test Documentation
- ISO/IEC 29119-4: Test Techniques
- ISO/IEC 29119-5: Keyword Driven Testing
ISO/IEC/IEEE 29119-1
ISO/IEC 29119-1 covers the following topics:
- 5.1 Introduction to Software Testing
- 5.1.1 The Role of Testing in Verification and Validation
- 5.1.2 Exhaustive Testing
- 5.1.3 Testing as a Heuristic
- 5.2 Software Testing in an Organizational and Project Context
- 5.2.1 The Test Process
- 5.3 Generic Testing Processes in the Software Life cycle
- 5.3.1 Development Project Sub-processes and their Results
- 5.3.2 On-going Maintenance and its Results
- 5.3.3 Support Processes for the Software Development Life Cycle
- 5.4 Risk-based Testing
- 5.4.1 Using Risk-Based Testing in the Organizational Test Process
- 5.4.2 Using Risk-Based Testing in the Test Management processes
- 5.4.3 Using Risk-Based Testing in the Dynamic Testing processes
- 5.5 Test Sub-process
- 5.5.1 Test Objectives
- 5.5.2 Test Item
- 5.5.3 Testing of Quality Characteristics
- 5.5.4 Test Basis
- 5.5.5 Retesting and Regression Testing
- 5.5.6 Test Design Techniques
- 5.6 Test Practices
- 5.6.1 Introduction
- 5.6.2 Requirements-Based Testing
- 5.6.3 Model-Based Testing
- 5.6.4 Mathematical-Based Testing
- 5.6.5 Experience-Based Testing
- 5.6.6 Scripted and Unscripted Testing
- 5.7 Automation in Testing
- 5.8 Defect Management
- Annex A (informative) The Role of Testing in Verification and Validation
- Annex B (informative) Metrics and Measures
- B.1 Metrics and Measures
- Annex C (informative) Testing in Different Life Cycle Models
- C.1 Overview
- C.2 Agile Development and Testing
- C.2.1 Agile Development Principles
- C.2.2 Test Management in Agile Development
- C.2.3 Test Sub-processes in Agile Development
- C.3 Sequential Development and Testing
- C.3.1 Sequential Development Principles
- C.3.2 Test Management in Sequential Development
- C.3.3 Test Sub-processes in Sequential Development
- C.4 Evolutionary Development and Testing
- C.4.1 Evolutionary Development Principles
- C.4.2 Test Management in Evolutionary Development
- C.4.3 Test Sub-processes in Evolutionary Development
ISO/IEC/IEEE 29119-2
ISO/IEC/IEEE 29119-2 is a normative standard that defines a generic process model for software testing that can be used within any software development life cycle and in any organisation. The model specifies test processes for governing, managing and implementing software testing. The standard presents a three-layer process model that covers:
- Organizational Test Specifications (e.g. Organizational Test Policy, Organizational Test Strategy)
- Test Management
- Dynamic Testing
A risk-based approach to testing is presented for strategizing and managing testing, allowing all testing activities to be prioritzed to the most important aspects of each system.
ISO/IEC/IEEE 29119-3
.png)
ISO/IEC/IEEE 29119-3 provides standard templates for test documentation that cover the entire software testing life cycle. Each template can be tailored to suit the unique needs of each organisation and life cycle model. All templates align with the test process defined in ISO/IEC/IEEE 29119-2 and can be produced by applying the processes that are defined in that standard. The templates within the standard can be implemented in a wide variety of ways, including as formal documents, in testing tools, or even as whiteboard photos or mind maps.
The documents that are defined in ISO/IEC/IEEE 29119-3 are as follows:
Organizational Test Process Documentation:
- Test Policy
- Organizational Test Strategy
Test Management Process Documentation:
- Test Plan (including a Test Strategy)
- Test Status Report
- Test Completion Report
Dynamic Test Process Documentation:
- Test Design Specification
- Test Case Specification
- Test Procedure Specification
- Test Data Requirements
- Test Data Readiness Report
- Test Environment Requirements
- Test Environment Readiness Report
- Actual Results
- Test Result
- Test Execution Log
- Test Incident Report
ISO/IEC/IEEE 29119-4 covers a variety of commonly used dynamic software testing techniques:
- Specification-Based Testing Techniques:
- - Equivalence Partitioning
- - Classification Tree Method
- - Boundary Value Analysis
- - State Transition Testing
- - Decision Table Testing
- - Cause-Effect Graphing
- - Syntax Testing
- - Combinatorial Test Techniques, including:
- -> All Combinations
- -> Pairwise Testing
- -> Each Choice Testing
- -> Base Choice Testing
- - Scenario Testing (including Use Case Testing)
- - Random Testing
- Structure-Based Testing Techniques:
- - Statement Testing
- - Branch Testing
- - Decision Testing
- - Condition Testing, including:
- -> Branch Condition Testing
- -> Branch Condition Combination Testing
- -> Modified Condition Decision Condition (MCDC) Testing
- - Data Flow Testing, including:
- -> All definitions
- -> All-c-uses
- -> All-p-uses
- -> all-uses
- -> all-du-paths
- Experience-Based Testing Techniques:
- - Error Guessing
Part 4 also provides informative definitions of a variety of quality-related types of testing and provides examples of how the test case design techniques in the lists above can be applied within the following types of testing:
- Accessibility Testing
- Backup/Recovery Testing
- Compatibility Testing
- Conversion Testing
- Disaster Recovery Testing
- Functional Testing
- Installability Testing
- Interoperability Testing
- Localization Testing
- Maintainability Testing
- Performance-Related Testing (e.g. Load Testing, Stress Testing and Endurance Testing)
- Portability Testing
- Procedure Testing
- Reliability Testing
- Security Testing
- Stability Testing
- Usability Testing
ISO/IEC/IEEE 29119-5 provides an international standard for supporting Keyword-Driven Testing. Keyword-Driven Testing is a way of describing test cases by using a predefined set of Keywords. These Keywords are names which are associated with a set of actions that are required to perform a specific step in a test case. By using keywords to describe test steps instead of natural language, test cases can be easier to understand, to maintain and to automate.
The standard covers the following topics:
- Introduction in Keyword-Driven Testing - provides an introduction to Keyword-Driven Testing addressing the benefits and explaining how keywords can be organized into several layers, which types of keywords are common and how keywords are associated with data
- Application of Keyword-Driven Testing - covers how to identify Keywords, how they are used to compose test cases, what needs to be taken into regard for maintenance of defined sets of Keywords and how Keyword-Driven Testing and data driven testing are related
- Keyword-Driven Testing Frameworks - Keyword-Driven Testing can be supported by frameworks, consisting of, for example, commercial software-tools, individual scripts and documentation. This clause addresses how a suitable Keyword-Driven Test framework can be composed and which properties are required and helpful
- Data Interchange - describes what data needs to be exchanged between tools for Keyword-Driven Testing, and describes a data format for this task
- Benefits & Issues with Keyword-Driven Testing
- Getting Started with Keyword-Driven Testing
- Roles and Tasks - describes several roles with associated tasks in Keyword-Driven Testing. Team members in a project can be assigned roles with tasks matching their qualification
- Basic Keywords
- Examples
