Metrics & measurement

Defect density of a component of the test automation code.

The degree to which a component or system is operational and accessible when required for use. Often expressed as a percentage.

The process of intentionally adding defects to those already in the component or system for the purpose of monitoring the rate of detection and removal, and …

The percentage of boundary values that have been exercised by a test suite.

The coverage of all possible combinations of all single condition outcomes within one statement.

The coverage of condition outcomes.

The percentage of branches that have been exercised by a test suite. 100% branch coverage implies both 100% decision coverage and 100% statement coverage.

A set of automated tests which validates the integrity of each new build and verifies its key/core functionality, stability and testability.

The coverage of sequences of N+1 transitions.

An analysis method that determines which parts of the software have been executed (covered) by the test suite and which parts have not been executed, e.g., …

A software product that is developed for the general market, i.e. for a large number of customers, and that is delivered to many customers in identical format.

The percentage of all single condition outcomes that independently affect a decision outcome that have been exercised by a test case suite. 100% modified …

In managing project risks, the period of time within which a contingency action must be implemented in order to be effective in reducing the impact of the …

A statistical process control tool used to monitor a process and determine whether it is statistically controlled. It graphically depicts the average value and …

An approach to structure-based testing in which test cases are designed to execute specific sequences of events. Various techniques exist for control flow …

A metric that shows progress toward a defined criterion, e.g., convergence of the total number of tests executed to the total number of tests planned for …

The degree, expressed as a percentage, to which a specified coverage item has been exercised by a test suite.

An entity or property used as a basis for test coverage, e.g., equivalence partitions or code statements.

The maximum number of linear, independent paths through a program. Cyclomatic complexity may be computed as L = N + 2P, where L = the number of edges/links in …

The percentage of decision outcomes that have been exercised by a test suite. 100% decision coverage implies both 100% branch coverage and 100% statement …

The number of defects identified in a component or system divided by the size of the component or system (expressed in standard measurement terms, e.g., …

The number of defects found by a test level, divided by the number found by that test level and any other means afterwards.

The coverage of equivalence partitions.

The ratio of the number of failures of a given category to a given unit of measure, e.g., failures per unit of time, failures per number of transactions, …

An approach to software measurement using a three-level model conceptual level (goal), operational level (question) and quantitative level (metric).

A measure that can be used to estimate or predict another measure.

A metric that supports the judgment of process performance.

Documentation defining a designated number of virtual users who process a defined set of transactions in a specified time period that a component or system …

The number or category assigned to an attribute of an entity by making a measurement.

The process of assigning a number or category to an entity to describe an attribute of that entity.

A measurement scale and the method used for measurement.

The degree, expressed as a percentage, to which model elements are planned to be or have been exercised by a test suite.

An indicator of psychological preference representing the different personalities and communication styles of people.

A statistical technique in decision making that is used for selection of a limited number of factors that produce significant overall effect. In terms of …

The percentage of paths that have been exercised by a test suite.

The percentage of defects that are removed in the same phase of the software lifecycle in which they were introduced.

The effect on the component or system by the measurement instrument when the component or system is being measured, e.g., by a performance testing tool or …

The ability of the software product to perform its required functions under stated conditions for a specified period of time, or for a specified number of …

A model that shows the growth in reliability over time during continuous testing of a component or system as a result of the removal of defects that result in …

A questionnaire-based usability test technique for measuring software quality from the end user's point of view.

The percentage of executable statements that have been exercised by a test suite.

Coverage measures based on the internal structure of a component or system.

The disproportionate growth of the number of test cases with growing size of the test basis, when using a certain test design technique. Test case explosion …

A document specifying the test conditions (coverage items) for a test item, the detailed test approach and identifying the associated high-level test cases.

The calculated approximation of a result related to various aspects of testing (e.g., effort spent, completion date, costs involved, number of test cases, …

A two-dimensional table, which correlates two entities (e.g., requirements and test cases). The table allows tracing back and forth the links of one entity to …

A test session in usability testing in which a usability test participant is executing tests, moderated by a moderator and observed by a number of observers.

Confirmation by examination and through provision of objective evidence that specified requirements have been fulfilled.

A questionnaire-based usability test technique for measuring web site software quality from the end user's point of view.