The unintended termination of the execution of a component or system prior to completion.
The unintended termination of the execution of a component or system prior to completion.
A test case without concrete (implementation level) values for input data and expected results. Logical operators are used: instances of the actual values are not yet defined and/or available.
A use case in which some actors with malicious intent are causing harm to the system or to other actors.
The exit criteria that a component or system must satisfy in order to be accepted by a user, customer, or other authorized entity.
A collaborative approach to development in which the team and customers are using the customers own domain language to understand their requirements, which forms the basis for testing a component or system.
Formal testing with respect to user needs, requirements, and business processes conducted to determine whether or not a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system.
The degree to which a product or system can be used by people with the widest range of characteristics and capabilities to achieve a specified goal in a specified context of use.
Testing to determine the ease by which users with disabilities can use a component or system.
The process of obtaining user account information based on trial and error with the intention of using that information in a security attack.
The degree to which the actions of an entity can be traced uniquely to that entity.
The capability of the software product to provide the right or agreed results or effects with the needed degree of precision.
Testing to determine the accuracy of a software product.
The phase within the IDEAL model where the improvements are developed, put into practice, and deployed across the organization. The acting phase consists of the activities: create solution, pilot/test solution, refine solution and implement solution.
A scripting technique that uses data files to contain not only test data and expected results, but also keywords related to the application being tested. The keywords are interpreted by special supporting scripts that are called by the control script for the test.
User or any other person or system that interacts with the test object in a specific way.
The behavior produced/observed when a component or system is tested.
The behavior produced/observed when a component or system is tested.
A review technique performed informally without a structured process.
A review technique carried out by independent reviewers informally, without a structured process.
The capability of the software product to be adapted for different specified environments without applying actions or means other than those provided for this purpose for the software considered.
A statement on the values that underpin Agile software development. The values are: individuals and interactions over processes and tools, working software over comprehensive documentation, customer collaboration over contract negotiation, responding to change over following a plan.
A group of software development methodologies based on iterative incremental development, where requirements and solutions evolve through collaboration between self-organizing cross-functional teams.
Testing practice for a project using Agile software development methodologies, incorporating techniques and methods, such as extreme programming (XP), treating development as the customer of testing and emphasizing the test-first design paradigm.
Simulated or actual operational testing by potential users/customers or an independent test team at the developers' site, but outside the development organization. Alpha testing is often employed for commercial off-the-shelf software as a form of internal acceptance testing.
A test strategy whereby the test team analyzes the test basis to identify the test conditions to cover.
Testing based on a systematic analysis of e.g., product risks or requirements.
The capability of the software product to be diagnosed for deficiencies or causes of failures in the software, or for the parts to be modified to be identified.
A tool that carries out static analysis.
Any condition that deviates from expectation based on requirements specifications, design documents, user documents, standards, etc., or from someone's perception or experience. Anomalies may be found during, but not limited to, reviewing, testing, analysis, compilation, or use of software products or applicable documentation.
Software that is used to detect and inhibit malware.
Repeated action, process, structure or reusable solution that initially appears to be beneficial and is commonly used but is ineffective and/or counterproductive in practice.
Testing performed by submitting commands to the software under test using programming interfaces of the application directly.
A type of interface in which the components or systems involved exchange information in a defined formal structure.
The degree to which users can recognize whether a component or system is appropriate for their needs.
A document summarizing the assessment results, e.g., conclusions, recommendations and findings.
A person who conducts an assessment. Any member of an assessment team.
A condition that cannot be decomposed, i.e., a condition that does not contain two or more single conditions joined by a logical operator (AND, OR, XOR).
Directed and focused attempt to evaluate a specific quality characteristic of a test object by attempting to force specific failures to occur.
A path or means by which an attacker can gain access to a system for malicious purposes.
A person or process that attempts to access data, functions or other restricted areas of the system without authorization, potentially with malicious intent.
The capability of the software product to be attractive to the user.
Testing to determine if the game music and sound effects will engage the user in the game and enhance the game play.
An independent evaluation of software products or processes to ascertain compliance to standards, guidelines, specifications, and/or procedures based on objective criteria, including documents that specify: the form or content of the products to be produced, the process by which the products shall be produced, and how compliance to standards or guidelines shall be measured.
A procedure determining whether a person or a process is, in fact, who or what it is declared to be.
The degree to which the identity of a subject or resource can be proved to be the one claimed.
Permission given to a user or process to access resources.
Defect density of a component of the test automation code.
One of four levels that specify the item's or element's necessary requirements of ISO 26262 and safety measures to avoid an unreasonable residual risk.
A process reference model and an associated process assessment model in the automotive industry that conforms with the requirements of ISO/IEC 33002:2015.
The degree to which a component or system is operational and accessible when required for use. Often expressed as a percentage.
Testing to compare two or more variants of a test item or a simulation model of the same test item by executing the same test cases on all variants and comparing the results.
A strategic tool for measuring whether the operational activities of a company are aligned with its objectives in terms of business vision and strategy.
The process of intentionally adding defects to those already in the component or system for the purpose of monitoring the rate of detection and removal, and estimating the number of remaining defects. Fault seeding is typically part of development (pre-release) testing and can be performed at any test level (component, integration, or system).
The response of a component or system to a set of input values and preconditions.
A collaborative approach to development in which the team is focusing on delivering expected behavior of a component or system for the customer, which forms the basis for testing.
Software developed specifically for a set of users or customers. The opposite is commercial off-the-shelf software.
A superior method or innovative practice that contributes to the improved performance of an organization under given context, usually recognized as "best" by other peer organizations.
Operational testing by potential and/or existing users/customers at an external site not otherwise involved with the developers, to determine whether or not a component or system satisfies the user/customer needs and fits within the business processes. Beta testing is often employed as a form of external acceptance testing for commercial off-the-shelf software in order to acquire feedback from the market.
Procedure to derive and/or select test cases based on an analysis of the specification, either functional or non-functional, of a component or system without reference to its internal structure.
Procedure to derive and/or select test cases based on an analysis of the specification, either functional or non-functional, of a component or system without reference to its internal structure.
A procedure to derive and/or select test cases based on an analysis of the specification, either functional or non-functional, of a component or system without reference to its internal structure.
A network of compromised computers, called bots or robots, which is controlled by a third party and used to transmit malware or spam, or to launch attacks.
An input value or output value which is on the edge of an equivalence partition or at the smallest incremental distance on either side of an edge, for example the minimum or maximum value of a range.
A black-box test design technique in which test cases are designed based on boundary values.
The percentage of boundary values that have been exercised by a test suite.
A basic block that can be selected for execution based on a program construct in which one of two or more alternative program paths is available, e.g., case, jump, go to, if-then-else.
A logical expression that can be evaluated as True or False.
The coverage of all possible combinations of all single condition outcomes within one statement.
A white-box test design technique in which test cases are designed to execute combinations of single condition outcomes (within one statement).
The coverage of condition outcomes.
The percentage of branches that have been exercised by a test suite. 100% branch coverage implies both 100% decision coverage and 100% statement coverage.
A white-box test technique in which test cases are designed to exercise branches.
A flaw in a component or system that can cause the component or system to fail to perform its required function, e.g., an incorrect statement or data definition. A defect, if encountered during execution, may cause a failure of the component or system.
An approach to testing in which gamification and awards for defects found are used as a motivator.
A document reporting on any flaw in a component or system that can cause the component or system to fail to perform its required function.
A system of (hierarchical) categories designed to be a useful aid for reproducibly classifying defects.
A set of automated tests which validates the integrity of each new build and verifies its key/core functionality, stability and testability.
A publicly displayed chart that depicts the outstanding effort versus time in an iteration. It shows the status and trend of completing the tasks of the iteration. The X-axis typically represents days in the sprint, while the Y-axis is the remaining effort (usually either in ideal engineering hours or story points).
An abstract representation of calling relationships between subroutines in a program.
A framework that describes the key elements of an effective product development and maintenance process. The Capability Maturity Model Integration covers best-practices for planning, engineering and managing product development and maintenance.
The degree to which the maximum limits of a component or system parameter meet requirements.
Testing to evaluate the capacity of a system.
A test automation approach, where inputs to the test object are recorded during manual testing in order to generate automated test scripts that could be executed later (i.e. replayed).
A type of test execution tool where inputs are recorded during manual testing in order to generate automated test scripts that can be executed later (i.e. replayed). These tools are often used to support automated regression testing.
A test automation approach, where inputs to the test object are recorded during manual testing in order to generate automated test scripts that could be executed later (i.e. replayed).
A type of test execution tool where inputs are recorded during manual testing in order to generate automated test scripts that can be executed later (i.e. replayed). These tools are often used to support automated regression testing.
Acronym for Computer Aided Software Engineering.
Acronym for Computer Aided Software Testing.
An analysis technique aimed at identifying the root causes of defects. By directing corrective measures at root causes, it is hoped that the likelihood of defect recurrence will be minimized.
A black-box test technique in which test cases are designed from cause-effect graphs.
A table showing combinations of inputs and/or stimuli (causes) with their associated outputs and/or actions (effects), which can be used to design test cases.
A graphical representation used to organize and display the interrelationships of various possible root causes of a problem. Possible causes of a real or potential defect or failure are organized in categories and subcategories in a horizontal tree-structure, with the (potential) defect or failure as the root node.
A graphical representation of inputs and/or stimuli (causes) with their associated outputs (effects), which can be used to design test cases.
A black-box test technique in which test cases are designed from cause-effect graphs.
The process of confirming that a component, system or person complies with its specified requirements.
(1) A structured approach to transitioning individuals and organizations from a current state to a desired future state. (2) Controlled way to effect a change, or a proposed change, to a product or service.
A type of testing initiated by modification to a component or system.
The capability of the software product to enable specified modifications to be implemented.
A statement of test objectives, and possibly test ideas about how to test. Test charters are used in exploratory testing.
The person involved in the review that identifies and describes anomalies in the product or project under review. Reviewers can be chosen to represent different viewpoints and roles in the review process.
A review technique guided by a list of questions or required attributes.
A review technique guided by a list of questions or required attributes.
An experience-based test design technique whereby the experienced tester uses a high-level list of items to be noted, checked, or remembered, or a set of rules or criteria against which a product has to be verified.
The coverage of sequences of N+1 transitions.
A tree showing equivalence partitions hierarchically ordered, which is used to design test cases in the classification tree method.
A black-box test design technique in which test cases, described by means of a classification tree, are designed to execute combinations of representatives of input and/or output domains.
A black-box test technique in which test cases are designed using a classification tree.
Testing based on an analysis of the internal structure of the component or system.
Acronym for Command-Line Interface.
Testing performed by submitting commands to the software under test using a dedicated command-line interface.
A system in which the controlling action or input is dependent on the output or changes in output.
The capability of the software product to co-exist with other independent software in a common environment sharing common resources.
An analysis method that determines which parts of the software have been executed (covered) by the test suite and which parts have not been executed, e.g., statement coverage, decision coverage or condition coverage.
A type of security attack performed by inserting malicious code at an interface into an application to exploit poor handling of untrusted data.
Testing based on an analysis of the internal structure of the component or system.
Excessive emotional or psychological dependence on another person, specifically in trying to change that person's current (undesirable) behavior while supporting them in continuing that behavior. For example, in software testing, complaining about late delivery to test and yet enjoying the necessary "heroism", working additional hours to make up time when delivery is running late, therefore reinforcing the lateness.
A standard that describes the characteristics of a design or a design description of data or program components.
A decision table in which combinations of inputs that are impossible or lead to the same outputs are merged into one column (rule), by setting the conditions that do not influence the outputs to don't care.
A black-box test design technique in which test cases are designed to execute specific combinations of values of several parameters.
A type of interface in which the information is passed in form of command lines.
A software product that is developed for the general market, i.e. for a large number of customers, and that is delivered to many customers in identical format.
A test tool to perform automated test comparison of actual results with expected results.
The degree to which a component or system can exchange information with other components or systems, and/or perform its required functions while sharing the same hardware or software environment.
A software tool that translates programs expressed in a high-order language into their machine language equivalents.
A test approach in which the test suite comprises all combinations of input values and preconditions.
The set of generic and specific conditions, agreed upon with the stakeholders for permitting a process to be officially completed. The purpose of exit criteria is to prevent a task from being considered completed when there are still outstanding parts of the task which have not been finished. Exit criteria are used to report against and to plan when to stop testing.
The degree to which a component or system has a design and/or internal structure that is difficult to understand, maintain and verify.
The capability of the software product to adhere to standards, conventions or regulations in laws and similar prescriptions.
Testing to determine the compliance of the component or system.
A minimal software item that can be tested in isolation.
Testing performed to expose defects in the interfaces and interactions between integrated components.
The testing of individual software components.
Two or more single conditions joined by means of a logical operator.
The computing-based processes, techniques, and tools to support testing.
The practice of determining how a security attack has succeeded and assessing the damage caused.
A test case with concrete (implementation level) values for input data and expected results. Logical operators from high-level test cases are replaced by actual values that correspond to the objectives of the logical operators.
The simultaneous execution of multiple independent threads by a component or system.
Testing to evaluate if a component or system involving concurrency behaves as specified.
A logical expression that can be evaluated as True or False.
The coverage of all possible combinations of all single condition outcomes within one statement.
A white-box test design technique in which test cases are designed to execute combinations of single condition outcomes (within one statement).
The coverage of condition outcomes.
The percentage of all single condition outcomes that independently affect a decision outcome that have been exercised by a test case suite. 100% modified condition / decision coverage implies 100% decision condition coverage.
A white-box test technique in which test cases are designed to exercise single condition outcomes that independently affect a decision outcome.
A white-box test design technique in which test cases are designed to execute condition outcomes.
In managing project risks, the period of time within which a contingency action must be implemented in order to be effective in reducing the impact of the risk.
A test suite that covers the main functionality of a component or system to determine whether it works properly before planned testing begins.
The degree to which a component or system ensures that data are accessible only to those authorized to have access.
The composition of a component or system as defined by the number, nature, and interconnections of its constituent parts.
An aggregation of hardware, software or both, that is designated for configuration management and treated as a single entity in the configuration management process.
A discipline applying technical and administrative direction and surveillance to identify and document the functional and physical characteristics of a configuration item, control changes to those characteristics, record and report change processing and implementation status, and verify compliance with specified requirements.
Testing to determine the portability of a software product.
Testing that runs test cases that failed the last time they were run, in order to verify the success of corrective actions.
Testing to determine the compliance of the component or system.
The degree to which a component or system can connect to other components or systems.
A test strategy whereby the test team relies on the input of one or more key stakeholders to determine the details of the strategy.
A process model providing a detailed description of good engineering practices, e.g., test practices.
A process model providing a detailed description of good engineering practices, e.g., test practices.
Users, tasks, equipment (hardware, software and materials), and the physical and social environments in which a software product is used.
A software development procedure merging, integrating and testing all changes as soon as they are committed within an automated process.
A capability maturity model structure wherein capability levels provide a recommended order for approaching process improvement within specified process areas.
An approach that involves a process of testing early, testing often, test everywhere, and automate to obtain feedback on the business risks associated with a software release candidate as rapidly as possible.
Acceptance testing conducted to verify whether a system satisfies its contractual requirements.
A statistical process control tool used to monitor a process and determine whether it is statistically controlled. It graphically depicts the average value and the upper and lower control limits (the highest and lowest values) of a process.
A sequence of events (paths) in the execution through a component or system.
A form of static analysis based on a representation of unique paths (sequences of events) in the execution through a component or system. Control flow analysis evaluates the integrity of control flow structures, looking for possible control flow anomalies such as closed loops or logically unreachable process steps.
An abstract representation of all possible sequences of events (paths) in the execution through a component or system.
A sequence of consecutive edges in a directed graph.
An approach to structure-based testing in which test cases are designed to execute specific sequences of events. Various techniques exist for control flow testing, e.g., decision testing, condition testing, and path testing, that each have their specific approach and level of control flow coverage.
A metric that shows progress toward a defined criterion, e.g., convergence of the total number of tests executed to the total number of tests planned for execution.
A dashboard-style representation of the status of corporate performance data.
The total costs incurred on quality activities and issues and often split into prevention costs, appraisal costs, internal failure costs and external failure costs.
The degree, expressed as a percentage, to which a specified coverage item has been exercised by a test suite.
An entity or property used as a basis for test coverage, e.g., equivalence partitions or code statements.
An element necessary for an organization or project to achieve its mission. Critical success factors are the critical factors or activities required for ensuring the success.
A content-based model for test process improvement built around twelve critical processes. These include highly visible processes, by which peers and management judge competence and mission-critical processes in which performance affects the company's profits and reputation.
The degree to which a website or web application can function across different browsers and degrade gracefully when browser features are absent or lacking.
A vulnerability that allows attackers to inject malicious code into an otherwise benign website.
An approach to testing in which testing is distributed to a large group of testers.
Software developed specifically for a set of users or customers. The opposite is commercial off-the-shelf software.
A software tool developed specifically for a set of users or customers.
The maximum number of linear, independent paths through a program. Cyclomatic complexity may be computed as L = N + 2P, where L = the number of edges/links in a graph, N = the number of nodes in a graph, P = the number of disconnected parts of the graph (e.g., a called graph or subroutine).
The maximum number of linear, independent paths through a program. Cyclomatic complexity may be computed as L = N + 2P, where L = the number of edges/links in a graph, N = the number of nodes in a graph, P = the number of disconnected parts of the graph (e.g., a called graph or subroutine).
A software development activity in which a system is compiled and linked daily so that it is consistently available at any time including all the latest changes.
A representation of dynamic measurements of operational performance for some organization or activity, using metrics represented via metaphors such as visual dials, counters, and other devices resembling those on the dashboard of an automobile, so that the effects of events or activities can be easily understood and related to operational goals.
An executable statement where a variable is assigned a value.
The sequence of possible changes to the state of data objects.
A form of static analysis based on the definition and usage of variables.
A white-box test technique in which test cases are designed to execute definition-use pairs of variables.
Data transformation that makes it difficult for a human to recognize the original data.
The protection of personally identifiable information or otherwise sensitive information from undesired disclosure.
A scripting technique that stores test input and expected results in a table or spreadsheet, so that a single control script can execute all of the tests in the table. Data-driven testing is often used to support the application of test execution tools such as capture/playback tools.
Code that cannot be reached and therefore is impossible to execute.
A tool used by programmers to reproduce failures, investigate the state of programs and find the corresponding defect. Debuggers enable programmers to execute programs step by step, to halt a program at any program statement and to set and examine program variables.
The process of finding, analyzing and removing the causes of failures in software.
A tool used by programmers to reproduce failures, investigate the state of programs and find the corresponding defect. Debuggers enable programmers to execute programs step by step, to halt a program at any program statement and to set and examine program variables.
A program point at which the control flow has two or more alternative routes. A node with two or more links to separate branches.
The percentage of all condition outcomes and decision outcomes that have been exercised by a test suite. 100% decision condition coverage implies both 100% condition coverage and 100% decision coverage.
A white-box test design technique in which test cases are designed to execute condition outcomes and decision outcomes.
The percentage of decision outcomes that have been exercised by a test suite. 100% decision coverage implies both 100% branch coverage and 100% statement coverage.
The result of a decision (which therefore determines the branches to be taken).
A table showing combinations of inputs and/or stimuli (causes) with their associated outputs and/or actions (effects), which can be used to design test cases.
A black-box test design technique in which test cases are designed to execute the combinations of inputs and/or stimuli (causes) shown in a decision table.
A white-box test design technique in which test cases are designed to execute decision outcomes.
A flaw in a component or system that can cause the component or system to fail to perform its required function, e.g., an incorrect statement or data definition. A defect, if encountered during execution, may cause a failure of the component or system.
The number of defects identified in a component or system divided by the size of the component or system (expressed in standard measurement terms, e.g., lines-of-code, number of classes or function points).
The number of defects found by a test level, divided by the number found by that test level and any other means afterwards.
The process of recognizing, investigating, taking action and disposing of defects. It involves recording defects, classifying them and identifying the impact.
A cross-functional team of stakeholders who manage reported defects from initial detection to ultimate resolution (defect removal, defect deferral, or report cancellation). In some cases, the same team as the configuration control board.
A document reporting on any flaw in a component or system that can cause the component or system to fail to perform its required function.
A system of (hierarchical) categories designed to be a useful aid for reproducibly classifying defects.
A cross-functional team of stakeholders who manage reported defects from initial detection to ultimate resolution (defect removal, defect deferral, or report cancellation). In some cases, the same team as the configuration control board.
A procedure to derive and/or select test cases targeted at one or more defect types, with tests being developed from what is known about the specific defect type.
A procedure to derive and/or select test cases targeted at one or more defect types, with tests being developed from what is known about the specific defect type.
A test technique in which test cases are developed from what is known about a specific defect type.
The set of generic and specific conditions, agreed upon with the stakeholders for permitting a process to be officially completed. The purpose of exit criteria is to prevent a task from being considered completed when there are still outstanding parts of the task which have not been finished. Exit criteria are used to report against and to plan when to stop testing.
The set of generic and specific conditions for permitting a process to go forward with a defined task, e.g., test phase. The purpose of entry criteria is to prevent a task from starting which would entail more (wasted) effort compared to the effort needed to remove the failed entry criteria.
The association of a definition of a variable with the subsequent use of that variable. Variable uses include computational (e.g., multiplication) or to direct the execution of a path (predicate use).
A physical or logical subnetwork that contains and exposes an organization's external-facing services to an untrusted network, commonly the Internet.
An iterative four-step problem-solving process (plan-do-check-act) typically used in process improvement.
A security attack that is intended to overload the system with requests such that legitimate requests cannot be serviced.
Any event occurring that requires investigation.
A document reporting on any event that occurred, e.g., during the testing, which requires investigation.
A type of testing in which test suites are executed on physical or virtual devices.
The phase within the IDEAL model where it is determined where one is, relative to where one wants to be. The diagnosing phase consists of the activities to characterize current and desired states and develop recommendations.
A test strategy whereby the test team relies on the input of one or more key stakeholders to determine the details of the strategy.
Testing a component or system in a way for which it was not intended to be used.
A black-box test design technique that is used to identify efficient and effective test cases when multiple variables can or should be tested together. It builds on and generalizes equivalence partitioning and boundary values analysis.
A software component or test tool that replaces a component that takes care of the control and/or the calling of a component or system.
The process of evaluating behavior, e.g., memory performance, CPU usage, of a system or component during execution.
Testing that involves the execution of the software of a component or system.
A type of testing in which business processes are tested from start to finish under production-like circumstances.
The capability of producing an intended result.
(1) The capability of the software product to provide appropriate performance, relative to the amount of resources used, under stated conditions. (2) The capability of a process to produce the intended outcome, relative to the amount of resources used.
The ability, capacity, and skill to identify, assess, and manage the emotions of one's self, of others, and of groups.
A device, computer program, or system that accepts the same inputs and produces the same outputs as a given system.
The process of encoding information so that only authorized parties can retrieve the original information, usually by means of a specific decryption key or process.
A type of testing in which business processes are tested from start to finish under production-like circumstances.
Testing to determine the stability of a system under a significant load over a significant period of time within the system's operational context.
The set of generic and specific conditions for permitting a process to go forward with a defined task, e.g., test phase. The purpose of entry criteria is to prevent a task from starting which would entail more (wasted) effort compared to the effort needed to remove the failed entry criteria.
An executable statement or process step which defines a point at which a given process is intended to begin.
An abstraction of the real environment of a component or system including other components, processes, and environment conditions, in a real-time simulation.
A large user story that cannot be delivered as defined within a single iteration or is large enough that it can be split into smaller user stories.
A portion of an input or output domain for which the behavior of a component or system is assumed to be the same, based on the specification.
A portion of an input or output domain for which the behavior of a component or system is assumed to be the same, based on the specification.
The coverage of equivalence partitions.
A black-box test design technique in which test cases are designed to execute representatives from equivalence partitions. In principle, test cases are designed to cover each partition at least once.
Effort required for running tests manually.
A human action that produces an incorrect result.
A test design technique where the experience of the tester is used to anticipate what defects might be present in the component or system under test as a result of errors made, and to design tests specifically to expose them.
The process of intentionally adding defects to those already in the component or system for the purpose of monitoring the rate of detection and removal, and estimating the number of remaining defects. Fault seeding is typically part of development (pre-release) testing and can be performed at any test level (component, integration, or system).
A tool for seeding (i.e., intentionally inserting) faults in a component or system.
The degree to which a component or system can continue normal operation despite the presence of erroneous inputs.
A defect that was not detected in a previous test level which is supposed to find such type of defects.
The phase within the IDEAL model where the specifics of how an organization will reach its destination are planned. The establishing phase consists of the activities set priorities, develop approach and plan actions.
A security tester using hacker techniques.
A non-prescriptive framework for an organization's quality management system, defined and owned by the European Foundation for Quality Management, based on five 'Enabling' criteria (covering what an organization does), and four 'Results' criteria (covering what an organization achieves).
A source code statement that, when translated into object code, can be executed in a procedural manner.
A test approach in which the test suite comprises all combinations of input values and preconditions.
The set of generic and specific conditions, agreed upon with the stakeholders for permitting a process to be officially completed. The purpose of exit criteria is to prevent a task from being considered completed when there are still outstanding parts of the task which have not been finished. Exit criteria are used to report against and to plan when to stop testing.
An executable statement or process step which defines a point at which a given process is intended to cease.
The behavior predicted by the specification, or another source, of the component or system under specified conditions.
The behavior predicted by the specification, or another source, of the component or system under specified conditions.
Procedure to derive and/or select test cases based on the tester's experience, knowledge and intuition.
Procedure to derive and/or select test cases based on the tester's experience, knowledge and intuition.
A procedure to derive and/or select test cases based on the tester's experience, knowledge and intuition.
Testing based on the tester's experience, knowledge and intuition.
An informal usability review in which the reviewers are experts. Experts can be usability experts or subject matter experts, or both.
An informal test design technique where the tester actively controls the design of the tests as those tests are performed and uses information gained while testing to design new and better tests.
A software engineering methodology used within Agile software development whereby core practices are programming in pairs, doing extensive code review, unit testing of all code, and simplicity and clarity in code.
The leader and main person responsible for an inspection or other review process.
A test is deemed to fail if its actual result does not match its expected result.
The status of a test result in which the actual result does not match the expected result.
The backup operational mode in which the functions of a system that becomes unavailable are assumed by a secondary system.
Testing by simulating failure modes or actually causing failures in a controlled environment. Following a failure, the failover mechanism is tested to ensure that data is not lost or corrupted and that any agreed service levels are maintained (e.g., function availability or response times).
Deviation of the component or system from its expected delivery, service or result.
The physical or functional manifestation of a failure.
A systematic approach to risk identification and analysis of identifying possible modes of failure and attempting to prevent their occurrence.
The ratio of the number of failures of a given category to a given unit of measure, e.g., failures per unit of time, failures per number of transactions, failures per number of computer runs.
A test result in which a defect is reported although no such defect actually exists in the test object.
A test result which fails to identify the presence of a defect that is actually present in the test object.
A test result which fails to identify the presence of a defect that is actually present in the test object.
A test result in which a defect is reported although no such defect actually exists in the test object.
A flaw in a component or system that can cause the component or system to fail to perform its required function, e.g., an incorrect statement or data definition. A defect, if encountered during execution, may cause a failure of the component or system.
Directed and focused attempt to evaluate a specific quality characteristic of a test object by attempting to force specific failures to occur.
The number of defects identified in a component or system divided by the size of the component or system (expressed in standard measurement terms, e.g., lines-of-code, number of classes or function points).
The number of defects found by a test level, divided by the number found by that test level and any other means afterwards.
The process of intentionally adding defects to a system for the purpose of finding out whether the system can detect, and possibly recover from, a defect. Fault injection is intended to mimic failures that might occur in the field.
The process of intentionally adding defects to those already in the component or system for the purpose of monitoring the rate of detection and removal, and estimating the number of remaining defects. Fault seeding is typically part of development (pre-release) testing and can be performed at any test level (component, integration, or system).
A tool for seeding (i.e., intentionally inserting) faults in a component or system.
The capability of the software product to maintain a specified level of performance in cases of software faults (defects) or of infringement of its specified interface.
A technique used to analyze the causes of faults (defects). The technique visually models how logical relationships between failures, human errors, and external events can combine to cause specific faults to disclose.
A path for which a set of input values and preconditions exists which causes it to be executed.
A distinguishing characteristic of a component or system.
An iterative and incremental software development process driven from a client-valued functionality (feature) perspective. Feature-driven development is mostly used in Agile software development.
A type of testing conducted to evaluate the system behavior under productive connectivity conditions in the field.
A result of an evaluation that identifies some important issue, problem, or opportunity.
A computational model consisting of a finite number of states and transitions between those states, possibly with accompanying actions.
A black-box test design technique in which test cases are designed to execute valid and invalid state transitions.
A component or set of components that controls incoming and outgoing network traffic based on predetermined security rules.
A graphical representation used to organize and display the interrelationships of various possible root causes of a problem. Possible causes of a real or potential defect or failure are organized in categories and subcategories in a horizontal tree-structure, with the (potential) defect or failure as the root node.
The exploration of a target area aiming to gain information that can be useful for an attack.
A review characterized by documented procedures and requirements, e.g., inspection.
A type of evaluation designed and used to improve the quality of a component or system, especially when it is still being designed.
The degree to which a component or system mitigates the potential risk to economic status, living things, health, or the environment.
Method aiming to measure the size of the functionality of an information system. The measurement is independent of the technology. This measurement may be used as a basis for the measurement of productivity, the estimation of the needed resources, and project control.
The degree to which the functions facilitate the accomplishment of specified tasks and objectives.
The degree to which the set of functions covers all the specified tasks and user objectives.
The degree to which a component or system provides the correct results with the needed degree of precision.
An integration approach that combines the components or systems for the purpose of getting a basic functionality working early.
A requirement that specifies a function that a component or system must perform.
The absence of unreasonable risk due to hazards caused by malfunctioning behavior of Electric/Electronic(E/E) - Systems.
The degree to which a component or system provides functions that meet stated and implied needs when used under specified conditions.
Testing based on an analysis of the specification of the functionality of a component or system.
The capability of the software product to provide functions which meet stated and implied needs when the software is used under specified conditions.
A software testing technique used to discover security vulnerabilities by inputting massive amounts of random data, called fuzz, to the component or system.
A software testing technique used to discover security vulnerabilities by inputting massive amounts of random data, called fuzz, to the component or system.
Representation of the layers, components, and interfaces of a test automation architecture, allowing for a structured and modular approach to implement test automation.
Testing based on an analysis of the internal structure of the component or system.
An approach to software measurement using a three-level model conceptual level (goal), operational level (question) and quantitative level (metric).
A type of interface that allows users to interact with a component or system through graphical icons and visual indicators.
Testing performed by interacting with the software under test via the graphical user interface.
A person or organization who is actively involved in security attacks, usually with malicious intent.
Dynamic testing performed using real hardware with integrated software in a simulated environment.
Testing performed to expose defects in the interfaces and interaction between hardware and software components.
Transformation of a variable length string of characters into a usually shorter fixed-length value or key. Hashed values, or hashes, are commonly used in table or database lookups. Cryptographic hash functions are used to secure data.
A technique used to characterize the elements of risk. The result of a hazard analysis will drive the methods used for development and testing of a system.
A generally recognized rule of thumb that helps to achieve a goal.
A usability review technique that targets usability problems in the user interface or user interface design. With this technique, the reviewers examine the interface and judge its compliance with recognized usability principles (the "heuristics").
A test case without concrete (implementation level) values for input data and expected results. Logical operators are used: instances of the actual values are not yet defined and/or available.
The tracing of requirements for a test level through the layers of test documentation (e.g., test plan, test design specification, test case specification and test procedure specification or test script).
An approach to design that aims to make software products more usable by focusing on the use of the software products and applying human factors, ergonomics, and usability knowledge and techniques.
A pointer within a web page that leads to other web pages.
A tool used to check that no broken hyperlinks are present on a web site.
An organizational improvement model that serves as a roadmap for initiating, planning, and implementing improvement actions. The IDEAL model is named for the five phases it describes: initiating, diagnosing, establishing, acting, and learning.
The damage that will be caused if the risk becomes an actual outcome or event.
The assessment of change to the layers of development documentation, test documentation and components, in order to implement a given change to specified requirements.
Any event occurring that requires investigation.
The process of recognizing, investigating, taking action and disposing of incidents. It involves logging incidents, classifying them and identifying the impact.
A tool that facilitates the recording and status tracking of incidents. They often have workflow-oriented facilities to track and control the allocation, correction and re-testing of incidents and provide reporting facilities.
A document reporting on any event that occurred, e.g., during the testing, which requires investigation.
A development lifecycle where a project is broken into a series of increments, each of which delivers a portion of the functionality in the overall project requirements. The requirements are prioritized and delivered in priority order in the appropriate increment. In some (but not all) versions of this lifecycle model, each subproject follows a mini V-model with its own design, coding and testing phases.
Separation of responsibilities, which encourages the accomplishment of objective testing.
An organization responsible to test and certify that the software, hardware, firmware, platform, and operating system follow all the jurisdictional rules for each location where the product will be used.
A measure that can be used to estimate or predict another measure.
A path that cannot be executed by any set of input values and preconditions.
A review not based on a formal (documented) procedure.
Measures that protect and defend information and information systems by ensuring their availability, integrity, authentication, confidentiality, and non-repudiation. These measures include providing for restoration of information systems by incorporating protection, detection, and reaction capabilities.
Attributes of software products that bear on its ability to prevent unauthorized access, whether accidental or deliberate, to programs and data.
The phase within the IDEAL model where the groundwork is laid for a successful improvement effort. The initiating phase consists of the activities: set context, build sponsorship and charter infrastructure.
A variable (whether stored within a component or outside) that is read by a component.
A security threat originating from within the organization, often by an authorized system user.
Testing performed by people who are co-located with the project team but are not fellow employees.
A type of peer review that relies on visual examination of documents to detect defects, e.g., violations of development standards and non-conformance to higher level documentation. The most formal review technique and therefore always based on a documented procedure.
The leader and main person responsible for an inspection or other review process.
The person involved in the review that identifies and describes anomalies in the product or project under review. Reviewers can be chosen to represent different viewpoints and roles in the review process.
The capability of the software product to be installed in a specified environment.
Supplied instructions on any suitable media, which guides the installer through the installation process. This may be a manual guide, step-by-step procedure, installation wizard, or any other similar process description.
Supplied software on any suitable media which leads the installer through the installation procedure.
A test suite that covers the main functionality of a component or system to determine whether it works properly before planned testing begins.
The process of combining components or systems into larger assemblies.
Testing performed to expose defects in the interfaces and in the interactions between integrated components or systems.
The degree to which a component or system allows only authorized access and modification to a component, a system or data.
A type of integration testing performed to determine whether components or systems pass data and control correctly to one another.
The capability of the software product to interact with one or more specified components or systems.
Testing to determine the interoperability of a software product.
A system which monitors activities on the 7 layers of the OSI model from network to application level, to detect violations of the security policy.
Testing using input values that should be rejected by the component or system.
Testing a component or system in a way for which it was not intended to be used.
A graphical representation used to organize and display the interrelationships of various possible root causes of a problem. Possible causes of a real or potential defect or failure are organized in categories and subcategories in a horizontal tree-structure, with the (potential) defect or failure as the root node.
A type of software development lifecycle model in which the component or system is developed through a series of repeated cycles.
A metric that supports the judgment of process performance.
A scripting technique that uses data files to contain not only test data and expected results, but also keywords related to the application being tested. The keywords are interpreted by special supporting scripts that are called by the control script for the test.
The person who leads an assessment. In some cases, for instance CMMI and TMMi when formal assessments are conducted, the lead assessor must be accredited and formally trained.
On large projects, the person who reports to the test manager and is responsible for project management of a particular test level or a particular set of testing activities.
The capability of the software product to enable the user to learn its application.
The phase within the IDEAL model where one learns from experiences and improves one's ability to adopt new processes and technologies in the future. The learning phase consists of the activities: analyze and validate, and propose future actions.
The level to which a test object is modified by adjusting it for testability.
A test plan that typically addresses one test level.
A partitioning of the life of a product or project into phases.
The activities performed at each stage in software development, and how they relate to one another logically and chronologically.
The probability that a risk will become an actual outcome or event.
A simple scripting technique without any control structure in the test scripts.
Testing performed to expose defects in the interfaces and interactions between integrated components.
The process of simulating a defined set of activities at a specified load to be submitted to a component or system.
A tool that generates a load for a system under test.
The control and execution of load generation, and performance monitoring and reporting of the component or system.
Documentation defining a designated number of virtual users who process a defined set of transactions in a specified time period that a component or system being tested may experience in production.
A type of performance testing conducted to evaluate the behavior of a component or system with increasing load, e.g., numbers of parallel users and/or numbers of transactions, to determine what load can be handled by the component or system.
Testing based on an analysis of the internal structure of the component or system.
Testing based on an analysis of the internal structure of the component or system.
A test case without concrete (implementation level) values for input data and expected results. Logical operators are used: instances of the actual values are not yet defined and/or available.
A test case with concrete (implementation level) values for input data and expected results. Logical operators from high-level test cases are replaced by actual values that correspond to the objectives of the logical operators.
The ease with which a software product can be modified to correct defects, modified to meet new requirements, modified to make future maintenance easier, or adapted to a changed environment.
Testing to determine the maintainability of a software product.
Modification of a software product after delivery to correct defects, to improve performance or other attributes, or to adapt the product to a modified environment.
Testing the changes to an operational system or the impact of a changed environment to an operational system.
Software that is intended to harm a system or its components.
Static analysis aiming to detect and remove malicious code received at an interface.
The interception, mimicking and/or altering and subsequent relaying of communications (e.g., credit card transactions) by a third party such that a user remains unaware of that third party's presence.
A systematic evaluation of software acquisition, supply, development, operation, or maintenance process, performed by or on behalf of management that monitors progress, determines the status of plans and schedules, confirms requirements and their system allocation, or evaluates the effectiveness of management approaches to achieve fitness for purpose.
A view of quality, whereby quality is measured by the degree to which a product or service conforms to its intended design and requirements. Quality arises from the process(es) used.
A test plan that typically addresses multiple test levels.
Testing to determine the correctness of the pay table implementation, the random number generator results, and the return to player computations.
(1) The capability of an organization with respect to the effectiveness and efficiency of its processes and work practices. (2) The capability of the software product to avoid failure as a result of defects in the software.
Degree of process improvement across a predefined set of process areas in which all goals in the set are attained.
A structured collection of elements that describe certain aspects of maturity in an organization, and aid in the definition and understanding of an organization's processes.
Any model used in model-based testing.
The percentage of all single condition outcomes that independently affect a decision outcome that have been exercised by a test case suite. 100% modified condition / decision coverage implies 100% decision condition coverage.
The average time between failures of a component or system.
The average time a component or system will take to recover from a failure.
The number or category assigned to an attribute of an entity by making a measurement.
The process of assigning a number or category to an entity to describe an attribute of that entity.
A memory access failure due to a defect in a program's dynamic store allocation logic that causes it to fail to release memory after it has finished using it, eventually causing the program and/or other concurrent processes to fail due to lack of memory.
A table containing different test approaches, testing techniques and test types that are required depending on the Automotive Safety Integrity Level (ASIL) and on the context of the test object.
A test strategy whereby the test team uses a pre-determined set of test conditions such as a quality standard, a checklist, or a collection of generalized, logical test conditions which may relate to a particular domain, application or type of testing.
A measurement scale and the method used for measurement.
A point in time in a project at which defined (intermediate) deliverables and results should be ready.
A diagram arranged around a general theme that represents ideas, tasks, words or other items.
A human action that produces an incorrect result.
The degree, expressed as a percentage, to which model elements are planned to be or have been exercised by a test suite.
Dynamic testing performed using a simulation model of the system in a simulated environment.
A test strategy whereby the test team derives testware from models.
Testing based on or involving models.
A tool that supports the creation, amendment, and verification of models of the component or system.
The leader and main person responsible for an inspection or other review process.
The degree to which a component or system can be changed without introducing defects or degrading existing product quality.
The percentage of all single condition outcomes that independently affect a decision outcome that have been exercised by a test case suite. 100% modified condition / decision coverage implies 100% decision condition coverage.
A white-box test technique in which test cases are designed to exercise single condition outcomes that independently affect a decision outcome.
The percentage of all single condition outcomes that independently affect a decision outcome that have been exercised by a test case suite. 100% modified condition / decision coverage implies 100% decision condition coverage.
A white-box test technique in which test cases are designed to exercise single condition outcomes that independently affect a decision outcome.
The degree to which a system is composed of discrete components such that a change to one component has minimal impact on other components.
A minimal software item that can be tested in isolation.
The testing of individual software components.
A software tool or hardware device that runs concurrently with the component or system under test and supervises, records and/or analyzes the behavior of the component or system.
Multiple heterogeneous, distributed systems that are embedded in networks at multiple levels and in multiple interconnected domains, addressing large-scale inter-disciplinary common problems and purposes, usually without a common management structure.
Testing to determine if many players can simultaneously interact with the casino game world, with computer-controlled opponents, game servers, and with each other, as expected according to the game design.
Two or more single conditions joined by means of a logical operator.
The coverage of all possible combinations of all single condition outcomes within one statement.
A white-box test design technique in which test cases are designed to execute combinations of single condition outcomes (within one statement).
An indicator of psychological preference representing the different personalities and communication styles of people.
The coverage of sequences of N+1 transitions.
Testing a component or system in a way for which it was not intended to be used.
A form of integration testing where all of the nodes that connect to a given node are the basis for the integration testing.
A sub-network with a defined level of trust. For example, the Internet or a public zone would be considered to be untrusted.
Testing the attributes of a component or system that do not relate to functionality, e.g., reliability, efficiency, usability, maintainability and portability.
The degree to which actions or events can be proven to have taken place, so that the actions or events cannot be repudiated later.
A software product that is developed for the general market, i.e. for a large number of customers, and that is delivered to many customers in identical format.
Model-based test approach whereby test cases are generated into a repository for future execution.
Model-based test approach whereby test cases are generated and executed simultaneously.
Model-based test approach whereby test cases are generated and executed simultaneously.
A software tool that is available to all potential users in source code form, usually via the internet. Its users are permitted, usually under license, to study, change, improve and, at times, to distribute the software.
A system in which controlling action or input is independent of the output or changes in output.
The capability of the software product to enable the user to operate and control it.
Operational testing in the acceptance test phase, typically performed in a (simulated) operational environment by operations and/or systems administration staff focusing on operational aspects, e.g., recoverability, resource-behavior, installability and technical compliance.
The intended environment for a component or system to be used in production.
The representation of a distinct set of tasks performed by the component or system, possibly based on user behavior when interacting with the component or system, and their probabilities of occurrence. A task is logical rather that physical and can be executed over several machines or be executed in non-contiguous time segments.
The process of developing and implementing an operational profile.
Testing performed to evaluate a component or system in its operational environment.
A source to determine expected results to compare with the actual result of the software under test. An oracle may be the existing system (for a benchmark), other software, a user manual, or an individual's specialized knowledge, but should not be the code.
A high-level document describing the principles, approach and major objectives of the organization regarding testing.
A high-level description of the test levels to be performed and the testing within those levels for an organization or programme (one or more projects).
A 2-dimensional array constructed with special mathematical properties, such that choosing any two columns in the array provides every pair combination of each number in the array.
A systematic way of testing all-pair combinations of variables using orthogonal arrays. It significantly reduces the number of all combinations of variables to test all pair combinations.
The consequence/outcome of the execution of a test.
A variable (whether stored within a component or outside) that is written by a component.
Testing performed by people who are not co-located with the project team and are not fellow employees.
The defined time delay between iterations of the test scenario execution.
A software development approach whereby lines of code (production and/or test) of a component are written by two programmers sitting at a single computer. This implicitly means ongoing real-time code reviews are performed.
An approach in which two team members simultaneously collaborate on testing a work product.
A form of integration testing that targets pairs of components that work together, as shown in a call graph.
A black-box test design technique in which test cases are designed to execute all possible discrete combinations of each pair of input parameters.
Testing to determine that the game returns the correct mathematical results to the screen, to the players' accounts, and to the casino account.
A statistical technique in decision making that is used for selection of a limited number of factors that produce significant overall effect. In terms of quality improvement, a large majority of problems (80%) are produced by a few key causes (20%).
A black-box test design technique in which test cases are designed to execute representatives from equivalence partitions. In principle, test cases are designed to cover each partition at least once.
A test is deemed to pass if its actual result matches its expected result.
Decision rules used to determine whether a test item has passed or failed.
The status of a test result in which the actual result matches the expected result.
A security attack recovering secret passwords stored in a computer system or transmitted over a network.
A sequence of consecutive edges in a directed graph.
The percentage of paths that have been exercised by a test suite.
A white-box test design technique in which test cases are designed to execute paths.
The maximum operating capacity of a component or system.
A review of a software work product by colleagues of the producer of the product for the purpose of identifying defects and improvements. Examples are inspection, technical review and walkthrough.
A testing technique aiming to exploit security vulnerabilities (known or unknown) to gain unauthorized access.
The degree to which a system or component accomplishes its designated functions within given constraints regarding processing time and throughput rate.
The degree to which a component or system uses time, resources and capacity when accomplishing its designated functions.
A metric that supports the judgment of process performance.
Testing to determine the performance of a software product.
A test tool that generates load for a designated test item and that measures and records its performance during test execution.
A review technique whereby reviewers evaluate the work product from different viewpoints.
A review technique whereby reviewers evaluate the work product from different viewpoints.
A security attack intended to redirect a web site's traffic to a fraudulent web site without the user's knowledge or consent.
The percentage of defects that are removed in the same phase of the software lifecycle in which they were introduced.
An attempt to acquire personal or sensitive information by masquerading as a trustworthy entity in an electronic communication.
A consensus-based estimation technique, mostly used to estimate effort or relative size of user stories in Agile software development. It is a variation of the Wideband Delphi method using a deck of cards with values representing the units in which the team estimates.
Testing done by testers from a player's perspective to validate player satisfaction.
A data item that specifies the location of another data item.
The ease with which the software product can be transferred from one hardware or software environment to another.
Testing to determine the portability of a software product.
A meeting at the end of a project during which the project team members evaluate the project and learn lessons that can be applied to the next project.
A type of testing to ensure that the release is performed correctly and the application can be deployed.
Environmental and state conditions that must be fulfilled after the execution of a test or test procedure.
Environmental and state conditions that must be fulfilled before the component or system can be executed with a particular test or test procedure.
A logical expression which evaluates to true or false to direct the execution path.
The behavior predicted by the specification, or another source, of the component or system under specified conditions.
The level of (business) importance assigned to an item, e.g., defect.
A systematic approach to risk-based testing that employs product risk identification and analysis to create a product risk matrix based on likelihood and impact. Term is derived from Product RISk MAnagement.
The effect on the component or system by the measurement instrument when the component or system is being measured, e.g., by a performance testing tool or monitor. For example performance may be slightly worse when performance testing tools are being used.
An unknown underlying cause of one or more incidents.
A set of interrelated activities, which transform inputs into outputs.
A disciplined evaluation of an organization's software processes against a reference model.
A program of activities designed to improve the performance and maturity of the organization's processes, and the result of such a program.
A framework in which processes of the same nature are classified into an overall model.
A process model providing a generic body of best practices and how to improve a process in a prescribed step-by-step manner.
A test strategy whereby the test team follows a set of predefined processes, whereby the processes address such items as documentation, the proper identification and use of the test basis and test oracle(s), and the organization of the test team.
A scripting technique where scripts are structured into scenarios which represent use cases of the software under test. The scripts can be parameterized with test data.
A risk directly related to the test object.
A view of quality, wherein quality is based on a well-defined set of quality attributes. These attributes must be measured in an objective and quantitative way. Differences in the quality of products of the same type can be traced back to the way the specific quality attributes have been implemented.
Operational testing in the acceptance test phase, typically performed in a (simulated) operational environment by operations and/or systems administration staff focusing on operational aspects, e.g., recoverability, resource-behavior, installability and technical compliance.
A project is a unique set of coordinated and controlled activities with start and finish dates undertaken to achieve an objective conforming to specific requirements, including the constraints of time, cost and resources.
A structured way to capture lessons learned and to create specific action plans for improving on the next project or next project phase.
A risk related to management and control of the (test) project, e.g., lack of staffing, strict deadlines, changing requirements, etc.
A set of conventions that govern the interaction of processes, devices, and other components within a system.
A type of testing to confirm that sensors can detect nearby objects without physical contact.
A series which appears to be random but is in fact generated according to some prearranged sequence.
The process of demonstrating the ability to fulfill specified requirements. Note the term "qualified" is used to designate the corresponding status.
The degree to which a component, system or process meets specified requirements and/or user/customer needs and expectations.
Part of quality management focused on providing confidence that quality requirements will be fulfilled.
A feature or characteristic that affects an item's quality.
A category of product attributes that bears on quality.
A set of activities designed to evaluate the quality of a component or system.
A facilitated workshop technique that helps determine critical characteristics for new product development.
A special milestone in a project. Quality gates are located between those phases of a project strongly depending on the outcome of a previous phase. A quality gate includes a formal check of the documents of the previous phase.
Coordinated activities to direct and control an organization with regard to quality that include establishing a quality policy and quality objectives, quality planning, quality control, quality assurance, and quality improvement.
A product risk related to a quality attribute.
A matrix describing the participation by various roles in completing tasks or deliverables for a project or process. It is especially useful in clarifying roles and responsibilities. RACI is an acronym derived from the four key responsibilities most typically used: Responsible, Accountable, Consulted, and Informed.
A technique for decreasing the load on a system in a measurable and controlled way.
A technique for increasing the load on a system in a measurable and controlled way.
A black-box test technique in which test cases are designed by generating random independent inputs to match an operational profile.
A proprietary adaptable iterative software development process framework consisting of four project lifecycle phases: inception, elaboration, construction and transition.
Testing that runs test cases that failed the last time they were run, in order to verify the success of corrective actions.
A test strategy whereby the test team waits to design and implement tests until the software is received, reacting to the actual system under test.
Testing that dynamically responds to the behavior of the test object and to test results being obtained.
The exploration of a target area aiming to gain information that can be useful for an attack.
A test automation approach, where inputs to the test object are recorded during manual testing in order to generate automated test scripts that could be executed later (i.e. replayed).
A type of test execution tool where inputs are recorded during manual testing in order to generate automated test scripts that can be executed later (i.e. replayed). These tools are often used to support automated regression testing.
The person who records each defect mentioned and any suggestions for process improvement during a review meeting, on a logging form. The scribe should ensure that the logging form is readable and understandable.
The capability of the software product to re-establish a specified level of performance and recover the data directly affected in case of failure.
Testing to determine the recoverability of a software product.
Testing to determine the recoverability of a software product.
A degradation in the quality of a component or system due to a change.
Testing of a previously tested program following modification to ensure that defects have not been introduced or uncovered in unchanged areas of the software, as a result of the changes made. It is performed when the software or its environment is changed.
A test strategy whereby the test team applies various techniques to manage the risk of regression such as functional and/or non-functional regression test automation at one or more levels.
Testing using various techniques to manage the risk of regression, e.g., by designing re-usable testware and by extensive automation of testing at one or more test levels.
Testing to determine the compliance of the component or system.
Acceptance testing conducted to verify whether a system conforms to relevant laws, policies and regulations.
The ability of the software product to perform its required functions under stated conditions for a specified period of time, or for a specified number of operations.
A model that shows the growth in reliability over time during continuous testing of a component or system as a result of the removal of defects that result in reliability failures.
Testing to determine the reliability of a software product.
A facility that provides remote access to a test environment.
The capability of the software product to be used in place of another specified software product for the same purpose in the same environment.
A condition or capability needed by a user to solve a problem or achieve an objective that must be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed document.
A tool that supports the recording of requirements, requirements attributes (e.g., priority, knowledge responsible) and annotation, and facilitates traceability through layers of requirements and requirements change management. Some requirements management tools also provide facilities for static analysis, such as consistency checking and violations to pre-defined requirements rules.
An approach to testing in which test cases are designed based on test objectives and test conditions derived from requirements, e.g., tests that exercise specific functions or probe non-functional attributes such as reliability or usability.
The capability of the software product to use appropriate amounts and types of resources, for example the amounts of main and secondary memory used by the program and the sizes of required temporary or overflow files, when the software performs its function under stated conditions.
The process of testing to determine the resource-utilization of a software product.
The consequence/outcome of the execution of a test.
A meeting at the end of a project during which the project team members evaluate the project and learn lessons that can be applied to the next project.
The degree to which a work product can be used in more than one system, or in building other work products.
An evaluation of a product or project status to ascertain discrepancies from planned results and to recommend improvements. Examples include management review, informal review, technical review, inspection, and walkthrough.
A document describing the approach, resources and schedule of intended review activities. It identifies, amongst others: documents and code to be reviewed, review types to be used, participants, as well as entry and exit criteria to be applied in case of formal reviews, and the rationale for their choice. It is a record of the review planning process.
A tool that provides support to the review process. Typical features include review planning and tracking support, communication support, collaborative reviews and a repository for collecting and reporting of metrics.
The person involved in the review that identifies and describes anomalies in the product or project under review. Reviewers can be chosen to represent different viewpoints and roles in the review process.
A factor that could result in future negative consequences.
The process of assessing identified project or product risks to determine their level of risk, typically by estimating their impact and probability of occurrence (likelihood).
The process of identifying and subsequently analyzing the identified project or product risk to determine its level of risk, typically by assigning likelihood and impact ratings.
A set of risks grouped by one or more common factors such as a quality attribute, cause, location, or potential effect of risk. A specific set of product risk types is related to the type of testing that can mitigate (control) that risk type. For example, the risk of user interactions being misunderstood can be mitigated by usability testing.
The importance of a risk as defined by its characteristics impact and likelihood. The level of risk can be used to determine the intensity of testing to be performed. A risk level can be expressed either qualitatively (e.g., high, medium, low) or quantitatively.
The process of identifying risks using techniques such as brainstorming, checklists and failure history.
The damage that will be caused if the risk becomes an actual outcome or event.
The importance of a risk as defined by its characteristics impact and likelihood. The level of risk can be used to determine the intensity of testing to be performed. A risk level can be expressed either qualitatively (e.g., high, medium, low) or quantitatively.
The probability that a risk will become an actual outcome or event.
Systematic application of procedures and practices to the tasks of identifying, analyzing, prioritizing, and controlling risk.
The process through which decisions are reached and protective measures are implemented for reducing risks to, or maintaining risks within, specified levels.
A set of risks grouped by one or more common factors such as a quality attribute, cause, location, or potential effect of risk. A specific set of product risk types is related to the type of testing that can mitigate (control) that risk type. For example, the risk of user interactions being misunderstood can be mitigated by usability testing.
An approach to testing to reduce the level of product risks and inform stakeholders of their status, starting in the initial stages of a project. It involves the identification of product risks and the use of risk levels to guide the test process.
The degree to which a component or system can function correctly in the presence of invalid inputs or stressful environmental conditions.
Testing to determine the robustness of the software product.
A review technique where reviewers evaluate a work product from the perspective of different stakeholder roles.
A source of a defect such that if it is removed, the occurrence of the defect type is decreased or removed.
An analysis technique aimed at identifying the root causes of defects. By directing corrective measures at root causes, it is hoped that the likelihood of defect recurrence will be minimized.
A methodology whereby objectives are defined very specifically rather than generically. SMART is an acronym derived from the attributes of the objective to be defined: Specific, Measurable, Attainable, Relevant and Timely.
The absence of unreasonable risk due to hazards caused by malfunctioning behavior of Electric/Electronic(E/E) - Systems.
A system whose failure or malfunction may result in death or serious injury to people, or loss or severe damage to equipment, or environmental harm.
A cryptographic technique that adds random data (salt) to the user data prior to hashing.
A test suite that covers the main functionality of a component or system to determine whether it works properly before planned testing begins.
The degree to which a component or system can be adjusted for changing capacity.
Testing to determine the scalability of the software product.
A black-box test design technique in which test cases are designed to execute scenarios of use cases.
A review technique in which a work product is evaluated to determine its ability to address specific scenarios.
A review technique where the review is guided by determining the ability of the work product to address specific scenarios.
The person who records each defect mentioned and any suggestions for process improvement during a review meeting, on a logging form. The scribe should ensure that the logging form is readable and understandable.
A person who executes security attacks that have been created by other hackers rather than creating one's own attacks.
Testing (manual or automated) that follows a test script.
An iterative incremental framework for managing projects commonly used with Agile software development.
Attributes of software products that bear on its ability to prevent unauthorized access, whether accidental or deliberate, to programs and data.
An attempt to gain unauthorized access to a component or system, resources, information, or an attempt to compromise system integrity.
An audit evaluating an organization's security processes and infrastructure.
A high-level document describing the principles, approach and major objectives of the organization regarding security.
A set of steps required to implement the security policy and the steps to be taken in response to a security incident.
A quality risk related to security.
Testing to determine the security of the software product.
A tool that supports operational security.
A weakness in the system that could allow for a successful security attack.
A type of development lifecycle model in which a complete system is developed in a linear way of several discrete and successive phases with no overlap between them.
A technique to enable virtual delivery of services which are deployed, accessed and managed remotely.
Testing to determine the maintainability of a software product.
A method for measuring and managing session-based testing.
An approach in which test activities are planned as test sessions.
The degree of impact that a defect has on the development or operation of a component or system.
A statistical process control tool used to monitor a process and determine whether it is statistically controlled. It graphically depicts the average value and the upper and lower control limits (the highest and lowest values) of a process.
A programming language/interpreter technique for evaluating compound conditions in which a condition on one side of a logical operator may not be evaluated if the condition on the other side is sufficient to determine the final outcome.
The representation of selected behavioral characteristics of one physical or abstract system by another system.
A device, computer program or system used during testing, which behaves or operates like a given system when provided with a set of controlled inputs.
A test suite that covers the main functionality of a component or system to determine whether it works properly before planned testing begins.
An attempt to trick someone into revealing information (e.g., a password) that can be used to attack systems or networks.
Computer programs, procedures, and possibly associated documentation and data pertaining to the operation of a computer system.
The activities performed at each stage in software development, and how they relate to one another logically and chronologically.
A systematic approach to risk identification and analysis of identifying possible modes of failure and attempting to prevent their occurrence.
A technique used to analyze the causes of faults (defects). The technique visually models how logical relationships between failures, human errors, and external events can combine to cause specific faults to disclose.
A distinguishing characteristic of a component or system.
Dynamic testing performed using real software in a simulated environment or with experimental hardware.
The degree to which software complies or must comply with a set of stakeholder-selected software and/or software-based system characteristics (e.g., software complexity, risk assessment, safety level, security level, desired performance, reliability or cost) which are defined to reflect the importance of the software to its stakeholders.
The period of time that begins when a software product is conceived and ends when the software is no longer available for use. The software lifecycle typically includes a concept phase, requirements phase, design phase, implementation phase, test phase, installation and checkout phase, operation and maintenance phase, and sometimes, retirement phase. Note these phases may overlap or be performed iteratively.
A program of activities designed to improve the performance and maturity of the organization's software processes and the results of such a program.
A feature or characteristic that affects an item's quality.
Testing performed on completed, integrated software to provide evidence for compliance with software requirements.
A feature or characteristic that affects an item's quality.
Any event occurring that requires investigation.
A document reporting on any event that occurred, e.g., during the testing, which requires investigation.
A questionnaire-based usability test technique for measuring software quality from the end user's point of view.
An entity in a programming language, which is typically the smallest indivisible unit of execution.
Documentation that provides a detailed description of a component or system for the purpose of developing and testing it.
A development technique in which the specification is defined by examples.
Procedure to derive and/or select test cases based on an analysis of the specification, either functional or non-functional, of a component or system without reference to its internal structure.
Procedure to derive and/or select test cases based on an analysis of the specification, either functional or non-functional, of a component or system without reference to its internal structure.
Testing to determine the ability of a system to recover from sudden bursts of peak loads and return to a steady state.
A type of code injection in the structured query language (SQL).
A model structure wherein attaining the goals of a set of process areas establishes a maturity level; each level builds a foundation for subsequent levels.
Formal, possibly mandatory, set of requirements developed and used to prescribe consistent approaches to the way of working or to provide guidelines (e.g., ISO/IEC standards, IEEE standards, and organizational standards).
A test strategy whereby the test team follows a standard. Standards followed may be valid e.g., for a country (legislation standards), a business domain (domain standards), or internally (organizational standards).
Testing to determine the compliance of the component or system.
A diagram that depicts the states that a component or system can assume, and shows the events or circumstances that cause and/or result from a change from one state to another.
A grid showing the resulting transitions for each state combined with each possible event, showing both valid and invalid transitions.
A transition between two states of a component or system.
A diagram that depicts the states that a component or system can assume, and shows the events or circumstances that cause and/or result from a change from one state to another.
A black-box test design technique in which test cases are designed to execute valid and invalid state transitions.
An entity in a programming language, which is typically the smallest indivisible unit of execution.
The percentage of executable statements that have been exercised by a test suite.
A white-box test design technique in which test cases are designed to execute statements.
Analysis of software development artifacts, e.g., requirements or code, carried out without execution of these software development artifacts. Static analysis is usually carried out by means of a supporting tool.
A tool that carries out static analysis.
A tool that carries out static analysis.
The analysis of source code carried out without execution of that software.
Testing of a software development artifact, e.g., requirements, design or code, without execution of these artifacts, e.g., reviews or static analysis.
The capability of the software product to use appropriate amounts and types of resources, for example the amounts of main and secondary memory used by the program and the sizes of required temporary or overflow files, when the software performs its function under stated conditions.
The process of testing to determine the resource-utilization of a software product.
A type of performance testing conducted to evaluate a system or component at or beyond the limits of its anticipated or specified workloads, or with reduced availability of resources such as access to memory or servers.
A tool that supports stress testing.
Coverage measures based on the internal structure of a component or system.
Procedure to derive and/or select test cases based on an analysis of the internal structure of a component or system.
Testing based on an analysis of the internal structure of the component or system.
Procedure to derive and/or select test cases based on an analysis of the internal structure of a component or system.
Procedure to derive and/or select test cases based on an analysis of the internal structure of a component or system.
Testing based on an analysis of the internal structure of the component or system.
A scripting technique that builds and utilizes a library of reusable (parts of) scripts.
A step-by-step presentation by the author of a document in order to gather information and to establish a common understanding of its content.
A skeletal or special-purpose implementation of a software component, used to develop or test a component that calls or is otherwise dependent on it. It replaces a called component.
The capability of the software product to provide an appropriate set of functions for specified tasks and user objectives.
Testing to determine the suitability of a software product.
A type of evaluation designed and used to gather conclusions about the quality of a component or system, especially when a substantial part of it has completed design.
A collection of components organized to accomplish a specific function or set of functions.
The step-by-step process of reducing the security vulnerabilities of a system by applying a security policy and different layers of protection.
Testing the integration of systems and packages; testing interfaces to external organizations (e.g., Electronic Data Interchange, Internet).
Multiple heterogeneous, distributed systems that are embedded in networks at multiple levels and in multiple interconnected domains, addressing large-scale inter-disciplinary common problems and purposes, usually without a common management structure.
Testing performed on the completed, integrated system of software components, hardware components, and mechanics to provide evidence for compliance with system requirements and that the complete system is ready for delivery.
Testing an integrated system to verify that it meets specified requirements.
The amount of data passing through a component or system in a given time period.
A simple, ten-item attitude scale giving a global view of subjective assessments of usability.
A structured testing methodology, also used as a content-based model for improving the testing process. Systematic Test and Evaluation Process (STEP) does not require that improvements occur in a specific order.
A peer group discussion activity that focuses on achieving consensus on the technical approach to be taken.
A set of one or more test cases.
The layer in a test automation architecture which provides the necessary code to adapt test scripts on an abstract level to the various components, configuration or interfaces of the SUT.
The process of analyzing the test basis and defining test objectives.
The implementation of the test strategy for a specific project. It typically includes the decisions made that follow based on the (test) project's goal and the risk assessment carried out, starting points regarding the test process, the test design techniques to be applied, exit criteria and test types to be performed.
(1) A person who provides guidance and strategic direction for a test organization and for its relationship with other disciplines. (2) A person who defines the way testing is structured for a given system, including topics such as test tools and test data management.
The use of software to perform or support test activities, e.g., test management, test design, test execution and results checking.
An instantiation of the generic test automation architecture to define the architecture of a test automation solution, i.e., its layers, components, services and interfaces.
A person who is responsible for the design, implementation and maintenance of a test automation architecture as well as the technical evolution of the resulting test automation solution.
A tool that provides an environment for test automation. It usually includes a test harness and test libraries.
A person who is responsible for the planning and supervision of the development and evolution of a test automation solution.
A realization/implementation of a test automation architecture, i.e., a combination of components implementing a specific test automation assignment. The components may include commercial off-the-shelf test tools, test automation frameworks, as well as test hardware.
A high-level plan to achieve long-term objectives of test automation under given boundary conditions.
All documents from which the requirements of a component or system can be inferred. The documentation on which the test cases are based. If a document can be amended only by way of formal amendment procedure, then the test basis is called a frozen test basis.
An environment containing hardware, instrumentation, simulators, software tools, and other support elements needed to conduct a test.
A set of input values, execution preconditions, expected results and execution postconditions, developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement.
Procedure used to derive and/or select test cases.
The disproportionate growth of the number of test cases with growing size of the test basis, when using a certain test design technique. Test case explosion may also happen when applying the test design technique systematically for the first time.
A document specifying a set of test cases (objective, inputs, test actions, expected results, and execution preconditions) for a test item.
A set of several test cases for a component or system under test, where the post condition of one test is often used as the precondition for the next one.
A statement of test objectives, and possibly test ideas about how to test. Test charters are used in exploratory testing.
During the test closure phase of a test process data is collected from completed activities to consolidate experience, testware, facts and numbers. The test closure phase consists of finalizing and archiving the testware and evaluating the test process, including preparation of a test evaluation report.
A test tool to perform automated test comparison of actual results with expected results.
The activity that makes test assets available for later use, leaves test environments in a satisfactory condition and communicates the results of testing to relevant stakeholders.
The set of generic and specific conditions, agreed upon with the stakeholders for permitting a process to be officially completed. The purpose of exit criteria is to prevent a task from being considered completed when there are still outstanding parts of the task which have not been finished. Exit criteria are used to report against and to plan when to stop testing.
An item or event of a component or system that could be verified by one or more test cases, e.g., a function, transaction, feature, quality attribute, or structural element.
A test management task that deals with developing and applying a set of corrective actions to get a test project on track when monitoring shows a deviation from what was planned.
The degree, expressed as a percentage, to which a specified coverage item has been exercised by a test suite.
An instance of the test process against a single identifiable version of the test object.
Data that exists (for example, in a database) before a test is executed, and that affects or is affected by the component or system under test.
The activity to select data from existing databases or create, generate, manipulate and edit data for testing.
A type of test tool that enables data to be selected from existing databases or created, generated, manipulated and edited for use in testing.
The layer in a generic test automation architecture which supports test implementation by supporting the definition of test suites and/or test cases, e.g., by offering templates or guidelines.
The process of transforming general test objectives into tangible test conditions and test cases.
A document specifying the test conditions (coverage items) for a test item, the detailed test approach and identifying the associated high-level test cases.
Procedure used to derive and/or select test cases.
A tool that supports the test design activity by generating test inputs from a specification that may be held in a CASE tool repository, e.g., requirements management tool, from specified test conditions held in the tool itself, or from code.
A senior manager who manages test managers.
A software component or test tool that replaces a component that takes care of the control and/or the calling of a component or system.
An environment containing hardware, instrumentation, simulators, software tools, and other support elements needed to conduct a test.
The calculated approximation of a result related to various aspects of testing (e.g., effort spent, completion date, costs involved, number of test cases, etc.) which is usable even if input data may be incomplete, uncertain, or noisy.
The process of running a test on the component or system under test, producing actual result(s).
The use of software, e.g., capture/playback tools, to control the execution of tests, the comparison of actual results to expected results, the setting up of test preconditions, and other test control and reporting functions.
The layer in a generic test automation architecture which supports the execution of test suites and/or test cases.
A scheme for the execution of test procedures. Note: The test procedures are included in the test execution schedule in their context and in the order in which they are to be executed.
A test tool that executes tests against a designated test item and evaluates the outcomes against expected results and postconditions.
The layer in a generic test automation architecture which supports manual or automated design of test suites and/or test cases.
A type of test tool that enables data to be selected from existing databases or created, generated, manipulated and edited for use in testing.
A test environment comprised of stubs and drivers needed to execute a test.
A customized software interface that enables automated testing of a test object.
The process of developing and prioritizing test procedures, creating test data and, optionally, preparing test harnesses and writing automated test scripts.
A plan for achieving organizational test process improvement objectives based on a thorough understanding of the current strengths and weaknesses of the organization's test processes and test process assets.
Any event occurring that requires investigation.
A document reporting on any event that occurred, e.g., during the testing, which requires investigation.
The organizational artifacts needed to perform testing, consisting of test environments, test tools, office environment and procedures.
The data received from an external source by the test object during test execution. The external source can be hardware, software or human.
The individual element to be tested. There usually is one test object and many test items.
On large projects, the person who reports to the test manager and is responsible for project management of a particular test level or a particular set of testing activities.
A group of test activities that are organized and managed together. A test level is linked to the responsibilities in a project. Examples of test levels are component test, integration test, system test and acceptance test.
A chronological record of relevant details about the execution of tests.
The process of recording information about tests executed into a test log.
The planning, estimating, monitoring and control of test activities, typically carried out by a test manager.
A tool that provides support to the test management and control part of a test process. It often has several capabilities, such as testware management, scheduling of tests, the logging of results, progress tracking, incident management and test reporting.
The person responsible for project management of testing activities and resources, and evaluation of a test object. The individual who directs, controls, administers, plans and regulates the evaluation of a test object.
A five-level staged framework for test process improvement, related to the Capability Maturity Model Integration (CMMI), that describes the key elements of an effective test process.
The purpose of testing for an organization, often documented as part of the test policy.
A model describing testware that is used for testing a component or a system under test.
A test management task that deals with the activities related to periodically checking the status of a test project. Reports are prepared that compare the actuals to that which was planned.
The component or system to be tested.
A reason or purpose for designing and executing a test.
A source to determine expected results to compare with the actual result of the software under test. An oracle may be the existing system (for a benchmark), other software, a user manual, or an individual's specialized knowledge, but should not be the code.
The consequence/outcome of the execution of a test.
A distinct set of test activities collected into a manageable phase of a project, e.g., the execution activities of a test level.
A document describing the scope, approach, resources and schedule of intended test activities. It identifies amongst others test items, the features to be tested, the testing tasks, who will do each task, degree of tester independence, the test environment, the test design techniques and entry and exit criteria to be used, and the rationale for their choice, and any risks requiring contingency planning. It is a record of the test planning process.
The activity of establishing or updating a test plan.
A formula based test estimation method based on function point analysis.
A high-level document describing the principles, approach and major objectives of the organization regarding testing.
A sequence of test cases in execution order, and any associated actions that may be required to set up the initial preconditions and any wrap up activities post execution.
A document specifying a sequence of actions for the execution of a test. Also known as test script or manual test script.
The fundamental test process comprises test planning and control, test analysis and design, test implementation and execution, evaluating exit criteria and reporting, and test closure activities.
A collection of (test) specialists who facilitate the definition, maintenance, and improvement of the test processes used by an organization.
A program of activities undertaken to improve the performance and maturity of the organization's test processes.
A statement that echoes the Agile manifesto, and defines values for improving the testing process. The values are: flexibility over detailed processes, best practices over templates, deployment orientation over process orientation, peer reviews over quality assurance (departments), business driven over model-driven.
A person implementing improvements in the test process based on a test improvement plan.
A document summarizing testing activities and results, produced at regular intervals, to report progress of testing activities against a baseline (such as the original test plan) and to communicate risks and alternatives requiring a decision to management.
A graphical model representing the relationship of the amount of testing per level, with more at the bottom than at the top.
A chronological record of relevant details about the execution of tests.
The process of recording information about tests executed into a test log.
Documentation summarizing test activities and results.
Collecting and analyzing data from testing activities and subsequently consolidating the data in a report to inform stakeholders.
An item or event of a component or system that could be verified by one or more test cases, e.g., a function, transaction, feature, quality attribute, or structural element.
The consequence/outcome of the execution of a test.
An environment containing hardware, instrumentation, simulators, software tools, and other support elements needed to conduct a test.
The execution of a test suite on a specific version of the test object.
A chronological record of relevant details about the execution of tests.
A document specifying a sequence of actions for the execution of a test. Also known as test script or manual test script.
A list of activities, tasks or events of the test process, identifying their intended start and finish dates and/or times, and interdependencies.
Commonly used to refer to a test procedure specification, especially an automated one.
A sequence of test cases in execution order, and any associated actions that may be required to set up the initial preconditions and any wrap up activities post execution.
The criteria used to guide the generation of test cases or to select test cases in order to limit the size of a test.
An uninterrupted period of time spent in executing tests.
A set of several test cases for a component or system under test, where the post condition of one test is often used as the precondition for the next one.
An item or event of a component or system that could be verified by one or more test cases, e.g., a function, transaction, feature, quality attribute, or structural element.
A document that consists of a test design specification, test case specification and/or test procedure specification.
Procedure used to derive and/or select test cases.
A group of test activities that are organized and managed together. A test level is linked to the responsibilities in a project. Examples of test levels are component test, integration test, system test and acceptance test.
A document summarizing testing activities and results, produced at regular intervals, to report progress of testing activities against a baseline (such as the original test plan) and to communicate risks and alternatives requiring a decision to management.
A high-level description of the test levels to be performed and the testing within those levels for an organization or programme (one or more projects).
A set of several test cases for a component or system under test, where the post condition of one test is often used as the precondition for the next one.
A document summarizing testing activities and results. It also contains an evaluation of the corresponding test items against exit criteria.
A procedure used to derive and/or select test cases.
A software product that supports one or more test activities, such as planning and control, specification, building initial files and data, test execution and test analysis.
A group of test activities aimed at testing a component or system focused on a specific test objective, i.e. functional test, usability test, regression test etc. A test type may take place on one or more test levels or test phases.
A way of developing software where the test cases are developed, and often automated, before the software is developed to run those test cases.
An approach to software development in which the test cases are designed and implemented before the associated component or system is developed.
The capability of the software product to enable modified software to be tested.
A skilled professional who is involved in the testing of a component or system.
The process consisting of all lifecycle activities, both static and dynamic, concerned with planning, preparation and evaluation of software products and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects.
A classification model of test types/levels in four quadrants, relating them to two dimensions of test goals: supporting the team vs. critiquing the product, and technology-facing vs. business-facing.
Artifacts produced during the test process required to plan, design, and execute tests, such as documentation, scripts, inputs, expected results, set-up and clear-up procedures, files, databases, environment, and any additional software or utilities used in testing.
A usability testing technique where test participants share their thoughts with the moderator and observers by thinking aloud while they solve usability test tasks. Think aloud is useful to understand the test participant.
The amount of time required by a user to determine and execute the next action in a sequence of actions.
A test estimation method using estimated values for the "best case", "worst case", and "most likely case" of the matter being estimated, to define the degree of certainty associated with the resultant estimate.
The degree to which a component or system can perform its required functions within required response times, processing times and throughput rates.
An organization-wide management approach centered on quality, based on the participation of all members of the organization and aiming at long-term success through customer satisfaction, and benefits to all members of the organization and to society. Total Quality Management consists of planning, organizing, directing, control, and assurance.
A set of exploratory tests organized around a special focus.
A continuous business-driven framework for test process improvement that describes the key elements of an effective and efficient test process.
The ability to identify related items in documentation and software, such as requirements with associated tests.
A two-dimensional table, which correlates two entities (e.g., requirements and test cases). The table allows tracing back and forth the links of one entity to the other, thus enabling the determination of coverage achieved and the assessment of impact of proposed changes.
The analysis of transactions between people and within people's minds; a transaction is defined as a stimulus plus a response. Transactions take place between people and between the ego states (personality segments) within one person's mind.
A view of quality, wherein quality cannot be precisely defined, but we know it when we see it, or are aware of its absence when it is missing. Quality depends on the perception and affective feelings of an individual or group of individuals toward a product.
The capability of the software product to enable the user to understand whether the software is suitable, and how it can be used for particular tasks and conditions of use.
A minimal software item that can be tested in isolation.
A tool that provides an environment for unit or component testing in which a component can be tested in isolation or with suitable stubs and drivers. It also provides other support for the developer, such as debugging capabilities.
The testing of individual software components.
Code that cannot be reached and therefore is impossible to execute.
The capability of the software to be understood, learned, used and attractive to the user when used under specified conditions.
A process through which information about the usability of a system is gathered in order to improve the system (known as formative evaluation) or to assess the merit or worth of a system (known as summative evaluation).
A test facility in which unintrusive observation of participant reactions and responses to software takes place.
A requirement on the usability of a component or system.
A representative user who solves typical tasks in a usability test.
A document specifying a sequence of actions for the execution of a usability test. It is used by the moderator to keep track of briefing and pre-session interview questions, usability test tasks, and post-session interview questions.
A test session in usability testing in which a usability test participant is executing tests, moderated by a moderator and observed by a number of observers.
A usability test execution activity specified by the moderator that needs to be accomplished by a usability test participant within a given period of time.
Testing to determine the extent to which the software product is understood, easy to learn, easy to operate and attractive to the users under specified conditions.
A sequence of transactions in a dialogue between an actor and a component or system with a tangible result, where an actor can be a user or anything that can exchange information with the system.
A black-box test design technique in which test cases are designed to execute scenarios of use cases.
Acceptance testing carried out by future users in a (simulated) operational environment focusing on user requirements and needs.
The degree to which a component or system protects users against making errors.
A person's perceptions and responses resulting from the use or anticipated use of a software product.
All components of a system that provide information and controls for the user to accomplish specific tasks with the system.
The degree to which a user interface enables pleasing and satisfying interaction for the user.
A low-level, specific rule or recommendation for user interface design that leaves little room for interpretation so designers implement it similarly. It is often used to ensure consistency in the appearance and behavior of the user interface of the systems produced by an organization.
A black-box test design technique in which test cases are designed to execute scenarios of use cases.
A high-level user or business requirement commonly used in Agile software development, typically consisting of one sentence in the everyday or business language capturing what functionality a user needs and the reason behind this, any non-functional criteria, and also includes acceptance criteria.
A black-box test design technique in which test cases are designed based on user stories to verify their correct implementation.
A usability evaluation whereby a representative sample of users are asked to report subjective evaluation into a questionnaire based on their experience in using a component or system.
A type of testing in which a test client is used to switch the user agent string and identify itself as a different client while executing test suites.
A view of quality, wherein quality is the capacity to satisfy needs, wants and desires of the user(s). A product or service that does not fulfill user needs is unlikely to find any users. This is a context dependent, contingent approach to quality since different business characteristics require different qualities of a product.
A framework to describe the software development lifecycle activities from requirements specification to maintenance. The V-model illustrates how testing activities can be integrated into each phase of the software development lifecycle.
Confirmation by examination and through provision of objective evidence that the requirements for a specific intended use or application have been fulfilled.
A view of quality wherein quality is defined by price. A quality product or service is one that provides desired performance at an acceptable cost. Quality is determined by means of a decision process with stakeholders on trade-offs between time, effort and cost aspects.
An element of storage in a computer that is accessible by a software program by referring to it by a name.
Confirmation by examination and through provision of objective evidence that specified requirements have been fulfilled.
A simulation of activities performed according to a user operational profile.
A static analyzer that is used to detect particular security vulnerabilities in the code.
A step-by-step presentation by the author of a document in order to gather information and to establish a common understanding of its content.
A part of a series of web accessibility guidelines published by the Web Accessibility Initiative (WAI) of the World Wide Web Consortium (W3C), the main international standards organization for the internet. They consist of a set of guidelines for making content accessible, primarily for people with disabilities.
A questionnaire-based usability test technique for measuring web site software quality from the end user's point of view.
Procedure to derive and/or select test cases based on an analysis of the internal structure of a component or system.
Procedure to derive and/or select test cases based on an analysis of the internal structure of a component or system.
A procedure to derive and/or select test cases based on an analysis of the internal structure of a component or system.
Testing based on an analysis of the internal structure of the component or system.
An expert-based test estimation technique that aims at making an accurate estimation using the collective wisdom of the team members.
A pointer that references a location that is out of scope for that pointer or that does not exist.
An arrangement of work elements and their relationship to each other and to the end product.
A generalized term for dynamic testing in different virtual test environments.