Test tools & automation

A scripting technique that uses data files to contain not only test data and expected results, but also keywords related to the application being tested. The …

A tool that carries out static analysis.

Testing performed by submitting commands to the software under test using programming interfaces of the application directly.

Permission given to a user or process to access resources.

Defect density of a component of the test automation code.

A strategic tool for measuring whether the operational activities of a company are aligned with its objectives in terms of business vision and strategy.

A framework that describes the key elements of an effective product development and maintenance process. The Capability Maturity Model Integration covers …

A test automation approach, where inputs to the test object are recorded during manual testing in order to generate automated test scripts that could be …

A type of test execution tool where inputs are recorded during manual testing in order to generate automated test scripts that can be executed later (i.e. …

A tree showing equivalence partitions hierarchically ordered, which is used to design test cases in the classification tree method.

Acronym for Command-Line Interface.

Testing performed by submitting commands to the software under test using a dedicated command-line interface.

An analysis method that determines which parts of the software have been executed (covered) by the test suite and which parts have not been executed, e.g., …

A standard that describes the characteristics of a design or a design description of data or program components.

A software tool that translates programs expressed in a high-order language into their machine language equivalents.

A test suite that covers the main functionality of a component or system to determine whether it works properly before planned testing begins.

Testing that runs test cases that failed the last time they were run, in order to verify the success of corrective actions.

A statistical process control tool used to monitor a process and determine whether it is statistically controlled. It graphically depicts the average value and …

A sequence of consecutive edges in a directed graph.

The degree, expressed as a percentage, to which a specified coverage item has been exercised by a test suite.

A software tool developed specifically for a set of users or customers.

A scripting technique that stores test input and expected results in a table or spreadsheet, so that a single control script can execute all of the tests in the …

A tool used by programmers to reproduce failures, investigate the state of programs and find the corresponding defect. Debuggers enable programmers to execute …

The process of finding, analyzing and removing the causes of failures in software.

A software component or test tool that replaces a component that takes care of the control and/or the calling of a component or system.

A device, computer program, or system that accepts the same inputs and produces the same outputs as a given system.

Effort required for running tests manually.

A tool for seeding (i.e., intentionally inserting) faults in a component or system.

A non-prescriptive framework for an organization's quality management system, defined and owned by the European Foundation for Quality Management, based on five …

A source code statement that, when translated into object code, can be executed in a procedural manner.

A test is deemed to fail if its actual result does not match its expected result.

The process of intentionally adding defects to a system for the purpose of finding out whether the system can detect, and possibly recover from, a defect. Fault …

Representation of the layers, components, and interfaces of a test automation architecture, allowing for a structured and modular approach to implement test …

A type of interface that allows users to interact with a component or system through graphical icons and visual indicators.

Testing performed by interacting with the software under test via the graphical user interface.

A tool used to check that no broken hyperlinks are present on a web site.

A tool that facilitates the recording and status tracking of incidents. They often have workflow-oriented facilities to track and control the allocation, …

The level to which a test object is modified by adjusting it for testability.

The activities performed at each stage in software development, and how they relate to one another logically and chronologically.

A simple scripting technique without any control structure in the test scripts.

A tool that generates a load for a system under test.

The ease with which a software product can be modified to correct defects, modified to meet new requirements, modified to make future maintenance easier, or …

Modification of a software product after delivery to correct defects, to improve performance or other attributes, or to adapt the product to a modified …

A measurement scale and the method used for measurement.

Testing based on or involving models.

A software tool or hardware device that runs concurrently with the component or system under test and supervises, records and/or analyzes the behavior of the …

A software tool that is available to all potential users in source code form, usually via the internet. Its users are permitted, usually under license, to …

The intended environment for a component or system to be used in production.

The consequence/outcome of the execution of a test.

A test is deemed to pass if its actual result matches its expected result.

A review of a software work product by colleagues of the producer of the product for the purpose of identifying defects and improvements. Examples are …

A test tool that generates load for a designated test item and that measures and records its performance during test execution.

A consensus-based estimation technique, mostly used to estimate effort or relative size of user stories in Agile software development. It is a variation of the …

The ease with which the software product can be transferred from one hardware or software environment to another.

A meeting at the end of a project during which the project team members evaluate the project and learn lessons that can be applied to the next project.

The effect on the component or system by the measurement instrument when the component or system is being measured, e.g., by a performance testing tool or …

A framework in which processes of the same nature are classified into an overall model.

A scripting technique where scripts are structured into scenarios which represent use cases of the software under test. The scripts can be parameterized with …

A risk directly related to the test object.

A structured way to capture lessons learned and to create specific action plans for improving on the next project or next project phase.

The degree to which a component, system or process meets specified requirements and/or user/customer needs and expectations.

A proprietary adaptable iterative software development process framework consisting of four project lifecycle phases: inception, elaboration, construction and …

Testing of a previously tested program following modification to ensure that defects have not been introduced or uncovered in unchanged areas of the software, …

A test strategy whereby the test team applies various techniques to manage the risk of regression such as functional and/or non-functional regression test …

The capability of the software product to use appropriate amounts and types of resources, for example the amounts of main and secondary memory used by the …

A factor that could result in future negative consequences.

The process of identifying and subsequently analyzing the identified project or product risk to determine its level of risk, typically by assigning likelihood …

The process through which decisions are reached and protective measures are implemented for reducing risks to, or maintaining risks within, specified levels.

A person who executes security attacks that have been created by other hackers rather than creating one's own attacks.

Testing (manual or automated) that follows a test script.

An iterative incremental framework for managing projects commonly used with Agile software development.

A technique to enable virtual delivery of services which are deployed, accessed and managed remotely.

A device, computer program or system used during testing, which behaves or operates like a given system when provided with a set of controlled inputs.

Formal, possibly mandatory, set of requirements developed and used to prescribe consistent approaches to the way of working or to provide guidelines (e.g., …

Analysis of software development artifacts, e.g., requirements or code, carried out without execution of these software development artifacts. Static analysis …

A type of performance testing conducted to evaluate a system or component at or beyond the limits of its anticipated or specified workloads, or with reduced …

A scripting technique that builds and utilizes a library of reusable (parts of) scripts.

A skeletal or special-purpose implementation of a software component, used to develop or test a component that calls or is otherwise dependent on it. It …

See test object.

The layer in a test automation architecture which provides the necessary code to adapt test scripts on an abstract level to the various components, …

(1) A person who provides guidance and strategic direction for a test organization and for its relationship with other disciplines. (2) A person who defines the …

The use of software to perform or support test activities, e.g., test management, test design, test execution and results checking.

An instantiation of the generic test automation architecture to define the architecture of a test automation solution, i.e., its layers, components, services …

A person who is responsible for the design, implementation and maintenance of a test automation architecture as well as the technical evolution of the …

A tool that provides an environment for test automation. It usually includes a test harness and test libraries.

A person who is responsible for the planning and supervision of the development and evolution of a test automation solution.

A realization/implementation of a test automation architecture, i.e., a combination of components implementing a specific test automation assignment. The …

A high-level plan to achieve long-term objectives of test automation under given boundary conditions.

The disproportionate growth of the number of test cases with growing size of the test basis, when using a certain test design technique. Test case explosion …

The activity to select data from existing databases or create, generate, manipulate and edit data for testing.

A type of test tool that enables data to be selected from existing databases or created, generated, manipulated and edited for use in testing.

The layer in a generic test automation architecture which supports test implementation by supporting the definition of test suites and/or test cases, e.g., by …

The process of running a test on the component or system under test, producing actual result(s).

The use of software, e.g., capture/playback tools, to control the execution of tests, the comparison of actual results to expected results, the setting up of …

The layer in a generic test automation architecture which supports the execution of test suites and/or test cases.

A test tool that executes tests against a designated test item and evaluates the outcomes against expected results and postconditions.

The layer in a generic test automation architecture which supports manual or automated design of test suites and/or test cases.

A test environment comprised of stubs and drivers needed to execute a test.

A customized software interface that enables automated testing of a test object.

The organizational artifacts needed to perform testing, consisting of test environments, test tools, office environment and procedures.

The process of recording information about tests executed into a test log.

A tool that provides support to the test management and control part of a test process. It often has several capabilities, such as testware management, …

The person responsible for project management of testing activities and resources, and evaluation of a test object. The individual who directs, controls, …

A five-level staged framework for test process improvement, related to the Capability Maturity Model Integration (CMMI), that describes the key elements of an …

A model describing testware that is used for testing a component or a system under test.

A reason or purpose for designing and executing a test.

Collecting and analyzing data from testing activities and subsequently consolidating the data in a report to inform stakeholders.

Commonly used to refer to a test procedure specification, especially an automated one.

A software product that supports one or more test activities, such as planning and control, specification, building initial files and data, test execution and …

The capability of the software product to enable modified software to be tested.

The process consisting of all lifecycle activities, both static and dynamic, concerned with planning, preparation and evaluation of software products and …

Artifacts produced during the test process required to plan, design, and execute tests, such as documentation, scripts, inputs, expected results, set-up and …

A continuous business-driven framework for test process improvement that describes the key elements of an effective and efficient test process.

A two-dimensional table, which correlates two entities (e.g., requirements and test cases). The table allows tracing back and forth the links of one entity to …

A tool that provides an environment for unit or component testing in which a component can be tested in isolation or with suitable stubs and drivers. It also …

A document specifying a sequence of actions for the execution of a usability test. It is used by the moderator to keep track of briefing and pre-session …

A framework to describe the software development lifecycle activities from requirements specification to maintenance. The V-model illustrates how testing …

Confirmation by examination and through provision of objective evidence that specified requirements have been fulfilled.