Types of Test Tools

Tool support for software testing

Important Terms:

Types of test tool

Configuration management tool, debugging tool, dynamic analysis tool, incident management tool, load testing tool, modeling tool, monitoring tool, performance testing tool, probe effect, requirement management tool, review tool, static analysis tool, stress testing tool, test comparator, test data preparation tool, test design tool, test harness, test execution tool, test management tool, unit test framework tool.

 Effective use of tools: potential benefits and risks

Data driven (testing), keyword-driven (testing), scripting language

 introducing a tool into an organization

Types of test tool

management of testing and tests	Requirement management tools Incident management tools Configuration management tools
Static testing	Review tools Static analysis tools (D) Modeling tools (D)
Test specification	§         Test design tools §         Test data preparation tools
Test execution and logging	Test execution tools Test harness/unit test framework tools (D) Test comparators Coverage measurement tools (D) Security tools
Performance and monitoring	Dynamic analysis tools Performance/Load/Stress Testing tools Monitoring tools
Specific application areas	Special tools for web-based applications Special tools for specific development flat forms Special tools for embedded systems
Tool support using other tools

Test Tools and their purposes

Requirement management tools	Store requirements, check for consistency, allow requirements to be prioritized, trace changes, coverage of requirements etc.
Incident management tools	Store and manage incident reports, facilitating prioritization, assessment of actions to people and attribution of status etc.
Configuration management tools	Store information about versions and builds of software and testware; enable traceability between testware and software work products etc.
Review tools	Store information, store and communicate review comments etc.
Static analysis tools (D)	The enforcement of coding standards, the analysis of structures and dependencies, aiding in understanding the code etc.
Modeling tools (D)	Validate models of the software, find defects in data model, state model or an object model etc.
Test design tools	Generate test inputs or executable tests, generate expected out comes etc.
Test data preparation tools	Preparing test data, Manipulate databases, files or data transmissions to set up test data etc.
Test execution tools	Record tests, Automated test execution, use inputs and expected outcomes, compare results with expected outcomes, repeat tests, dynamic comparison, manipulate the tests using scripting language etc.
Test harness/unit test framework tools (D)	Test components or part of a system by simulating the environment, provide an execution framework in middleware etc.
Test comparators	Determine differences between files, databases or test results post-execution comparison, may use test oracle if it is automated etc.
Coverage measurement tools (D)	Measure the percentage of specific types of code structure (ex: statements, branches or decisions, and module or function calls)
Security tools	Check for computer viruses and denial of service attacks, search for specific vulnerabilities of the system etc.
Dynamic analysis tools (D)	Detect memory leaks, identify time dependencies and identify pointer arithmetic errors.
Performance/Load/Stress Testing tools	Measure load or stress, Monitor and report on how a system behaves a variety of simulated usage conditions, simulate a load on an application/a database/or a system environment, repetitive execution of tests etc.
Monitoring tools	Continuously analyze, verify and report on specific system resources; store information about the version and build of the software and testware, and enable traceability.
Tool support using other tools	Some tools use other tools (Ex: QTP uses excel sheet and SQL tools)

Potential benefits and risks of tool support for testing

Benefits:

• Repetitive work is reduced
• Greater consistency and repeatability
• Objective assessment
• Ease of access to information about tests or testing

Risks:

• Unrealistic expectations for the tool
• Underestimating the time and effort needed to achieve significant and continues benefits from the tool
• Underestimating the effort required to maintain the test assets generated by the tool
• Over-reliance on the tool

Special considerations for some types of tools

Following tools have special considerations         

ü       Test execution tools

ü       Performance testing tools

ü       Static testing tools

ü       Test management tools

                            Introducing a tool into an organization               

The following factors are important in selecting a tool:

• Assessment of the organization maturity
• Identification of the areas within the organization where tool support will help to improve testing process
• Evaluation of tools against clear requirements and objective criteria
• Proof-of-concept to see whether the product works as desired and meets the requirements and objectives defined for it
• Evaluation of the vendor (training, support and other commercial aspects) or open-source network of support
• Identifying and planning internal implementation (including coaching and mentoring for those new to the use of the tool)

The objectives for a pilot project for a new tool

• To learn more about the tool
• To see how the tool would fit with existing processes or documentation
• To decide on standard ways of using the tool that will work for all potential users
• To evaluate the pilot project agonist its objectives

Successes factors for the deployment of the new tool within an organization

• Rolling out the tool to the rest of the organization incrementally
• Adapting and improving process to fit with the use of the tool
• Providing training and coaching/mentoring for new users.
• Defining usage guidelines
• Implementing a way to learn lessons from tool use.
• Monitoring tool use and benefits.