Glossary of Software Testing Terms Provided by Testing Realms

Glossary of Software Testing Terms: S

This glossary of software testing terms and conditions is a compilation of knowledge, gathered over time, from many different sources. It is provided “as-is” in good faith, without any warranty as to the accuracy or currency of any definition or other information contained herein. If you have any questions or queries about the contents of this glossary, please contact Project Realms directly.

Safety Testing Specification Structural Test Case Design
Sanity Testing Specification Testing Structural Testing
Scalability Testing Specified Input Structured Basis Testing
Schedule Stabilization Period Structured Walkthrough
Scrambling State Transition Stub
Scribe State Transition Testing Subgoal
Script Statement Subpath
Security Statement Coverage Suitability
Security Requirements Statement Testing Suspension Criteria
Security Testing Static Analysis Symbolic Execution
Simple Subpath Static Analyzer Symbolic Processing
Simulation Static Code Analysis Syntax Testing
Simulator Static Testing System
Smoke Testing Statistical Testing System Integration Testing
Soak Testing Storage Testing System Testing
Software Requirements Specification Stress Testing System Under Test (SUT)
Software Testing Structural Coverage  

Safety Testing
The process of testing to determine the safety of a software product.


Sanity Testing
Brief test of major functional elements of a piece of software to determine if it's basically operational. See also Smoke Testing.

Scalability Testing
Scalability Testing is designed to determine the ability of a system to expand as the usage increases. It measures individual components of the system to determine their performance characteristics under increasing user loads. It also examines inter-component and multiple component behavior.

A scheme for the execution of test procedures.
The test procedures are included in the test execution schedule in the order in which they are to be executed.

Data obfuscation routine to de-identify sensitive data in test data environments to meet the requirements of the Data Protection Act and other legislation.

The person who has to record each defect mentioned and any suggestions for improvement during a review meeting, on a logging form. The scribe has to make ensure that the logging form is understandable.

When the user's actions are recorded, the output that is generated and stored in a script.

The preservation of availability, integrity, and confidence of information:
  • Availability is ensuring that authorized users have access to information and associated assets when required.
  • Integrity is safeguarding the accuracy and completeness of information and processing methods.
  • Confidentiality is ensuring that information is accessible only to those authorized to have access.
Security Requirements
A specification of the required security for the system or software.

Security Testing
This is a process to determine that an IS (Information System) protects data and maintains functionality as intended.

The six concepts that need to be covered by security testing are: confidentiality, integrity, authentication, authorization, availability, and non-repudiation.

A security measure which protects against the disclosure of information to parties other than the intended recipient(s). Often ensured by means of encoding, using a defined algorithm and some secret information known only to the originator of the information and the intended recipient(s) (a process known as cryptography) but that is by no means the only way of ensuring confidentiality.

A measure intended to allow the receiver to determine that the information which it receives has not been altered in transit or by other than the originator of the information. Integrity schemes often use some of the same underlying technologies as confidentiality schemes, but they usually involve adding additional information to a communication to form the basis of an algorithmic check rather than encoding all of the communication.

A measure designed to establish the validity of a transmission, message, or originator. It allows a receiver to have confidence that the information it receives originated from a specific known source.

The process of determining that a requester is allowed to receive a service or perform an operation.

Assuring information and communications services will be ready for use when expected. Information must be kept available to authorized persons when they need it.

A measure intended to prevent the later denial that an action happened, or a communication took place, etc. In communication terms, this often involves the interchange of authentication information combined with some form of provable time stamp.

Security Testing focuses on these key areas of security
  • Application-level Security; including access to the Data or Business Functions
  • System-level Security; including logging into / remote access to the system

Application-level security ensures that, based upon the desired security, users are restricted to specific functions / use cases or are limited in the data that is available to them. For example, every user may be permitted to enter data and create new accounts, but only managers can delete them. If there is security at the data level, testing ensures that all users can see all customer information, including financial data; however, managers can also see the demographic data for the client.

System-level security ensures that only those users granted access to the system are capable of accessing the applications and only through the appropriate gateways.
Security testing is a broad category of testing. Based on the two types listed above, there are some areas of focus to consider:

  • Application Based Security - These are home grown solutions in which the programmers create and maintain users and user security permissions within a given application. This means the users, their security roles and their security permissions are written in the application. Security could be applied at a field level, screen level, feature level or system level. One common subset of this type is to utilize operating system based security (see below) and integrate it with application based security. This means the basic authorization for the user is managed at the operating system level, but if authorized, then application specific permissions are managed by the programmers. A review of the system design and requirements should help define the testing effort for these types of security systems
  • Operating System Based Security - This security is defined by the Operating System (OS) of the system. Microsoft uses an OS based security that is managed by an internal application called Active Directory (AD). AD allows system administrators to create and manage users on the system. It defines what level of access the user has to system objects and Microsoft system hardware. It allows the customization of user access by utilizing security groups and the definition of custom user security properties. It is recommended that you study Active Directory from Microsoft to understand how to test this type of security
  • Device Based Security - There are many hardware devices on the market dedicated to security. Hardware firewalls, Security Card Readers and biometric devices are just a few. Each of these are some combination of a physical device and software designed to interpret the information passing through the physical device while applying security rules to or based on the data flowing through the device. Testing of these devices requires knowledge of what the device is for and how it works. The device manufacturer will typically make this information available, though sometimes they require specialized training
  • Web Based Security - This is a combination of all three security types; Application, OS and Device based security. As such it can be tested from various points of view. One could test the browser security and the code presented in the browser. Both efforts would be considered application based security testing. One could also test the web server security. This testing attempts to uncover vulnerabilities at the web server tier. This is considered both application and OS based security testing as web servers are typically distributed with an OS. Another level of testing could be external, intrusion testing. This probes for vulnerabilities from outside the web based system. This will include testing all three securities simultaneously. The security testing of web systems is a discipline and a career for some people.

Simple Subpath
A subpath of the control flow graph in which no program part is executed more than necessary.

The representation of selected behavioral characteristics of one physical or abstract system by another system.

A device, computer program or system used during software verification, which behaves or operates like a given system when provided with a set of controlled inputs.

Smoke Testing
Smoke testing is a quick test (sometimes called a sanity check) often used to validate a new build. It validates that the configuration is available, functional, and no gross defects are apparent.

Key points to include in a smoke test are:
  • If the System has a User Interface, then request each screen once to:
    • Validate the screen exists
    • Validate key points on the screen to ensure you are not getting an error message screen and that the screen is usable. This is subject to interpretation, but the guiding principal is gross functionality is available
    • Do not validate all the data on the screen. This is the job of other tests
  • Request each system interface to verify the following:
    • The connection is active
    • The response was not an error response
    • The response was timely
    • The data is accurate for a single scenario. This can occur via a User Interface or directly through the database
  • Smoke test should be no longer than 30 minutes. They are typically 5 minutes in duration
  • Create a single Test Case, a group of Test Cases called Smoke Tests or a check list identifying each component of the Smoke Test to ensure repeatability
  • Use automated tools whenever possible

Soak Testing
Soak testing involves testing a system with a significant load extended over a significant period of time, to discover how the system behaves under sustained use.

For example, in software testing, a system may behave exactly as expected when tested for 1 hour. However, when it is tested for 3 hours, problems such as memory leads cause the system to fail or behave randomly.

Soak tests are used primarily to check the reaction of a subject under test under a possible simulated environment for a given duration and for a given threshold. Observations made during the soak test are used to improve the characteristics of the subject under test further.

Software Requirements Specification
This is a deliverable that describes all data, functional, and behavioral requirements, all constraints, and all validation requirements for software.

Software Testing
The process used to measure the quality of developed computer software. Usually, quality is constrained to such topics as correctness, completeness, security, but can also include more technical requirements as described under the ISO standard ISO 9126, such as capability, reliability, efficiency, portability, maintainability, compatibility, and usability. Testing is a process of technical investigation, performed on behalf of stakeholders, that is intended to reveal quality-related information about the product with respect to the context in which it is intended to operate. This includes, but is not limited to, the process of executing a program or application with the intent of finding errors. Quality is not an absolute; it I a value to some person.

With that in mind, testing can never completely establish the correctness of arbitrary computer software; testing furnishes a criticism or comparison that compares the state and behavior of the product against a specification. An important point is that software testing should be distinguished from the separate discipline of Software Quality Assurance (SQA), which encompasses all business process areas, and not just testing.

Today, software has grown in complexity and size. The software product developed by a developer is according to the System Requirement Specification. Every software product has a target audience. For example, a video game software has its audience completely different from baking software. Therefore, when an organization invests large sums in making a software product, it must ensure that the software product must be acceptable to the end users or its target audience. This is where Software Testing comes into play. Software testing is not merely finding defects or bugs in the software, it is the completely dedicated discipline of evaluating the quality of the software.

There are many approaches to software testing, but effective testing of complex products is essentially a process of investigation, not merely a matter of creating and following routine procedures. One definition of testing is "the process of questioning a product in order to evaluate it", where the "questions" are operations the test attempts to execute with the product, and the product answers with its behavior in reaction to the probing of the tester. Although most of the intellectual processes of testing are nearly identical to that of review or inspection, the word testing is also used to connote the dynamic analysis of the product - putting the product through its paces. Sometimes one therefore refers to reviews, walkthroughs or inspections as "static testing". Whereas actually running the program with a given set of test cases in a given development stage is often referred to as "dynamic testing", to emphasize the fact that formal review processes form part of the overall testing scope.

A description, in any suitable form, of requirements.

Specification Testing
An approach to testing wherein the testing is restricted to verifying the system/software meets an agreed specification.

Specified Input
An input for which the specification predicts an outcome.

Stabilization Period
A period of time after the system is promoted to production, during which the project team may need to support post-deployment patches. The amount of time will vary depending on each project, but it may span a few days to a few weeks. It is wise to ensure that testing personnel are available to support these patches before moving on to the next testing project. This period of time is sometimes called Code Freeze.

State Transition
A transition between two allowable states of a system or component.

State Transition Testing
A test case design technique in which test cases are designed to execute state transitions.

An entity in a programming language which is typically the smallest indivisible unit of execution.

Statement Coverage
The percentage of executable statements in a component that have been exercised by a test case suite.

Statement Testing
A test case design technique for a component in which test cases are designed to execute statements. Statement testing is a structural or white box technique, because it is conducted with reference to the code. Statement testing comes under dynamic analysis.

In an ideal world every statement of every component would be fully tested. However, in the real world this hardly ever happens. In statement testing every possible statement is tested.

Static Analysis
Analysis of a program carried out without executing the program.

Static Analyzer
A tool that carries out static analysis.

Static Code Analysis
The analysis of computer software that is performed without actually executing programs built from that software. In most cases the analysis is performed on some version of the source code and in the other cases some form of the object code. The term is usually applied to the analysis performed by an automated tool, with human analysis being called program understanding or program comprehension's.

Static Testing
This form of testing involves any activity that looks for defects by inspecting, reviewing, and analyzing any static component of the software development process. Multiple testing roles should participate in the static testing exercise in order to gather diverse points of view. Static components include code, design documents, models, requirements, test plans, test cases, etc. Investing time in static testing activities offers a high return on that investment rather than finding defects and managing change later in dynamic testing (i.e., Unit Testing, System Testing, etc.).

Statistical Testing
A test case design technique in which a model is used of the statistical distribution of the input to construct representative test cases.

Storage Testing
Testing that verifies the program under test stores data files in the correct directories and that it reserves sufficient space to prevent unexpected termination resulting from lack of space. This is external storage as opposed to internal storage.

Stress Testing
Stress testing determines the stability of an application to perform under low and/or excessive loads to ensure that critical information and services are available when and how the end user expects them. It identifies conditions under which the system fails to continue functioning properly, including the peak workload the System Under Test can handle. It involves testing beyond normal operational capacity, often to a breaking point, in order to observe the results. In addition, Stress Testing is used to find defects that result from competition for shared resources.

Structural Coverage
Coverage measures based on the internal structure of the component.

Structural Test Case Design
Test case selection that is based on an analysis of the internal structure of the component.

Structural Testing
See Structural Test Case Design.

Structured Basis Testing
A test case design technique in which test cases are derived from the code logic to achieve % branch coverage.

Structured Walkthrough
See Walkthrough.

A skeletal or special-purpose implementation of a software module, used to develop or test a component that calls or is otherwise.

An attribute which becomes a temporary intermediate goal for the inference engine. Subgoal values need to be determined because they are used in the premise of rules that can determine higher level goals.

A sequence of executable statements within a component.

The capability of the software product to provide an appropriate set of functions for specified tasks and user objectives.

Suspension Criteria
The criteria used to (temporarily) stop all or a portion of the testing activities on the test items.

Symbolic Execution
A static analysis technique used to analyze if and when errors in the code may occur. It can be used to predict what code statements do to specified inputs and outputs. It is also important for considering path traversal. It struggles when dealing with statements which are not purely mathematical.

Symbolic Processing
Use of symbols, rather than numbers, combined with rules-of-thumb (or heuristics), in order to process information and solve problems.

Syntax Testing
A test case design technique for a component
or system in which test case design is based upon the syntax of the input.

The application software and hardware related to the testing effort. This includes application code, database software and hardware, operating system software, server hardware and software, network equipment and connectivity, load balancing software and hardware, and third party software included in processing of the simulated transactions. It does not include external systems that are deemed in scope of the testing effort.

System Integration Testing
System Integration Testing is the process of testing the system as a whole with other applications, systems, and interfaces. It ensures that the various pieces interact properly and function together. System Integration Testing assumes that system testing of the allocated features and functions has previously occurred and that those features and functions operate correctly. If certain features or functions are inoperable, then there must be a practical and documented workaround. This type of testing is sometimes called Integration Testing or End-To-End testing.

Testing a collection of systems and ensuring that they communicate with each other is a part of this type of testing. As this is a communications test, the verifications should focus on integration points. Verifications could include:
  • Verifying successful transfer and processing of information through a set of systems
  • Verifying Error Condition behaviors when part of the system chain is broken
  • Determine the system behavior when immediate systems are no longer connected
  • Determine the system behavior when secondary systems are no longer connected

System Testing
System Testing is a suite of testing that consists of, but is not limited to, Configuration Testing, Security Testing, Data and Database Integrity Testing, User Interface Testing, Failover Testing, Functional Testing, Installation Testing, Recovery Testing, and Stress Testing. System Testing assumes that testing of the allocated features and functions has occurred previously and that those features and functions operate correctly. If certain features or functions are inoperable, then there must be a practical and documented workaround. System testing is aimed at proving that the application or system meets the stated requirements and objectives.

System Under Test (SUT)
See System.

| Contact us for more info