Black box systems

System
Black box, Oracle machine
Methods and techniques
Black-box testing, Blackboxing
Related techniques
Feed forward, Obfuscation, Pattern recognition, White box, White-box testing, Gray-box testing, System identification
Fundamentals
A priori information, Control systems, Open systems, Operations research, Thermodynamic systems
v t e

Gray-box testing (International English spelling: grey-box testing) is a combination of white-box testing and black-box testing. The aim of this testing is to search for the defects, if any, due to improper structure or improper usage of applications.^[1]^[2]

Overview

A black-box tester is unaware of the internal structure of the application to be tested, while a white-box tester has access to the internal structure of the application. A gray-box tester partially knows the internal structure, which includes access to the documentation of internal data structures as well as the algorithms used.^[3]

Gray-box testers require both high-level and detailed documents describing the application, which they collect in order to define test cases.^[4]

Need for gray-box testing

Gray-box testing is beneficial because it takes the straightforward technique of black-box testing and combines it with the code-targeted systems in white-box testing.

Gray-box testing is based on requirement test case generation because it presents all the conditions before the program is tested by using the assertion method. A requirement specification language is used to make it easy to understand the requirements and verify its correctness.^[5]

Gray-box testing assumptions for object-oriented software

Object-oriented software consists primarily of objects; where objects are single indivisible units having executable code and/or data. Some assumptions are stated below which are needed for the application of use gray-box testing.

Activation of Methods^[6]
State Reporting in Class Under Test (CUT).
Report Testing is inherent in Class Under Test.^[5]

Examples

Architectural model
Unified Modeling Language - UML Design Model
Finite-state machine - State Model.^[7]^[8]

Techniques

Cem Kaner defines "gray-box testing as involving inputs and outputs, but test design is educated by information about the code or the program operation of a kind that would normally be out of view of the tester".^[9] Gray-box testing techniques are:

Matrix Testing: states the status report of the project.
Regression testing: it implies rerunning of the test cases if new changes are made.
Pattern Testing: verify the good application for its design or architecture and patterns.
Orthogonal array testing: used as subset of all possible combination.^[10]

Effects

Positive Effects

Offers combined benefits: As Gray-box testing is combination of white-box and black-box testing, it serves advantages from both the testings.
Non Intrusive: It is based on functional specification, architectural view whereas not on source code or binaries which makes it invasive too.
Intelligent Test Authoring: Gray-box tester handles intelligent test scenario, for example, data type handling, communication protocol, exception handling.
Unbiased Testing: In spite of all above advantages and functionalities, Gray-box testing maintains boundary for testing between tester and developer.^[11]

Negative Effects

Partial code coverage: In gray-box testing, source code or binaries are missing because of limited access to internal or structure of the applications which results in limited access for code path traversal.
Defect Identification: In distributed applications, it is difficult to associate defect identification. Still, Gray-box testing is a boon to find how appropriate these systems throw exceptions and how fine are these exceptions handled in distributed systems having web services environment.^[11]^[12]

Applications

Gray-box testing is well suited for web applications. Web applications have distributed network or systems; due to absence of source code or binaries it is not possible to use white-box testing. Black-box testing is also not used due to just contract between customer and developer, so it is more efficient to use gray-box testing as significant information is available in Web Services Description Language (WSDL).^[13]
Gray-box testing is suited for functional or business domain testing. Functional testing is done basically a test of user interactions with may be external systems. Gray-box testing is well-suited for functional testing due to its characteristics; it also helps to confirm that software meets the requirements defined for the software.^[14]^[15]^[16]^[17]

Future scope

The distributed nature of Web services allows gray-box testing to detect defects within a service-oriented architecture (SOA). As we know, white-box testing is not suitable for Web services as it deals directly with the internal structures. White-box testing can be used for state art methods; for example, message mutation which generates the automatic tests for large arrays to help exception handling states, flow without source code or binaries. Such a strategy is useful to push gray-box testing nearer to the outcomes of white-box testing.

References

^ "Microsoft Research – Emerging Technology, Computer, and Software Research" (PDF).
^ "Archived copy" (PDF). Archived from the original (PDF) on 29 March 2012. Retrieved 17 October 2011.((cite web)): CS1 maint: archived copy as title (link)
^ "Gray Box Testing". Software Testing Fundamentals. 4 November 2011. Retrieved 19 January 2012.
^ "Example of grey box testing with definition". Geekinterview.com. Retrieved 19 January 2012.
^ ^a ^b Jake Rogers (8 August 2016). "Common Questions Regarding Grey-Box Testing". cgsec.co.uk. Retrieved 8 August 2016.^{[permanent dead link]}
^ "Object-Oriented Extensions to Pascal". Pascal-central.com. Archived from the original on 5 June 2021. Retrieved 19 January 2012.
^ Patton, Ron (26 July 2005). Software Testing. Sams. p. 2. ISBN 978-0-672-32798-8.
^ "Archived copy" (PDF). Archived from the original (PDF) on 3 April 2012. Retrieved 17 October 2011.((cite web)): CS1 maint: archived copy as title (link)
^ Nguyen, Hung Q (2001). Testing Applications on the Web: Test Planning for Internet-Based Systems. John Wiley & Sons. ISBN 9780471437642.
^ "Explore the World of Gray Box Testing". Extremesoftwaretesting.com. Retrieved 19 January 2012.
^ ^a ^b "SOA Testing Tools for Black, White and Gray Box SOA Testing Techniques". Crosschecknet.com. Archived from the original on 1 October 2018. Retrieved 19 January 2012.
^ "E33 Gray Box Testing.PDF" (PDF).
^ Ramdeo, Anand (5 May 2011). "Gray Box Testing - Software". Testing Geek. Retrieved 19 January 2012.
^ Bach, James (31 December 2001). Lessons Learned in Software Testing. Wiley Computer Publishing. ISBN 978-0-471-08112-8.
^ Falk, Jack (12 April 1999). Testing Computer Software, 2nd Edition. Wiley Computer Publishing. ISBN 978-0-471-35846-6.
^ http://legacy.cleanscape.net/docs_lib/paper_graybox.pdf^{[bare URL PDF]}
^ Li, Z. J.; Tan, H. F.; Liu, H. H.; Zhu, J.; Mitsumori, N. M. (6 April 2010). "Business-process-driven gray-box SOA testing". IBM Systems Journal. 47 (3): 457–472. doi:10.1147/sj.473.0457.

v t e Software testing
The "box" approach	Black-box testing All-pairs testing Exploratory testing Fuzz testing Model-based testing Scenario testing Grey-box testing White-box testing API testing Mutation testing Static testing
Testing levels	Acceptance testing Integration testing System testing Unit testing
Testing types, techniques, and tactics	A/B testing Benchmark Compatibility testing Concolic testing Concurrent testing Conformance testing Continuous testing Destructive testing Development testing Differential testing Dynamic program analysis Installation testing Negative testing Random testing Regression testing Security testing Smoke testing (software) Software performance testing Stress testing Symbolic execution Test automation Usability testing
See also	Graphical user interface testing Manual testing Orthogonal array testing Pair testing Soak testing Software reliability testing Stress testing Web testing