Types and levels of testing
Testing levels, black, white and grey box, functional and non-functional testing, and change-related testing.
There are dozens of named testing types, and the names overlap. The easiest way to keep them straight is to sort them along a few independent axes: at what level we test, how much we know about the internals, what we test, and why we test right now.
Levels: what size of thing we test
| Level | Scope | Typically done by |
|---|---|---|
| Unit (component) | A function, class or component in isolation | Developers |
| Integration | How modules or services work together, APIs, database | Developers and QA |
| System | The complete product against requirements | QA |
| Acceptance | Whether the product is ready for users and the business | Business, users, QA |
Methods: how much we know about the inside
| Method | Knowledge of code | Typical use |
|---|---|---|
| Black box | None; we test through inputs and outputs | Functional and acceptance testing |
| White box | Full; tests are based on code structure | Unit tests, coverage of branches and paths |
| Grey box | Partial: architecture, database, API | Integration and API testing, data checks |
Functional and non-functional
Functional testing checks what the system does: business rules, calculations, workflows. Non-functional testing checks how well it does it.
- Performance: load (expected traffic), stress (beyond limits), endurance (hours of steady load), spike (sudden jumps)
- Security: access control, input validation, data exposure, dependency vulnerabilities
- Usability and accessibility: can everyone, including people using screen readers or keyboards, complete tasks?
- Compatibility: browsers, devices, operating systems, screen sizes
- Localisation: translations, date and number formats, text that becomes longer in German or right-to-left languages
- Reliability and recovery: what happens after a crash, a lost connection or a failed deploy
Change-related testing
| Type | When | Goal |
|---|---|---|
| Smoke | Right after a new build | Is the build stable enough to test at all? |
| Sanity | After a small fix or change | Does the changed area work, at a glance? |
| Re-testing (confirmation) | After a bug fix | Is this exact bug really fixed? |
| Regression | Before each release | Did the change break anything that worked before? |
Approaches: scripted, exploratory and risk-based
Scripted testing follows prepared test cases and is good for repeatability and audits. Exploratory testing combines learning, test design and execution at the same time, guided by a mission and a time box; it is excellent at finding unexpected problems. Risk-based testing decides where to spend effort: areas with high business impact and high likelihood of failure get the most attention.
Other names you'll meet
- Alpha and beta testing: internal testing before release, then testing by a limited group of real users
- A/B testing: comparing two versions with real users to see which performs better (a product experiment more than a bug hunt)
- Monkey testing: random inputs to find crashes
- Installation and update testing: fresh installs, upgrades, rollbacks
- Data and migration testing: correctness and completeness of moved or transformed data