Non-functional requirements: verifying and acceptance tests¶

statement¶

Consider the following non-functional requirements:

This is a very vague statement. It can not be verified.
‘user-friendly’ or ‘easy-to-use’ is not very specific, and it can mean anything from a simple, clean UI to a UI full of images and complex animations.

This requirement is very specific. It can be verified.
Since this term can be measured, a set of acceptance tests can be written. Here are some of them:
- The tester can manually count the number of mouse clicks to navigate the system’s user interface window, and if they find a journey with more than 10 clicks, the test fails.
- Using an automated testing tool, the tester can randomly follow link paths and count the number of mouse clicks to navigate the system’s user interface window, and if they find a journey with more than 10 clicks, the test fails.
- An automated crawler will crawl every possible link in the system’s user interface and count the number of mouse clicks to navigate the system’s user interface window. The test fails if they find a journey with more than 10 clicks (calculating the average number of clicks used in navigation may be a possible and good metric here).

This is a very vague statement. It can not be verified.
The acceptance tests for such a requirement will significantly change as we answer the questions below:
- How much training is considered ‘minimum’?
- How much flexibility does the company have regarding the training program?
- How much domain (and technology) do the target users have?
- What parts of the system have more flexibility regarding the training program?
There are still some variables regarding this requirement, even after answering the questions above; hence, following the current statement, it is very hard to generate any meaningful acceptance tests.

This statement is in between, not very clear, nor a vague one.
Although it seems specific since it contains a number in the statement (1 second), it is not. The broadband connection is not a fixed-term; it can be anything from a dial-up to a high-speed internet connection.
The connection speed significantly impacts the latency and, therefore, the test results.
We can partially verify this statement by writing acceptance tests for the most common connection speeds, ignoring the very low (or high) connection speeds.
It is important to agree on this requirement before commencing to avoid surprises later.
some acceptance tests we might consider:
- choose the most common connection speeds (e.g., dial-up, DSL, cable, etc.), start measuring the time it takes to load pages (an automated tool might choose a sample of pages) then validating the results to see if the condition (less than 1 second) is met.
- Start with the highest connection speed, then go down to the lowest one, and measure the time it takes to load pages (on a randomly selected set of pages and links). Determine the highest speed where the test starts to fail and see if this is approved by the client policies.

This statement is also in between, not very clear nor vague.
The definition of ‘loss of important data’ is not very clear.
The important data may include anything, from the user’s personal data to the system’s to the current working session’s data.
The definition of system failure is not very clear.
Once the important data and system failure are defined, we can start developing acceptance tests for this requirement.
acceptance tests may include:
- Deliberately cause the system failure, then examine the data after recovery and see if this matches the user expectations.