5. TDD and system specifications¶

2.7.1 Overview of Software Testing¶

fault: also called “defect” or “bug,” is an erroneous hardware or software element of a system that can cause the system to fail
test-driven development (TDD), one of the practices of Extreme Programming (XP).
testing shows only the presence of faults, not their absence. Showing the absence of faults requires exhaustively trying all possible combinations of inputs
test approaches:
- brute force: try every possible combination of inputs.
- prove correctness by reasoning (theorem proving): hard to automate unless the problem is defined on mathematical or logical grounds.
testing is expensive, the goal is:
- find faults as quickly and cheaply as possible.
- write less of unsuccessful tests cases: cases that do not expose any faults.
- sometimes, other methods may be more useful than testing, like: code reviews, reasoning, static analysis.
- automate as much testing as possible to increase frequency and coverage of tests.
- test early and often.
tests organization:
1. unit tests:
  - works on individual units of code (a method in a class, a function).
  - finds the difference between the object design model and its implementation
2. integration tests: works on a composition of units (eg. a procedure that integrates a few functions together).
3. system tests: works on the whole system:
  - function test: verifies that requirements are met.
  - quality test: verifies that non-functional requirements are met.
  - acceptance test: customer verifies that requirements are met
  - installation test: test in end-user environment.
4. regression testing:
  - expose new errors (regressions) in an existing functionality after modifications.
  - new test is added for every discovered fault.
  - tests must run after every change.
  - protects against reversions that reintroduce faults.
Black box testing: testing a running program without looking to implementations, usually done by customers in acceptance tests
white box testing: aware of the implementations.

2.7.2 Test Coverage and Code Coverage¶

test coverage: the degree of which the code/specification is tested
code coverage: the degree of which the source code is tested. and includes criteria:
- equivalence testing
- boundary testing
- control-flow testing
- state-based testing

equivalence testing¶

equivalence testing is black box testing that divides the space of possible inputs into equivalence groups where the program acts similarly on each of these groups.
finding equivalence groups is hard, so finding test cases depends on personal experience.
Equivalence classes defined for an input parameter must satisfy the following criteria:
1. Coverage: Every possible input value belongs to an equivalence class.
2. Disjointedness: No input value belongs to more than one equivalence class.
3. Representation: If an operation is invoked with one element of an equivalence class as an input parameter and returns a particular result, then it must return the same result if any other element of the class is used as input.
For example, the square root function f(x) = sqrt(x) accepts all real numbers as an input; we can divide them into two groups: 1. negative numbers. 2. positive numbers (including zero). For the negative numbers, sqrt should throw an error. For positive numbers, it should find the square root.

Boundary testing¶

Boundary testing is a special case of equivalence testing, were after generating the equivalence groups we are interested in the boundary values of these groups.
For example, the function f(x) = x * x which accepts all real numbers as input. To test this function, we can choose values from the left boundaries (most high negative numbers), e.g. a number approaches -Infinity, then some values from the right boundaries, e.g. a number approaches +Infinity; then some values in the middle, e.g. around zero.

Control Flow Testing¶

Statement coverage: the test cases must ensure that all branches (e.g. if statements, conditions …) is executed (there is a test case suitable for this branch)
Edge coverage: every branch of the program is being traversed at least once, eg. for nested branches, you need at least one test case that traverse every possible case of nested branches top to bottom
condition coverage: for boolean expressions, your test cases must include both cases, true and false.
path coverage: calculates the number of unique paths in the program that must be traversed, to ensure correctness that all of these paths are covered by tests.

State-based testing¶

State-based testing : testing the behavior of an object by supplying a fixed state to it.
steps:
- test driver initializes the target object (unit) with the test case state
- the test driver exercises the target unit by invoking methods on it.
- the test driver compares the current unit state (after exercises) with the expected state.
- if the unit state matches the expected state, the unit is considered correct, regardless of how it got to that state.

2.7.3 Practical Aspects of Unit Testing¶

the target unit must be isolated from the rest of the system, this is not possible practically so we use test driver to prepare all parts that this unit may require before running the tests.
test driver: simulates the part of the system that invokes operations on the target object.
test stub: minimal implementation that simulates components that are called by the tested object, e.g. hardcode a return value that must be returned after invoking a specific function.
fixture: the thing to be tested
Each testing method follows this cycle:
1. Create the thing to be tested (fixture), the test driver, and the test stub(s)
2. Have the test driver invoke an operation on the fixture
3. Evaluate that the results are as expected
test case functions can be named as: ${testedMethodName}_${startingState}_${expectedResult}
isolation is an important part of unit testing, otherwise, your unit tests will look like integration tests.

2.7.4 Integration and Security Testing¶

in traditional development cycles, integration tests happens late in the project, and components put together in a horizontal fashion. integration tests varies depending on how components are integrated.
in agile method, integration tests happen on every change form the beginning of the project, and components put together in a vertical fashion to implement end-to-end functionality. each vertical slice corresponds to user story and user stories are implemented and tested in parallel.

Horizontal Integration Testing Strategies¶

big bang integration: tries linking all components at once and testing the combination.
bottom-up integration testing:
- starts by combining the units at the lowest level of the hierarchy (leafs), which usually have not dependencies to other units.
- start by doing unit tests on the bottmmost level of the hierarchy.
- then every time the integration test goes up one level.
- no need to implement test stubs since the previous level has been tested already and we have values that can supply the current level.
- we still need to implement test drivers but they are relatively simple.
- in real world, systems do not form a hierarchy, so levels may not be obvious.
Top-down integration testing:
- starts by nit testing the units on the highest level of the hierarchy that no other units depend on.
- we dont need test drivers, but we need test stubs.

horizontal integration testing

sandwich integration testing:
- combines top-down and bottom-up by starting from both ends and incrementally using components of the middle level in both directions
- the middle level is called the target level, its components works as driver for bottom-up tests on components higher than the target level and also a driver for top-down tests on components lower than the target level.
- The advantages of sandwich testing include no need to write stubs or drivers and the ability of early testing of the user interface and thus early involvement of end users.
- A drawback is that sandwich testing does not thoroughly test the units of the target (middle) level before integration.

Vertical Integration Testing Strategies¶

used in agile, user stories can be developed in parallel.
each user story has a feedback loop where developers use unit tests in the inner loop and customer runs acceptance tests in the outer loop.
each story starts by the customer defining the acceptance tests, then based on those acceptance tests the developer can generate the unit tests

Security Testing¶

test for “negatives,” such as abuse cases, to determine how the system behaves under attack.
Security tests are often driven by known attack patterns.

2.7.6 Refactoring: Improving the Design of Existing Code¶

refactoring: improving the current code design while preserving its behavior.
process of refactoring includes:
- remove duplicate code
- simplify complex code
- clarify unclear code

Using Polymorphism Instead of Conditional Logic¶

Polymorphism allows avoiding explicit conditionals when you have objects whose behavior varies depending on their types.

summary

3. Modeling and system specification¶

The term “system specification” is used both for the process of deriving the properties of the software system as well as for the document that describes those properties

3.1.1 World Phenomena and Their Abstractions¶

individuals in phenomena:
- event: individual happening at specific point of time, has no internal structure and has no time to happen.
- entity: individual with distinct existence, has time to live and can change its properties, may initiate events.
- value: individual that exists outside of time and space,
relations in phenomena (relations between different individuals):
- state: relationship between an entity and a value.
- truth: fixed relationship amon individuals that can not change over time. e.g. an event has happened at specific time in the past, this fact can not be changed with time.
- role: relation between event and individual that participate in the event.

3.1.2 States and State Variables¶

state: is a relation on a set of objects, which simply selects a subset of the set
System state is defined as a tuple of state variables containing any valid combination of state relations.
microstate: The states that are directly observable at a given level of detail (coarse graining).
macrostate: A group of microstates.

3.1.3 Events, Signals, and Messages¶

events:
- indivisible happenings, a sharp boundary between two different states
- any happening (or performance, or action) that has an internal time structure must be regarded as two or more events.
- no two events occur simultaneously. All events happen sequentially, and between successive events there are intervals of time in which nothing happens (no events)
- events mark transitions between successive states.
message:
- a signal sent from one entity to another entity.
- message does not imply state change, it can happen without state change (unlike events).

3.1.4 Context Diagrams and Domains¶

context : the environment in which it will work.
context diagrams:
- not part of UML.
- the context of the problem that the developer sets out to solve.
domain:
- A domain is a part of the world that can be distinguished because it is conveniently considered as a whole, and can be considered—to some extent—separately from other parts of the world.
- in software, there are 2 domains: 1. application domain 2. machine domain
domain types:
- given domain: the domain that is given to the system and developers are not allowed to design (or change) it.
- lexical domain: is a physical representation of data—that is, of symbolic phenomena

3.1.5 Systems and System Descriptions¶

system:
- any domain is a system.
- a system is an organized or complex whole, assemblage of things or parts interacting in a coordinated way.
- all systems are affected by the environment in which they are run. both internal environment (under organization’s control) and external environment.
Dynamical systems theory describes properties of solutions to models that are prevalent across the sciences.

3.2 Notations for System Specification¶

proposition: declarative sentence (a sentence that declares a fact)
compound proposition: a sequence of propositions grouped together using logical operators.
conditional statement:
- obtained by combining two propositions p and q to a compound proposition “if p, then q.” It is also written as p => q and can be read as “p implies q.”
- in condition p=>q, p is the antecedent (premise, hypothesis) and q is the consequent (conclusion).

3.3 Problem Frames¶

Problem frames decompose the original complex problem into simple, known subproblems. Each frame captures a problem class stylized enough to be solved by a standard method and simple enough to present clearly separated concerns.