7. Test Organization
7.1. General considerations
This section will discuss an approach to organizing an AUnit test harness, considering some possibilities offered by Ada language features.
The general idea behind this approach to test organization is that making the test case a child of the unit under test gives some useful facilities. The test case gains visibility to the private part of the unit under test. This offers a more ‘white box’ approach to examining the state of the unit under test than would, for instance, accessor functions defined in a separate fixture that is a child of the unit under test. Making the test case a child of the unit under test also provides a way to make the test case share certain characteristics of the unit under test. For instance, if the unit under test is generic, then any child package (here the test case) must be also generic: any instantiation of the parent package will require an instantiation of the test case in order to accomplish its aims.
Another useful concept is matching the test case type to that of the unit under test, for example:
When testing a generic package, the test package should also be generic.
When testing a tagged type, then test routines should be dispatching, and the test case type for a derived tagged type should be a derivation of the test case type for the parent.
Maintaining such similarity of properties between the test case and unit under test can facilitate the testing of units derived in various ways.
The following sections will concentrate on applying these concepts to the testing of tagged type hierarchies and to the testing of generic units.
A full example of this kind of test organization is available in the AUnit
installation directory:
<AUnit-root>/share/examples/aunit/calculator
, or
from the AUnit source distribution
aunit-<version>-src/examples/calculator
.
7.2. OOP considerations
When testing a hierarchy of tagged types, one will often want to run tests for parent types against their derivations without rewriting those tests.
We will illustrate some of the possible solutions available in AUnit, using the following simple example that we want to test:
First we consider a Root
package defining the Parent
tagged type, with two procedures P1
and P2
.
package Root is
type Parent is tagged private;
procedure P1 (P : in out Parent);
procedure P2 (P : in out Parent);
private
type Parent is tagged record
Some_Value : Some_Type;
end record;
end Root;
We will also consider a derivation from type Parent
:
with Root;
package Branch is
type Child is new Root.Parent with private;
procedure P2 (C : in out Child);
procedure P3 (C : in out Child);
private
type Child is new Root.Parent with null record;
end Branch;
Note that Child
retains the parent implementation of P1
,
overrides P2
and adds P3
. Its test will override
Test_P2
when we override P2
(not necessary, but certainly
possible).
7.2.1. Using AUnit.Test_Fixtures
Using type Test_Fixture
, we first test Parent
using the
following test case:
with AUnit; use AUnit;
with AUnit.Test_Fixtures; use AUnit.Test_Fixtures;
-- We make this package a child package of Parent so that it can have
-- visibility to its private part
package Root.Tests is
type Parent_Access is access all Root.Parent'Class;
-- Reference an object of type Parent'Class in the test object, so
-- that test procedures can have access to it.
type Parent_Test is new Test_Fixture with record
Fixture : Parent_Access;
end record;
-- This will initialize P.
procedure Set_Up (P : in out Parent_Test);
-- Test routines. If derived types are declared in child packages,
-- these can be in the private part.
procedure Test_P1 (P : in out Parent_Test);
procedure Test_P2 (P : in out Parent_Test);
end Root.Tests;
package body Root.Tests is
Fixture : aliased Parent;
-- We set Fixture in Parent_Test to an object of type Parent.
procedure Set_Up (P : in out Parent_Test) is
begin
P.Fixture := Parent_Access (Fixture'Access);
end Set_Up;
-- Test routines: References to the Parent object are made via
-- P.Fixture.all, and are thus dispatching.
procedure Test_P1 (P : in out Parent_Test) is ...;
procedure Test_P2 (P : in out Parent_Test) is ...;
end Root.Tests;
The associated test suite will be:
with AUnit.Test_Caller;
with Root.Tests;
package body Root_Suite is
package Caller is new AUnit.Test_Caller with (Root.Tests.Parent_Test);
function Suite return AUnit.Test_Suites.Access_Test_Suite is
Ret : Access_Test_Suite := AUnit.Test_Suites.New_Suite;
begin
AUnit.Test_Suites.Add_Test
(Ret, Caller.Create ("Test Parent : P1", Root.Tests.Test_P1'Access));
AUnit.Test_Suites.Add_Test
(Ret, Caller.Create ("Test Parent : P2", Root.Tests.Test_P2'Access));
return Ret;
end Suite;
end Root_Suite;
Now we define the test suite for the Child
type. To do this,
we inherit a test fixture from Parent_Test
,
overriding the Set_Up
procedure to initialize Fixture
with
a Child
object. We also override Test_P2
to adapt it
to the new implementation. We define a new Test_P3
to test
P3
. And we inherit Test_P1
, since P1
is unchanged.
with Root.Tests; use Root.Tests;
with AUnit; use AUnit;
with AUnit.Test_Fixtures; use AUnit.Test_Fixtures;
package Branch.Tests is
type Child_Test is new Parent_Test with null record;
procedure Set_Up (C : in out Child_Test);
-- Test routines:
-- Test_P2 is overridden
procedure Test_P2 (C : in out Child_Test);
-- Test_P3 is new
procedure Test_P3 (C : in out Child_Test);
end Branch.Tests;
package body Branch.Tests is
use Assertions;
Fixture : Child;
-- This could also be a field of Child_Test
procedure Set_Up (C : in out Child_Test) is
begin
-- The Fixture for this test will now be a Child
C.Fixture := Parent_Access (Fixture'Access);
end Set_Up;
-- Test routines:
procedure Test_P2 (C : in out Child_Test) is ...;
procedure Test_P3 (C : in out Child_Test) is ...;
end Branch.Tests;
The suite for Branch.Tests will now be:
with AUnit.Test_Caller;
with Branch.Tests;
package body Branch_Suite is
package Caller is new AUnit.Test_Caller with (Branch.Tests.Parent_Test);
-- In this suite, we use Ada 2005 distinguished receiver notation to
-- simplify the code.
function Suite return Access_Test_Suite is
Ret : Access_Test_Suite := AUnit.Test_Suites.New_Suite;
begin
-- We use the inherited Test_P1. Note that it is
-- Branch.Tests.Set_Up that will be called, and so Test_P1 will be run
-- against an object of type Child
Ret.Add_Test
(Caller.Create ("Test Child : P1", Branch.Tests.Test_P1'Access));
-- We use the overridden Test_P2
Ret.Add_Test
(Caller.Create ("Test Child : P2", Branch.Tests.Test_P2'Access));
-- We use the new Test_P2
Ret.Add_Test
(Caller.Create ("Test Child : P3", Branch.Tests.Test_P3'Access));
return Ret;
end Suite;
end Branch_Suite;
7.2.2. Using AUnit.Test_Cases
Using an AUnit.Test_Cases.Test_Case
derived type, we obtain the
following code for testing Parent
:
with AUnit; use AUnit;
with AUnit.Test_Cases;
package Root.Tests is
type Parent_Access is access all Root.Parent'Class;
type Parent_Test is new AUnit.Test_Cases.Test_Case with record
Fixture : Parent_Access;
end record;
function Name (P : Parent_Test) return Message_String;
procedure Register_Tests (P : in out Parent_Test);
procedure Set_Up_Case (P : in out Parent_Test);
-- Test routines. If derived types are declared in child packages,
-- these can be in the private part.
procedure Test_P1 (P : in out Parent_Test);
procedure Test_P2 (P : in out Parent_Test);
end Root.Tests;
The body of the test case will follow the usual pattern, declaring one or
more objects of type Parent
, and executing statements in the
test routines against them. However, in order to support dispatching to
overriding routines of derived test cases, we need to introduce class-wide
wrapper routines for each primitive test routine of the parent type that
we anticipate may be overridden. Instead of registering the parent’s
overridable primitive operations directly using Register_Routine
,
we register the wrapper using Register_Wrapper
. This latter routine
is exported by instantiating
AUnit.Test_Cases.Specific_Test_Case_Registration
with the actual
parameter being the parent test case type.
with AUnit.Assertions; use AUnit.Assertions
package body Root.Tests is
-- Declare class-wide wrapper routines for any test routines that will be
-- overridden:
procedure Test_P1_Wrapper (P : in out Parent_Test'Class);
procedure Test_P2_Wrapper (P : in out Parent_Test'Class);
function Name (P : Parent_Test) return Message_String is ...;
-- Set the fixture in P
Fixture : aliased Parent;
procedure Set_Up_Case (P : in out Parent_Test) is
begin
P.Fixture := Parent_Access (Fixture'Access);
end Set_Up_Case;
-- Register Wrappers:
procedure Register_Tests (P : in out Parent_Test) is
package Register_Specific is
new Test_Cases.Specific_Test_Case_Registration (Parent_Test);
use Register_Specific;
begin
Register_Wrapper (P, Test_P1_Wrapper'Access, "Test P1");
Register_Wrapper (P, Test_P2_Wrapper'Access, "Test P2");
end Register_Tests;
-- Test routines:
procedure Test_P1 (P : in out Parent_Test) is ...;
procedure Test_P2 (C : in out Parent_Test) is ...;
-- Wrapper routines. These dispatch to the corresponding primitive
-- test routines of the specific types.
procedure Test_P1_Wrapper (P : in out Parent_Test'Class) is
begin
Test_P1 (P);
end Test_P1_Wrapper;
procedure Test_P2_Wrapper (P : in out Parent_Test'Class) is
begin
Test_P2 (P);
end Test_P2_Wrapper;
end Root.Tests;
The code for testing the Child type will now be:
with Parent_Tests; use Parent_Tests;
with AUnit; use AUnit;
package Branch.Tests is
type Child_Test is new Parent_Test with private;
function Name (C : Child_Test) return Message_String;
procedure Register_Tests (C : in out Child_Test);
-- Override Set_Up_Case so that the fixture changes.
procedure Set_Up_Case (C : in out Child_Test);
-- Test routines:
procedure Test_P2 (C : in out Child_Test);
procedure Test_P3 (C : in out Child_Test);
private
type Child_Test is new Parent_Test with null record;
end Branch.Tests;
with AUnit.Assertions; use AUnit.Assertions;
package body Branch.Tests is
-- Declare wrapper for Test_P3:
procedure Test_P3_Wrapper (C : in out Child_Test'Class);
function Name (C : Child_Test) return Test_String is ...;
procedure Register_Tests (C : in out Child_Test) is
package Register_Specific is
new Test_Cases.Specific_Test_Case_Registration (Child_Test);
use Register_Specific;
begin
-- Register parent tests for P1 and P2:
Parent_Tests.Register_Tests (Parent_Test (C));
-- Repeat for each new test routine (Test_P3 in this case):
Register_Wrapper (C, Test_P3_Wrapper'Access, "Test P3");
end Register_Tests;
-- Set the fixture in P
Fixture : aliased Child;
procedure Set_Up_Case (C : in out Child_Test) is
begin
C.Fixture := Parent_Access (Fixture'Access);
end Set_Up_Case;
-- Test routines:
procedure Test_P2 (C : in out Child_Test) is ...;
procedure Test_P3 (C : in out Child_Test) is ...;
-- Wrapper for new routine:
procedure Test_P3_Wrapper (C : in out Child_Test'Class) is
begin
Test_P3 (C);
end Test_P3_Wrapper;
end Branch.Tests;
Note that inherited and overridden tests do not need to be explicitly
re-registered in derived test cases - one just calls the parent version of
Register_Tests
. If the application tagged type hierarchy is organized
into parent and child units, one could also organize the test cases into a
hierarchy that reflects that of the units under test.
7.3. Testing generic units
When testing generic units, one would like to apply the same generic tests to all instantiations in an application. A simple approach is to make the test case a child package of the unit under test (which then must also be generic).
For instance, suppose the generic unit under test is a package (it could be a subprogram, and the same principle would apply):
generic
-- Formal parameter list
package Template is
-- Declarations
end Template;
The corresponding test case would be:
with AUnit; use AUnit;
with AUnit.Test_Fixtures;
generic
package Template.Gen_Tests is
type Template_Test is new AUnit.Test_Fixtures.Test_Fixture with ...;
-- Declare test routines
end Template.Gen_Tests;
The body will follow the usual patterns with the fixture based on the
parent package Template
. Note that due to an Ada AI, accesses to
test routines, along with the test routine specifications, must be defined
in the package specification rather than in its body.
Instances of Template
will automatically define the Tests
child
package that can be directly instantiated as follows:
with Template.Gen_Test;
with Instance_Of_Template;
package Instance_Of_Template.Tests is new Instance_Of_Template.Gen_Test;
The instantiated test case objects are added to a suite in the usual manner.