Producing source traces with gnatcov instrument¶
The production of source traces is performed by an instrumented version of the original program running in its regular execution environment. A gnatcov instrument command produces alternative sources for units of interest, with additional data structures and code statements solely aimed at tracking coverage related information. Coverage data is output in accordance with a user selectable policy, for example upon exit of the program, for which the program main unit gets instrumented as well.
To gather coverage data while the program is running and to output this data in the form of a source trace file eventually, the instrumented code relies on common data types and subprograms provided by a coverage runtime library. The first thing to do for a given project is then to build and install the coverage runtime so it becomes available to the instrumented sources afterwards, as we will describe in the Setting up the coverage runtime library section of this chapter.
Once the coverage runtime is setup, the simplest possible use of the instrumentation is on entire programs at once, with their main unit(s) included. The process essentially goes like:
Instrument, both units-of-interest and main units;
Build the instrumented code;
Execute the program to produce a trace.
GNATcoverage also allows partial instrumentations to be combined together, which we will describe in the Composed instrumentation part.
The instrumenter is generally careful not to introduce constructs specific to a version of Ada more recent than the original one, only assuming that this is at least Ada 95. In order to support Pure or Preelaborable units, we currently assume that compilations are performed with a GNAT Pro toolchain. Parts of the effects reach into the coverage runtime, so this also concerns application programs which do not have so categorized units.
GNATcoverage supports two direct ways of outputing coverage data from the program:
Produce a trace file directly, or
Output a base64 encoded version of the data on standard output, using
Ada.Text_IO
. Coverage data is emitted between markers and a trace file can be produced offline from a captured output thanks to the gnatcov extract-base64-trace command.
The gnatcov instrument command allows instrumenting main units according to a specified mode, with a choice of possible insertion points. It also features a manual mode so users can implement a custom strategy, which only requires that the output format abides by one of the two variants we support.
Thread safety on accesses to coverage data structures is achieved by writing at most one status Boolean at a time in data structures voluntarily not packed.
Setting up the coverage runtime library¶
GNATcoverage comes with two variants of the coverage runtime, distributed
as a set of sources and two project files (one project for each variant)
located in the share/gnatcoverage/gnatcov_rts
subdirectory of your
GNATcoverage installation.
gnatcov_rts_full.gpr
is intended for programs which have access to a full Ada runtime, with support for both the base64 output and the direct creation of trace files;gnatcov_rts.gpr
is intended for programs which would operate in more restricted environments, with support for only the base64 output. The base runtime we provide relies onAda.Text_IO
to output the data. This can be tailored if needed.
To set up the coverage runtime for use by your project, first copy the
gnatcov_rts
subdirectory tree to a local spot of your choice where you will
perform the build. For example, using a Unix shell like syntax:
$ rm -rf gnatcov_rts-build # Remove a possible previous copy
$ cp -r <gnatcov-install>/share/gnatcoverage/gnatcov_rts/ gnatcov_rts-build
Then switch to the local directory to build the library, and install it to another location of your choice:
$ cd gnatcov_rts-build
$ gprbuild -P<gnatcov_rts_gpr> [-XLIBRARY_TYPE=<library-type>]
$ gprinstall -P<gnatcov_rts_gpr> --prefix=<gnatcov_rts-install>
Here, pick <gnatcov_rts_gpr>
as one of the two variants introduced
previously, depending on the capabilities of your execution environment and
the kind of coverage data output you will want to do.
Regardless of the variant, the library needs to be compiled with the same toolchain as the one which will be used to build the application code.
The full library can be built for different possible use modes afterwards,
typically, static, shared, or static-pic. The choice of a mode is controlled
by the LIBRARY_TYPE
scenario variable as hinted in the example commands
above. The default is to build a static library.
That’s essentially it; we will explain later in this chapter how the installed library is to be used.
Instrumenting programs¶
Instrumentation is performed upfront for an intended strictest coverage criterion on a given set of units of interest. The production of a coverage report afterwards might restrict the report to a subset of those units, or lower to a less strict criterion. Instrumenting programs is achieved with gnatcov instrument commands, which might involve two distinct kinds of operations:
Modify the code in units-of-interest so the program records, while it is running, facts of relevance to the coverage metrics to be assessed,
Modify the main unit(s) to output the so gathered coverage data to an externally readable channel, typically either a source trace file or some communication port.
gnatcov instrument command line¶
As for other commands, help on the command line interface is displayed
by gnatcov instrument --help
. The general sysopsis is as follows:
gnatcov instrument --level=<> <units-of-interest> [OPTIONS]
--level
states the strictest criterion that this instrumentation
will allow assessing afterwards and the <units-of-interest>
switches
specify the set of units for which such assessment will be possible. The
latter may only resort to project file facilities, as described in the
Using project files (-P, --projects, --units) section of this manual. Projects marked Externally_Built
in the closure are not instrumented or otherwise modified.
A few [OPTIONS]
allow controlling the instrumentation of main units,
if any are designated by the root project:
--dump-trigger
selects the execution point at which the output of coverage data should be injected in main units. This is
manual
by default, leaving to users the responsibility to emit the coverage data as they see fit. Other possible choices areatexit
,main-end
andravenscar-task-termination
.--dump-channel
selects the mechanism used to output coverage data at the selected triggering point, if any. The possible choices are
bin-file
, to create a source trace file, orbase64-stdout
to emit a base64 encoded version of the data throughAda.Text_IO
.bin-file
is the default.--externally-built-projects
instructs the instrumenter to look into projects marked as externally built when computing the list of units of interest (they are ignored by default), for the sole purpose of instrumenting mains.
In addtion, for trace files produced automatically from a bin-file
dump-channel, the --dump-filename-<>
family of switches provides control
over the name of trace files. See Controlling trace file names for more details on
the default behavior and possibilities to alter it.
Output strategies for main units¶
The choice of a dump-trigger/dump-channel pair for main units depends on the runtime environment available to your program.
For a native program with access to a full Ada runtime and the associated
coverage runtime, bin-file
is the recommended channel as it produces a
trace in the most direct manner and separates the trace data from the regular
output. atexit
is a natural triggering choice in this case, as it takes
care of outputing the data automatically at a point where we know the program
is not going to execute more, regardless of how or why the program exits.
The main-end
alternative simply inserts the calls at the end of the main
subprogram bodies, which may be bypassed if the program exits abruptly, or
miss data if the program has tasks not terminated when execution of the main
subprogram/thread reaches its end.
For more restricted environments where, say, there is limited file IO
available to the program, a base64-stdout
kind of output is needed in
association with the restricted coverage runtime.
If none of the available automatic triggering option works out well, full
control is offered by the manual
policy where the instrumenter doesn’t
actually add any code to main units for emitting the collected coverage
data. You will have to emit this data somehow to allow analysing coverage
afterwards, still, and can of course experiment with other possibilities just
to get examples of possible ways to proceed.
Controlling trace file names¶
When an instrumented program produces a trace file through a bin-file
dump-channel, the file is by default created in the current working directory
at the data output point (for example, at exit time for an atexit
dump-trigger), and named as <ename>-<istamp>-<pid>-<estamp>.srctrace
,
where:
<ename>
is the executable name,<istamp>
is the instrumentation time stamp, representing the time at which the instrumentation took place,<pid>
is the execution process identifier,<estamp>
is an execution time stamp, representing the time at which coverage data was written out to the file.
The <estamp>
and <pid>
components are intented to ensure that parallel
executions of the program from the same working directory write out to
different files. The <istamp>
component allows distinguishing traces
issued from different versions of the program. These three components are
expressed as hexadecimal integers to limit the growth of file name lengths.
This default behavior can be influenced in several manners. First:
The
--dump-filename-prefix
switch to gnatcov instrument requests replacing the<ename>
component by the switch argument;The
--dump-filename-simple
switch requests the removal of the variable components (stamps and pid), so only the<ename>
component remains or the replacement provided by--dump-filename-prefix
if that switch is also used.
The use of a specific location for the file, or of a specific file name can be
requested at run time by setting the GNATCOV_TRACE_FILE
variable in the
program’s environment.
If the variable value ends with a /
or \
character, this value is
interpreted as the name of a directory where the trace file is to be produced,
following the rules we have just described for the file base name. The
directory reference may be a full or a relative path, resolved at the trace
file creation point and expected to exist at that time.
If the variable value does not end with a /
or \
character, the
value is used directly as the name of the file to create. This name may hold a
path specification, full or relative, also resolved at the trace file creation
point and the directories involved are expected to exist at that time.
For specific needs of programs wishing to output to different places from
within the same environment, the variable name for a program can actually be
tailored by passing a --dump-filename-env-var
switch to gnatcov instrument,
providing the variable name to use.
Building instrumented components¶
Compared to a regular build, the intermediate instrumentation process requires two specific actions:
For the units which have been instrumented (as main units or declared of-interest to coverage instrumentation time), arrange to use the instrumented sources instead of the original ones; and
Provide the instrumented code with access to the coverage runtime support.
Since release 20, our GPRbuild builder incorporates features allowing a direct reuse of a project hierarchy without replication of the directory structure, not even modification of the project files.
For each project in the closure of-interest, the instrumentation generates the
alternative sources in the gnatcov-instr
subdirectory of the project’s
object directory. Giving priority to this subdir when it exists is achieved
by passing a --src-subdirs
switch to gprbuild, naming
that particular relative subirectory.
Then gprbuild now supports a --implicit-with
option which
requests processing every project as if it started with a with
statement
for a given project, which we can use to designate the coverage runtime
project file so all the compiled code gets access to the support packages.
The build of instrumented components then proceeds almost exactly as a regular
one, only adding --src-subdirs=gnatcov-instr
and
--implicit-with=
<gnatcov_rts_gpr>
to the build options,
where <gnatcov_rts_gpr>
would be the coverage runtime project file
setup beforehand for the project, as described previously in this
chapter. This project file could be refered to with a full path specification,
or with a simple basename if the GPR_PROJECT_PATH
environment variable is
updated to designate the directory where the coverage runtime has been
installed.
While the scheme relies on the use of GNAT project files, it does not absolutely require gprbuild to build the instrumented programs, even though we have augmented that builder with a few features to make that process very efficient and straightforward.
Extracting a trace from standard output¶
With the base64-stdout
channel, coverage data is emitted
with Ada.Text_IO
on the program’s standard output stream. The actual
base64 encoded data is framed by start/end-of-coverage-data markers and GNATcoverage
provides the gnatcov extract-base64-trace command to extract this data from a captured output and
create a trace file offline (outside of the program’s execution context). The
extraction command line simply is:
gnatcov extract-base64-trace <captured-output> <output-trace-file>
The captured output may be used directly, there is no need to first extract the trace data section.
Composed instrumentation¶
To prevent unnecessary re-instrumentation and re-build of components which don’t change, GNATcoverage allows partial instrumentations to be combined together. A common use case would be the testing of library components, where the library doesn’t change and its coverage needs to be assessed incrementally as new tests get developed.
In such situations, the process would become something like:
Setup or reuse a separate project file for the library, which normally wouldn’t have any main unit;
Instrument the library using this project as the root project;
Build the instrumented library;
Then for each new test:
Setup or reuse a separate project file for the test, designating the main unit if you wish to leverage the instrumenter’s ability to insert the coverage coverage data output code. Setup a dependency from this project on the library project, with an
Externally_Built
attribute set to"True"
;Instrument the testing code main unit alone;
Build a program combining the library (instrumented for coverage measurement) and the testing code (instrumented to output the gathered coverage data);
Execute the program to produce a trace.
The following section illustrates such a use case.
Example use cases¶
Whole program instrumented at once, cross configuration, base64 output¶
Here we will consider examining the coverage achieved by the execution of the
very basic sample program below, assuming the existence of a Sensors
source unit providing access to some sensor values.
with Sensors; use Sensors;
with Ada.Text_IO; use Ada.Text_IO;
procedure Monitor is
Sensor_Value : Integer;
begin
for Sensor_Index in Sensor_Index_Range loop
Sensor_Value := Sensors.Value (Sensor_Index);
Put ("Sensor(" & Sensor_Index'Img & ") = " & Sensor_Value'Img & " ");
if (Sensor_Value > 1000) then
Put_Line ("!Alarm!");
else
Put_Line ("!Ok!");
end if;
end loop;
end;
We will consider a cross target environment, say PowerPC-VxWorks, using Real
Time Processes hence an rtp
Ada runtime library. We will assume we
don’t have a filesystem at hand, so will rely on the base64 encoded output of
trace data to standard output.
Setting up the coverage runtime¶
We just “build” the runtime library project as we would build a regular program for our target configuration, specifying the target name and the intended base Ada runtime library.
For our intended target environment, this would be something like:
# Copy the sources into a fresh local place for the build:
cp -rp <gnatcoverage-install>/share/gnatcoverage/gnatcov_rts <gnatcov_rts-build-dir>
# Build and install the library to a place of our choice. Pick gnatcov_rts.gpr as
# we won't be emitting source trace files directly:
cd <gnatcov_rts-build-dir>
gprbuild -Pgnatcov_rts.gpr --target=powerpc-wrs-vxworks7r2 --RTS=rtp -f -p
rm -rf <gnatcov_rts-ppc-install-dir>
gprinstall -Pgnatcov_rts.gpr --target=powerpc-wrs-vxworks7r2 --RTS=rtp \
-p --prefix=<gnatcov_rts-ppc-install-dir>
# Allow references to the coverage runtime project from other project files:
export GPR_PROJECT_PATH=<gnatcov_rts-ppc-install-dir>
Instrument and build¶
We setup a monitor.gpr
project file for our program, where we
Provide the main unit name, so it can be instrumented automatically, and…
State the target configuration name and Ada runtime library so we won’t have to pass explicit
--target
and--RTS
on every command line involving project files afterwards.
For example:
project Monitor is
for Target use "powerpc-wrs-vxworks7r2";
for Runtime ("Ada") use "rtp";
for Object_Dir use "obj-" & Project'Runtime("Ada");
for Main use ("monitor.adb");
end Monitor;
We can now instrument with:
gnatcov instrument -Pmonitor.gpr --level=stmt+decision
--dump-trigger=main-end --dump-channel=base64-stdout
This is VxWorks where we don’t necessarily have an atexit
service. Our
program doesn’t have tasks so main-end
is a suitable alternative. The
stmt+decision instrumentation will let us assess either statement coverage
alone or statement and decision coverage afterwards.
Building the instrumented version of the program is then achieved with:
gprbuild -p -Pmonitor.gpr
--src-subdirs=gnatcov-instr --implicit-with=gnatcov_rts.gpr
Execute, extract a trace and report¶
The steps required to execute are very environment specific. Symbolically, we do something like:
run-cross obj-rtp/monitor.vxe > monitor.stdout
In our case, we have stubbed 4 sensors and obtain an output such as:
Sensor( 1) = 1 !Ok!
Sensor( 2) = 5 !Ok!
Sensor( 3) = 3 !Ok!
Sensor( 4) = 7 !Ok!
== GNATcoverage source trace file ==
R05BVGNvdiBzb3VyY2UgdHJhY2UgZmlsZQAAAAAAAAAAAAAABAEAAAAAAAEAAAAHbW9
uaXRvcgAAAAACAAAACAAAAAAAAAAAAAAAAwAAAAAAAAAAAAAAAAAAAAcAAAAHAAAAAg
AAAAAAALNVLgQbmnY19sbrMoReNvzLLN1DAABtb25pdG9yAF8
== End ==
From which we can extract a source trace file like so:
gnatcov extract-base64-trace monitor.stdout mon.srctrace
And finally produce a report, with a gnatcov coverage command such as:
gnatcov coverage --level=stmt+decision --annotate=xcov mon.srctrace -Pmonitor.gpr
Library instrumented separately, native configuration, trace output¶
For this case we will consider a sample native software system with two source
directories: one code
directory with the sources of a library to test, and a
tests
directory with main programs verifying that the library services and
operate as intended.
For the sake of the example, we will consider that
The library source code is not going to change, and
We will be adding tests and assess the achieved coverage by each new test individually or for the current set of tests at a given point in time.
Setting up the coverage runtime¶
On a native system such as Linux or Windows, the simplest is to pick a
gnatcov_rts_full.gpr variant, thanks to which we will be able to produce
trace files directly. We go for a straightforward setup assuming we will use
the default full Ada runtime (no specific --RTS
option):
# Copy the sources into a fresh local place for the build:
cp -rp <gnatcoverage-install>/share/gnatcoverage/gnatcov_rts <gnatcov_rts-build-dir>
# Build and install the library to a place of our choice.
cd <gnatcov_rts-build-dir>
gprbuild -Pgnatcov_rts_full.gpr -f -p
rm -rf <gnatcov_rts-install-dir>
gprinstall -Pgnatcov_rts_full.gpr -p --prefix=<gnatcov_rts-install-dir>
# Allow references to the coverage runtime project from other project files:
export GPR_PROJECT_PATH=<gnatcov_rts-install-dir>
Project file architecture¶
A possible straightforward way to handle code + tests system when all the code is available upfront is to setup a single project file designating the two source dirs and the main units within the tests component.
When part of the code, as the set of tests in our case, is being developed and
the other is frozen, best is to isolate the frozen part as a separate project
and declare it Externally_Built
once the instrumented version has been built.
This would normally be achieved by gprinstall after the build,
except the support for instrumentation artifacts (--src-subdirs
option) may not be available.
One solution consists in setting up a separate library project file for the
library code
part, build the library, use the build tree in-place as the
installation prefix, and switch the Externally_Built
attribute to "True"
before proceeding with separate steps for the tests, instrumenting main units
in particular.
Using an scenario variable to influence the Externally_Built
status, we could
have something like the following project file for the library:
-- code.gpr
library project Code is
for Library_Name use "code";
for Library_Kind use "static";
for Library_Dir use "lib-" & Project'Name;
for Object_Dir use "obj-" & Project'Name;
for Source_Dirs use ("code");
type Mode is ("build", "instrument", "use");
LIB_MODE : Mode := external ("CODE_LIBMODE", "use");
case LIB_MODE is
when "build" => for Externally_Built use "False";
when "instrument" => for Externally_Built use "False";
when "use" => for Externally_Built use "True";
end case;
end Code;
And for the tests, a separate project file where we can list the main units and state that none of the test units are of interest to the coverage metrics:
-- tests.gpr
with "code.gpr";
project Tests is
for Source_Dirs use ("tests");
for Object_Dir use "obj-" & Project'Name;
for Main use ("test_inc.adb");
package Coverage is
for Units use ();
end Coverage;
end Tests;
Instrument and build the library¶
We would first instrument and build the instrumented library with commands such as:
gnatcov instrument -Pcode.gpr -XCODE_LIBMODE=instrument --level=stmt+decision
gprbuild -f -Pcode.gpr -XCODE_LIBMODE=build -p
--src-subdirs=gnatcov-instr --implicit-with=gnatcov_rts_full.gpr
Both commands proceed with Externally_Built
"False"
. There is no main unit
attached to the library per se, so no need for --dump-trigger
or
--dump-channel
at instrumentation time.
Then we can go on with the tests using the default CODE_LIBMODE
value,
implicitly switching the attribute to "True"
.
Instrument, build and run the tests to produce traces¶
Here the only point of the instrumentation phase is to instrument the main units, in our case to dump trace files when the test programs exit:
gnatcov instrument -Ptests.gpr --level=stmt+decision
--dump-trigger=atexit [--dump-method=bin-file] --externally-built-projects
The --externally-built-projects
option is required to consider units
from the library code project as contributing to the set of units of interest,
for the purpose of instrumenting mains, that is, so the instrumentation of
main considers coverage data from those units when producing the trace file.
The build of instrumented tests then proceeds as follows:
gprbuild -Ptests.gpr -p
--src-subdirs=gnatcov-instr --implicit-with=gnatcov_rts_full.gpr
And a regular execution in the host environment would produce a source trace in addition to performing the original functional operations.