7.3. How to Use GNATprove in a Team

The most common use of GNATprove is as part of a regular quality control or quality assurance activity inside a team. Usually, GNATprove is run every night on the current codebase, and during the day by developers either on their computer or on servers. For both nightly and daily runs, GNATprove results need to be shared between team members, either for viewing results or to compare new results with the shared results. These various processes are supported by specific ways to run GNATprove and share its results.

In all cases, the source code should not be shared directly (say, on a shared drive) between developers, as this is bound to cause problems with file access rights and concurrent accesses. Rather, the typical usage is for each user to do a check out of the sources/environment, and use therefore her own version/copy of sources and project files, instead of physically sharing sources across all users.

The project file should also always specify a local, non shared, user writable directory as object directory (whether explicitly or implicitly, as the absence of an explicit object directory means the project file directory is used as object directory).

7.3.1. Possible Workflows

Multiple workflows allow to use GNATprove in a team:

  1. GNATprove is run on a server or locally, and no warnings or check messages should be issued. Typically this is achieved by suppressing spurious warnings and justifying unproved check messages.
  2. GNATprove is run on a server or locally, and textual results are shared in Configuration Management.
  3. GNATprove is run on a server, and textual results are sent to a third-party qualimetry tool (like GNATdashboard, SonarQube, SQUORE, etc.)
  4. GNATprove is run on a server or locally, and the GNATprove session files are shared in Configuration Management.

In all workflows (but critically for the first workflow), messages can suppressed or justified. Indeed, like every sound and complete verification tool, GNATprove may issue false alarms. A first step is to identify the type of message:

Check messages from proof may also correspond to provable checks, which require interacting with GNATprove to find the correct contracts and/or analysis switches, see How to Investigate Unproved Checks.

The textual output in workflow 3 corresponds to the compiler-like output generated by GNATprove and controlled with switches --report and --warnings (see Running GNATprove from the Command Line). By default messages are issued only for unproved checks and warnings.

The textual output in workflow 2 comprises this compiler-like output, and possibly additional output generated by GNATprove in file gnatprove.out (see Effect of Mode on Output and Managing Assumptions).

Workflow 4 requires sharing session files used by GNATprove to record the state of formal verification on each source package. This is achieved by specifying in the Project Attributes the Proof_Dir proof directory, and sharing this directory under Configuration Management. To avoid conflicts, it is recommended that developers do not push their local changes to this directory in Configuration Management, but instead periodically retrieve an updated version of the directory. For example, a nightly run on a server, or a dedicated team member, can be responsible for updating the proof directory with the latest version generated by GNATprove.

A benefit of workflow 4 compared to other workflows is that it avoids reproving locally properties that were previously proved, as the shared session files keep track of which checks were proved.

7.3.2. Suppressing Warnings

GNATprove issues two kinds of warnings, which are controlled separately:

  • Compiler warnings are controlled with the usual GNAT compilation switches:

    • -gnatws suppresses all warnings

    • -gnatwa enables all optional warnings

    • -gnatw? enables a specific warning denoted by the last character

      See the GNAT User’s Guide for more details. These should passed through the compilation switches specified in the project file.

  • GNATprove specific warnings are controlled with switch --warnings:

    • --warnings=off suppresses all warnings

    • --warnings=error treats warnings as errors

    • --warnings=continue issues warnings but does not stop analysis (default)

      The default is that GNATprove issues warnings but does not stop.

Both types of warnings can be suppressed selectively by the use of pragma Warnings in the source code. For example, GNATprove issues three warnings on procedure Warn, which are suppressed by the three pragma Warnings in the source code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
pragma Warnings (Off, "unused initial value of ""X""",
                 Reason => "Parameter mode is mandated by API");

procedure Warn (X : in out Integer) with
  SPARK_Mode
is
   pragma Warnings (Off, "initialization has no effect",
                    Reason => "Coding standard requires initialization");
   Y : Integer := 0;
   pragma Warnings (On, "initialization has no effect");

begin
   pragma Warnings (Off, "unused assignment",
                    Reason => "Test program requires double assignment");
   X := Y;
   pragma Warnings (On, "unused assignment");
   X := Y;
end Warn;

Warnings with the specified message are suppressed in the region starting at pragma Warnings Off and ending at the matching pragma Warnings On or at the end of the enclosing scope. The Reason argument string is optional. A regular expression can be given instead of a specific message in order to suppress all warnings of a given form. Pragma Warnings Off can be added in a configuration file to suppress the corresponding warnings across all units in the project. Pragma Warnings Off can be specified for an entity to suppress all warnings related to this entity.

Pragma Warnings can also take a first argument of GNAT or GNATprove to specify that it applies only to GNAT compiler or GNATprove. For example, the previous example can be modified to use these refined pragma Warnings:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
pragma Warnings (GNATprove, Off, "unused initial value of ""X""",
                 Reason => "Parameter mode is mandated by API");

procedure Warn2 (X : in out Integer) with
  SPARK_Mode
is
   pragma Warnings (GNATprove, Off, "initialization has no effect",
                    Reason => "Coding standard requires initialization");
   Y : Integer := 0;
   pragma Warnings (GNATprove, On, "initialization has no effect");

begin
   pragma Warnings (GNATprove, Off, "unused assignment",
                    Reason => "Test program requires double assignment");
   X := Y;
   pragma Warnings (GNATprove, On, "unused assignment");
   X := Y;
end Warn2;

Besides the documentation benefit of using this refined version of pragma Warnings, it makes it possible to detect useless pragma Warnings, that do not suppress any warning, with switch -gnatw.w. Indeed, this switch can then be used both during compilation with GNAT and formal verification with GNATprove, as pragma Warnings that apply to only one tool can be identified as such.

See the GNAT Reference Manual for more details.

7.3.3. Suppressing Information Messages

Information messages can be suppressed by the use of pragma Warnings in the source code, like for warnings.

7.3.4. Justifying Check Messages

7.3.4.1. Direct Justification with Pragma Annotate

Check messages generated by GNATprove‘s flow analysis or proof can be selectively justified by adding a pragma Annotate in the source code. For example, the check message about a possible division by zero in the return expression below can be justified as follows:

return (X + Y) / (X - Y);
pragma Annotate (GNATprove, False_Positive,
                 "divide by zero", "reviewed by John Smith");

The pragma has the following form:

pragma Annotate (GNATprove, Category, Pattern, Reason);

where the following table explains the different entries:

Item Explanation
GNATprove is a fixed identifier
Category is one of False_Positive or Intentional
Pattern is a string literal describing the pattern of the check messages which shall be justified
Reason is a string literal providing a justification for reviews

All arguments should be provided.

The Category currently has no impact on the behavior of the tool but serves a documentation purpose:

  • False_Positive indicates that the check cannot fail, although GNATprove was unable to prove it.
  • Intentional indicates that the check can fail but that it is not considered to be a bug.

Pattern should be a substring of the check message to justify.

Reason is a string provided by the user as a justification for reviews. This reason may be present in a GNATprove report.

Placement rules are as follows: in a statement list or declaration list, pragma Annotate applies to the preceding item in the list, ignoring other pragma Annotate. If there is no preceding item, the pragma applies to the enclosing construct. For example, if the pragma is the first element of the then-branch of an if-statement, it will apply to condition in the if-statement.

If the preceding or enclosing construct is a subprogram body, the pragma applies to both the subprogram body and the spec including its contract. This allows to place a justification for a check message issued by GNATprove either on the spec when it is relevant for callers. Note that this placement of a justification is ineffective on subprograms analyzed only in the context of their calls (see details in Contextual Analysis of Subprograms Without Contracts).

As a point of caution, the following placements of pragma Annotate will apply the pragma to a possibly large range of source lines:

  • when the pragma appears in a statement list after a block, it will apply to the entire block (e.g. an if statement including all branches, or a loop including the loop body).
  • when the pragma appears directly after a subprogram body, it will apply to the entire body and the spec of the subprogram.

Users should take care to not justify checks which were not intended to be justified, when placing pragma Annotate in such places.

   procedure Do_Something_1 (X, Y : in out Integer) with
     Depends => ((X, Y) => (X, Y));
   pragma Annotate (GNATprove, Intentional, "incorrect dependency ""Y => X""",
                    "Dependency is kept for compatibility reasons");

or on the body when it is an implementation choice that need not be visible to users of the unit:

   procedure Do_Something_2 (X, Y : in out Integer) with
     Depends => ((X, Y) => (X, Y));
   procedure Do_Something_2 (X, Y : in out Integer) is
   begin
      X := X + Y;
      Y := Y + 1;
   end Do_Something_2;
   pragma Annotate (GNATprove, Intentional, "incorrect dependency ""Y => X""",
                    "Currently Y does not depend on X, but may change later");

Pragmas Annotate of the form above that do not justify any check message are useless and result in a warning by GNATprove. Like other warnings emitted by GNATprove, this warning is treated like an error if the switch --warnings=error is set.

7.3.4.2. Indirect Justification with Pragma Assume

Check messages generated by GNATprove‘s proof can alternatively be justified indirectly by adding a Pragma Assume in the source code, which allows the check to be proved. For example, the check message about a possible integer overflow in the assignment statement below can be justified as follows:

   procedure Next_Tick is
   begin
      pragma Assume (Clock_Ticks < Natural'Last,
                     "Device uptime is short enough that Clock_Ticks is less than 1_000 always");
      Clock_Ticks := Clock_Ticks + 1;
   end Next_Tick;

Using pragma Assume is more powerful than using pragma Annotate, as the property assumed may be used to prove more than one check. Thus, one should in general use pragma Annotate rather than pragma Assume to justify simple runtime checks. There are some cases though where using a pragma Assume may be preferred. In particular:

  • To keep assumptions local:

    pragma Assume (<External_Call's precondition>,
                   "because for these internal reasons I know it holds");
    External_Call;
    

    If the precondition of External_Call changes, it may not be valid anymore to assume it here, though the assumption will stay True for the same reasons it used to be. Incompatible changes in the precondition of External_Call will lead to a failure in the proof of External_Call’s precondition.

  • To sum up what is expected from the outside world so that it can be reviewed easily:

    External_Find (A, E, X);
    pragma Assume (X = 0 or (X in A'Range and A (X) = E),
                   "because of the documentation of External_Find");
    

    Maintenance and review is easier with a single pragma Assume than if it is spread out into various pragmas Annotate. If the information is required at several places, the pragma Assume can be factorized into a procedure:

    function External_Find_Assumption (A : Array, E : Element, X : Index) return Boolean
    is (X = 0 or (X in A'Range and A (X) = E))
    with Ghost;
    
    procedure Assume_External_Find_Assumption (A : Array, E : Element, X : Index) with
     Ghost,
     Post => External_Find_Assumption (A, E, X)
    is
       pragma Assume (External_Find_Assumption (A, E, X),
                      "because of the documentation of External_Find");
    end Assume_External_Find_Assumption;
    
    External_Find (A, E, X);
    Assume_External_Find_Assumption (A, E, X);
    

In general, assumptions should be kept as small as possible (only assume what is needed for the code to work). Indirect justifications with pragma Assume should be carefully inspected as they can easily introduce errors in the verification process.

7.3.5. Sharing Proof Results Via a Memcached Server

GNATprove can cache and share results between distinct runs of the tool, even across several computers, via a Memcached server. To use this feature, you need to setup a memcached server (see https://memcached.org/) on your network or on your local machine. Then, if you add the option --memcached-server=hostname:portnumber to your invocation of gnatprove (or use the Switches Attribute of the Prove Package of your project file), then caching will be used, and speedups should be observed in many cases.

7.3.6. Managing Assumptions

Because GNATprove analyzes separately subprograms and packages, its results depend on assumptions about other subprograms and packages. For example, the verification that a subprogram is free from run-time errors depends on the property that all the subprograms it calls implement their specified contract. If a program is completely analyzed with GNATprove, GNATprove will report messages on those other subprograms, if they might not implement their contract correctly. But in general, a program is partly in SPARK and partly in other languages, mostly Ada, C and assembly languages. Thus, assumptions on parts of the program that cannot be analyzed with GNATprove need to be recorded for verification by other means, like testing, manual analysis or reviews.

When switch --assumptions is used, GNATprove generates information about remaining assumptions in its result file gnatprove.out. These remaining assumptions need to be justified to ensure that the desired verification objectives are met. An assumption on a subprogram may be generated in various cases:

  • the subprogram was not analyzed (for example because it is marked SPARK_Mode => Off)
  • the subprogram was not completely verified by GNATprove (that is, some unproved checks remain)

Note that currently, only assumptions on called subprograms are output, and not assumptions on calling subprograms.

The following table explains the meaning of assumptions and claims which gnatprove may output:

Assumption Explanation
effects on parameters and global variables The subprogram does not read or write any other parameters or global variables than what is described in its spec (signature + data dependencies).
absence of run-time errors The subprogram is free from run-time errors.
the postcondition The postconditon of the subprogram holds after each call of the subprogram.