2. How to Run CodePeer

2.1. Running CodePeer

This section describes how to run CodePeer from the command line (for simple experiments or for running jobs automatically), as well as how to use CodePeer from the GNAT Programming Studio and GNATbench (Eclipse plug-in).

2.1.1. Running CodePeer from the Command Line

In order to run a code review with CodePeer from the command line, you need to perform the following steps:

  • Run codepeer

    See Command Line Invocation for more details. A typical invocation of codepeer will look like:

    $ codepeer -Pmy_project
    

    Note that if you want to run codepeer as part of a script, you should consider using also the -quiet switch.

    This command will first run codepeer-gprbuild to generate or update SCIL files (intermediate representation), and then will run the CodePeer main engine.

    This will update the CodePeer database under codepeer/my_project.db. You can then view the results using GPS, a Web browser, or from the command line, for example:

    $ codepeer -Pmy_project -output-msg-only
    alias.adb:10:11: high: divide by zero fails here: requires Page_Size /= 0
    

    Note that if you want to systematically generate messages from the command line, you can then use directly:

    $ codepeer -Pmy_project -output-msg
    alias.adb:10:11: high: divide by zero fails here: requires Page_Size /= 0
    

    See How to View CodePeer Output for more details.

    In addition, the codepeer switch -level can be used, based on the kind of analysis needed, and the size of the source code analyzed, see CodePeer Levels for more details.

2.1.2. Running CodePeer from GPS

When CodePeer is installed and found on your PATH, GPS will detect it automatically and provide a new CodePeer top level menu with the following contents.

The main entry point to run CodePeer and display a code review is the Analyze All menu.

  • Analyze All

    Generate SCIL for sources of the whole project tree if needed, then run CodePeer, displaying a progress bar giving an estimate of the review state, and then load the CodePeer report window. See Viewing CodePeer Output in GPS for more details on the report window. This menu item will use the CodePeer switches set in your project files, if any. By default, CodePeer annotations such as automatically generated contracts will be imported and displayed in the source editor. This step can take a significant amount of time for large projects and can be disabled via the GPS Preferences.

  • Analyze...

    _images/codepeer-analyze.jpg

    Open a dialog where you can select more precisely the kind of analysis performed and in particular to add or override CodePeer switches defined in your project file. You can specify any command line switch in the bottom text input, as listed in Command Line Invocation. In addition you can also select graphically some of these options:

    Analysis level Allows specifying the precision of analysis and corresponding time and memory. Maps to the -level switch.
    Messages Specify the verbosity and number of messages produced by CodePeer. Maps to the -messages switch. The default is -messages normal unless specified otherwise in your project file.
    Baseline run Sets the run as the baseline for future comparisons. Maps to the -baseline switch.
    No race condition Disable global race condition analysis. Maps to the -no-race-conditions switch.
    Root project only CodePeer will only analyze the source files associated with the root project file (presuming your application uses a tree of project files to represent all of its constituents). Maps to the --no-subprojects switch.
    Force analysis Force generation of all SCIL files and analysis of all files. Maps to the -f switch.
    GNATcheck Launch GNATcheck and collect its messages, see GNATcheck Integration. Maps to the --gnatcheck switch.
    GNAT warnings Launch GNAT front end and collect its warnings, see GNAT Warnings Integration. Maps to the --gnat-warnings switch.
    Cutoff Specify the reference (cut off) run to perform a comparison of messages. Maps to the -cutoff switch, see also CodePeer Baselines.
    Baseline review # Change the default baseline. Maps to the -set-baseline-id <id> switch.
    Multiprocessing Specify the number of processes to generate SCIL files and analyze files. Maps to the -j switch.

    Once the selection is done, click on the Execute button to get a behavior similar to Analyze All.

  • Analyze File

    Analyze the current source file only. This menu is available only when a source file is selected in GPS. Note that the analysis is performed without taking into account effects of calls outside the selected file, and the results of this run are not entered into the CodePeer historical database. In addition, the messages are displayed in the Locations view only, without opening the CodePeer report view. See Running CodePeer in Compiler Mode for more details on this capability. Note that analyzing a subunit (separate unit) or a generic unit will have no effect: in the case of subunits you need to analyze the enclosing unit, and analyze instantiations of generics to get messages on generic units.

    If you would like to get a CodePeer report view, even when analyzing only a single file, then you can use Analyze... instead, and append %F to the end of the command line, which requests an analysis of the current file only.

  • Analyze File by File...

    _images/codepeer-file-by-file.jpg

    Similar to Analyze File for all files of the project. Analysis is performed incrementally, so only files that have changed since the last run will be reanalyzed and messages produced. See Running CodePeer in Compiler Mode for more details on this capability. Similarly to Analyze..., you can specify relevant custom switches either manually via the text input at the bottom of the dialog, or via the graphical interface for some of the switches:

    Messages Specify the verbosity and number of messages produced by CodePeer. Maps to the -messages switch. The default in compiler mode is -messages min unless specified otherwise in your project file.
    Multiprocessing Specify the number of processes to generate SCIL files and analyze files. Maps to the -j switch.
    Root project only CodePeer will only analyze the source files associated with the root project file (presuming your application uses a tree of project files to represent all of its constituents). Maps to the --no-subprojects switch.
    Force analysis Force generation of all SCIL files and analysis of all files. Maps to the -f switch.
    Complete output Output messages for all files even the ones that have not been reanalyzed. Maps to the --complete-output switch.
  • Display Code Review

    Assuming CodePeer has already been run previously, GPS will load the CodePeer review information from the database and display a summary of the review. See Viewing CodePeer Output in GPS for more details.

  • Generate HTML Report...

    _images/codepeer-html.jpg

    Generate a CodePeer report in HTML format. This menu must be run after a CodePeer analysis has completed. See HTML Output for more information. The following switches can be specified to refine the information generated, as part of an intermediate dialog:

    Show annotations Show annotations in addition to messages. Maps to the -show-annotations switch.
    Show informationals Show informational messages. Maps to the -show-info switch.
    Show removed Show messages removed from baseline run. Maps to the -show-removed switch.
    Hide low messages Do not generate messages ranked low. Maps to the -hide-low switch.
    CWE Include CWE ids in message output. Maps to the -cwe switch.
    Current Override current run id. Maps to the -current switch.
    Cutoff Override baseline run id. Maps to the -cutoff switch.

    See Text Output for more details on these switches. Note that you will need to first run the CodePeer web server before viewing the HTML report.

  • Generate CSV Report...

    Generate a CodePeer report in CSV format, and open the report in a source editor. This menu must be run after a CodePeer analysis has completed. It will open an intermediate dialog which is described in the previous section (Generate HTML Report...). See also CSV Output for more details on the CSV output.

  • Advanced

    This submenu provides advanced capabilities, less often needed:

    • Regenerate Report...

      Run CodePeer to only regenerate reports by reading the database. Maps to the -output-only switch. This might be useful for e.g. changing the baseline or cutoff point of comparison.

    • Generate SCIL...

      Note: the Analyze All and Analyze... menus generate SCIL files automatically, so this menu is not needed in general. Open a dialog to generate SCIL for your project. The default switches should in general be sufficient, in which case, pressing enter or clicking on the Execute button will launch the SCIL generation. Alternatively, you might need to add some extra switches for better SCIL generation, depending on your sources.

    • CodePeer Log

      Open a text editor with the low level log file produced by CodePeer. This can be useful for example to understand why CodePeer did not generate the expected output, or to send CodePeer bug reports.

    • Remove Lock

      Remove the lock associated with a CodePeer run. In normal conditions, CodePeer will handle locks automatically. In case of an unexpected exit during CodePeer execution, a lock file may be left behind, which can be removed using this menu. This menu should not be used while CodePeer is still running.

    • Remove XML Code Review

      GPS generates an intermediate XML file containing the contents of the last code review as stored in the CodePeer database. In normal mode of operation, GPS will generate this file when needed. This menu can be used to manually remove this xml and force GPS to regenerate it the next time e.g. the Display Code Review menu is used.

    • Remove SCIL

      Remove all SCIL directories associated with the project. This can be useful when e.g. upgrading to a new version of CodePeer or when you want to start a clean analysis from scratch.

    • Remove SCIL & DB

      Same as above, and in addition, also removes the CodePeer historical database and any other CodePeer artefacts.

Note that most of the menus will launch commands that can be configured or tuned using the Edit ‣ Preferences menu, in the Build Targets section. See the GPS documentation for more details on the build targets.

GPS allows sharing the database among several users. Paths to database and output directories can be defined using the project properties editor (Edit ‣ Project Properties...). See Project Attributes.

In addition, new preferences specific to CodePeer are also defined, see GPS Preferences for more details.

2.1.3. Running CodePeer from GNATbench

When CodePeer is installed and found on your PATH, GNATbench will detect it automatically and provide a context sensitive menu when you right click on the project. The context menu will have a Codepeer entry with the following submenus:

  • Analyze All
  • Analyze...
  • Analyze File
  • Analyze File By File
  • Run CodePeer
  • Regenerate Report
  • Clear CodePeer Markers
  • Redisplay CodePeer Markers
  • Display HTML Report
  • View Race Condition
  • Generate SCIL

The main entry point to run CodePeer and display a code review is the Analyze All menu.

  • Analyze All

    Right click on the Project and then click the Codepeer ‣ Analyze All menu. Analyze All generates SCIL for sources of the whole project tree if needed and run CodePeer. When CodePeer starts running, a Console window will come up in the bottom pane; this window will contain any errors or informational messages generated by CodePeer while it is running. At the end of the analysis, the CodePeer Problem Details tab will be highlighted and the messages can be seen in the bottom pane under the CodePeer messages heading. See Viewing CodePeer Output in GNATbench for more details on the report window. You can view the HTML report by clicking the Display HTML Report menu.

  • Analyze...

    Open a dialog where you can select more precisely the kind of analysis performed. In particular you can select to analyze the root project only, and you can change the analysis level. Once the selection is done, click on the Execute button to get a behavior similar to Analyze All. In order to perform a quick analysis, set the Analysis level to 0 or 1. In order to analyze only the root project, enable the Analyze root project check box.

  • Analyze File

    Analyze the current source file only. This menu is available only when a source file is selected in GB. Note that the analysis is performed without taking into account effects of calls outside the selected file, and the run is not taken into account in the CodePeer historical database. See Running CodePeer in Compiler Mode for more details on this capability.

  • Analyze File by File...

    Similar to Analyze File for all files of the project. Analysis is performed incrementally, so only files that have changed since the last run will be reanalyzed and messages produced. See Running CodePeer in Compiler Mode for more details regarding this.

  • Run CodePeer

    Right click on the Project and select the Codepeer->Run CodePeer menu. The Run CodePeer dialog will pop up. The default switches should be sufficient for running. You can choose extra switches if needed. See Command Line Invocation for more details on CodePeer switches.

    _images/codepeer-run-gb.jpg

    Click ‘Proceed’ button to start the analysis. You will see some messages from the CodePeer run in the Console tab. At the end of the run, the CodePeer Problem Details tab will be highlighted and the messages can be seen in the bottom pane under CodePeer Messages. Click on an arrow for a file to expand the messages for that file.

  • Regenerate Report

    Right click on the Project and then select CodePeer-> Regenerate Report menu to regenerate reports by reading the database. A dialog box will appear containing various switches. This might be useful for e.g. changing the baseline or cutoff point of comparison, or filtering messages to generate HTML report. Once you select the switches, then you can press Proceed to regenerate reports without running another analysis of the sources.

  • Clear CodePeer Markers

    Right click on the Project and then select Clear CodePeer Markers menu. It will remove all the CodePeer markers both annotation and problem markers from the editor window area.

  • Redisplay CodePeer Markers

    Right click on the Project and then select Redisplay CodePeer Markers menu. It will recreate all the CodePeer markers that have been cleared for this project without running the CodePeer. It is assumed that CodePeer has been run before for this project.

  • Display HTML Report

    Right click on the Project and then select Display HTML Report menu. It will open a default browser on CodePeer’s HTML report, assuming CodePeer was run previously (via e.g. Analyze All menu). See HTML Output for more information.

  • View Race Condition

    Right click on the Project and then select Codepeer->View Race Condition menu. It will open a new view titled “Race Condition Details” in the bottom pane. In this view you can see results associated with race condition analysis.

  • Generate SCIL

    Note: the Analyze All menu generates SCIL files automatically, so this menu is not needed in general. Right click on the Project and then select the Codepeer->Generate SCIL menu which will open a dialog. Select this menu to generate SCIL files for your project. The default switches should be sufficient in general. If needed, you can select extra switches for better SCIL generation (e.g. -gnatI). At the end, press enter or Proceed button to generate SCIL files.

2.1.4. Running CodePeer from Jenkins

Install <install_dir>/share/jenkins/plugins/codepeer*.hpi into your Jenkins installation. See the Jenkins documentation for more information.

When the jenkins-codepeer-plugin is loaded in your Jenkins instance, it will automatically register a new Build Step available from the project configuration view.

The global configuration will have a CodePeer entry with the following setting:

  • CodePeer executable

The per-project configuration will have a CodePeer Build Step with the following regular settings:

  • Project file
  • Working directory
  • GPR_PROJECT_PATH
  • Analysis level

And the following Advanced settings:

  • Messages verbosity level
  • Jobs count
  • CodePeer executable
  • Ignore exit code
  • Additional CodePeer options

Access the global configuration by navigating from the main dashboard to Manage Jenkins ‣ Configure System, section CodePeer Analyser.

  • CodePeer executable

    Location of the CodePeer executable to use at the system level. Leave empty to use the default CodePeer located in PATH. This setting is overridden by the equivalent per-project setting (see the per-project CodePeer Executable setting).

Access the per-project configuration by navigating from the main dashboard to <your-project> ‣ Configure.

Create a new “build step” by selecting Add build step ‣ Analyse with CodePeer.

  • Project file

    Path to the project file to run the analysis on. Relative to Jenkins workspace directory (e.g. the root of the repository).

  • Working directory

    The directory from where to execute CodePeer, also relative to Jenkins workspace directory. If left empty, the workspace directory is used.

  • GPR_PROJECT_PATH

    Overrides the environment variable GPR_PROJECT_PATH used during CodePeer execution. The original value can be extended, e.g.:

    /gpr/project/path:$GPR_PROJECT_PATH
    
  • Analysis level

    The level of analysis to be performed. This is equivalent to using -level on the command line.

  • Messages verbosity level (Advanced Options)

    The level of verbosity for generated messages. This is equivalent to using –messages on the command line.

  • Jobs count (Advanced Options)

    The number of simultaneous processes launched during an analysis. This is equivalent to using -j on the command line.

  • CodePeer executable (Advanced Options)

    Location of the CodePeer executable to use, defined at the project level. Overrides the global setting of the same name.

  • Ignore exit code (Advanced Options)

    Any exit code from CodePeer other than 0 is treated as an error and the build will fail. Enable this option to ignore the exit code.

  • Additional CodePeer options (Advanced Options)

    This plugin does not expose all CodePeer options. Use this setting to configure any additional option to add to the command line.

2.2. Getting the Right CodePeer Settings

We recommend that you first get familiar with System Requirements for a proper recommended configuration to run CodePeer.

Then there are two cases:

  • If you do not have a project file set up (.gpr) for e.g. building with GNAT or from a previous use of CodePeer, then our recommendation is to start running CodePeer with the -level 0 switch. This can be achieved by creating a simple project file as described in Basic Project File Setup.

  • Once you have a fully working project file setup (either pre existing for GNAT build or after having followed Project File Setup), then use -level 1:

    project My_Project is
       for Source_Dirs use ...
    
       package CodePeer is
          for Switches use ("-level", "1");
       end CodePeer;
    end My_Project;
    

Then run codepeer to produce messages, e.g:

codepeer -Pmy_project -output-msg

2.2.1. Analyze messages

If the run is mostly clean or contains mainly interesting messages, then try running at the next level (e.g. -level 2) and iterate until a point of diminishing interest in terms of useful messages vs number of false alarms, as well as timing.

If the run already contains many messages, start analyzing them and identify if there are groups of uninteresting or incorrect messages. You can then exclude categories of messages producing uninteresting results via the --be-messages switch.

For example if messages related to access check are not producing interesting results in your code base, you can disable these messages using --be-messages=-access_check (see Filtering Messages for more details).

If many uninteresting or incorrect messages are issued for a file, you can exclude this file from analysis via the CodePeer'Excluded_Source_Files project attribute or via pragma Annotate (CodePeer, Skip_Analysis) as explained in Partial Analysis.

Also choose the relevant messages based on ranking: CodePeer ranks messages in categories depending on how interesting and likely to happen (see Categorization of Messages).

While analysing existing code, you should start looking at High messages first, then Medium, and finally if it makes sense, Low messages.

A recommended setting is to consider High and Medium messages, which is what is displayed by default in the GPS and HTML interfaces.

2.2.2. Run CodePeer Faster

If runs are too slow for your needs (e.g. quick user feedback as opposed to nightly runs) then you should consider first lowering the level of analysis, making sure you are using the -j0 switch (enabled by default unless you are using another -jxx switch explicitly) and using a machine with as many cores as possible as explained in System Requirements.

If this isn’t sufficient, the next step is to identify the files that are taking a long time to analyze and exclude the most costly files from the analysis as per Partial Analysis.

To identify the files that are taking a long time to analyze, you can save codepeer’s default output and sort it on the 4th column (timing info):

analyzed main.scil in 0.05 seconds
analyzed main__body.scil in 620.31 seconds
analyzed pack1__body.scil in 20.02 seconds
analyzed pack2__body.scil in 5.13 seconds

For example using the sort utility:

codepeer -Pprj |& tee codepeer-output.txt
sort -r -g -k 4 codepeer-output.txt

You’ll get:

analyzed main__body.scil in 620.31 seconds
analyzed pack1__body.scil in 20.02 seconds
analyzed pack2__body.scil in 5.13 seconds
analyzed main.scil in 0.05 seconds

In this simple example, skipping the analysis of the body of unit main (e.g. main.adb) will produce a 25 seconds analysis instead of taking more than 10 minutes.

2.3. Running CodePeer in Compiler Mode

An alternate way to run CodePeer is to use the -compiler-mode switch, where CodePeer will behave more like a compiler by analyzing files one by one, and only reanalyzing files which have changed since the last run. For example:

$ codepeer -compiler-mode -Pmy_project

Will analyze all files in my_project and generate messages in the standard output for each file analyzed.

You can also access this capability from GPS via the menu CodePeer ‣ Analyze File by File..., see Analyze File by File.

Most CodePeer switches are relevant in this mode, in particular:

  • -file
  • -j
  • –no-subprojects
  • -U
  • –complete-output

CodePeer switches from the command line and from the project file are taken into account, whenever relevant.

In addition, the -m switch for the builder can be relevant, as described in Ignoring Source File Timestamp Mismatch.

Compiler mode has some advantages:

  • Provides incremental analysis by taking advantage of gprbuild’s ability to only recompile/reanalyze files that have been modified since the last run. Note that messages from unchanged files analyzed from the previous run are not displayed by default, to allow concentrating on the direct impact of a change. You can request messages from the previous run to be displayed for files that have not been reanalyzed via the --complete-output switch.
  • Each file is analyzed in parallel when using the -j switch
  • Uses less memory
  • Skips several extra phases done by CodePeer (generation of HTML files, text listings, update of the historical database).

As well as several limitations:

  • Analysis is limited per unit, so no global analysis is performed (roughly equivalent to -level 1)
  • No extra information is generated (no html, no historical database, no annotations), which means in particular less information available to understand messages
  • Limited integration in the IDEs (no CodePeer report window, no display of extra information such as backtraces or possible values of variables)
  • No history of previous runs is stored
  • The only way to review messages is via pragma Annotate

2.4. Reducing the Number of False Alarms

By default CodePeer will try to detect as many potential issues as possible, at the expense of generating false alarms. You can tell CodePeer to concentrate on detecting only the most likely errors and generate many fewer false alarms (also known as false positives) via the -messages min switch. Note that this switch is enabled by default already at -level 1, -level 2, and when using -compiler-mode.

2.5. CodePeer Levels

You can configure CodePeer to perform its analysis at different levels via the -level switch, to tune the accuracy of analysis needed and depending on the size of the source code analyzed.

-level sets the level of analysis, which impacts both time and accuracy. Note that messages for lower levels are not necessarily a subset of those for higher levels. The switch -messages can be used to filter messages that have a higher probability of being false positives, in particular via -messages min.

Level Description Recommended Code Characteristics
0 or min Light and fast analysis similar to compiler warnings or linters. This level only requires a basic project setup. CodePeer will look in the project’s source directories and analyze all files with the following extensions by default: .ada, .ads, .adb, .spc, .bdy. This level only runs external checkers based on libadalang, and does not run the core engine (so most switches are ignored). No code size or complexity limit.
1 In addition to running the light checkers from level 0, also runs the core engine of codepeer, subprogram by subprogram. Implies -messages min -no-race-conditions -steps 60 -dbg-partition-limit 1000. No code size or complexity limit.
2 Analyze by groups of units. Slower analysis, more precise. Implies -messages min -no-race-conditions -steps 100 -dbg-partition-limit 3000. No code size or complexity limit.
3 Semi-global analysis. Units will be analyzed globally as much as possible, and an automatic partitioning is performed to complete the analysis within the memory constraints of the machine. Implies -steps 1000. Suitable for less than 1 million SLOC and subprograms with a cyclomatic complexity below 40.
4 or max Global analysis. Analyze all units together with no partitioning. May require large amounts of memory and time. Recommended only for very accurate and complete analysis to detect all potential cases of run-time errors exhaustively at the expense of many false positives. Implies -no-presumptions -messages max -gnateF -global -steps 3000. Suitable for less than 200K SLOC and subprograms with a cyclomatic complexity below 20.

So typical use cases for each level would include:

Level Use Case
0

Used for getting initial static analysis results without having to perform a complex setup of your project. Quickly runs light checkers and linters on any Ada sources, including very large source code bases. This is ideal for getting familiar with CodePeer and for running CodePeer at each developer’s desk or after each commit.

If you want to override the default file extensions, you can add to your project file the project attribute File_Patterns, as explained in Project Attributes.

1 When you have set up your project file so that CodePeer can more fully analyze your source files (as per Project File Setup), you can use level 1, which includes the light checkers from level 0, and in addition will run the CodePeer main engine in a local mode (analyzing each subprogram independently). This is intended for running at each developer’s desk or after each commit, with more detailed results than level 0.
2 After having useful and mostly clean results at level 1, running at level 2 will provide a more detailed analysis and in particular perform some level of interprocedural analysis, keeping only the most useful messages. This level can be used either at each developer’s desk for small to medium code bases (e.g. less than 100K SLOC), and on servers automatically for larger code bases.
3 Default level. Can be used for performing semi-global analysis, on code bases typically no larger than 1 million lines of code. This level will provide a more detailed interprocedural analysis and also come with more false positives. This level is suitable for automatic runs on servers.
4 This level is only suitable for small to medium code bases (typically less than 200K SLOC) for a sound (messages reported with high rank are certain) and complete (all possible errors are reported) analysis, where a systematic review of all potential run-time checks is required, including reviewing many false positives and where time of analysis is not important. This is the only level where a complete/exhaustive detection of all potential run-time failures (that would have raised an exception at run time) is performed. In terms of memory requirements, a 64-bit configuration with at least 1 GB of memory per 10K SLOC is required. See also Provide Evidence for Program Verification for more details on the use of this level.

In order to ensure reproducibility of results, CodePeer uses a notion of steps instead of timeouts. While analyzing a subprogram, CodePeer will deterministically increment the current number of steps. If this number exceeds the limit specified with the option -steps, CodePeer will stop analyzing this subprogram and continue analyzing the rest of the source code.

Note that steps are not perfectly linear with the time, which means that one subprogram can take 10 seconds and 100 steps whereas another subprogram might take 10 seconds but 300 steps. The important thing to note here is that the same subprogram will take the same number of steps in any system configuration.

2.6. Provide Evidence for Program Verification

CodePeer’s core is a sound and complete static analyzer, which means that it is designed to consider all possible run-time errors and not miss any due to approximations in the analysis. See How does CodePeer Work? for more details on CodePeer’s internals.

For each possible run-time error in Ada, corresponding to raising an exception (e.g. Constraint_Error) during execution, CodePeer either outputs an error message classified as High, Medium, or Low depending on the likelihood of an error occurring, or it outputs no message, meaning that no error is possible at this program point. An error ranked High corresponds to a certain error, while an error ranked Medium or Low corresponds to a possible error depending on the context (or a false positive). The distinction between Medium and Low facilitates prioritizing the most probable errors for review.

By default, a set of prioritizing rules, both internal and external (in a user modifiable file) can hide errors and/or warnings that have a very low probability of causing actual errors. In order to recover all errors and warnings, e.g. to meet a certification requirement, you can run CodePeer in a full-report mode (option -level max, see other relevant options at Report File). Because a run of CodePeer can exercise as much code as exhaustive testing of the same program, inspecting the output of such a run gives exhaustive guarantees about the behavior of the program. The output consists both of prioritized errors and warnings and generated preconditions/postconditions for subprograms.

The results of CodePeer’s analysis are with respect to the preconditions that it generates so that a manual review of the preconditions generated for top-level subprograms (main program or external subprograms for the analysis of a library) is needed to ensure no error or warning is missing due to a too restrictive context for the analysis. This is not needed for all other subprograms, as CodePeer automatically checks that the generated preconditions are verified at calls. This is also not needed if you specify your main subprograms in your project file since CodePeer will flag all preconditions on such main subprograms automatically. See Understanding the differences between preconditions and run-time errors for more details on preconditions and runtime errors.

In the context of software certification, CodePeer can be used in particular to perform:

  • Runtime error analysis (or Boundary value analysis). CodePeer can be used to automatically detect attempts to dereference a pointer that could be null, values outside the bounds of an Ada type or subtype, buffer overflows, numeric overflow or wraparound, and division by zero.
  • Control flow analysis. CodePeer can be used to detect suspicious and potentially incorrect control flows, such as unreachable code, redundant conditionals, loops that either run forever or fail to terminate normally, and subprograms that never return.
  • Data flow analysis. CodePeer can be used to detect suspicious and potentially incorrect data flows, such as variables read before they are written (uninitialized variables), variables written more than once without being read (redundant assignments), and variables that are written but never read.

2.7. CodePeer Baselines

CodePeer maintains a historical database of all the analysis performed on a specific project and all of the associated messages. This database allows CodePeer to specify if a message is new, unchanged or removed. Instead of classifying the message by comparing the current analysis to the previous one, CodePeer uses a more elaborate notion of baseline run or simply baseline. A baseline is the run of reference against which the new analysis is compared to. Each analysis level has its baseline which is, by default, the first run performed at this level (CodePeer will not compare two runs performed at two different analysis levels).

The baseline stays the same from one run to another unless specified otherwise with either -baseline or -set-baseline-id <run_id> switches:

  • -baseline

    A run with the -baseline switch will keep the same baseline as the previous run but will become the new baseline for the following runs.

  • -set-baseline-id <run_id>

    Using -set-baseline-id <run_id> makes the run with the specified run_id the new baseline starting from the current run. For a specific level each run is given a unique id (a run with id 3 corresponds to the third analysis done at its analysis level). GPS displays the ids of the current run and its associated baseline (or cutoff if specified, see below) at the top of the “CodePeer report” tab (see here).

In addition, the user can temporarily ignore the baseline by setting a cutoff with the switch -cutoff <run_id>.

  • -cutoff <run_id>

    The current run is compared to the run with id run_id, without impacting the baseline.

All these switches can be used in combination with -output-msg-only.

  • -output-msg-only

    Adding -output-msg-only lets you display the current run messages without performing a new analysis. You can then compare the current run with any other runs (using -cutoff), or manipulate the baseline (using -baseline or -set-baseline-id) after the analysis is done. In GPS setting -output-msg-only corresponds to using the “Regenerate Report...” menu (in “Codepeer” -> “Advanced”).

In GPS you can set -baseline by checking the “Baseline run” box in either “Analyze...” menu or “Regenerate Report...” menu. In the same menus, you can set -set-baseline-id <run_id> with the “Baseline review #” option and -cutoff <run_id> with the “Cutoff” option (See Running CodePeer from GPS).

The typical workflow is to make successive runs until reaching a satisfactory state. At this point, run CodePeer with -output-only -baseline to set this run as the new baseline. Afterwards, you can always compare the current run with other runs with -output-only -cutoff <run_id>. If later on you find that the baseline is not satisfactory, set a specific one with -set-baseline-id <id>.

Note:

By default the status of the messages are not displayed in the text output of CodePeer (as opposed to GPS and the html output where you can filter the messages with their status). In order to display the status of the messages in the text output you can either tell CodePeer to generate the output in CSV format with the switch -csv, or get textual indicators via the -show-removed and -show-added swiches.

In the CSV format, CodePeer specifies if a message is added, unchanged and also removed if you specify the -show-removed switch. Furthermore, you can compare two arbitrary runs by setting a cutoff and manually override the current run via the -current switch. See Text Output for a complete list of switches supported with -output-msg(-only).

2.8. External Tools Integration

CodePeer can launch external tools to collect their messages, report them and add them to its database. This is done in addition to CodePeer’s core analysis.

As for other messages, you can use the Message Patterns mechanism to filter the external messages (by their source and by their category). A default ranking is chosen for each category of external messages in the MessagePatterns.xml file. See Format of MessagePatterns.xml File for more details.

2.8.1. GNAT Warnings Integration

In order to tell CodePeer to collect the GNAT front-end warnings, the switch --gnat-warnings must be specified.

When specified, CodePeer calls the GNAT front end with the warnings enabled by default by GNAT, and optionally specified as compiler switches in your project file (when using project files also for compilation). You can refine this default by specifying the -gnatw parameters explicitly as optional parameters to --gnat-warnings, as in:

$ codepeer -Pmy_project --gnat-warnings=a.u

These switches will be enabled in addition to the ones specified elsewhere in your project. If you want to cancel the effect of other switches, you can specify the letter A (which corresponds to the -gnatwA switch), e.g.:

$ codepeer -Pmy_project --gnat-warnings=Au

Note that:

  • You need to use pragma Warnings (Off) instead of pragma Annotate to review messages in the source code.
  • Project attributes in the CodePeer package (in particular Excluded_Source_Dirs and Excluded_Source_Files) are ignored when generating GNAT warnings. You need to use the corresponding project level attributes instead if needed.

2.8.2. GNATcheck Integration

In order to tell CodePeer to collect the GNATcheck messages, the switch --gnatcheck must be specified.

Note that GNATcheck requires a file that describes the rules that it should check. Such a file can be created with the help of GPS through the menu: Analyze ‣ Coding Standard ‣ Edit Rules File (see here for more details). Refer to the GNATcheck Reference Manual for the complete GNATcheck documentation.

2.8.3. Libadalang Light Checkers Integration

As mentioned in CodePeer Levels, CodePeer runs a selection of light checkers based on libadalang in addition to its main analysis. Contrary to the other external tools, these checkers are run by default. They can be disabled with the switch --no-lal-checkers.

The selection of checkers used at level 0 can be found in the file install/share/lal-checkers/level_0_checkers.txt and for the other levels in the file install/share/lal-checkers/default_checkers.txt.

You can make your own selection with the --lal-checkers=xxx switch where xxx is a list of checkers separated by commas. You can also use the switch --lal-checkers-from=<filename> where filename specify a file that contains a list of checkers, one checker per line.

Some checkers can be fine tuned with specific arguments. When using the switch --lal-checkers-from=<filename>, add the desired arguments at the end of the line corresponding to the checker you want to tune. See the duplicate_branches checker in install/share/lal-checkers/default_checkers.txt. You can also specify the arguments directly on the command line with the switch --lal-checkers by enclosing in quotes the list of checkers with the arguments inlined, e.g., --lal-checkers="duplicate_branches --size-threshold 100,dead_code".

The switch --list-categories makes CodePeer print the list of the different messages issued by the libadalang checkers (among others), the checkers that issue them as well as a small description, see Listing All Message Kinds.

2.9. Partial Analysis

This section describes how to perform a partial analysis, and in particular, how to exclude entities (projects, directories, files, packages, subprograms) from analysis.

2.9.1. Excluding Subprograms or Packages From Analysis

To selectively disable CodePeer’s analysis of a particular subprogram or set of subprograms, you can add in your Ada source code a pragma Annotate of the form:

pragma Annotate (CodePeer, Skip_Analysis);

When used in this way, an Annotate pragma takes exactly these two arguments. The pragma may be placed at the beginning (possibly preceded only by different Annotate pragmas) of the declaration list of a subprogram body, a package body, or package visible part. In the case of a subprogram body, CodePeer’s analysis of that subprogram body (and anything declared therein, including nested subprograms) will not be performed. In the case of a package body, CodePeer’s analysis of any subprogram body occurring (directly or indirectly) within the package body will not be performed; this includes any such subprogram bodies that are implicitly generated by the compiler (e.g., the compiler might choose to generate a default initialization subprogram for a type declaration). In the case of a package visible part, analysis of subprogram bodies occurring in the package specification (as well as the package body) will not be performed; note that a subprogram body can occur in a package specification as part of an instantiation of a generic unit. The subprogram or package in question may be a generic unit or may occur within a generic unit; in this case, the pragma will be replicated (as described in Ada RM 12.3(13)) when an instance of the generic unit is declared and will affect CodePeer’s analysis of the instance as described in this section.

When an Annotate pragma is used in this way to prevent CodePeer’s analysis of a subprogram body, the analysis of clients of the subprogram will be affected in the same way as if the body of the subprogram were unavailable for some other reason (e.g. a missing .adb file).

Note that an annotate pragma of this form is ignored by the compiler when generating code; it only affects CodePeer’s analysis.

Here is an example of usage to disable analysis of a specific subprogram:

procedure Complex_Subprogram (...) is
   pragma Annotate (CodePeer, Skip_Analysis);
begin
   ...
end Complex_Subprogram;

And similarly for a package:

package Complex_Package is
   pragma Annotate (CodePeer, Skip_Analysis);

   ...
end Complex_Package;

2.9.2. Excluding Files From Analysis

If you want to exclude some files from analysis (because e.g. CodePeer is taking lots of time analyzing them, and these files do not contain useful subprograms for CodePeer to analyze), you can use the Excluded_Source_Files project attribute in package CodePeer, for example:

package CodePeer is
   for Excluded_Source_Files use ("xxx.adb");
   -- Analysis of xxx.adb generates lots of timeouts, skip it for now
end CodePeer;

Note that you should always specify the basename of the excluded files, with no path information. The path information is computed automatically by CodePeer based on other project properties.

CodePeer will still generate SCIL for the excluded file, but it will not attempt to analyze it. If you want to also exclude files from SCIL generation and these files are not in the transitive closure of other files (e.g. a body file, as opposed to a spec file used by other Ada units), then you can use instead the project level Excluded_Source_Files attribute:

project My_Project is
   for Excluded_Source_Files use ("xxx.adb");
end My_Project;

2.9.3. Excluding Directories From Analysis

If you want to exclude from analysis all source files from a given set of directories, you can use the Excluded_Source_Dirs project attribute in package codePeer, for example:

package CodePeer is
   for Excluded_Source_Dirs use ("directory1", "directory2");
end CodePeer;

You can specify absolute directories or directories relative to the location of the project file (same as for source directories).

2.9.4. Excluding Projects From Analysis

If you have a tree of project files (.gpr) and want to exclude some entire projects (aka subsystems) from CodePeer analysis, you can do it by adding the following project attributes to each of the projects involved:

for Externally_Built use "True";

This attribute tells CodePeer to not build/generate SCIL files for this project, thus disabling analysis on all source files for this project.

Projects using files (e.g. spec) from this project will still be analyzed, as well as generic instantiations made in other projects.

2.10. Source Annotations

CodePeer’s analysis and reporting can be affected by adding certain pragmas to your source code. In particular, pragma Annotate can be applied in a few different ways:

  • pragma Annotate (CodePeer, False_Positive|Intentional, “<check name>”, “<review message>”);

    This form can be used to suppress messages by classifying them as false positives or as intentional errors. See the section on reviewing messages Through Pragma Annotate in Source Code.

  • pragma Annotate (CodePeer, Skip_Analysis);

    Pragma Annotate can also be used to skip the analysis of specific subprograms or packages, as described in Partial Analysis.

  • pragma Annotate (CodePeer, Single_Thread_Entry_Point|Multiple_Thread_Entry_Point, “entry point”);

    Pragma Annotate can be used to aid in identifying race conditions, as discussed in Identify Possible Race Conditions.

  • pragma Annotate (CodePeer, Mutex, “lock subprogram”, “unlock subprogram”);

    Also used to Identify Possible Race Conditions.

Note that pragma Annotate is ignored by the compiler when generating code, so is only taken into account by CodePeer.

In addition, the contract-related pragmas Assert, Assume, Precondition, and Postcondition can be added in the source code, serving as manual checks that can aid or enhance CodePeer’s analysis, as documented in the section on User Checks. See the section Improve Your Code Specification for examples of using some of these pragmas to enhance error detection.