5. Viewing Results

5.1. GNAT SAS Report Command

The gnatsas analyze command does not print any results. Instead, the gnatsas report command should be used, along with a third argument representing the desired output format. If no format is specified, text is used as default.

The following switch can be used to save results to a specific file:

--out FILE

Output the results to FILE instead of the standard output.

See also

See GNAT SAS CLI Reference for more information about available command-line switches.

Examples

$ gnatsas report -P prj.gpr

will report results for the analysis of project prj.gpr in text format to standard output.

$ gnatsas report csv -P prj.gpr --out output_file.csv

will output results in CSV format to the file output_file.csv.

5.1.1. Supported Report Formats

GNAT SAS supports multiple report formats:

The text, security, html formats are the main outputs formats, and are intended for "immediate use". The HTML format allows the user to browse the messages emitted by GNAT SAS, in a similar way as in GNAT Studio (see Viewing GNAT SAS Output in GNAT Studio).

The csv format is suitable for importation in a spreadsheet, typically to generate statistics. The code-climate format can be used to print reports in a format that can be ingested by Gitlab. The sarif format should be used when running analyses for viewers that accept inputs in the SARIF OASIS format..

The exit-code format has a very simple output: the exit code returned by this command will be the number of messages emitted, up to 255 (the restriction comes from a Linux restriction on the return code size). Its sole purpose is to be used in automation pipelines, e.g. to detect new messages.

The gnat-studio format will be automatically used when running an analysis from GNAT Studio, and so will be the gnathub format. Those are internal formats, not meant to be required by users directly.

See also

5.1.2. Selecting the Results to Display

Running the default report command gnatsas report <format> -P <prj> displays the results of the last gnatsas analyze -P <prj> run that was performed, regardless of the switches used for that run.

In order to generate a report for another run, you may use any of the following options:

  • Specify a timeline, to display the results of the last run made in that timeline (see Timelines):

    gnatsas report <format> -P <prj> --timeline <timeline>

  • Directly generate a report from a SAM file, with the command:

    gnatsas report <format> <sam-file>

    Note that you do not have to specify a project file in this case.

As seen in Comparing GNAT SAS Runs, you can add the --compare-with <sam-file> switch to compare the selected run with any other run.

5.1.3. Filtering Messages

Messages can be filtered using the --show switch. The switch is fully documented in gnatsas report --help in the section SHOW OPTION FORMATTING, see also the section SHOW OPTION DEFAULT VALUE and SHOW OPTION EXAMPLES. We present here a few use cases:

  • --show all, to display even the messages that would be hidden by default.

  • --show age=removed, to display only the messages that disappeared since the previous analysis.

  • --show rank+low, to add low messages to the messages displayed by default.

  • --show kind=annotation, to display only Inspector's annotations.

  • --show kind=info, to display only Inspector's info messages

The basic format of the switch is either:

  • <category>=<constraint> to restrict the report to the messages for which category (e.g. age) satisfies constraint (e.g. removed).

  • <category>+<constraint> to add to the report the messages for which category (e.g. rank) satisfies constraint (e.g. low).

  • <category>-<constraint> to remove from the report the messages for which category (e.g. kind) satifies constraint (e.g. precondition).

You can add multiple constraint to one category, as in --show age=added+unchanged which constrains the messages to be either added or unchanged (this is the default for text output).

The full list of categories and their associated constraints is shown in gnatsas report --help.

Multiple such filters can be combined as in:

gnatsas report -P<prj> --show all,age=added,rank=high,review_status=none

The command above makes GNAT SAS only report added high messages with no user review.

Specifying --show multiple times performs a union of the results of each filter. For example, the following switch will only show Infer and high Inspector messages:

gnatsas report -P<prj> --show tool=inspector,rank=high --show tool=infer

5.1.3.1. Default filtering per report format

Some messages are hidden by default in the output depending on the report format. This is because some formats are expected to be used in environments providing advanced filtering capabilities compared to others (e.g. it is easier to sort and filter messages in CSV in a spreadsheet software, compared to text output).

Note

It is possible to deactivate the default filtering of any format (except for the security report) and print all messages, with the command --show all.

The applicable default filtering settings are described in the table below for all report formats.

Format

Hidden messages by default

Displaying hidden messages

Text

All messages with any of the following conditions:

  • age is removed, or

  • kind is annotation or info,

  • review_kind is not_a_bug

Hidden messages can be displayed back using the corresponding re-enabling filter:

  • --show=age+removed

  • --show=kind+info (or/and +annotation)

  • --show=review_kind+not_a_bug

or a combination of those.

CodeClimate

Same as text

Exit Code

Same as text

Security

Same as text. Additionally, only messages with associated potential security vulnerabilities (i.e., messages corresponding to a CWE weakness) are displayed. Among those, potential dynamic errors such as access checks which are mitigated by Ada's built-in run-time checks and exceptions are also hidden by default.

Potential dynamic errors mitigated by Ada's built-in run-time checks and exceptions can be reported using the --show-mitigated-cwes switch. Other hidden messages cannot be displayed and other report formats should be used instead.

CSV

All messages with the following condition:

  • kind is annotation or info,

Hidden messages can be displayed back using the corresponding re-enabling filter:

  • --show=kind+info (or/and +annotation)

SARIF

Same as CSV

HTML

All messages

See also

See Report Formats in detail for detailed information about each report format.

5.1.3.2. Filtering by message kind

When filtering messages by their kinds (see section GNAT SAS Messages Reference for their list and descpritions) you need to replace any space with an underscore. For example, to reference the conditional check category, use conditional_check; similarly for useless reassignment, use useless_reassignment.

In addition, you can use kinds groups. The groups of messages for --show kind= are:

Kind

Effect

check

References all Run-Time Checks, User Checks and Uninitialized and Invalid Variables (Specific to Inspector and Infer)

warning

References all Warning Messages (all engines)

race_condition

References all Race Condition Messages (Inspector-specific)

info

References all Information Messages (Inspector-specific)

annotation

References all Annotations (Inspector-specific)

To tell GNAT SAS to only emit checks, except for precondition checks, use a dash character (interpreted as a minus):

--show kind=check-precondition

5.2. Comparing GNAT SAS Runs

For each analysis mode, GNAT SAS keeps the results of a baseline run, set by default to the first run performed at the corresponding mode (refer to Timelines for more details). This allows GNAT SAS to specify, for each subsequent run, if a message is new, unchanged or removed, in comparison to the baseline.

This behavior also extends to runs that apply to one specific file with the --file <file> switch, or the --files-from <file> switch. In both cases a specific baseline is stored so that successive runs made on a file are compared to a baseline made on the same file.

The baseline stays the same from one run to another, unless the gnatsas baseline command is ran. This command accepts the following switches:

--bump-baseline

Makes the last run of the selected mode the new baseline for the following runs.

--set-baseline <sam_file>

Sets the baseline to sam_file, starting from the current run. GNAT Studio displays the name of the current run and its associated baseline at the top of the "GNAT SAS report" tab (see Viewing GNAT SAS Output in GNAT Studio).

Warning

Registering a new baseline overrides any pre-existing baseline's SAM file. In case of a mistake, GNAT SAS makes one backup of the previous baseline before doing so.

You can use the switch --compare-with of the gnatsas report command to temporarily test a potential baseline before setting it up (see below). If you want to save the previous baseline before setting a new one, you have to do so manually, see GNAT SAS Files Reference to locate the file to backup.

In addition, the user can temporarily ignore the baseline and do a one-off gnatsas report comparison with the switch --compare-with <sam_file>.

--compare-with <sam_file>

The current run is compared to the specified run, without impacting the baseline.

In GNAT Studio you can bump the baseline with the menu GNATSAS ‣ Baseline ‣ Bump Baseline to Current Run. The baseline can be set to a specific file with the GNATSAS ‣ Baseline ‣ Set Baseline to Run.... See GNAT SAS menu.

5.2.1. Classifying Messages as Unchanged, Added, and Removed

In order to classify messages from the current run as unchanged, added, and removed relatively to the baseline run, for each message from the current run, GNAT SAS decides whether it matches an existing message in the baseline SAM file. GNAT SAS tries to match two messages even if the surrounding source code changes, but it can happen that GNAT SAS fails to do so and a message from the current run is marked as added even if there exists a corresponding message in the baseline run (false added). Indeed, GNAT SAS needs to be careful and not match an added message with an existing message which does not correspond to it (false existing) since such message would be "lost".

To decide whether a message from the current run matches a message in existing one, the following information is used:

  • full name of the procedure in which the message is emitted

  • analysis engine emitting the message

  • kind of the message

  • selected parts of the message text depending on the message kind

  • if all the above matches multiple messages, GNAT SAS uses their order of appearance in the code.

Note

By default, GNAT SAS reports messages with the following settings:

  • Only added and unchanged messages are reported.

  • Unchanged messages do not have a specific mention; whereas added messages are explicitly marked as such.

In order to display removed messages, you can tell GNAT SAS to display them via the --show age+removed report switch. See gnatsas report --help for a complete list of supported --show values.

5.3. Viewing GNAT SAS Output in IDEs

5.3.1. In GNAT Studio

GNAT SAS output can be displayed in a user-friendly report interface in the GNAT Studio IDE. The GNAT SAS Report Window lists the messages for a given run and enables interactive filtering and source navigation. Comparison between runs is also supported, amongst other features.

See also

Refer to Viewing GNAT SAS Output in GNAT Studio for a complete guide.

5.3.2. In Visual Studio Code

GNAT SAS output generated in SARIF format can be displayed by any SARIF-viewing capabilities tool, in particular in Visual Studio Code with SARIF dedicated extensions.

See also

Refer to Using GNAT SAS in Visual Studio Code for more details.

5.4. Report Formats in detail

5.4.1. Text Output

You can get a compiler-like listing of messages generated by GNAT SAS with the gnatsas report text command (used by default when simply running gnatsas report without a format), e.g.:

$ gnatsas analyze -Pmy_project
gnatcheck 24.0w (20230603)
[...]
Compiler
   [Ada]          alias.adb
$ gnatsas report -Pmy_project
alias.adb:10:11: high: validity check (Inspector): Int is uninitialized here

The messages are formatted such as:

filename:line:column: rank: kind (analysis engine name): message contents

The analysis engine name is one of:

  • Inspector

  • Infer

  • GNATcheck

  • -gnatw<X> for GNAT Warnings, where <X> is the GNAT compiler switch corresponding to the warning message. Refer to GNAT Warnings Messages for more details.

You can add more switches to affect text output, e.g.:

--show-backtraces

Show backtrace information

--show-reviews

Show review associated to messages

See also

See GNAT SAS CLI Reference for more information about available command-line switches.

5.4.2. HTML Output

In order to generate HTML output, you need to use the gnatsas report html command after having run a gnatsas analyze command.

For example, to perform both a GNAT SAS analysis and generate HTML, you can use:

$ gnatsas analyze -Pprj
$ gnatsas report html -Pprj

For historical reasons, the HTML report is generated in a directory named gnathub/html-report inside of the project's object directory. However, the HTML report is not generated by the GNAThub tool anymore, and the output directory can be controlled with the --out or -o switch, e.g. gnatsas report html -Pprj --out my_report.

Once the report has been generated, you can visualize it by opening the file named index.html in a web browser.

For more details on the HTML interface, see the documentation of GNATdashboard web interface.

5.4.3. CSV Output

You can use the gnatsas report csv command to generate messages in a CSV format, suitable for use by other tools such as spreadsheets.

When the switch --out is used to specify an output file, the CSV output is emitted in that file. Otherwise, the report is printed on the standard output.

For example:

$ gnatsas report csv -Pprj --out messages.csv

will generate a file messages.csv with the following contents: first a heading listing all columns:

project,basename,path,subp,line,column,category,history,ranking,tool,message,cwe,kind,related_checks,key,key_seq,review_from_source,review_kind,review_date,review_status,review_author,review_text

then one line for each message, with each field separated by a comma, e.g:

prj.gpr,reprod.adb,reprod.adb,reprod.test,7,19,check,unchanged,high,Inspector,"overflow check fails here: requires I /= Integer_32'Last",,overflow check,,52218f775ac40790bcc25156dc7536d3,1,false,uncategorized,,,,

CSV format specification

The available columns are listed in the table below.

  • Except for the review_* fields, all fields are emitted as output of the GNAT SAS report and are not meant to be edited by users when importing CSV files.

  • Regarding review_* fields:

    • They are emitted when GNAT SAS finds review data associated to a message, in which case the last review is displayed.

    • They can also be used to import user reviews as described in Through a CSV File.

Field name

Description

project

GPR project name.

basename

Basename of the file the message is coming from.

path

Relative path of the file from the GPR project.

subp

Fully qualified name of the enclosing subprogram.

line

Line number of the message.

column

Column number of the message.

category

Category of the message, as listed in GNAT SAS Messages Reference.

history

added if message is new relative to the baseline, removed if message has been removed relative to the baseline, and unchanged otherwise, see also Comparing GNAT SAS Runs.

ranking

Ranking of the message (info, low, medium, high, suppressed).

tool

Tool that emitted the message.

message

Text of the message, surrounded by double quotes.

cwe

List of relevant Common Weakness Enumeration ids for the given message. See GNAT SAS Messages Reference more information about CWE.

kind

Message kind (check, warning, info, annotation).

related_checks

Only relevant for Precondition_Checks. This is a list of the checks that contributed to the associated precondition (that is, checks that might fail if the precondition is violated by a caller).

key

Internal field to uniquely identify a message.

key_seq

Related to Key to uniquely identify a message.

review_from_source

Whether the review message comes from the source.

review_kind

Kind of the review associated to the review status.

review_date

Date of the latest review, if relevant.

review_status

Status of the review (uncategorized, pending, false_positive, not_a_bug, bug).

review_author

Name of the review author, if relevant.

review_text

Last review comment, if relevant.

The statements below also apply to the CSV format supported by GNAT SAS.

  • CSV files are exported and imported in the UTF-8 encoding.

  • The separator is always expected to be a comma ,, regardless of the locale.

  • By default, double-quotes surrounding field values are optional. Unless the value contains a double-quote, in which case:

    • the value may be surrounded by double-quotes and the inner double-quotes escaped by doubling them. E.g. "Some value with ""inner double-quoted text"".", or

    • the value may be surrounded by single-quotes and the inner double-quotes do not need escaping. E.g. 'Some value with "inner double-quoted text".'.

5.4.4. Security Report

GNAT SAS can emit a security-oriented analysis report in HTML format via the --security-report switch, which can be combined with other switches. For example:

$ gnatsas report security -Pprj

The security report contains the following potential vulnerabilities (see GNAT SAS Messages Reference for more details):

  • unprotected access

  • unprotected shared access

  • mismatched protected access

  • validity check

  • dead code

  • test always false

  • test always true

  • unused assignment

  • unused out parameter

  • useless reassignment

  • loop does not complete normally

The location of the generated report can be controlled with the --out switch. Potential dynamic errors such as access checks are mitigated by Ada's built-in run-time checks and exception and are not reported by default. They can be shown as well using the --show-mitigated-cwes switch.

This HTML file contains the following sections:

  1. Potential Vulnerabilities Detected by GNAT SAS

  2. Security Vulnerabilities Not Present

  3. List of Ada Source Files Analyzed by GNAT SAS

A sample security report is available here or in the directory share/examples/gnatsas/gnat_sas_by_example of your installation.

5.4.5. CodeClimate Export

GNAT SAS can emit a report in the Code Climate format with the gnatsas report code-climate switch. The Code Climate format enables integrating GNAT SAS' output with various tools such as BitBucket, Github or Gitlab. The paths in the Code Climate report will be relative to the analyzed GPR file by default. You can make the paths relative to any other directory by using the --root switch. For example,

$ gnatsas report code-climate -Pprj --root .

Will make all paths relative to the directory GNAT SAS is being launched from and

$ gnatsas report code-climate -Pprj --root /

Will make all paths relative to the root of the filesystem.

5.4.6. SARIF Export

GNAT SAS can emit a report in the SARIF format with the gnatsas report sarif command. The SARIF format enables integrating GNAT SAS's output with any SARIF viewer tool. In particular, it is the format used for GNAT SAS integration with Visual Studio Code.

$ gnatsas report sarif -Pprj

By default, the URIs in file locations in the generated SARIF file will be relative to the project directory. The directory to which those URIs should be relative can be controlled using the --root DIR switch, to ensure portability of the generated report.

See also

Refer to GNAT SAS CLI Reference for more information about command-line switches.

5.4.7. Exit Code

The exit-code format has a very simple output: the exit code returned by this command will be the number of messages emitted, up to 255 (the restriction comes from a Linux restriction on the return code size). Its sole purpose is to be used in automation pipelines, e.g. to detect new messages.

$ gnatsas analyze -Pmy_project
[...]
$ gnatsas report exit-code -Pmy_project
$ echo $?
6

indicates that six messages were reported by GNAT SAS.