.. _How_To_View_GNAT_SAS_Output: *************** Viewing Results *************** GNAT SAS Report Command ======================= The ``gnatsas analyze`` command does not print any results. Instead, the ``gnatsas report`` command should be used, along with a third argument representing the desired output format. If no format is specified, ``text`` is used as default. The following switch can be used to save results to a specific file: ``--out FILE`` Output the results to FILE instead of the standard output. .. seealso:: See :ref:`CLI_Reference` for more information about available command-line switches. .. rubric:: Examples .. code-block:: sh $ gnatsas report -P prj.gpr will report results for the analysis of project :file:`prj.gpr` in text format to standard output. .. code-block:: sh $ gnatsas report csv -P prj.gpr --out output_file.csv will output results in CSV format to the file :file:`output_file.csv`. Supported Report Formats ------------------------ GNAT SAS supports multiple report formats: - ``text`` (see :ref:`Text_Output`) - ``security`` (see :ref:`Security_Report`) - ``html`` (see :ref:`HTML_Output`) - ``csv`` (see :ref:`CSV_Output`) - ``code-climate`` (see :ref:`CodeClimate`) - ``SARIF`` (see :ref:`SARIF`) - ``exit-code`` (see :ref:`Exit_Code`) - ``gnat-studio`` (see :ref:`Viewing_GNAT_SAS_Output_In_GNAT_Studio`) - ``gnathub`` The ``text``, ``security``, ``html`` formats are the main outputs formats, and are intended for "immediate use". The HTML format allows the user to browse the messages emitted by GNAT SAS, in a similar way as in GNAT Studio (see :ref:`Viewing_GNAT_SAS_Output_In_GNAT_Studio`). The ``csv`` format is suitable for importation in a spreadsheet, typically to generate statistics. The ``code-climate`` format can be used to print reports in a format that can be ingested by Gitlab. The ``sarif`` format should be used when running analyses for viewers that accept inputs in the `SARIF OASIS format. `_. The ``exit-code`` format has a very simple output: the exit code returned by this command will be the number of messages emitted, up to 255 (the restriction comes from a Linux restriction on the return code size). Its sole purpose is to be used in automation pipelines, e.g. to detect new messages. The ``gnat-studio`` format will be automatically used when running an analysis from GNAT Studio, and so will be the ``gnathub`` format. Those are internal formats, not meant to be required by users directly. .. seealso:: * More information about supported formats is available in :ref:`Report_Formats`. * See :ref:`GNAT_SAS_Workflows` for examples of using those formats in integration workflows. .. _Selecting run: Selecting the Results to Display -------------------------------- Running the default report command ``gnatsas report -P `` displays the results of the last ``gnatsas analyze -P `` run that was performed, regardless of the switches used for that run. In order to generate a report for another run, you may use any of the following options: * Specify a timeline, to display the results of the last run made in that timeline (see :ref:`Timeline`): .. code-block:: sh gnatsas report -P --timeline * Directly generate a report from a SAM file, with the command: .. code-block:: sh gnatsas report Note that you do not have to specify a project file in this case. As seen in :ref:`GNAT_SAS_Baseline`, you can add the ``--compare-with `` switch to compare the selected run with any other run. .. _Filtering_Messages: Filtering Messages ------------------ Messages can be filtered using the ``--show`` switch. The switch is fully documented in ``gnatsas report --help`` in the section `SHOW OPTION FORMATTING`, see also the section `SHOW OPTION DEFAULT VALUE` and `SHOW OPTION EXAMPLES`. We present here a few use cases: * ``--show all``, to display even the messages that would be hidden by default. * ``--show age=removed``, to display only the messages that disappeared since the previous analysis. * ``--show rank+low``, to add low messages to the messages displayed by default. * ``--show kind=annotation``, to display only Inspector's annotations. * ``--show kind=info``, to display only Inspector's info messages The basic format of the switch is either: * ``=`` to *restrict* the report to the messages for which `category` (e.g. `age`) satisfies `constraint` (e.g. `removed`). * ``+`` to *add* to the report the messages for which `category` (e.g. `rank`) satisfies `constraint` (e.g. `low`). * ``-`` to *remove* from the report the messages for which `category` (e.g. `kind`) satifies `constraint` (e.g. `precondition`). You can add multiple `constraint` to one category, as in ``--show age=added+unchanged`` which constrains the messages to be either added or unchanged (this is the default for text output). The full list of `categories` and their associated `constraints` is shown in ``gnatsas report --help``. Multiple such filters can be combined as in: .. code-block:: sh gnatsas report -P --show all,age=added,rank=high,review_status=none The command above makes GNAT SAS only report added high messages with no user review. Specifying ``--show`` multiple times performs a union of the results of each filter. For example, the following switch will only show Infer and high Inspector messages: .. code-block:: sh gnatsas report -P --show tool=inspector,rank=high --show tool=infer .. _Default_Filtering: Default filtering per report format ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Some messages are hidden by default in the output depending on the report format. This is because some formats are expected to be used in environments providing advanced filtering capabilities compared to others (e.g. it is easier to sort and filter messages in CSV in a spreadsheet software, compared to text output). .. note:: It is possible to deactivate the default filtering of any format (except for the security report) and print all messages, with the command ``--show all``. The applicable default filtering settings are described in the table below for all report formats. .. table:: :class: longtable :widths: 10 60 30 ============ ================================================== =================================== Format Hidden messages by default Displaying hidden messages ============ ================================================== =================================== Text All messages with any of the following conditions: * ``age`` is ``removed``, or * ``kind`` is ``annotation`` or ``info``, * ``review_kind`` is ``not_a_bug`` Hidden messages can be displayed back using the corresponding re-enabling filter: * ``--show=age+removed`` * ``--show=kind+info`` (or/and ``+annotation``) * ``--show=review_kind+not_a_bug`` or a combination of those. CodeClimate Same as text Exit Code Same as text Security Same as text. Additionally, only messages with associated potential security vulnerabilities (i.e., messages corresponding to a :ref:`CWE weakness`) are displayed. Among those, potential dynamic errors such as access checks which are mitigated by Ada's built-in run-time checks and exceptions are also hidden by default. Potential dynamic errors mitigated by Ada's built-in run-time checks and exceptions can be reported using the ``--show-mitigated-cwes`` switch. Other hidden messages cannot be displayed and other report formats should be used instead. CSV All messages with the following condition: * ``kind`` is ``annotation`` or ``info``, Hidden messages can be displayed back using the corresponding re-enabling filter: * ``--show=kind+info`` (or/and ``+annotation``) SARIF Same as CSV HTML All messages ============ ================================================== =================================== .. seealso:: See :ref:`Report_Formats` for detailed information about each report format. Filtering by message kind ^^^^^^^^^^^^^^^^^^^^^^^^^ When filtering messages by their `kinds` (see section :ref:`Messages_and_Annotations` for their list and descpritions) you need to replace any space with an underscore. For example, to reference the `conditional check` category, use `conditional_check`; similarly for `useless reassignment`, use `useless_reassignment`. In addition, you can use kinds `groups`. The groups of messages for ``--show kind=`` are: .. table:: :widths: 20 80 ================ ======================================================= **Kind** **Effect** `check` References all :ref:`Run-Time_Checks`, :ref:`User_Checks` and :ref:`Validity_Check` *(Specific to Inspector and Infer)* `warning` References all :ref:`Warning_Messages` *(all engines)* `race_condition` References all :ref:`Race_Condition_Messages` *(Inspector-specific)* `info` References all :ref:`Information_Messages` *(Inspector-specific)* `annotation` References all :ref:`Annotations` *(Inspector-specific)* ================ ======================================================= To tell GNAT SAS to only emit checks, except for `precondition` checks, use a dash character (interpreted as a minus): .. code-block:: sh --show kind=check-precondition .. _GNAT_SAS_Baseline: Comparing GNAT SAS Runs ======================= For each :ref:`analysis mode`, GNAT SAS keeps the results of a `baseline run`, set by default to the first run performed at the corresponding mode (refer to :ref:`Timeline` for more details). This allows GNAT SAS to specify, for each subsequent run, if a message is `new`, `unchanged` or `removed`, in comparison to the baseline. This behavior also extends to runs that apply to one specific file with the ``--file `` switch, or the ``--files-from `` switch. In both cases a specific baseline is stored so that successive runs made on a file are compared to a baseline made on the same file. The baseline stays the same from one run to another, unless the ``gnatsas baseline`` command is ran. This command accepts the following switches: ``--bump-baseline`` Makes the last run of the selected mode the new baseline for the following runs. ``--set-baseline `` Sets the baseline to `sam_file`, starting from the current run. GNAT Studio displays the name of the current run and its associated baseline at the top of the "GNAT SAS report" tab (see :ref:`Viewing_GNAT_SAS_Output_In_GNAT_Studio`). .. warning:: Registering a new baseline overrides any pre-existing baseline's SAM file. In case of a mistake, GNAT SAS makes one backup of the previous baseline before doing so. You can use the switch ``--compare-with`` of the ``gnatsas report`` command to temporarily test a potential baseline before setting it up (see below). If you want to save the previous baseline before setting a new one, you have to do so manually, see :ref:`GNAT_SAS_Files` to locate the file to backup. In addition, the user can temporarily ignore the baseline and do a one-off ``gnatsas report`` comparison with the switch ``--compare-with ``. ``--compare-with `` The current run is compared to the specified run, without impacting the baseline. In GNAT Studio you can bump the baseline with the menu :menuselection:`GNATSAS --> Baseline --> Bump Baseline to Current Run`. The baseline can be set to a specific file with the :menuselection:`GNATSAS --> Baseline --> Set Baseline to Run...`. See :ref:`Invocation_from_GNAT_Studio`. .. _Classifying_Messages_As_Unchanged_Added_Removed: Classifying Messages as Unchanged, Added, and Removed ----------------------------------------------------- In order to classify messages from the current run as ``unchanged``, ``added``, and ``removed`` relatively to the baseline run, for each message from the current run, GNAT SAS decides whether it matches an existing message in the baseline SAM file. GNAT SAS tries to match two messages even if the surrounding source code changes, but it can happen that GNAT SAS fails to do so and a message from the current run is marked as added even if there exists a corresponding message in the baseline run (``false added``). Indeed, GNAT SAS needs to be careful and not match an added message with an existing message which does not correspond to it (``false existing``) since such message would be "lost". To decide whether a message from the current run matches a message in existing one, the following information is used: * full name of the procedure in which the message is emitted * analysis engine emitting the message * kind of the message * selected parts of the message text depending on the message kind * if all the above matches multiple messages, GNAT SAS uses their order of appearance in the code. .. note:: By default, GNAT SAS reports messages with the following settings: * Only added and unchanged messages are reported. * Unchanged messages do not have a specific mention; whereas added messages are explicitly marked as such. In order to display removed messages, you can tell GNAT SAS to display them via the ``--show age+removed`` report switch. See ``gnatsas report --help`` for a complete list of supported ``--show`` values. Viewing GNAT SAS Output in IDEs =============================== In GNAT Studio -------------- GNAT SAS output can be displayed in a user-friendly report interface in the GNAT Studio IDE. The GNAT SAS Report Window lists the messages for a given run and enables interactive filtering and source navigation. Comparison between runs is also supported, amongst other features. .. seealso:: Refer to :ref:`Viewing_GNAT_SAS_Output_In_GNAT_Studio` for a complete guide. In Visual Studio Code --------------------- GNAT SAS output generated in SARIF format can be displayed by any SARIF-viewing capabilities tool, in particular in Visual Studio Code with SARIF dedicated extensions. .. seealso:: Refer to :ref:`Using_GNAT_SAS_in_VSCode` for more details. .. _Report_Formats: Report Formats in detail ======================== .. _Text_Output: Text Output ----------- You can get a compiler-like listing of messages generated by GNAT SAS with the ``gnatsas report text`` command (used by default when simply running ``gnatsas report`` without a format), e.g.: .. code-block:: sh $ gnatsas analyze -Pmy_project gnatcheck 24.0w (20230603) [...] Compiler [Ada] alias.adb $ gnatsas report -Pmy_project alias.adb:10:11: high: validity check (Inspector): Int is uninitialized here The messages are formatted such as: .. code-block:: text filename:line:column: rank: kind (analysis engine name): message contents The analysis engine name is one of: * ``Inspector`` * ``Infer`` * ``GNATcheck`` * ``-gnatw`` for GNAT Warnings, where ```` is the GNAT compiler switch corresponding to the warning message. Refer to :ref:`GNAT_Warnings_Messages` for more details. You can add more switches to affect text output, e.g.: ``--show-backtraces`` Show backtrace information ``--show-reviews`` Show review associated to messages .. seealso:: See :ref:`CLI_Reference` for more information about available command-line switches. .. _HTML_Output: HTML Output ----------- In order to generate HTML output, you need to use the ``gnatsas report html`` command after having run a ``gnatsas analyze`` command. For example, to perform both a GNAT SAS analysis and generate HTML, you can use: .. code-block:: sh $ gnatsas analyze -Pprj $ gnatsas report html -Pprj By default, the HTML report is generated under the :term:`output directory` as :file:`/html-report`. The location of that directory can be controlled with the ``--out`` or ``-o`` switch, e.g. ``gnatsas report html -Pprj --out my_report``. Once the report has been generated, you can visualize it by opening the file named :file:`index.html` in a web browser. For more details on the HTML interface, see the documentation of `GNATdashboard web interface `_. .. _CSV_Output: CSV Output ---------- You can use the ``gnatsas report csv`` command to generate messages in a *CSV* format, suitable for use by other tools such as spreadsheets. When the switch ``--out`` is used to specify an output file, the CSV output is emitted in that file. Otherwise, the report is printed on the standard output. For example: .. code-block:: sh $ gnatsas report csv -Pprj --out messages.csv will generate a file :file:`messages.csv` with the following contents: first a heading listing all columns: .. code-block:: none project,basename,path,subp,line,column,category,history,ranking,tool,message,cwe,kind,related_checks,key,key_seq,review_from_source,review_kind,review_date,review_status,review_author,review_text then one line for each message, with each field separated by a comma, e.g: .. code-block:: none prj.gpr,reprod.adb,reprod.adb,reprod.test,7,19,check,unchanged,high,Inspector,"overflow check fails here: requires I /= Integer_32'Last",,overflow check,,52218f775ac40790bcc25156dc7536d3,1,false,uncategorized,,,, .. rubric:: CSV format specification The available columns are listed in the table below. * Except for the ``review_*`` fields, all fields are emitted as output of the GNAT SAS report and are not meant to be edited by users when importing CSV files. * Regarding ``review_*`` fields: * They are emitted when GNAT SAS finds review data associated to a message, in which case the last review is displayed. * They can also be used to import user reviews as described in :ref:`Through_CSV_File`. .. table:: :class: longtable :widths: 20 80 +--------------------+------------------------------------------------------+ | Field name | Description | +====================+======================================================+ | project | GPR project name. | +--------------------+------------------------------------------------------+ | basename | Basename of the file the message is coming from. | +--------------------+------------------------------------------------------+ | path | Relative path of the file from the GPR project. | +--------------------+------------------------------------------------------+ | subp | Fully qualified name of the enclosing subprogram. | +--------------------+------------------------------------------------------+ | line | Line number of the message. | +--------------------+------------------------------------------------------+ | column | Column number of the message. | +--------------------+------------------------------------------------------+ | category | Category of the message, as listed in | | | :ref:`Messages_and_Annotations`. | +--------------------+------------------------------------------------------+ | history | *added* if message is new relative to the baseline, | | | *removed* if message has been removed relative to | | | the baseline, and *unchanged* otherwise, see also | | | :ref:`GNAT_SAS_Baseline`. | +--------------------+------------------------------------------------------+ | ranking | Ranking of the message (info, low, medium, | | | high, suppressed). | +--------------------+------------------------------------------------------+ | tool | Tool that emitted the message. | +--------------------+------------------------------------------------------+ | message | Text of the message, surrounded by double quotes. | +--------------------+------------------------------------------------------+ | cwe | List of relevant Common Weakness Enumeration ids | | | for the given message. See | | | :ref:`Messages_and_Annotations` more information | | | about CWE. | +--------------------+------------------------------------------------------+ | kind | Message kind (check, warning, info, annotation). | +--------------------+------------------------------------------------------+ | related_checks | Only relevant for Precondition_Checks. This is a | | | list of the checks that contributed to the | | | associated precondition (that is, checks that might | | | fail if the precondition is violated by a caller). | +--------------------+------------------------------------------------------+ | key | Internal field to uniquely identify a message. | +--------------------+------------------------------------------------------+ | key_seq | Related to Key to uniquely identify a message. | +--------------------+------------------------------------------------------+ | review_from_source | Whether the review message comes from the source. | +--------------------+------------------------------------------------------+ | review_kind | Kind of the review associated to the review status. | +--------------------+------------------------------------------------------+ | review_date | Date of the latest review, if relevant. | +--------------------+------------------------------------------------------+ | review_status | Status of the review (uncategorized, pending, | | | false_positive, not_a_bug, bug). | +--------------------+------------------------------------------------------+ | review_author | Name of the review author, if relevant. | +--------------------+------------------------------------------------------+ | review_text | Last review comment, if relevant. | +--------------------+------------------------------------------------------+ The statements below also apply to the CSV format supported by GNAT SAS. * CSV files are exported and imported in the UTF-8 encoding. * The separator is always expected to be a comma ``,``, regardless of the locale. * By default, double-quotes surrounding field values are optional. Unless the value contains a double-quote, in which case: * the value may be surrounded by double-quotes and the inner double-quotes escaped by doubling them. E.g. ``"Some value with ""inner double-quoted text""."``, or * the value may be surrounded by single-quotes and the inner double-quotes do not need escaping. E.g. ``'Some value with "inner double-quoted text".'``. .. _Security_Report: Security Report --------------- GNAT SAS can emit a security-oriented analysis report in HTML format via the ``--security-report`` switch, which can be combined with other switches. For example: .. code-block:: sh $ gnatsas report security -Pprj This command generates the file :file:`security-report.html` in the GNAT SAS :term:`output directory`. The location of the report can be controlled with the ``--out`` switch. The security report contains the following potential vulnerabilities (see :ref:`Messages_and_Annotations` for more details): * unprotected access * unprotected shared access * mismatched protected access * validity check * dead code * test always false * test always true * unused assignment * unused out parameter * useless reassignment * loop does not complete normally Potential dynamic errors such as access checks are mitigated by Ada's built-in run-time checks and exception and are not reported by default. They can be shown as well using the ``--show-mitigated-cwes`` switch. This HTML file contains the following sections: #. Potential Vulnerabilities Detected by GNAT SAS #. Security Vulnerabilities Not Present #. List of Ada Source Files Analyzed by GNAT SAS A :download:`sample security report <_static/gnatsas-security-report.html>` is available here or in the directory :file:`share/examples/gnatsas/gnat_sas_by_example` of your installation. .. _CodeClimate: CodeClimate Export ------------------ GNAT SAS can emit a report in the Code Climate format with the ``gnatsas report code-climate`` switch. The Code Climate format enables integrating GNAT SAS' output with various tools such as BitBucket, Github or Gitlab. The paths in the Code Climate report will be relative to the analyzed GPR file by default. You can make the paths relative to any other directory by using the ``--root`` switch. For example, .. code-block:: sh $ gnatsas report code-climate -Pprj --root . Will make all paths relative to the directory GNAT SAS is being launched from and .. code-block:: sh $ gnatsas report code-climate -Pprj --root / Will make all paths relative to the root of the filesystem. .. Coverity Export =============== GNAT SAS can emit a report in the input format of the Synopsis Coverity tool with the ``gnatsas report coverity`` command. The Coverity format enables integrating GNAT SAS' output with Coverity Connect. .. code-block:: sh $ gnatsas report coverity -Pprj .. _SARIF: SARIF Export ------------ GNAT SAS can emit a report in the SARIF format with the ``gnatsas report sarif`` command. The SARIF format enables integrating GNAT SAS's output with any SARIF viewer tool. In particular, it is the format used for :ref:`GNAT SAS integration with Visual Studio Code `. .. code-block:: sh $ gnatsas report sarif -Pprj By default, the URIs in file locations in the generated SARIF file will be relative to the project directory. The directory to which those URIs should be relative can be controlled using the ``--root DIR`` switch, to ensure portability of the generated report. .. seealso:: Refer to :ref:`CLI_Reference` for more information about command-line switches. .. _Exit_Code: Exit Code --------- The ``exit-code`` format has a very simple output: the exit code returned by this command will be the number of messages emitted, up to 255 (the restriction comes from a Linux restriction on the return code size). Its sole purpose is to be used in automation pipelines, e.g. to detect new messages. .. code-block:: sh $ gnatsas analyze -Pmy_project [...] $ gnatsas report exit-code -Pmy_project $ echo $? 6 indicates that six messages were reported by GNAT SAS.