7. Workflows

There are many different ways to take advantage of GNAT SAS capabilities, depending on the stage of the project's lifecycle. During development phases, GNAT SAS can either be run every day or as often as there is a build of some part of the system. Using GNAT SAS to find defects before a project enters run time testing minimizes the number of crashes encountered during testing, allowing functional tests to run to completion and thereby providing more usable output for test engineers. On existing code being maintained, it can be used to find potential problems that are not revealed by testing and which can be extremely difficult to track under deployment. It can also be used to perform a change impact analysis. In high-integrity and certification-related environment, it can be used to demonstrate the absence of certain kind of errors or improve the quality of code review activities.

In all cases, the source code should not be shared directly (say, on a shared drive) between developers, as this is bound to cause problems with file access rights and concurrent accesses. Rather, the typical usage is for each user to do a check-out of the sources/environment, and use therefore their own version/copy of sources and project files, instead of physically sharing sources across all users.

This section will describe in detail various ways to put GNAT SAS in production and give it to the hand of all team members, or only a few selected ones.

Note

In many workflows below where GNAT SAS is to be run as part of a script, you might want to use the --quiet switch to suppress info output.

7.1. Use on a developer machine

This is the most basic usage: each developer runs GNAT SAS on their own machine, in fast mode (see Analysis modes). Reviews are versioned along the sources, and are updated by developers when new messages appear.

Thanks to fast mode's incrementality, the developer's machine is not required to be overly powerful (once an initial analysis has been performed).

7.2. Nightly Runs on a Server

In this workflow, GNAT SAS is run nightly on a dedicated server with lots of resources available (e.g. 16 cores as per System Requirements), in deep mode (set with the switch --mode=deep).

These runs will typically be run nightly to take into account all commits of the day, and provide results to users the next morning.

The next day, developers can analyze the results, either by checking them out if they are versioned, or by retrieving them as described in Importing GNAT SAS Results.

Developers may then review the messages and depending on the outcome, decide to fix the code or justify the relevant messages, directly in the code or using a review file to be committed. Refer to Reviewing Messages for the message review methods, and to Importing and Sharing GNAT SAS User Reviews for managing reviews from multiple users.

7.3. Continuous Runs on a Server after Each Change

In this workflow, GNAT SAS is run on a dedicated server with lots of resources available (e.g. 16 cores as per System Requirements) for performing runs sufficiently rapidly as to provide quick results to users after each commit. The idea of these runs is not to be exhaustive, but to focus on the differences from the previous run.

These continuous runs will trigger on a new repository change, typically integrated with a continuous integration framework such as Jenkins.

At the end of a run, a summary is sent to developers via email or a web interface, or even directly through GNAT SAS' report formats that are compatible with forges (e.g. SARIF, or Code Climate for Gitlab).

Once a first analysis has been performed, it is possible to filter the report of the subsequent analyses in a way that only new messages are shown, for example with the following command:

$ gnatsas report text -Pprj --show age=added

Developers may then review the messages and depending on the outcome, decide to fix the code or justify the relevant messages, directly in the code or using a review file to be committed. Refer to Reviewing Messages for the message review methods, and to Importing and Sharing GNAT SAS User Reviews for managing reviews from multiple users.

7.4. Importing GNAT SAS Results

As stated in Message Files, GNAT SAS stores the results of a run in a SAM file, in the output directory. Depending on the use case, you may only have to retrieve this SAM file, or you may need additional elements. We describe here a few use cases:

Assuming all project dependencies are available, simply use:
```
gnatsas report <format> -P <project>
```
When all dependencies required in the GPR project are not available in the environment, the above command may fail with an explicit error message. It is still possible to generate a report from a SAM file without specifying the project file by running the command below.
```
gnatsas report <format> <sam-file>
```
Warning
- The <format> argument is mandatory.
- Only the following formats are supported: text, csv, exit-code, security, gnathub. Other formats require an explicit -P <project> switch.
In order to use the results of a distant run as a baseline, or to compare local runs to it, retrieve the distant SAM file and use it as argument to, either
```
gnatsas baseline -P<prj> --set-baseline <sam-file>
```
or
```
gnatsas report -P<prj> --compare-with <sam-file>
```
You can specify --mode <mode> on both commands to change the affected mode (see Analysis modes).
In order to handle a distant analysis as if it was run locally, retrieve the distant SAM file and use it as argument to
```
gnatsas baseline -P<prj> --set-current <sam-file>
```
This way, the imported run is set as the current run (specify --mode <mode> to change the affected mode). The local baseline and reviews are applied. You can then display the results with, e.g. gnatsas report -P<prj>, add new reviews, etc…
In order to duplicate the whole state of a distant machine (comprising all analysis, baselines and reviews), replace your local output directory and review file with the distant ones. Note that you then lose your local state.

Note

You can identify the SAM file of a run with:

gnatsas report -P<prj> --show-header

7.6. Using GNAT SAS with a Jenkins Automation Server

The recommended way of integrating GNAT SAS with the Jenkins Automation Server is using a pipeline script. The script can be source controlled alongside the code, or saved in a pipeline project's configuration.

7.6.1. Jenkins Agent Setup

To set-up a Jenkins Agent with the capability to run a GNAT SAS analysis follow the installation instructions corresponding to the agent's operating system; the instructions for installing GNAT Studio can be omitted.

7.6.2. Running the Analysis

To run the analysis add a gnatsas-analyze stage to your project's pipeline after any stages which make the code available on the agent's filesystem (the checkout stage in the Example Pipeline script).

Within this stage add a step that executes a host shell command invoking gnatsas analyze with the desired command line switches.

7.6.3. Getting Analysis Results

To retrieve the results of a GNAT SAS run outside of the Jenkins log console, you will need to setup a workflow following section Importing GNAT SAS Results.

You can add a step to the following pipeline that stores the SAM/SAR file(s) in a specific location where you can easily download it.

7.6.4. Example Pipeline script

An example of pipeline script which accesses the internet to clone a git repository from GitHub and run a GNAT SAS analysis on the checkout is:

pipeline {
   agent any
   stages {
      stage('checkout') {
         steps {
            url: 'https://github.com/...'
         }
      }
      stage('gnatsas-analyze') {
         steps {
            sh 'gnatsas analyze -P my_project.gpr'
         }
      }
   }
}

7.6.5. Warnings Next Generation usage

Warnings Next Generation is a Jenkins plugin that displays analysis results within Jenkins itself. See https://plugins.Jenkins.io/warnings-ng/ for more details on this plugin. This plugin supports SARIF, thus you can emit the messages in this format to import them in warnings ng.

Here is what the pipeline would look like using this plugin to publish the results.

pipeline {
   agent any
   stages {
      stage('checkout') {
         steps {
            url: 'https://github.com/...'
         }
      }
      stage('gnatsas-analyze') {
         steps {
            sh 'gnatsas analyze -P my_project.gpr'
         }
      }
   }
   post {
      always {
         sh 'gnatsas report sarif -P my_project.gpr -o output.sarif --show age-removed'
         recordIssues (
             enabledForFailure: true,
             tool: sarif(pattern: 'output.sarif')
         )
      }
   }
}

7.7. Easier Review of SAM files stored in Git

In order to make reviewing the SAM files you commit in your Git repositories easier, you can configure Git to use GNAT SAS to display SAM files as a textual report.

First, in your Git repository, define a Git textconv option for SAM files, like this:

git config --local diff.sam.textconv 'gnatsas report text'

Then, create a .gitattributes file whose content is as follows:

*.sam diff=sam

This will result in Git displaying diffs on SAM files as a textual format:

diff --git a/results.sam b/results.sam
--- a/results.sam
+++ b/results.sam
@@ -1,3 +0,0 @@
-file.adb:9:6: medium warning: unused assignment [CWE 563] (Inspector): into X
+file.adb:9:11: high: overflow check [CWE 190] (Inspector): fails here: requires X > 0

Instead of a JSON file.

Note

The displayed diff will use the default filtering of the text format. This can be changed using the --show option as described in Filtering Messages.

7.8. Using GNAT SAS in a GitLab Pipeline

The recommended way of integrating GNAT SAS in a GitLab Pipeline is using a pipeline script. The script can be source controlled alongside the code, or saved in a pipeline project's configuration.

7.8.1. Pre-requisites

In order to enable running GNAT SAS in a CI, one requirement is to make it available to the GitLab Pipeline. To do so, one simple approach is to embed it in a Docker image. This is not detailed here and the next section assumes that such an image has been created and published to your GitLab's Containing Registry, and is available as $REGISTRY_URL/$PROJECT_PATH/gnatsas:24.0w.

7.8.2. Running the Analysis

To run the analysis, simply run the gnatsas analyze command on the project's GPR file:

script:
   - gnatsas analyze -P my_project.gpr

7.8.3. Getting Analysis Results

To display the results of the analysis in text format in the CI log, run the following command:

script:
   - gnatsas report -P my_project.gpr

Furthermore, GNAT SAS provides the CodeClimate Export report format which enables displaying results in the GitLab UI. One needs to declare the report's path (here, code_quality_report.json) so that GitLab takes it into account:

script:
   - gnatsas
      report
      code-climate
      -P my_project.gpr
      --out $CI_PROJECT_DIR/gnatsas/code_quality_report.json
      --root $CI_PROJECT_DIR
      --long-desc

artifacts:
   - paths: gnatsas/
reports:
   codequality: gnatsas/code_quality_report.json

Note the --long-desc switch specified to the CodeClimate report format. This switch generates messages with a description that contains more information than usual, such as check name, analysis engine, CWE... as such information is not displayed by default in the CodeQuality widget integrated in GitLab.

7.8.4. Example Pipeline script

Here is a full example combining the steps described above. The script below uses a Docker image with GNAT SAS available to run a GNAT SAS analysis and report the results in the "Code Quality" tab of GitLab's interface.

This script may be committed to the root of the project's repository as .gitlab-ci.yml, for example.

default:
   image: $REGISTRY_URL/$PROJECT_PATH/gnatsas:24.0w

GNATSAS:
   script:
      - gnatsas analyze -P my_project.gpr
      - gnatsas report -P my_project.gpr
      - gnatsas
            report
            code-climate
            -P my_project.gpr
            --out $CI_PROJECT_DIR/gnatsas/code_quality_report.json
            --root $CI_PROJECT_DIR
            --long-desc

   artifacts:
      - paths: gnatsas/
   reports:
      codequality: gnatsas/code_quality_report.json

7.8.5. Advanced integration

7.8.5.1. Effective Code Quality report comparison

GitLab Code Quality engine provides its own report comparison mechanism. The report in CodeClimate format generated for the source branch of a Merge Request (MR) is automatically compared to the report of the target branch, when it exists.

Filters can still be used (see Filtering Messages) to customize the messages that are reported. Note that the default filtering for CodeClimate format is that removed messages and messages reviewed as "Not a bug" are not displayed.

Therefore, our recommendation is to configure GNAT SAS to report all messages found for the source branch. In practice, this means that

Review files should be committed so "Not a bug" messages are not reported
it is not necessary to commit the SAM files for comparison; but you may do so to version them and keep a history, and bump baselines when appropriate.

7.8.5.2. Incremental analysis

To benefit from incremental analysis and speed up CI time, we recommend to use caching capabilities in GitLab runners. There are a few things to consider when setting up the project:

GNAT SAS should be run in fast mode (see Analysis modes).
Default recommended switches for gnatsas analyze are:
- use -j0
- use --incrementality-method=minimal. This switch will not trigger re-compilation if sources timestamps differ but checksums match.
- If any pragma configuration files are used to compile the project (usually .adc files passed with the -gnatec= switch, or with the Global_Configuration_Pragmas of the Builder package), ensure that the switch -gnateb is specified as well to all projects with:
  for Global_Compilation_Switches ("Ada") use ("-gnateb");
  This way, those files are not recompiled when only their timestamps change.

The next step is to prepare for using caching in the GitLab CIs.

The gnatsas directory under the object directory (defined with the Object_Dir switch in the GPR project file), or the object directory directly, should be cached.
Importantly, as described in Effective Code Quality report comparison, the cache should not contain SAM files, otherwise they would be used as analysis baselines and reported messages may not be the ones expected for the branch in some cases if the cache is outdated. We thus recommend to define an output directory Output_Dir (see Analyzer package attributes) outside of the cached directories.
The caching policy is up to you; for example, a pull-push policy can be used in MRs and after merging, with a fallback key allowing the GitLab runner to restore cache from the target branch when cache does not exist yet for the source branch.
Finally, the CI job must do the following:
- restore cache (automatically done by GitLab)
- move/copy cache to the expected location with respect to the project (if locations differ)
- analyze the project and generate a report in CodeClimate format with GNAT SAS
- copy back the updated cached directories to the cache location (if locations differ)
- save cache (automatically done by GitLab)
- save artifacts: at least the report. More artifacts can be saved e.g. for debugging.
- tell GitLab to use the report as a codequality report.

7.8.5.3. Aggregating multiple reports

It is possible to analyze multiple projects and aggregate their reports in a single pipeline. However, there is a GitLab limitation that only one codequality report can be defined for a given job. Therefore, it is necessary to define a job per report and GitLab will automatically aggregate them.

For example, one could define a single gnatsas job analyzing two projects, and two parallel report jobs reporting each codequality report:

# Define a gnatsas CI job doing the analysis
gnatsas:
   # ...
   stage: gnatsas
   script:
      - gnatsas analyze -P my_project.gpr
      - gnatsas
          report
          code-climate
          -P my_project.gpr
          --out $CI_PROJECT_DIR/gnatsas/my_project.json
          --root $CI_PROJECT_DIR
          --long-desc

      - gnatsas analyze -P another_project.gpr
      - gnatsas
          report
          code-climate
          -P another_project.gpr
          --out $CI_PROJECT_DIR/gnatsas/another_project.json
          --root $CI_PROJECT_DIR
          --long-desc

   artifacts:
      - paths: gnatsas/

# Only one code-quality report per job can be uploaded, so use a matrix of jobs
# to report each report. Artifacts from previous job are available in subsequent
# ones.
gnatsas-report:
   stage: gnatsas
   interruptible: true
   needs:
      - job: gnatsas
         optional: true
   rules:
      ...
   parallel:
      matrix:
         - REPORT_PATH:
            [
               my_project.json,
               other_project.json
            ]
   script:
      # A non-empty script is required, otherwise artifacts is not recognized as a key
      - ls .
   artifacts:
      when: always
      reports:
         codequality: $CI_PROJECT_DIR/gnatsas/$REPORT_PATH

Alternatively, one could define two different analysis jobs, each analyzing a single project and outputting its report.

7. Workflows

7.1. Use on a developer machine

7.2. Nightly Runs on a Server

7.3. Continuous Runs on a Server after Each Change

7.4. Importing GNAT SAS Results

7.5. Importing and Sharing GNAT SAS User Reviews

7.6. Using GNAT SAS with a Jenkins Automation Server

7.6.1. Jenkins Agent Setup

7.6.2. Running the Analysis

7.6.3. Getting Analysis Results

7.6.4. Example Pipeline script

7.6.5. Warnings Next Generation usage

7.7. Easier Review of SAM files stored in Git

7.8. Using GNAT SAS in a GitLab Pipeline

7.8.1. Pre-requisites

7.8.2. Running the Analysis

7.8.3. Getting Analysis Results

7.8.4. Example Pipeline script

7.8.5. Advanced integration

7.8.5.1. Effective Code Quality report comparison

7.8.5.2. Incremental analysis

7.8.5.3. Aggregating multiple reports