GNATDAS Manuals [23.0w (20220529)]

• 1. GNATcoverage User’s Guide
  • 1.1. Getting Started
    • 1.1.1. Framework concepts and process overview
    • 1.1.2. Selecting a trace mode; supported features and environments
    • 1.1.3. A brief introduction to the main process steps
      • 1.1.3.1. Example production of a binary trace for a bareboard environment
      • 1.1.3.2. Example production of a source trace for a native environment
      • 1.1.3.3. Example production of a coverage report
    • 1.1.4. Going Further
    • 1.1.5. Conventions used in the rest of this manual
  • 1.2. Specifying Units Of Interest
    • 1.2.1. Providing coverage obligation files (--scos|--sid)
    • 1.2.2. Using project files (-P, --projects, --units)
      • 1.2.2.1. Conveying projects of interest
      • 1.2.2.2. Conveying units of interest within projects
      • 1.2.2.3. Other switches or attributes of importance
    • 1.2.3. Conveying source files to ignore / handling Ada subunits
    • 1.2.4. Compilation unit vs source file names
  • 1.3. Producing binary traces with gnatcov run
    • 1.3.1. Compilation prerequisites
    • 1.3.2. gnatcov run command line
    • 1.3.3. Execution environment
    • 1.3.4. Program selection and Arguments passing
    • 1.3.5. Trace file contents
  • 1.4. Producing source traces with gnatcov instrument
    • 1.4.1. Setting up the coverage runtime library
    • 1.4.2. Instrumenting programs
      • 1.4.2.1. gnatcov instrument command line
      • 1.4.2.2. Output strategies for main units
      • 1.4.2.3. Controlling trace file names
      • 1.4.2.4. gnatcov instrument limitations
    • 1.4.3. Building instrumented components
    • 1.4.4. Extracting a trace from standard output
    • 1.4.5. Composed instrumentation
    • 1.4.6. Instrumentation of a multi-languages project
    • 1.4.7. Instrumentation and coverage of SPARK code
      • 1.4.7.1. Inhibiting SPARK related pragmas
      • 1.4.7.2. Coverage obligations for SPARK code
    • 1.4.8. Example use cases
      • 1.4.8.1. Whole program instrumented at once, cross configuration, base64 output
      • 1.4.8.2. Library instrumented separately, native configuration, trace output
  • 1.5. Source coverage analysis with gnatcov coverage
    • 1.5.1. gnatcov coverage command line
    • 1.5.2. Output report formats (--annotate)
      • 1.5.2.1. Annotated sources, text (=xcov[+])
      • 1.5.2.2. Annotated sources, html (=html[+])
      • 1.5.2.3. Annotated sources, dynamic html (=dhtml)
      • 1.5.2.4. Violations summary, text (=report)
    • 1.5.3. Statement Coverage analysis
      • 1.5.3.1. Core notions and Reporting (--level=stmt)
      • 1.5.3.2. Example program and assessments
    • 1.5.4. Decision Coverage analysis
      • 1.5.4.1. Core notions and Reporting (--level=stmt+decision)
      • 1.5.4.2. Example program and assessments
    • 1.5.5. Modified Condition/Decision Coverage analysis
      • 1.5.5.1. Core notions and Reporting (--level=stmt+mcdc)
      • 1.5.5.2. Example decisions and conditions
      • 1.5.5.3. Handling non short-circuit operators
      • 1.5.5.4. MCDC variants
      • 1.5.5.5. Example program and assessments
    • 1.5.6. Synthetic metrics
    • 1.5.7. Handling source relocation for annotated sources output formats
    • 1.5.8. Inlining & Ada generic units
      • 1.5.8.1. Combined coverage on generics (default behavior)
      • 1.5.8.2. Separated coverage on generics
    • 1.5.9. Handling Ada assertions and contracts
    • 1.5.10. Processing of C macros
    • 1.5.11. Optimization and non-coverable items
  • 1.6. Object coverage analysis with gnatcov coverage
    • 1.6.1. gnatcov coverage command line
    • 1.6.2. Output report formats (--annotate)
      • 1.6.2.1. Machine level reports (=asm)
      • 1.6.2.2. Annotated sources, text (=xcov[+])
      • 1.6.2.3. Annotated sources, html (=html[+]) or dynamic html (=dhtml)
    • 1.6.3. Object Instruction Coverage analysis (--level=insn)
    • 1.6.4. Object Branch Coverage analysis (--level=branch)
    • 1.6.5. Focusing on subprograms of interest (--routines)
    • 1.6.6. Inlining & Ada Generic units
    • 1.6.7. Full object coverage considerations
  • 1.7. Object/Source level metrics considerations
    • 1.7.1. Main differences examplified
    • 1.7.2. Full branch coverage vs MCDC
  • 1.8. Coverage Consolidation
    • 1.8.1. Consolidation from trace files
      • 1.8.1.1. Single unit tested by different programs
      • 1.8.1.2. Disjoint units tested by different programs
      • 1.8.1.3. Multiple executions of a single program
      • 1.8.1.4. Special care needed with binary traces
    • 1.8.2. Consolidation from coverage checkpoints
      • 1.8.2.1. Efficient storage of test results
      • 1.8.2.2. Incremental coverage analysis
      • 1.8.2.3. Handle incidental coverage effects
      • 1.8.2.4. Compatibility considerations
  • 1.9. Coverage Exemptions
    • 1.9.1. Defining Exemption Regions
    • 1.9.2. Reporting about coverage exemptions
    • 1.9.3. Object coverage exemptions
  • 1.10. Using project files
    • 1.10.1. General considerations
    • 1.10.2. Specifying gnatcov command Switches
    • 1.10.3. Specifying the Target and language Runtime
  • 1.11. Importing binary traces with gnatcov convert
    • 1.11.1. gnatcov convert command line
    • 1.11.2. Generating Coverage Information from Nexus Traces on MPC5634M
      • 1.11.2.1. Nexus Program Trace Data: the Hardware Perspective
      • 1.11.2.2. Configuring the winIDEA Analyzer
      • 1.11.2.3. Exporting the Trace Data
    • 1.11.3. Generating Coverage Information from Lauterbach Trace32
      • 1.11.3.1. Enabling Trace Data
      • 1.11.3.2. Running the executable
      • 1.11.3.3. Exporting the Trace Data
      • 1.11.3.4. Automation
  • 1.12. Using GNATcoverage from GNAT Studio
    • 1.12.1. Basic usage
    • 1.12.2. Binary traces usage
    • 1.12.3. GNATtest scenario
  • 1.13. Appendices
    • 1.13.1. Sample html index
    • 1.13.2. Sample html annotated source
    • 1.13.3. Target specific points of note with Binary traces
    • 1.13.4. Known Limitations with Binary traces
      • 1.13.4.1. Incomplete statement coverage on lines with multiple statements
      • 1.13.4.2. MCDC inaccuracies with interrupts and multi-threaded applications
  • 1.14. Glossary of terms & concepts
• 2. GNATtest User’s Guide
  • 2.1. Running gnattest
    • 2.1.1. Framework Generation Mode
    • 2.1.2. Test Execution Mode
  • 2.2. Switches for gnattest in framework generation mode
  • 2.3. Switches for gnattest in test execution mode
  • 2.4. Project Attributes for gnattest
  • 2.5. Simple Example
  • 2.6. Setting Up and Tearing Down the Testing Environment
  • 2.7. Regenerating Tests
  • 2.8. Default Test Behavior
  • 2.9. Testing Primitive Operations of Tagged Types
  • 2.10. Testing Inheritance
  • 2.11. Tagged Type Substitutability Testing
  • 2.12. Testing with Contracts
  • 2.13. Additional Tests
  • 2.14. Individual Test Drivers
  • 2.15. Stubbing
  • 2.16. Integration with GNATcoverage
  • 2.17. Putting Tests under Version Control
  • 2.18. Current Limitations
• 3. Using GNATtest with GNATcoverage
  • 3.1. Using the Makefile generated by GNATtest
    • 3.1.1. Special considerations for cross environment targets
    • 3.1.2. Usage example
    • 3.1.3. Instrumenting test harnesses for a SPARK project
  • 3.2. Developing a custom solution
    • 3.2.1. Single test driver
    • 3.2.2. Separate test drivers
• 4. GNATfuzz User’s Guide
  • 4.1. Introduction
    • 4.1.1. Fuzz testing in a nutshell
    • 4.1.2. AFL++ basics
    • 4.1.3. GNATfuzz overview
  • 4.2. GNATfuzz command line interface
    • 4.2.1. GNATfuzz “analyze” mode command line interface
    • 4.2.2. GNATfuzz “generate” mode command line interface
    • 4.2.3. GNATfuzz “corpus-gen” mode command line interface
    • 4.2.4. GNATfuzz “fuzz” mode command line interface
  • 4.3. GNATfuzz by example
    • 4.3.1. Prerequisites and possible configurations
      • 4.3.1.1. AFL++ requirements
      • 4.3.1.2. GNATfuzz requirements
      • 4.3.1.3. GNATfuzz configurations
      • 4.3.1.4. GNATfuzz and the AFL++ execution modes
    • 4.3.2. GNATfuzz automation capabilities and limitations
    • 4.3.3. Fuzzing a simple Ada subprogram
      • 4.3.3.1. Using analyze mode
      • 4.3.3.2. Using generate mode
      • 4.3.3.3. Using corpus-gen mode
      • 4.3.3.4. Decoding an auto-generated test case
      • 4.3.3.5. Using fuzz mode
    • 4.3.4. Fuzzing an autogenerated test-harness with user-provided seeds
    • 4.3.5. Fuzzing instantiations of generic subprograms
    • 4.3.6. Fuzzing global and private state
    • 4.3.7. Fuzzing Access Types
    • 4.3.8. Filtering exceptions
    • 4.3.9. Fuzzing C functions through Ada bindings
    • 4.3.10. Resetting dynamically allocated memory when fuzz-testing with the AFL++ persistent mode
  • 4.4. The “User_Configuration” package
  • 4.5. GNATfuzz stopping criteria
  • 4.6. GNATfuzz integration in GNAT Studio
    • 4.6.1. The GNATfuzz menu
    • 4.6.2. Viewing analysis results and generating a fuzz-testing harness
    • 4.6.3. Running a fuzz session
    • 4.6.4. Viewing and debugging fuzz crashes
  • 4.7. FAQ for GNATfuzz
  • 4.8. GNATfuzz diagnostics and issue reporting