Skip to content

Skims functional tests

To develop a functional test, follow the following steps:

  1. Generate a configuration file: Generate a configuration file that emulates the desired execution scenario. Store this file in skims/test/data/config.

  2. Implement test logic: In skims/test/functional, write the basic test logic using the skims utility function. This function executes the CLI with the provided configuration file, analyzing specified modules and test files. It facilitates leveraging pytest features like mocking and patching. The utility function runs the equivalent of m . /skims scan config.yaml within a test environment.

    code, stdout, stderr = skims("scam", path)
    assert code == 0, stdout
    assert "[INFO] Startup work dir is:" in stdout
    assert "[INFO] An output file has been written:" in stdout
    assert not stderr, stderr
    check_that_csv_results_match(finding)
  3. Assign test group: Assign the test to a group and add it to skims/test_groups.json. After updating this file, run the test locally using the command m . /testPython/skims@group_name.

  4. Update CI/CD pipeline: Execute m . /pipelineOnGitlab/skimsTest to update the CI/CD pipeline YAML configuration on GitLab, ensuring its execution on the platform.

For adding tests specifically for new methods, integrate mock data into the appropriate folder within skims/test/data/lib_ and verify the target finding is defined in skims/test/functional/test_findings.py. These tests utilize the common config.yaml template to automatically generate and execute functional tests on specific paths.