Skip to content
Fluid Attacks Development Portal

Unit tests

Philosophy

Unit tests focus on verifying the functionality of individual units or components of our software. A unit would be the smallest testable part of a software, such as a function, method or class. Unit tests in Fluid Attacks must be:

  • Repeatable: Regardless of where they are executed, the result must be the same.
  • Fast: Unit tests should take little time to execute because, being the first level of testing, where you have isolated functions/methods and classes, the answers should be immediate. A unit test should take at most two (2) seconds.
  • Independent: The functions or classes to be tested should be isolated, no secondary effect behaviors should be validated, and, if possible, we should avoid calls to external resources such as databases; for this, we use mocks.
  • Descriptive: For any developer, it should be evident what is being tested in the unit test, what the result should be, and in case of an error, what is the source of the error.

Architecture

  • Location: To be discover by the testing framework, test files must be located next to the file to be tested with the _test suffix and every test method must start with the test_ prefix. Take a look at add group tests for reference.
  • Utilities: Some utilities are added to simplify tasks like to populate database, mock comfortably and include test files. It allows developers to focus on actually testing the code.
  • Coverage: Coverage is a module-scoped integer between 0 and 100. Current coverage for a given module con be found at <module-path>/coverage. For example, api/coverage.

Writing tests

See the following examples to understand how to write tests:

from integrates.dataloaders import Dataloaders, get_new_context
from integrates.db_model.organizations.types import Organization
from integrates.organizations.utils import get_organization
from integrates.testing.aws import IntegratesAws, IntegratesDynamodb
from integrates.testing.fakers import OrganizationFaker
from integrates.testing.mocks import mocks


@mocks(
    aws=IntegratesAws(
        dynamodb=IntegratesDynamodb(
            organizations=[
                OrganizationFaker(id="test-org-1"),
            ],
        ),
    ),
)
async def test_get_organization() -> None:
    # Arrange
    organization_id = "test-org-1"


    # Act
    loaders: Dataloaders = get_new_context()
    organization: Organization = await get_organization(
        loaders,
        organization_id,
    )


    # Assert
    assert organization is not None

@mocks decorator allows to populate the database with test data using aws parameter. A clean database will be created and populated for each parameter provided via @utils.parametrize. We got deeper into this decorators and helper methods in the next sections.

DynamoDB

Integrates database is populated using IntegratesAws.dynamodb in the @mocks decorator. This parameter is an instance of IntegratesDynamodb, a helper class to populate the main tables with valid data:

@mocks(
    aws=IntegratesAws(
        dynamodb=IntegratesDynamodb(
            organizations=[OrganizationFaker(id=ORG_ID)],
            stakeholders=[
                StakeholderFaker(email=ORGANIZATION_MANAGER_EMAIL),
                StakeholderFaker(email=ADMIN_EMAIL),
            ],
            organization_access=[
                OrganizationAccessFaker(
                    organization_id=ORG_ID,
                    email=ORGANIZATION_MANAGER_EMAIL,
                    state=OrganizationAccessStateFaker(has_access=True, role="organization_manager"),
                ),
                OrganizationAccessFaker(
                    organization_id=ORG_ID,
                    email=ADMIN_EMAIL,
                    state=OrganizationAccessStateFaker(has_access=True, role="admin"),
                ),
            ],
        ),
    ),
    others=[
        Mock(logs_utils, "cloudwatch_log", "sync", None),
    ],
)

In the example above, we are populating the database with one organization, two stakeholders, and giving access to both stakeholders to the Organization with a different role.

Every faker is a fake data generator for one element. Parameters are optional to modify your data for your tests (e.g., assigned role in OrganizationAccessStateFaker or the email in StakeholderFaker). Faker name gives a hint about where it should be used in the IntegratesDynamodb parameters.

The others parameter is a way to list all the startup mocks that you require in your test. In the example above, we are mocking the cloudwatch_log function from logs_utils module to avoid to call CloudWatch directly and return always a None value. Mock is a helper class that creates a mock based on module, function or variable name, mode (sync or async), and a return value.

This declarative approach ensures isolation. Each test will have its own data and will not conflict with other tests.

S3

Integrates buckets are created for testing at the same time when @mocks is called and no more actions are required. You can use the buckets in your tests and, also, load files to buckets automatically before every test run.

To load files to the buckets automatically, you must use:

@mocks(aws=IntegratesAws(s3=IntegratesS3(autoload=True)))

Use the following file structure as reference:

  • main.py (logic here)
  • main_test.py (tests here)
  • Directorytest_data/
    • Directorytest_name_1/
      • file_1.txt (It won’t be loaded)
      • file_2.txt (It won’t be loaded)
    • Directorytest_name_2/
      • Directoryintegrates.dev/
        • README.md (Loaded to integrates.dev bucket)
    • Directorytest_name_3/
      • Directoryintegrates/
        • README.md (Loaded to integrates bucket)

<test_name> directory is searched to load files into the corresponding buckets. For example, a README.md file will be loaded into integrates.dev for test_name_2, and a different README.md file will be loaded into integrates for test_name_3. This approach ensures both isolation and simplicity in the tests.

Utils

For easy testing, some utilities and decorators are provided.

Use @parametrize to include several test cases:

from integrates.testing.utils import parametrize


@parametrize(
    args=["arg", "expected"],
    cases=[
        ["a", "A"],
        ["b", "B"],
        ["c", "C"],
    ],
)
def test_capitalize(arg: str, expected: str) -> None:
    ...

Use raises to handle errors during tests:

from integrates.testing.utils import raises


def test_fail() -> None:
    with raises(ValueError):
       ...

Use get_file_abs_path to get the file’s absolute path in the test_data/<test_name> directory:

from integrates.testing.utils import get_file_abs_path




def test_name_1() -> None:
  abs_path = get_file_abs_path("file_1.txt")


  assert "/test_data/test_name_1/file_1.txt" in abs_path   # True

Use @freeze_time when you want to set the execution time (time-based features).

from integrates.testing.utils import freeze_time


@freeze_time("2024-01-01")
def test_accepted_until() -> None:
    ...

Running tests

You can run tests for specific modules with the following command:

Terminal window
integrates-back-test <module> [test-1] [test-2] [test-n]...

where:

  1. <module> is required and can be any Integrates module.
  2. [test-n] is optional and can be any test within that module.

If no specific tests are provided, this command will:

  1. Run all tests for the given module.
    1. Fail if any of the tests fail.
  2. Generate a coverage report.
    1. Fail if the new coverage is below the current one for the given module (Developer must add tests to at least keep the same coverage).
    2. Fail if the new coverage is above the current one for the given module (Developer must add new coverage to their commit).
    3. Pass if new and current coverage are the same.

If specific tests are provided, this command will:

  1. Only run the given tests.
    1. Fail if any of the provided tests fail.
  2. Skip the coverage report generation and evaluation.