Unit tests

Philosophy

Unit tests focus on verifying the functionality of individual units or components of our software. A unit would be the smallest testable part of a software, such as a function, method or class. Unit tests in Fluid Attacks must be:

• Repeatable: Regardless of where they are executed, the result must be the same.
• Fast: Unit tests should take little time to execute because, being the first level of testing, where you have isolated functions/methods and classes, the answers should be immediate. A unit test should take at most two (2) seconds.
• Independent: The functions or classes to be tested should be isolated, no secondary effect behaviors should be validated, and, if possible, we should avoid calls to external resources such as databases; for this, we use mocks.
• Descriptive: For any developer, it should be evident what is being tested in the unit test, what the result should be, and in case of an error, what is the source of the error.

Architecture

• Location: Unit tests for a given file are located right next to the file to be tested. For example, tests for mailmap/create.py are located at mailmap/create_test.py.
• Utilities: They use the fluidattacks-core library, which provides faking utilities so developers can focus on actually testing the code rather than faking/mocking data.
• Coverage: Current coverage for a given module con be found at <module-path>/coverage. For example, malmap/coverage.

Writing tests

Note

Waiting for tests to stabilize before giving an example.

Tip

Make sure you use other tests as a reference for creating your own.

Running tests

Running tests for all modules

You can run tests for all modules with the following command:

m . /integrates/back/test __all__

Running tests for a specific module

You can run tests for a specific module with the following command:

m . /integrates/back/test [module]

where [module] can be any Integrates module.

This command will:

1. Run all tests for the given module.
   1. Fail if any of the tests fail.
2. Generate a coverage report.
   1. Fail if the new coverage is below the current one for the given module (Developer must add tests to at least keep the same coverage).
   2. Fail if the new coverage is above the current one for the given module (Developer must add new coverage to their commit).
   3. Pass if new and current coverage are the same.

Tip

Try running tests for mailmap with:

m . /integrates/back/test mailmap

Old unit tests

Caution

These tests are currently being deprecated due to their poor architecture.

You can run tests using the following command:

m . /integrates/back/test/unit not_changes_db [module]

To run the ones that modify the mock database:

m . /integrates/back/test/unit changes_db [module]

Note

[module] is an optional argument. If you do not specify it, all the unit tests will be executed.

Currently, every time our unit tests run, we launch a mock stack that is populated with the necessary data required for our tests to execute. We utilize mocking to prevent race conditions and dependencies within the tests.

When writing unit tests, you can follow these steps to ensure that the test is repeatable, fast, independent, and descriptive:

• Test file: We store our tests using the same structure as our repository. Inside universe/integrates/back/test/unit/src you can find our unit tests. Look for the test_module_to_test.py file or add it if missing.
• Write the test: Once the file is ready, you can start writing the test. Consider the purpose of the function, method, or class that you want to test. Think about its behavior when different inputs are provided. Also, identify extreme scenarios to test within the test. These will form our test cases and are important for writing our assertions. We use the parametrize decorator if possible to declare different test cases.
• Mocks: What do you mock? A general guideline is to look for the await statement inside the function, method or class that you want to test. In most cases, await indicates that the awaited function requires an external resource, such as a database. To learn more about mocks, you can refer to the official documentation.
• Mock data: When using mocks, you need to provide the data required for your unit test to run. We accomplish this by using pytest fixtures, which allow us to have mock data available from conftest.py files.
• Assertions: Test the expected behavior. We use assertions to validate results, the number of function or mock calls, and the arguments used in mocks.