Tests for actors

The Leapp actors are covered by three types of tests - unit, component and e2e.

Unit and component tests

  • Both unit and component tests are to be placed in the actor‘s tests folder.
  • Unit and component tests modules should have unique names
  • Tutorial on How to write unit and component tests

Naming conventions

Test module names should match the following regex:

  • test_*.py
  • unit_test_*.py
  • component_test_*.py

Unit tests

  • These tests deal with individual actor‘s functions/methods.
  • It‘s not possible to unit test any method/function within the actor.py. You can write unit tests only for functions/methods within the actor‘s libraries.
  • Thus, to be able to write unit tests for an actor, ideally the only thing in the actor.py‘s process() method is calling the entry-point function of the actor‘s library python module.
  • Example of unit tests

Component tests

  • These tests provide fabricated input messages for the actor, check the outputs stated in the actor‘s description.
  • These tests should not be written based on the actor‘s code but rather based on the behavior stated in the actor‘s description. They could be written by somebody who didn‘t write the code.
  • Example of component tests

End to end (e2e) tests

  • The Leapp QA team maintains an internal testing framework facilitating e2e tests.
  • [Members of the oamg GitHub organization] can trigger execution of the e2e tests by adding the comment ‘e2e tests‘ under an opened leapp or leapp-repository PR

Bug fixing of actor

Each bug found and fixed in an actor should be covered by tests.