Welcome to developer documentation for Leapp!

Leapp is an OS and application modernization framework.

Terminology

Actor

An actor in terms of the Leapp project is a step that is executed within a workflow. Actors define what kind of data they expect, and what they produce. Actors also provide a list of tags, with which actors mark their use cases.

Actors scan the system and produce the information found as messages. Other actors consume those messages to make decisions, or process the data to produce new messages. Some actors might apply changes to the system based on the information gathered earlier.

Message

A message is produced by an actor, and the payload follows the definition of the Model it is named after. Messaging is a term used to describe the data exchange between actors.

Model

Models are the definition of the data model used in message payloads.

Phase

Phases are sections in a workflow dedicated to some specific area of execution. A phase consists of three stages: Before, Main, and After. Phases are defined by assigning one or more tags to them, which will be used to find actors in the repositories loaded.

Repository

A repository is the place where all actors, models, tags, topics, and workflows are defined. Additionally to that shared files, libraries and tools can be put into the repository.

Stage

Stage is a part of a phase. There are three defined stages:

• Before
• Main
• After

Before and After stages can be used to allow an actor to be run before or after any other actors in the phase. This should be useful in some hooking scenarios, where an action is supposed to be happening before or after another action. This way, other actors could be influenced.

Tag

A tag allows the framework to find actors in the repository and group their execution based on that tag.

Topic

Topics are assigned to models and are used for grouping the data into areas of interest.

Workflow

Workflows describe what work is going to be done and when. A workflow is describing a sequence of phases, where one phase has assigned filters with which the framework selects actors that should be executed from the repositories on the system.

Workflow APIs

Workflow APIs are custom API classes that actors can use and automatically inherit their consumed and produced messages. This way one can write a stable API for third party actor writers, without being affected by changes of message model layout, name changes etc.

Tutorials

Installing the development environment

RPM packages installation

If you do not want to modify the framework itself, install it from the RPM packages provided by the Copr build system, which automatically builds packages with every commit merged into master. Packages are built for EPEL and Fedora.

• On CentOS/RHEL:

  # yum install -y yum-utils
  # yum-config-manager --add-repo https://copr.fedorainfracloud.org/coprs/g/oamg/leapp/repo/epel-7/group_oamg-leapp-epel-7.repo
  

• On Fedora:

  # dnf install -y dnf-plugins-core
  # dnf copr enable @oamg/leapp
  

For the actor development, install the leapp and snactor tools. This pulls in also leapp-repository with already existing actors, models, topic, tags and workflows.

# yum install snactor leapp

For the actor development, install the snactor tool, and if you want to use actors, install also leapp-repository.

# yum install snactor leapp-repository

Virtualenv installation

To keep your environment clean, use a virtualenv.

First, create a new virtual environment called “tut“ and activate it:

$ cd ~/devel
$ virtualenv -p /usr/bin/python2.7 tut
$ . tut/bin/activate

Then, install the framework by using the pip package management system:

$ pip install git+https://github.com/oamg/leapp

Once the framework is installed, you can use the snactor tool.

$ snactor --help
usage: snactor [-h] [--version] [--logger-config LOGGER_CONFIG]
               [--config CONFIG] [--verbose] [--debug]
               ...

Optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  --logger-config LOGGER_CONFIG
                        Allows to override the logger.conf location
  --config CONFIG       Allows to override the leapp.conf location
  --verbose             Enables verbose logging
  --debug               Enables debug mode

Main commands:
  
    new-tag             Create a new tag
    new-model           Creates a new model
    run                 Execute the given actor
    workflow            Workflow related commands
    new-topic           Creates a new topic
    messages            Messaging related commands
    discover            Discovers all available entities in the current
                        repository
    new-project         [DEPRECATED] Creates a new repository
    repo                Repository related commands
    new-actor           Creates a new actor

A screen cast of the steps above

Creating a new repository for actor development

Leapp uses repositories with a defined filesystem layout. The snactor tool helps you to create a repository repository, in which you can develop and test your actors, tags, models, and topics.

To create a new repository called tutorial, run the snactor tool:

    $ snactor repo new tutorial

Enter the tutorial folder where the repository has been initialized, and see its content:

    $ cd tutorial
    $ ls -a

Now, you can start creating your actors, tags, models, and topics.

A screen cast of the steps above

Creating your first actor

We will go through the steps necessary to create a new actor in a completely clean environment. The purpose of the actor in this tutorial is to retrieve the hostname of the system and to send it as a message into the system, so that other actors can consume it.

We will start at the very beginning, so we will assume that all things have to be created.

Getting started

First, create and go to your repository directory. See Creating a new repository tutorial.

Creating a tag

Create a tag. Since we are scanning the system, let‘s call this tag ‘Scan‘. Use the snactor tool.

    $ snactor new-tag Scan

This will create a subdirectory called tags with a scan.py file. The file contains all the necessary code, and it creates the ScanTag class, which we will use later.

Screencast

Creating a topic

Create the SystemInfo topic.

    $ snactor new-topic SystemInfo

The topics directory has been created with a systeminfo.py file, which provides the complete code and definition for the SystemInfoTopic class used in the model.

Screencast

Creating a model

Create a model for sending a message. We will call the model Hostname and have it assigned to the SystemInfoTopic class.

   $ snactor new-model Hostname

The model boiler plate is available at the models/hostname.py file:

from leapp.models import Model, fields


class Hostname(Model):
    topic = None #  TODO: import appropriate topic and set it here

As the comment says, import the SystemInfoTopic class and assign it to the topic variable of the Hostname model.

Import the SystemInfoTopic class:

from leapp.topics import SystemInfoTopic

After the topic has been assigned, create a new field for the message called name, which is supposed to be a string. This can be accomplished by setting the name field:

class Hostname(Model):
    topic = SystemInfoTopic
    name = fields.String()

Add a default value of ‘localhost.localdomain‘, in case the name is not specified. Default values are initializing the values in the construction of the class object, if no other value has been determined.

class Hostname(Model):
    topic = SystemInfoTopic
    name = fields.String(default='localhost.localdomain')

Save the file and write an actor.

Screencast

Creating an actor

We are creating an actor that retrieves the system hostname and sends it as a message. Call the actor HostnameScanner.

    $ snactor new-actor HostnameScanner

We created the actors/hostnamescanner/ directory with an actor.py file and a tests subdirectory. Let‘s look at the pregenerated actor.py file:

from leapp.actors import Actor


class HostnameScanner(Actor):
    """
    No documentation has been provided for the hostname_scanner actor.
    """
    name = 'hostname_scanner'
    consumes = ()
    produces = ()
    tags = ()

    def process(self):
        pass

Import the model and the tag we have previously created to be able to assign them.

from leapp.models import Hostname
from leapp.tags import ScanTag

Assign Hostname to the produces attribute as a tuple element and do the same with the ScanTag and tags attributes. Do not forget the trailing commas.

     consumes = ()
     produces = (Hostname,)
     tags = (ScanTag,)

Now, we can start writing the actor code. The actor code has to be added in the process method.

To retrieve the hostname, we will use the python socket module, which has a function called getfqdn, which will retrieve the hostname.

For that, add import socket at the top of the file.

A very minimal implementation for this actor can look like this:

    def process(self):
        self.produce(Hostname(name=socket.getfqdn()))

But we would also like to do some logging, so that we can see our actor at work.

   def process(self):
       self.log.info("Starting to scan for the hostname")
       hostname = socket.getfqdn()
       self.produce(Hostname(name=hostname))
       self.log.info("Finished scanning for the hostname, found = %s",
                     hostname)

You can edit the description of the actor now.

Save the file, and it is ready to be run from the commandline:

    $ snactor run --debug HostnameScanner
    2018-03-20 13:24:06.20  INFO     PID: 6256 leapp: Logging has been initialized
    2018-03-20 13:24:06.22  INFO     PID: 6256 leapp.repository.tutorial: New repository 'tutorial' initialized at /home/evilissimo/devel/tutorial
    2018-03-20 13:24:06.67  INFO     PID: 6273 leapp.actors.hostname_scanner: Starting to scan for the hostname
    2018-03-20 13:24:16.188 INFO     PID: 6273 leapp.actors.hostname_scanner: Finished scanning for the hostname, found = actor-developer

To see the message it generated, use the –print-output option:

    $ snactor run --debug --print-output HostnameScanner
    2018-03-20 13:24:32.333 INFO     PID: 6300 leapp: Logging has been initialized
    2018-03-20 13:24:32.335 INFO     PID: 6300 leapp.repository.tutorial: New repository 'tutorial' initialized at /home/evilissimo/devel/tutorial
    2018-03-20 13:24:32.372 INFO     PID: 6317 leapp.actors.hostname_scanner: Starting to scan for the hostname
    2018-03-20 13:24:42.492 INFO     PID: 6317 leapp.actors.hostname_scanner: Finished scanning for the hostname, found = actor-developer
    [
      {
        "stamp": "2018-03-20T13:24:37.434408Z",
        "hostname": "actor-developer",
        "actor": "hostname_scanner",
        "context": "TESTING-CONTEXT",
        "phase": "NON-WORKFLOW-EXECUTION",
        "message": {
          "hash": "fb5ce8e630a1b3171709c9273883b8eb499b6b2ba09e112832ad47fa4e3f62b7",
          "data": "{\"name\": \"actor-developer\"}"
        },
        "type": "Hostname",
        "topic": "system_info"
      }
    ]

Screencast

How to create a Leapp actor for RHEL 7 to 8 upgrade

Introduction

This document is intended for all people who want to contribute to the process of upgrading Red Hat Enterprise Linux (RHEL) 7 to RHEL 8 using Leapp tool. The upgrade is performed in place meaning that the RHEL 7 installation is replaced by RHEL 8 on the same storage. After reading through this document, you will be able to transform your expertise in certain parts of RHEL into improvements of the RHEL 7 to 8 upgrade tooling.

Setting up the development environment

Leapp actors are written in Python 2.7+/3.6+ (the resulting code has to be both py2 and py3 compatible), so your usual Python development setup can be used during the process of creating a new actor.

Tools

The main tools you will use for the actor development are listed below.

leapp

The leapp framework provides the libraries required to be imported by any actor and also a binary tool used to control the execution of actors within a workflow.

snactor

Separate tool provided by Leapp to help the process of creating and executing an actor.

You can see snactor source code here.

Creating an actor

Every actor needs to be inside a so-called “Leapp repository”, otherwise it won’t be visible to Leapp. A Leapp repository groups actors and many other things which will be discussed later, like models, workflows, tags and topics. You can find all Leapp repositories under /usr/share/leapp-repository/repositories. A Leapp repository can be recognized by containing .leapp folder:

$ find -L /etc/leapp/repos.d/ -name ".leapp" -type d | xargs dirname
/etc/leapp/repos.d/common
/etc/leapp/repos.d/system_upgrade/el7toel8

First, you need to register repositories with snactor:

$ snactor repo find --path /etc/leapp/repos.d/
Registering /etc/leapp/repos.d/system_upgrade/el7toel8
Registering /etc/leapp/repos.d/common

After registering the repositories, you can move inside any of these repositories and use snactor to create a boilerplate of a new actor:

# cd /etc/leapp/repos.d/system_upgrade/el7toel8
# snactor new-actor MyNewActor
New actor MyNewActor has been created at /usr/share/leapp-repository/repositories/system_upgrade/el7toel8/actors/mynewactor/actor.py
# cd /usr/share/leapp-repository/repositories/system_upgrade/el7toel8/actors/mynewactor/
# tree
.
├── actor.py
└── tests

The main file of the actor is the actor.py in which you’ll write code for logic of the actor.

For further information about how create an actor read this document.

Including an actor in the RHEL 7 to 8 upgrade process

Until now, you have created boilerplate of a new actor and made it visible to Leapp. But, Leapp needs some more information about what to do with the actor. Specifically, in which “workflow” and in which “phase” the actor should be executed. A workflow is a sequence of phases. The only workflow available now is the one solving the upgrade of RHEL 7 to RHEL 8. Each phase is a set of actors that will be executed one after another before the next phase starts. To find out in which workflow and phase should the actor be executed, Leapp looks for “tags”. To be part of RHEL 7 to RHEL 8 upgrade workflow, an actor needs to be tagged with IPUWorkflowTag.

The phases of the IPUWorkflow (in order) are: Facts Collection, Checks, Report, Download, Upgrade RamDisk Preparation, Upgrade RamDisk Start, Late Tests, Preparation, RPM Upgrade, Application Upgrade, Third Party Applications, Finalization and First Boot. Each phase has a specific tag that marks an actor as being part of that phase. You can find descriptions of all the phases and their tags here and workflow diagram here.

For example, if an actor is to be executed within the Checks phase, it needs to be tagged both with IPUWorkflowTag and ChecksPhaseTag. The result after updating the boilerplate would be:

from leapp.actors import Actor
from leapp.tags import ChecksPhaseTag, IPUWorkflowTag

class MyNewActor(Actor):
    """ No description has been provided for the my_new_actor actor. """

    name = 'my_new_actor'
    consumes = ()
    produces = ()
    tags = (ChecksPhaseTag, IPUWorkflowTag)

    def process(self):
            pass

Inter-actor communication

Receiving data from other actors

All communication between actors in Leapp is carried out using “messages”. An actor can consume or produce messages. A message may contain any data, but the data needs to be in a specific format defined by a “model”. If an actor wants to consume a message produced by another actor, it needs to specify the specific model of the consumed messages. Leapp will make sure to execute such an actor only after some message of the specified model was produced by another actor. If no message of the specified model was produced in previous phases or in the current phase, the consuming actor will get no messages of that kind.

For further information about messaging see document.

One of the existing models in Leapp is ActiveKernelModulesFacts. Messages from this model contain data about the system on which Leapp has been started. For example, it contains installed kernel modules. If an actor wants to perform some action based on existing kernel modules on the system, the actor can get list of these modules by consuming the ActiveKernelModulesFacts messages. By extending the boilerplate, the code could look like this:

from leapp.actors import Actor
from leapp.models import ActiveKernelModulesFacts

from leapp.tags import ChecksPhaseTag, IPUWorkflowTag


class MyNewActor(Actor):
    """ No description has been provided for the my_new_actor actor. """

    name = 'my_new_actor'
    consumes = (ActiveKernelModulesFacts,)
    produces = ()
    tags = (ChecksPhaseTag, IPUWorkflowTag)

    def process(self):
        for fact in self.consume(ActiveKernelModulesFacts):
            for active_module in fact.kernel_modules:
                    self.log.info(active_module.filename)

By executing the above actor, all active kernel modules would be logged on output using log utilities inherited from the Actor class.

Asking user questions

In rare cases the actor can‘t choose a proper scenario of execution and leaves the final decision of how to proceed to the user. That‘s where dialogs come into play.

Please mind that using dialogs should be considered as last resort, when the situation absolutely can‘t resolve itself automatically. The rule of a thumb is to make upgrade procedure require as little user input as possible. But if you feel that there is no way to write a proper safe rhel7->rhel8 conversion logic and you need human to make a decision - you can go with dialogs.

The following restrictions apply:

• At the time only Yes/No questions can be asked. Effectively only leapp.dialogs.components.BooleanComponent can be used.
• Dialogs can‘t be codependent. Any question asked should be independent of previous question‘s answer and should not change the behavior of any other dialog that any actor might create.
• Dialogs can be used only at certain stages of the workflow - ChecksPhase and TargetTransactionChecksPhase.

For more information and real examples please check dialogs.

Producing data for other actors and reporting

An actor can produce some data interesting enough for other actors to consume. It could be some parsed data, or content that will be displayed to the user in a report or even shared info between a subset of actors.

The process is very similar to the one used to consume messages, but now the new actor will produce them. Similar to ActiveKernelModulesFacts, Leapp has a Report model. Messages from this model contain data that will be displayed to the user during ReportsPhase. For example, an actor can warn the user in case a btrfs kernel module is active on the system. Then, the actor could look like this:

from leapp import reporting
from leapp.actors import Actor
from leapp.models import ActiveKernelModulesFacts
from leapp.reporting import Report, create_report
from leapp.tags import ChecksPhaseTag, IPUWorkflowTag

class MyNewActor(Actor):
    """ No description has been provided for the my_new_actor actor. """

    name = 'my_new_actor'
    consumes = (ActiveKernelModulesFacts,)
    produces = (Report,)
    tags = (ChecksPhaseTag, IPUWorkflowTag)

    def process(self):
        for fact in self.consume(ActiveKernelModulesFacts):
            for active_module in fact.kernel_modules:
                if active_module.filename == 'btrfs':
                    create_report([
                        reporting.Title('Btrfs has been removed from RHEL8'),
                        reporting.Summary(
                            'The Btrfs file system was introduced as Technology Preview with the initial release'
                            ' of Red Hat Enterprise Linux 6 and Red Hat Enterprise Linux 7. As of versions 6.6'
                            ' and 7.4 this technology has been deprecated and removed in RHEL8.'),
                        reporting.ExternalLink(
                            title='Considerations in adopting RHEL 8 - btrfs has been removed.',
                            url='https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/'
                                'considerations_in_adopting_rhel_8file-systems-and-storage_considerations-in-'
                                'adopting-rhel-8#btrfs-has-been-removed_file-systems-and-storage'
                        ),
                        reporting.Severity(reporting.Severity.HIGH),
                        reporting.Groups([reporting.Groups.INHIBITOR, reporting.Groups.FILESYSTEM]),
                        reporting.RelatedResource('driver', 'btrfs')
                    ])
                    break

Final report is generated in “txt“ and “json“ format in /var/log/leapp directory at the end of either leapp preupgrade or leapp upgrade execution.

Reporting tips and good practices

Same type reports share the same title

To make your reports easier to read and aggregate please use same report title for checks that have the same semantics. For example, if your actor is checking for deprecated modules and creating a report entry each time one is found, the way to represent it in the report will be

create_report([
    reporting.Title(
-        'Upgrade process was interrupted because a deprecated module {0} is enabled'.format(module),
+        'Upgrade process was interrupted because a deprecated module is enabled',
    reporting.Summary(
        'Module {0} was surpassed by shiny-new-{0} and therefore it was '
        'removed from RHEL-8. Keeping it in the configuration may '
        'lock out the system thus it is necessary to disable it '
        'before the upgrade process can continue.'.format(module)
    ),
    ...
   ])

A better way, if applicable, would be to first collect all deprecated modules with their descendants and produce one report entry with a list of mappings, like

modules_map = {'moduleA': 'shiny-new-moduleA', 'moduleB': 'shiny-new-moduleB'}
...

create_report([
    reporting.Title(
         'Upgrade process was interrupted because deprecated modules are enabled',
    reporting.Summary(
        'Modules below were surpassed by new modules and '
        'removed from RHEL-8. Keeping them in the configuration may '
        'lock out the system thus it is necessary to disable them '
        'before the upgrade process can continue.\n{}'.format(
            '\n'.join("     - {0} -> {1}".format(old, new) for old, new in modules_map.iteritems()))
    ),
    ...
   ])

Remediations

Apart from the above example you can also suggest a remediation, which is a procedure intended to fix the discovered issue. Currently remediation can come in 3 flavors: a bash command to execute, a hint for manual action and a playbook.

reporting.Remediation(commands=[['alternatives', '--set', 'python', '/usr/bin/python3']])
reporting.Remediation(hint='Please remove the dropped options from your scripts.')
reporting.Remediation(playbook=<link_to_playbook>)

Available Groups

The following groups were originally known as Tags:

'accessibility', 'authentication', 'boot', 'communication', 'drivers', 'email', 'encryption',
'filesystem', 'firewall', 'high availability', 'kernel', 'monitoring', 'network', 'OS facts',
'post', 'python', 'repository', 'sanity', 'security', 'selinux', 'services', 'time management',
'tools', 'upgrade process'

The following groups were originally known as Flags:

'failure', 'inhibitor'

The failure Group is recommended to be used when the report is related to a command or other action failure.

If you need additional report groups, please open a GH issue or a PR, with the description why new required groups are needed.

Related resources

We recognize the following 6 types of resources:

reporting.RelatedResource('package', 'memcached')
reporting.RelatedResource('file', '/etc/passwd')
reporting.RelatedResource('service', 'postfix')
reporting.RelatedResource('directory', '/boot')
reporting.RelatedResource('repository', 'RHEL 7 Base')
reporting.RelatedResource('kernel-driver', 'vmxnet3')
reporting.RelatedResource('pam', 'pam_securetty')

The related resources are especially useful when you have a lot of accompanied objects like files or directories by your report and you would like to present it to the user in a specific way.

Testing your new actor

During development of your new actor, it is expected that you will test your work to verify that results match your expectations. You can do that by manually executing your actor, or writing tests on various levels (i.e unit tests, component tests, E2E tests).

Executing a single actor

You should use snactor tool to run a single actor and verify its output. Assuming that there are no errors, the actor was placed inside a valid leapp repository and snactor tool is aware of such repository, you can call snactor run to execute it. Below we are executing the existing OSReleaseCollector actor that provides information about operating system release from target system. For the snactor run command you can use either the actor’s folder name (osreleasecollector), the actor’s class name (OSReleaseCollector) or the value of the name attribute of the actor’s class (os_release_collector).

# pwd
/usr/share/leapp-repository/repositories/system_upgrade/el7toel8
# snactor run --verbose OSReleaseCollector
2018-11-23 11:16:25.126 INFO     PID: 4293 leapp: Logging has been initialized
2018-11-23 11:16:25.163 INFO     PID: 4293 leapp.repository.system_upgrade_el7toel8: A new repository 'system_upgrade_el7toel8' is initialized at /usr/share/leapp-repository/repositories/system_upgrade/el7toel8
2018-11-23 11:16:25.212 INFO     PID: 4293 leapp.repository.common: A new repository 'common' is initialized at /usr/share/leapp-repository/repositories/common

As you can see the actor is executed without errors. But, by default, snactor does only display data logged by the actor. In order to display messages generated by the actor you can re-run the above command with –print-output option.

# snactor run --verbose --print-output OSReleaseCollector
2018-11-23 11:32:42.193 INFO     PID: 4433 leapp: Logging has been initialized
2018-11-23 11:32:42.218 INFO     PID: 4433 leapp.repository.system_upgrade_el7toel8: A new repository 'system_upgrade_el7toel8' is initialized at /usr/share/leapp-repository/repositories/system_upgrade/el7toel8
2018-11-23 11:32:42.265 INFO     PID: 4433 leapp.repository.common: A new repository 'common' is initialized at /usr/share/leapp-repository/repositories/common
[
  {
    "stamp": "2019-04-30T13:00:13.836063Z",
    "hostname": "leapp-20190429150826",
    "actor": "os_release_collector",
    "topic": "system_info",
    "context": "0ac49430-1b29-4940-92bb-3e81da85f8af",
    "phase": "NON-WORKFLOW-EXECUTION",
    "message": {
      "hash": "8305f6a38dcd266ea02bbd2e7c0b799e871d7dbe8734ea4138da53f4779b993e",
      "data": "{\"id\": \"rhel\", \"name\": \"Red Hat Enterprise Linux Server\", \"pretty_name\": \"Red Hat Enterprise Linux\", \"variant\": \"Server\", \"variant_id\": \"server\", \"version\": \"7.6 (Maipo)\", \"version_id\": \"7.5\"}"
    },
    "type": "OSReleaseFacts"
  }
]

Now we can see that the OSReleaseCollector actor produced a message of the OSReleaseFacts model, containing data like OS Release name and version.

Executing a single actor that uses the workflow config

If you need to execute an actor on its own that requires the IPUConfig model you can execute the actor with the following command:

snactor run --actor-config IPUConfig ActorName

In order for this to work you have to run the IPUWorkflowConfig actor before and save its output, so that the config data is stored in the database for the current session:

snactor run --save-output IPUWorkflowConfig

Since the leapp upgrade repositories support several upgrade paths, the snactor needs to get additional data for the correct execution, like the release of the target system, the flavor specification, etc. If you see similar errors when running snactor

leapp.models.fields.ModelViolationError: The value of "target" field is None, but this is not allowed

please, set the following environment variables (adjust the LEAPP_UPGRADE_PATH_TARGET_RELEASE as needed):

export LEAPP_UPGRADE_PATH_TARGET_RELEASE=8.6
export LEAPP_UPGRADE_PATH_FLAVOUR=default
snactor run --save-output IPUWorkflowConfig

Executing the whole upgrade workflow with the new actor

Finally, you can make your actor part of the “leapp upgrade” process and check how it behaves when executed together with all the other actors in the workflow. Assuming that your new actor is tagged properly, being part of IPUWorkflow, and part of an existing phase, you can place it inside an existing leapp repository on a testing RHEL 7 system. All Leapp components (i.e actors, models, tags) placed inside /etc/leapp/repos.d/system_upgrade/el7toel8/ will be used by the “leapp upgrade” command during upgrade process.

Verifying correct communication between actors

Leapp provides another actor, named CheckOSRelease, that consumes messages from model OSReleaseFacts and produces an error message in case system OS Release is not supported by Leapp upgrade process. In order to consume such message, OSReleaseCollector actor needs to be executed before CheckOSRelease and its message needs to be stored inside Leapp database. This process is controlled by the framework during the execution of “leapp upgrade” command.

But, if you want to execute it manually, for test purposes, you can also use snactor for it. First we need to make sure that all messages that will be consumed are generated and stored. For this example, this means running OSReleaseCollector actor with the –save-output option of snactor:

# snactor run --verbose --save-output OSReleaseCollector
2018-11-23 13:06:30.706 INFO     PID: 17996 leapp: Logging has been initialized
2018-11-23 13:06:30.753 INFO     PID: 17996 leapp.repository.system_upgrade_el7toel8: A new repository 'system_upgrade_el7toel8' is initialized at /usr/share/leapp-repository/repositories/system_upgrade/el7toel8
2018-11-23 13:06:30.803 INFO     PID: 17996 leapp.repository.common: A new repository 'common' is initialized at /usr/share/leapp-repository/repositories/common

Now, you can execute CheckOSRelease actor and verify that it consumes the previously generated message and produces a message saying that the target system is not supported by Leapp upgrade process. You don’t need to specify which message will be consumed, snactor will take care of it.

# snactor run --verbose --print-output CheckOSRelease
2018-11-23 13:11:15.549 INFO     PID: 18126 leapp: Logging has been initialized
2018-11-23 13:11:15.578 INFO     PID: 18126 leapp.repository.system_upgrade_el7toel8: A new repository 'system_upgrade_el7toel8' is initialized at /usr/share/leapp-repository/repositories/system_upgrade/el7toel8
2018-11-23 13:11:15.617 INFO     PID: 18126 leapp.repository.common: A new repository 'common' is initialized at /usr/share/leapp-repository/repositories/common
[
  {
    "stamp": "2019-04-30T13:12:05.706317Z",
    "hostname": "leapp-20190429150826",
    "actor": "check_os_release",
    "topic": "report_topic",
    "context": "0ac49430-1b29-4940-92bb-3e81da85f8af",
    "phase": "NON-WORKFLOW-EXECUTION",
    "message": {
      "hash": "ceaf419907ec78a894334b2a331a9ebb0c5a7847c18afc6d7546ba6656959e0d",
      "data": "{\"report\": \"{\\\"audience\\\": \\\"sysadmin\\\", \\\"detail\\\": {\\\"related_resources\\\": [{\\\"scheme\\\": \\\"file\\\", \\\"title\\\": \\\"/etc/os-release\\\"}]}, \\\"flags\\\": [\\\"inhibitor\\\"], \\\"severity\\\": \\\"high\\\", \\\"summary\\\": \\\"The supported OS versions for the upgrade process: 7.6\\\", \\\"tags\\\": [\\\"sanity\\\"], \\\"title\\\": \\\"Unsupported OS version\\\"}\"}"
    },
    "type": "Report"
  }
]

To flush all saved messages from the repository database, run snactor messages clear.

Writing tests for an actor

Read the tutorial for writing and running unit and component tests

Best practices

Read the best practices document and Python guidelines.

Contributing actors to the Leapp project

Currently all Leapp elements (i.e. actors, models, tags) are stored under a public GitHub repository.

All new content that needs to be part of Leapp release distributed to all users should be proposed as a Pull Request in this repository.

Before submitting your work for review, make sure you have read and followed the contribution guidelines:

Contribution guidelines for writing actors

This pull request gives a good example of both guidelines-driven actor implementation and thorough test coverage.

FAQ

In which existing workflow phase should I place my new actor?

You can decide that based on the description of the phases this information is available in the code and diagram here. Please note that if your actor depends on some message generated by another actor, it cannot be executed in a phase before the phase of such actor. In a similar way, if your actor produces data, it needs to be executed before the actor consuming the data.

How to stop the upgrade in case my actor finds a problem with the system setup?

The process of inhibiting the upgrade is done by the VerifyCheckResult actor, executed during the ReportPhase. This actor consumes messages from the Report model and if any message with the flag “inhibitor” was generated it will inhibit the upgrade process. So, your actor needs to produce a Report message with the flag “inhibitor” before the upgrade process gets to the ReportPhase. Read more about inhibiting the upgrade process here.

How to stop execution of my actor in case of an unexpected error?

It’s good practice to code defensively so the code is robust. The actor should detect unexpected input or result of some operation and exit gracefully instead of tracebacking. In case you detect an unexpected behavior, let the framework know about it by raising StopActorExecutionError. Framework will act based on the setting of the upgrade workflow in one of the following three ways:

• end the upgrade process right away, or
• end the upgrade process after finishing the current phase, or
• do not end the upgrade process at all and continue with logging the issue only.

How does the logging work?

For logging of messages not to be visible to the user by default but rather for issue investigation purposes, use simply self.log.<level>(msg) within the actor. Or, within the actor’s library this way:

from leapp.libraries.stdlib import api
api.current_logger().<level>(msg)

The usual logging practice of Python’s logger library applies, i.e. the <level> can be for example debug, warning, error, critical, etc. Leapp framework will take care of these messages and provide them through appropriate channels (stdout/stderr, log files, journalctl, audit table in /var/lib/leapp/leapp.db).

What Python version my actor/tests code should be compatible with?

Python 2.7+/3.6+, but keep in mind that the resulting code has to be both py2 and py3 compatible.

How to add tests to my new actor?

Under “tests” folder, an actor can have Python files containing tests that will be executed using PyTest. Leapp provide tests utilities to simulate actor consuming and checking actor production. Please, refer to detailed Leapp documentation about how to write tests.

For further information read: Writing tests for actors.

How to use libraries or data files in my new actor?

An actor can have data files under the “files” folder and Python libraries under the “libraries” folder. Leapp does provide utilities to help an actor to access the files and libraries in these folders. Please, refer to detailed Leapp documentation about this.

Where can I report an issue or RFE related to the framework or other actors?

You can open a GitHub issues:

When filing an issue, include:

• How to reproduce it
• All files in /var/log/leapp.
• The /var/lib/leapp/leapp.db file.

Where can I seek help?

We’ll gladly answer your questions and lead you to through any troubles with the actor development. You can reach us, the OS and Application Modernization Group, at Libera.Chat IRC server in channel #leapp.

How to properly inhibit the RHEL 7 to 8 upgrade process

Process Inhibition

With latest changes on Leapp and with new actors added to the el7toel8 Leapp repository, any actor can inhibit the upgrade process by producing a specific message when a problem is found. The message model to use in this case is Report. If there is at least one Report message with the 'inhibitor' group produced before the Report phase, the upgrade will be stopped in the Reports phase, in which the messages are being collected. It means that any Report message produced after the Report phase will not have inhibiting effect. The details mentioned in the Report messages will be part of the report available to the user to review.

Sample Actor

Let’s start with a very simple actor that will verify if system architecture is supported (this actor may be removed in the future as more archs will be supported):

import platform

from leapp.actors import Actor
from leapp.tags import ChecksPhaseTag


class CheckSystemArch(Actor):
   """
    Check if system is running at a supported architecture. If no, inhibit the upgrade process.

    Base on collected system facts, verify if current architecture is supported, otherwise produces
    a message to inhibit upgrade process
    """

    name = 'check_system_arch'
    consumes = ()
    produces = ()
    tags = (ChecksPhaseTag,)

    def process(self):
        if platform.machine() != 'x86_64':
            self.log.info("Unsupported arch!")

If this actor is executed using snactor tool in a system with unsupported architecture, we will see the following output on log:

$ snactor run CheckSystemArch --verbose
2019-04-16 15:08:59.622 INFO     PID: 1996 leapp: Logging has been initialized
2019-04-16 15:08:59.638 INFO     PID: 1996 leapp.repository.sandbox: A new repository 'sandbox' is initialized at /home/leapp/sandbox
2019-04-16 15:08:59.695 INFO     PID: 2021 leapp.actors.check_system_arch: Unsupported arch!

If, instead of only adding a message to the log, the actor writer wants to make sure that the upgrade process will be stopped in case of unsupported arch, the actor needs to produce a Report message using one of the report_* functions from the reporting shared library with the 'inhibitor' group.

import platform

from leapp.actors import Actor
from leapp.reporting import Report, create_report
from leapp import reporting
from leapp.tags import ChecksPhaseTag, IPUWorkflowTag


class CheckSystemArch(Actor):
    """
    Check if system is running at a supported architecture. If no, inhibit the upgrade process.

    Base on collected system facts, verify if current architecture is supported, otherwise produces
    a message to inhibit upgrade process
    """

    name = 'check_system_arch'
    consumes = ()
    produces = (Report,)
    tags = (ChecksPhaseTag, IPUWorkflowTag)

    def process(self):
        if platform.machine() != 'x86_64':
            create_report([
                reporting.Title('Unsupported architecture'),
                reporting.Summary('Upgrade process is only supported on x86_64 systems.'),
                reporting.Severity(reporting.Severity.HIGH),
                reporting.Groups([reporting.Groups.INHIBITOR, reporting.Groups.SANITY]),
            ])

Running the actor again, it is possible to verify that a new message was generated. We will still use snactor tool to run the actor, but passing --print-output this time to output all generated messages by the actor:

$ snactor run CheckSystemArch --verbose --print-output
2019-04-16 15:20:32.74  INFO     PID: 2621 leapp: Logging has been initialized
2019-04-16 15:20:32.94  INFO     PID: 2621 leapp.repository.sandbox: A new repository 'sandbox' is initialized at /home/leapp/sandbox
[
  {
    "stamp": "2019-09-05T12:58:56.342095Z",
    "hostname": "leapp-20190904152934",
    "actor": "check_system_arch",
    "topic": "report_topic",
    "context": "9a064a30-5d16-44ba-a807-b7f08b3c4215",
    "phase": "NON-WORKFLOW-EXECUTION",
    "message": {
      "hash": "dc95adcfca56eae62b7fcceeb0477a6d8257c3dddd1b05b879ebdcf05f59d504",
      "data": "{\"report\": \"{\\\"audience\\\": \\\"sysadmin\\\", \\\"groups\\\": [\\\"inhibitor\\\", \\\"sanity\\\"], \\\"severity\\\": \\\"high\\\", \\\"summary\\\": \\\"Upgrade process is only supported on x86_64 systems.\\\", \\\"title\\\": \\\"Unsupported architecture\\\"}\"}"
    },
    "type": "Report"
  }
]

Or to inspect closely the message.data field, we could use jq tool:

snactor run CheckSystemArch --verbose --print-output | jq '.[] | .message.data | fromjson'
{
  "report": "{\"audience\": \"sysadmin\", \"groups\": [\"inhibitor\", \"sanity\"], \"severity\": \"high\", \"summary\": \"Upgrade process is only supported on x86_64 systems.\", \"title\": \"Unsupported architecture\"}"
}

This is all that an actor needs to do in order to verify if some condition is present on the system and inhibit the upgrade process based on that check.

After all the system checks are executed by different actors, an existing actor named VerifyCheckResults is scheduled to run in the Leapp upgrade workflow. If some Report message with the 'inhibitor' group was generated by some previous execution of another actor in any previous phase of the workflow, like the sample one we just wrote, the following output will be displayed to the user:

$ leapp upgrade
(...)
2019-04-16 15:36:54.696 INFO     PID: 7455 leapp.workflow: Starting phase Reports
2019-04-16 15:36:54.715 INFO     PID: 7455 leapp.workflow.Reports: Starting stage Before of phase Reports
2019-04-16 15:36:54.764 INFO     PID: 7455 leapp.workflow.Reports: Starting stage Main of phase Reports
2019-04-16 15:36:54.788 INFO     PID: 7455 leapp.workflow.Reports: Executing actor verify_check_results
2019-04-16 15:36:54.923 INFO     PID: 7455 leapp.workflow.Reports: Starting stage After of phase Reports
2019-04-16 15:36:54.970 INFO     PID: 7455 leapp.workflow: Workflow interrupted due to the FailPhase error policy

============================================================
                        ERRORS
============================================================

2019-04-16 15:36:54.871634 [ERROR] Actor: verify_check_results Message: Unsupported arch
2019-04-16 15:36:54.888818 [ERROR] Actor: verify_check_results Message: Ending process due to errors found during checks, see /var/log/leapp-report.txt for detailed report.

============================================================
                     END OF ERRORS
============================================================

Using messaging to send data between actors

The Leapp framework uses messages to send data to other actors that are executed afterward. Messages are defined through the models declared earlier. Actors can consume these messages and produce data based on their input.

As an example, the actors consume Hostname messages, resolve IPs for those hostnames, and create the ResolvedHostname model to send a new type of message.

Creating the ResolvedHostname model

Create the ResolvedHostname model by using the snactor tool.

$ snactor new-model ResolvedHostname

Assign the SystemInfoTopic to the new model and add two fields:

• The name field represents the hostname.
• The ips field contains a list of strings with IPv4 or IPv6 addresses.

from leapp.models import Model, fields
from leapp.topics import SystemInfoTopic


class ResolvedHostname(Model):
    topic = SystemInfoTopic
    name = fields.String()
    ips = fields.List(fields.String())

By default all fields which are not nullable e.g:

fields.Nullable(fields.String())

are required.

Creating a message consuming actor

Create a new actor that resolves the IPs for the hostnames:

$ snactor new-actor IpResolver

Import the ScanTag from leapp.tags, and the models Hostname and ResolvedHostname from leapp.models. To retrieve the Hostname messages to process their data, set it in the consumes tuple. The result will be ResolvedHostname, so set the type in the produces tuple.

The tags tuple gets extended with the ScanTag. Now, import the socket library.

To enable actors to consume messages, use the consume method, and pass the type of the message to be consumed. This is necessary to filter out the messages. In theory, all messages can be consumed, but it is not recommended. If you would like to change your code later and consume more types of messages, you might end up with unexpected results. Always specify the consume method for all types of messages to be consumed instead of retrieving all messages unfiltered.

Now, perform the resolving of the hostnames, and produce a new message.

See the example of the code:

import socket

from leapp.actors import Actor
from leapp.tags import ScanTag
from leapp.models import Hostname, ResolvedHostname


class IpResolver(Actor):
    """
    No description is provided for the ip_resolver actor.
    """
    name = 'ip_resolver'
    consumes = (Hostname,)
    produces = (ResolvedHostname,)
    tags = (ScanTag,)

    def process(self):
        self.log.info("Starting to resolve hostnames")
        for hostname in self.consume(Hostname):
            resolved = socket.getaddrinfo(
                    hostname.name, None, 0, socket.SOCK_STREAM,
                    socket.IPPROTO_TCP)
            # Filtering out link local IPv6 addresses which contain a %
            ips = [entry[4][0] for entry in resolved if not '%' in entry[4][0]]
            self.produce(ResolvedHostname(name=hostname.name, ips=ips))

Storing messages in the repository data for reuse

The snactor framework tool saves the output of actors as locally stored messages, so that they can be consumed by other actors that are being developed.

To make the data consumable, run the actor producing the data with the –save-output option:

$ snactor run --save-output HostnameScanner

The output of the actor is stored in the local repository data file, and it can be used by other actors. To flush all saved messages from the repository database, run snactor messages clear.

Testing the new actor

With the input messages available and stored, the actor can be tested.

$ snactor run --print-output IpResolver
2018-04-03 09:01:40.114 INFO     PID: 28841 leapp: Logging has been initialized
2018-04-03 09:01:40.115 INFO     PID: 28841 leapp.repository.tutorial: New repository 'tutorial' initialized at /home/evilissimo/devel/tutorial
2018-04-03 09:01:40.166 INFO     PID: 28860 leapp.actors.ip_resolver: Starting to resolve hostnames
[
  {
    "stamp": "2018-04-03T09:01:40.225635Z",
    "hostname": "actor-developer",
    "actor": "ip_resolver",
    "topic": "system_info",
    "context": "TESTING-CONTEXT",
    "phase": "NON-WORKFLOW-EXECUTION",
    "message": {
      "hash": "5fa31cac2237248f7c40df6a0190cc6acdd8a06c53c593aac2d93b8b3db58a70",
      "data": "{\"ips\": [\"fd15:4ba5:5a2b:1003:b14d:ed7:6c03:76cd\", \"192.168.89.153\"], \"name\": \"actor-developer\"}"
    },
    "type": "ResolvedHostname"
  }
]

Screencast

Asking user questions

Leapp framework uses dialogs to ask user for any additional information an actor needs that can not be deduced automatically. Dialogs contain Components which represent individual questions. Complete list of component types can be found in documentation.

As an example we will change IpResolver actor in a way that user will decide which hostnames will be resolved.

Creating the dialog

Import Dialog and MultipleChoiceComponent from leapp.dialog and leapp.dialog.components respectively. Create an instance of Dialog, specifying scope which is used to identify data in the answer file, reason to explain user what the data will be used for and components. For each component specify key which will be used to get answer to specific question, label and description. You can also specify default or choices if the component support them, but as choices in our example depend on consumed data, we will specify them later.

See the example of the code:

from leapp.dialogs import Dialog
from leapp.dialogs.components import MultipleChoiceComponent

class IpResolver(Actor):
    """
    No description has been provided for the ip_resolver actor.
    """
    name = 'ip_resolver'
    consumes = (Hostname,)
    produces = (ResolvedHostname,)
    tags = (ScanTag,)
    dialogs = (Dialog(scope='ipresolver', reason='Confirmation', components=(
        MultipleChoiceComponent(key='hostname', label='Please select hostnames to resolve',
                                description='No description'),)),)

Using the dialog and the answers

To pose a question that needs to be answered use get_answers method and pass the dialog containing the question.

Be aware that from actor writers‘ perspective get_answers function is not blocking workflow execution - no interactivity will be introduced by adding a get_answers call in actor‘s process() method; the value returned will correspond to the option saved in answerfile if one is found or an empty dict is returned otherwise.

You don‘t need to specifically inhibit execution in case user doesn‘t provide an option - the leapp framework will take care of that. A report entry with inhibitor type will be automatically created if the actor with a dialog is executed during leapp preupgrade or leapp upgrade run and no prerecorded user choice has been registered in answerfile. All interactivity that is necessary to record user options is part of leapp answer cli command; as an alternative you can modify the generated answerfile manually with an editor of choice to contain the desired choices.

For example, to get the hostnames selected by user from answers by component key and resolve them:

def process(self):
    self.log.info("Starting to resolve hostnames")
    component = self.dialogs[0].component_by_key('hostname')
    component.choices = [hostname.name for hostname in self.consume(Hostname)]
    component.default = component.choices
    answer = self.get_answers(self.dialogs[0])
    for hostname in answer.get('hostname'):
        resolved = socket.getaddrinfo(
                hostname, None, 0, socket.SOCK_STREAM,
                socket.IPPROTO_TCP)
        # Filtering out link local IPv6 addresses which contain a %
        ips = [entry[4][0] for entry in resolved if not '%' in entry[4][0]]
        self.produce(ResolvedHostname(name=hostname, ips=ips))

Explaining the dialogs processing mechanism during the upgrade

The upgrade itself, from the operator‘s point of view, consists of 3 distinct stages: preupgrade, remediate and upgrade.

Leapp preupgrade stage should be treated as “non-invasive system upgradeability analysis“, when the upgrade workflow stops right after preliminary system facts collection phases and a preupgrade report containing all the information about potential issues is generated. If an actor containing dialog is discovered during this stage, a specific message is added to the preupgrade report file saying that for the successful upgrade the operator should record their decision in the answerfile.

As far as the dialogs are concerned, leapp remediate stage is intended specifically for answerfile management. The operator has the option to manually edit the answerfile with editor of choice or use leapp answer command to fill the answerfile (usually located at /var/log/leapp/answerfile) with choices for the discovered dialogs. After modifying the answerfile you can check system upgradeability by rerunning leapp preupgrade.

Leapp upgrade stage should be run only when leapp preupgrade successfully passes. In case any unanswered/bad choice dialogs are encountered the upgrade process will stop and the report file will be generated telling the operator what has gone wrong.

Linking repositories

Snactor allows you to link repositories, which is needed if you want to use actors, tags, models, etc. from another repository.

Firstly, leave the current repository and create a new one called tutorial-linked and enter its folder.

To link the tutorial repository, run the snactor tool:

    $ snactor repo link --path ../tutorial

You can also link the repository using name or UUID of the repository. When using repository name, beware that the first matching name will be linked. Therefore it‘s recommended to rather link repositories by path or repository id.

Now, you will be able to use actors, tags, models, and topics from tutorial repository. You can check this using snactor discover command:

    $ snactor discover --all

A screen cast of the steps above

Working with workflows

Creating a workflow

To create a new workflow, create a tag with the same name, and then the workflow.

$ snactor workflow new Example

This procedure creates the Example workflow boilerplate:

from leapp.workflows import Workflow
from leapp.workflows.phases import Phase
from leapp.workflows.flags import Flags
from leapp.workflows.tagfilters import TagFilter
from leapp.workflows.policies import Policies
from leapp.tags import ExampleWorkflowTag


class ExampleWorkflow(Workflow):
    name = 'Example'
    tag =  ExampleWorkflowTag
    short_name = 'example'
    description = '''No description has been provided for the Example workflow.'''

    # Template for a phase definition. The order in which the phase classes are defined
    # within the Workflow class represents the execution order.
    #
    # class PhaseName(Phase):
    #    name = 'phase_name'
    #    filter = TagFilter(PhaseTag)
    #    policies = Policies(Policies.Errors.FailPhase,
    #                        Policies.Retry.Phase)
    #    flags = Flags()

Defining workflow phases

To add a phase, define a new subclass within the workflow deriving from the Phase class.

We will create a phase called ScanPhase, which is supposed to be handling all actors that are defining the ScanTag for the phase, and the ExampleWorkflowTag for the workflow.

Phases have policies that control the execution of the workflow. These policies can control the behavior in case of errors that are reported by actors. Additionally, the retry behavior can be specified. The retry behavior allows to specify how to recover from failing workflow executions without having to rerun the whole process or to disable the retry ability entirely.

In this scenario, we set the policy to fail the whole phase, but let all actors run, even if one of the actors fails. And for the retry policy, we specify to restart the whole phase from the beginning.

The definition of the ScanPhase class: (Note: ScanTag has to be imported)

class ScanPhase(Phase):
    name = 'scan phase'
    filter = TagFilter(ScanTag)
    policies = Policies(Policies.Errors.FailPhase,
                        Policies.Retry.Phase)
    flags = Flags()

Now, we will define an imaginary reports phase that would process the data produced by the ScanPhase and create one or more reports.

For this, the ReportsTag is used.

This time, we will make the phase fail immediately, and stop the workflow execution once one of the actors fails. And we disallow the retry by disabling it, which means that the phase cannot be recovered from. This implies that the workflow has to be restarted from the very beginning.

The definition of the ReportsPhase class: (Note: ReportsTag has to be imported)

class ReportsPhase(Phase):
    name = 'reports phase'
    filter = TagFilter(ReportsTag)
    policies = Policies(Policies.Errors.FailImmediately,
                        Policies.Retry.Disabled)
    flags = Flags()

The whole example workflow:

from leapp.workflows import Workflow
from leapp.workflows.phases import Phase
from leapp.workflows.flags import Flags
from leapp.workflows.tagfilters import TagFilter
from leapp.workflows.policies import Policies
from leapp.tags import ExampleWorkflowTag, ScanTag, ReportsTag


class ExampleWorkflow(Workflow):
    name = 'Example'
    tag =  ExampleWorkflowTag
    short_name = 'example'
    description = '''No description has been provided for the Example workflow.'''

    class ScanPhase(Phase):
        name = 'scan phase'
        filter = TagFilter(ScanTag)
        policies = Policies(Policies.Errors.FailPhase,
                            Policies.Retry.Phase)
        flags = Flags()

    class ReportsPhase(Phase):
        name = 'reports phase'
        filter = TagFilter(ReportsTag)
        policies = Policies(Policies.Errors.FailImmediately,
                            Policies.Retry.Disabled)
        flags = Flags()

Testing the workflow execution

To test the execution of workflows, use the snactor tool.

The snactor tool is run with the above workflow in the tutorial repository, which contains the HostnameScanner and IpResolver actors.

$ snactor workflow run Example
2018-04-04 09:23:54.767 INFO     PID: 38687 leapp: Logging has been initialized
2018-04-04 09:23:54.772 INFO     PID: 38687 leapp.repository.tutorial: New repository 'tutorial' initialized at /home/evilissimo/devel/tutorial
2018-04-04 09:23:54.797 INFO     PID: 38687 leapp.workflow: Starting workflow execution: Example - ID: c4615ed9-662b-49c6-8389-19f6128cdac5
2018-04-04 09:23:54.804 INFO     PID: 38687 leapp.workflow: Starting phase scan phase
2018-04-04 09:23:54.805 INFO     PID: 38687 leapp.workflow.scan phase: Starting stage Before of phase scan phase
2018-04-04 09:23:54.811 INFO     PID: 38687 leapp.workflow.scan phase: Starting stage Main of phase scan phase
2018-04-04 09:23:54.813 INFO     PID: 38687 leapp.workflow.scan phase: Executing actor hostname_scanner
2018-04-04 09:23:54.820 INFO     PID: 38695 leapp.workflow.scan phase.hostname_scanner: Starting to scan for the hostname
2018-04-04 09:24:05.153 INFO     PID: 38695 leapp.workflow.scan phase.hostname_scanner: Finished scanning for the hostname, found = actor-developer
2018-04-04 09:24:05.157 INFO     PID: 38687 leapp.workflow.scan phase: Executing actor ip_resolver
2018-04-04 09:24:05.165 INFO     PID: 38696 leapp.workflow.scan phase.ip_resolver: Starting to resolve hostnames
2018-04-04 09:24:10.325 INFO     PID: 38687 leapp.workflow.scan phase: Starting stage After of phase scan phase
2018-04-04 09:24:10.328 INFO     PID: 38687 leapp.workflow: Starting phase reports phase
2018-04-04 09:24:10.330 INFO     PID: 38687 leapp.workflow.reports phase: Starting stage Before of phase reports phase
2018-04-04 09:24:10.332 INFO     PID: 38687 leapp.workflow.reports phase: Starting stage Main of phase reports phase
2018-04-04 09:24:10.334 INFO     PID: 38687 leapp.workflow.reports phase: Starting stage After of phase reports phase

Adding an actor to a workflow

To have an actor added to a specific workflow phase, assign two tags:

1. The workflow tag In the Example workflow above this was the ExampleWorkflowTag.
2. The phase tag In case of the ScanPhase it is the ScanTag, in the Reports phase the ReportsTag.

In the actor, the tags field is filled like this:

    tags = (ExampleWorkflowTag, ScanTag)

To have an actor added to any workflow when a phase tag is used, add the .Common attribute of the tag:

    tags = (ScanTag.Common,)

Workflow APIs

Using Workflow APIs

To start using a Workflow API you have to know which API you are going to be using. In this case we are assuming that the repository defines a v2 namespace with an API definition of MyExampleAPI.

Now to use it, you have to import the API class object you want to use and specify the type of the class in the apis field. This is necessary so all the messages that are consumed and produced by the API can be picked up by the framework and the actor can depend on them.

Here is an example of the usage:

from leapp.actors import Actor
from leapp.workflows.api.v2 import MyExampleAPI

class MyWorkflowAPIUsingActor(Actor):
    """ An example actor consuming the v2.MyExampleAPI Workflow API """
    produces = ()
    consumes = ()
    apis = (MyExampleAPI,)

    def process(self):
        api = MyExampleAPI()
        api.second(value=10)

General workings of the Workflow API feature

In the leapp repository, you want to create the API in, create a directory called apis. Now within the apis directory anything that is in there can be imported with the leapp.workflows.api prefix. An example:

You created a module $repository/apis/example.py this module can now be imported like this:

from leapp.workflows.api import example

Defining Workflow APIs

A basic Workflow API is defined like this:

from leapp.workflows.api import WorkflowAPI

class MyExampleAPI(WorkflowAPI):
    pass

This of course is no good as this wouldn‘t do anything just yet. APIs are nothing else than python classes and one just adds a method to it to make it available.

Here is a simple function added to the API:

from leapp.workflows.api import WorkflowAPI

class MyExampleAPI(WorkflowAPI):
    def first(self):
        return 'First API function result'

Working with messages

Now if your API is going to consume or produce any messages, those messages have to be declared the same way as they are declared for Actors. Any Actor that specifies to use this API, will automatically inherit all the messages consumed or produced by the API.

Let‘s assume we have defined two models: Consumed and Produced where the definitions look like this:

from leapp.models import fields, Model
from leapp.topics import ExampleTopic

class Consumed(Model):
    topic = ExampleTopic


class Produced(Model):
    topic = ExampleTopic
    consumed = fields.Model(Consumed)
    value = fields.Integer()

We are now going to define a method second which expects one parameter called value. That function will produce a message of type Produced for each message of type Consumed.

from leapp.libraries.stdlib import api
from leapp.messages import Consumed, Produced
from leapp.workflows.api import WorkflowAPI

class MyExampleAPI(WorkflowAPI):
    consumes = (Consumed,)
    produces = (Produced,)

    def first(self):
        return 'First API function result'

    def second(self, value):
        # Creates a new message `Produced` for each message of type `Consumed` with the additional value passed by 
        # the caller.
        for consumed in api.consume(Consumed):
            self.produce(Produced(consumed=consumed, value=value))

Tests

Workflow APIs support having tests defined for them. We actually encourage you to define them for your APIs.

Tests for APIs are supposed to be defines in the apis/tests directory.

Dependencies

Workflow APIs can depend on another Workflow API, to allow API compositiion. Actors using APIs with dependencies on other APIs just have to specify the API they want to use and do not need to know that those depend on other APIs.

All consumes/produces are recursively summarized and joined with the Actors direct message dependencies. The dependency of a Workflow API on another Workflow API is expressed the same way as it is expressed for actors, using the apis field.

API definition best practises

Be always explicit about what messages the API consumes or produces

Even though a dependent API might consume or produce the message already, a modification of the dependent API might cause failures. Also from a readability point of view it is much clearer what kind of messages the API works on.

Keep API interfaces small and on the same topic

Do not try to make one API for all the things, you might quickly end up in a scenario where you will cause an actor being unable to produce a message that your API already consumes, which causes a dependency resolution failure in the framework.

Use lazy evaluation and caching for messages consumed to improve efficiency

Some messages need to be used multiple times during multiple API calls. Imagine an API interface like this:

class InstalledPackagesAPI(WorkflowAPI):
    consumes = (InstalledRPM,)

    def has_package(self, name): pass

Now an implementation of has_package could look like this:

def has_package(self, name):
    lookup = {rpm.name for rpm in leapp.stdlib.api.consume(InstalledRPM)}
    return name in lookup

Which is fine, if this is called only once per actor. However in case it needs to be run multiple times consider this:

class InstalledPackagesAPI(WorkflowAPI):
    consumes = (InstalledRPM,)

    def __init__(self):
        self._rpm_lookup = None

    @property
    def rpm_lookup(self):
        if not self._rpm_lookup:
            self._rpm_lookup = {rpm.name for rpm in leapp.stdlib.api.consume(InstalledRPM)}
        return self._rpm_lookup

    def has_package(self, name):
        return name in self.rpm_lookup

This way the API caches the package names and lazy loads them when needed for as long as the instance of this object lives.

Write tests that verify API contract compliance

We highly encouraged you to write tests for the API that ensure the contract of the API is fulfilled. That means that data types have guaranteed fields and data are, as they are documented.

This will allow you API breaking changes to be detected instantly once you try to commit changes that would break it, and enforce the API to be updated to be compliant. Not meaning that the tests should be fixed, but the code to be changed in a way it will stay compatible.

As an example if you rename a field in a Model that previously has been returned as a data type, instead you create a new data type or update the instance and make it be compatible with the previous version of the model.

# Before
class RPM(Model):
    package_name = fields.String()
    version = fields.String()

# After
class RPM(Model):
    name = fields.String()
    version = fields.List(fields.Integer)
    version_string = fields.String()

# Possible mitigation
class RPMAPIWrapper(object):
    def __init__(self, rpm):
        self._rpm = rpm

    @property
    def package_name(self):
        return self._rpm.name

    @property
    def version(self):
        return self._rpm.version_string

And on the fly when returning the data you wrap all new Models of RPM in that APIWrapper. This is only an example, of course. You can also just create a normal object.

The gist of this is to ensure API stability and that any one using the API is not caught by surprise and their code gets broken.

Writing tests for actors

The Leapp framework provides support for easily writing unit and component tests for actors and also allows easy execution of the whole actors within those tests. See this document to find out what is the difference between unit and component tests.

Getting started with writing tests

Tests are considered being part of the actor and we do not only encourage but basically require you to write tests if you want the actors to be accepted into the git repository. To read more about what we ask from you when submitting your work for our review, see Contributing guidelines for writing actors.

Tests for an actor are to be placed within the actor‘s directory, in a subdirectory called tests. The layout for an actor MyActor in the repository could look like this:

actors\
    myactor\
        actor.py
        tests\
            component_test_my_actor.py
            unit_test_my_actor.py

Naming conventions

To have the tests found and carried out by pytest framework, all test functions have to:

• reside in test_*.py or *_test.py files,
• be prefixed by test_.
• test modules should have unique names, and we use the following convention test_*_{actor_name}.py. For example: test_unit_sctpconfigread.py or component_test_sctpconfigread.py

See the pytest documentation.

Writing tests that execute the whole actor - component tests

Now let‘s assume you want to write a test that executes the actor. This is how your component_test_{actor_name}.py from above could look like:

def test_actor_execution(current_actor_context):
    current_actor_context.run()

This example makes use of the current_actor_context fixture and will execute the MyActor actor.

Now if you would want to check that it produced an imaginary model called ProducedExampleModel you can check this with the help of the consume method.

from leapp.models import ProducedExampleModel

def test_actor_execution(current_actor_context):
    current_actor_context.run()
    assert current_actor_context.consume(ProducedExampleModel)

If your actor requires input data that it can consume, you can specify the input data with the help of the feed method of the current_actor_context fixture.

from leapp.models import ConsumedExampleModel, ProducedExampleModel

def test_actor_execution(current_actor_context):
    current_actor_context.feed(
        ConsumedExampleModel(value=1),
        ConsumedExampleModel(value=2))
    current_actor_context.run()
    assert current_actor_context.consume(ProducedExampleModel)
    assert current_actor_context.consume(ProducedExampleModel)[0].value == 3

In case your actor uses ConfigModel for consuming workflow specific configuration, run the actor in the test as:

current_actor_context.run(config_model=ConfigModel(os_release=OSRelease()))

Fixtures

The unit testing support was first implemented with the help of pytest fixtures. Nowadays, we encourage you to use only the current_actor_context fixture mentioned above. However the other fixtures have been preserved and are still possible to use - see their documentation.

Testing actors that modify the OS

Replace the functions that read or modify the system with functions that do not alter the system and return what you specify in the test. This is called mocking. Currently it is not possible to mock any function while using the current_actor_context.run(). But, mocking is possible in an actor‘s library. For that, read further.

Testing private actor library - unit tests

Leapp allows actors to relocate their code into actor private library. This allows for better testability since the current implementation of Leapp does not allow tests to import anything from the actor.py. Thus the code that is supposed to be unit tested is necessary to move into the actor‘s private library. Modules from the private library can then be imported not only from the actor.py but also from the test modules.

Let‘s assume your actor has a private library module called private_{actor_name}.py.

actors\
    myactor\
        actor.py
        libraries\
            private_myactor.py
        tests\
            unit_test_my_actor.py

And the private_my_actor.py looks like this:

def my_function(value):
    return value + 42

You can easily write a test for this library like this:

    from leapp.libraries.actor import private_my_actor

    def test_my_actor_library():
        assert private.my_function(0) == 42
        assert private.my_function(1) == 43
        assert private.my_function(-42) == 0

Using repository resources during test runtime

It is possible to test other things in the repository your actor is in and in the linked repositories. For example you may want to test shared libraries, models, etc.

    from leapp.libraries.common import useful_library
    from leapp.models import ExampleModel, ProcessedExampleModel

    def my_repository_library_test():
        e = ExampleModel(value='Some string')
        result = shared.process_function(e)
        assert type(result) is ProcessedExampleModel

Actors‘s test dependencies

If your actor‘s tests require a special package for their execution, create a Makefile in the actor’s root directory with an install-deps target calling yum install -y.

$ cat actors/myactor/Makefile
install-deps:
	yum install -y my-tests-need-this-pkg

Note: Dependencies defined the way mentioned above is for test execution only. If your actor requires any package when executed as part of a workflow, it needs to be specified in a leapp-repository specfile.

Running the tests

Preparing the environment

To execute unit tests of actors from all Leapp repositories in the leapp-repository GitHub repository, you need to install test dependencies for all actors by running make install-deps.

Actor‘s tests

Makefile of leapp-repository provides target for testing your actors. Issue make test in the root directory of the leapp-repository GitHub repository to test all actors.

You can also run tests by simply running pytest

To test specific actor using makefile, set ACTOR environment variable:

ACTOR=myactor make test

or

pytest {PATH_TO_ACTOR}

It is also possible to run only selected tests based on their name:

pytest -k "vim"    # to run all tests contains vim in name
pytest -k "not vim"    # to run all tests, which not contains vim in name

More examples could be found in the pytest documentation

Shared libraries‘ tests

To run tests of all shared libraries (i.e. libraries stored in repo/system_upgrade/el7toel8/libraries) environment variable TEST_LIBS need to be set (only in case you use make command):

TEST_LIBS=y make test

It is also possible to test shared libraries using pytest. To run tests for all shared libraries:

pytest {PATH_TO_SHARED_LIBS}

To run tests for one specific module:

pytest libraries/tests/test_my_library.py

To run one specific test of module:

pytest libraries/tests/test_my_library.py::test_something

Debugging actors

Snactor

The snactor tool is used to debug your actors. You can execute actors and save their output, so that it can be consumed by other actors. See Storing messages in the repository data for reuse.

Snactor checks for the LEAPP_DEBUG environment variable and has also the –debug parameter which sets the environment variable to ‘1‘ when it is used. In that case, it enables the debug logging, so that any actor that logs to self.log.debug gets its output printed on the commandline.

PyCharm / rpdb

You can configure PyCharm to debug by pointing it to the snactor path and passing the arguments on the command line. The PyCharm debugger will also follow the child processes that are created by the snactor tool to execute the actor in a sandboxed environment.

Not everywhere you‘ll have PyCharm at hand but vim/nc lightweight tandem is already in place in majority of cases. It‘s possible to go minimal and debug actor execution with remote debugger like rpdb. The setup is as simple as:

1. Add breakpoint at the desired place in actor‘s code import rpdb; rpdb.set_trace()
2. Run snactor and wait till breakpoint is hit.
3. In a separate console connect to the debugger via network utility of your choice. The default port is 4444.

nc localhost 4444

Deprecation

The deprecation process is here to make (your) life of developers easier. It‘s not possible to write perfect solution for everything and as the project is evolving, it happens that some functionality needs to be changed, replaced or dropped completely. Such situations are inevitable. To reduce negative impact on your code, we introduce the deprecation process described below.

List of the deprecated functionality in leapp

The following lists cover deprecated functionality in the leapp utility, snactor, the leapp standard library, etc. But don‘t cover deprecated functionalities from particular leapp repositories (e.g. the elt7toel8 leapp repository). For such information, see Deprecated functionality in the el7toel8 repository.

current upstream development (till the next release + 6months)

• nothing yet...

v0.15.0 (till Mar 2023)

• Reporting primitives
  • leapp.reporting.Flags - The Flags report primitive has been deprecated in favor of the more general Groups one.
  • leapp.reporting.Tags - The Tags report primitive has been deprecated in favor of the more general Groups one.

What is covered by deprecation process in leapp?

In short, leapp entities that are supposed to be used by other developers. That means e.g.:

1. Models
2. Shared library classes and functions in leapp repository
3. Public APIs
4. Actors providing functionality that could be used by any developer (produce or consume messages)

In other words, private classes, private functions or anything in private libraries, may be modified or removed without the deprecation process. As well, it‘s possible we will need to change something (e.g. a behaviour of a function) that will not be possible to cover reasonably by the deprecation process (e.g. change output of the function...). We‘ll try our best to prevent it, but it may happen. To limit such problems, we recommend people to use APIs as much as possible.

What does it mean that something is deprecated?

When you deprecate something, the only thing that changes is that the deprecated entity is marked in the code as deprecated which can have additional impact, like messages produced on the user‘s terminal, in the report, ... But the rest of the functionality is the same as before, until the entity is removed completely.

What is the deprecation process for leapp?

In case a leapp entity covered by the deprecation process is to be removed for any reason, it needs to be marked as deprecated before the removal (if possible). The deprecation will be applied only for leapp entities that have been introduced in an official release in RHEL (IOW, a functionality that has been merged into the upstream, but has been removed before the release or was marked as experimental all the time, is going to be removed without the deprecation state). The time period during which the deprecated entity won‘t be removed is at least 6 months. That doesn‘t mean we will remove everything deprecated immediately after the 6 months, but it‘s to be expected that it will be dropped anytime between 6 and 12 months since the deprecation.

In case of issues, deprecated entities are not going to be fixed since they are deprecated (unless they are fixed e.g. as a side-effect of another problem fix).

How do I find out what is deprecated?

Mainly via release notes and changelogs. In the official leapp related projects (especially leapp and leapp-repository) the OAMG team takes care of release notes to ensure they inform about the dropped and deprecated functionality.

Additionally, when using leapp or snactor, user is notified via messages about deprecated entities in limited cases (see below). In case of the leapp utility, such messages are presented inside the generated reports. In case of the snactor utility, the information message is printed in the console output at the end of the snactor execution. See examples in this page for detail.

Please note, that the Deprecation warning is emitted only if:

• the deprecated class is instantiated
• the deprecated function is called

How to deprecate entities in leapp?

When you want to deprecate an entity in leapp projects, use the deprecated decorator from leapp.utils.deprecation above the definition of the entity. The decorator has three input parameters:

• since (mandatory) - specifying the start of the deprecation protection period
• message (mandatory) - explaining that particular deprecation (e.g. in case the deprecated functionality has a replacement, it is expected it will be mentioned in the msg.)
• stack_level_offset (optional) - useful to adjust the position of the reported usage in the deprecation message; e.g. in case of a base class or derived classes

Warning: possible change: It‘s possible the stack_level_offset parameter will be removed (or ignored) in future, if we discover a way to improve the deprecation of derived classes.

In case of a class deprecation, all derived classes are considered to be deprecated as well. However, the current reporting could be a little bit confusing. To improve that, the stack_level_offset option can be specified. See examples of the use of the @deprecated decorator for classes.

When you mark any entity as deprecated and this entity is then used in the code, users will be notified about that via a terminal and report messages (see the previous section). However, as the author of the deprecation, you know that the entity is deprecated and you do not want to notify people about the code that still uses the deprecated entity just for the reason to retain the original functionality. To suppress the deprecation messages in such cases, use the suppress_deprecation decorator taking as arguments objects that should not be reported by the deprecation. E.g. in case you use it above the definition of an actor, any use of the deprecated entity inside the actor will not be reported.

WARNING: It is strictly forbidden to use the suppress_deprecation decorator for any purposes but one - retaining the original official functionality over the deprecation protection period. If you see the message and you are not the provider of the functionality, you have to update your code to be independent on it.

Examples of a model deprecation

Imagine we want to deprecate a Foo model that is produced in an actor called FooProducer and consumed in an actor called FooConsumer. Let‘s keep this example simple and say that we do not want to set any replacement of this model. The first thing we have to do is to set the model definition as deprecated:

from leapp.models import Model, fields
from leapp.topics import SomeTopic
from leapp.utils.deprecation import deprecated


@deprecated(since='2020-06-20', message='This model has been deprecated.')
class Foo(Model):
    topic = SomeTopic
    value = fields.String()

If we do only this and execute actors that produce/consume messages of this model, we will obtain messages like these (just example from the actor producing the message after execution by snactor):

# snactor run fooproducer
============================================================
                 USE OF DEPRECATED ENTITIES
============================================================

Usage of deprecated Model "Foo" @ /path/to/repo/actors/fooproducer/actor.py:17
Near:         self.produce(Foo(value='Answer is: 42'))

Reason: This model has been deprecated.
------------------------------------------------------------

============================================================
             END OF USE OF DEPRECATED ENTITIES
============================================================

Apparently, the Reason is not so good. It‘s just example. In real world example, you would like to provide usually a little bit better explanation. Anyway, much more interesting is the point, that the message is now printed every time the actor is executed.

Obviously we do not want to remove the actor yet, because in such a case, the model could be hardly called as deprecated - we need to keep the same functionality during the protection period. But at the same time, we do not want the deprecation message to be produced in this case, as it would be kind of a spam for users who don‘t care about that model at all.

The warning messages are focused on a “custom“ use of the deprecated model - i.e. when a developer creates their own actor producing/consuming a message of the model. To fix this, suppress the deprecation message in this actor.

To do it, the only thing that has to be done is to set the suppress_deprecation decorator with the Foo as an argument (in this case) before the actor, e.g.:

from leapp.actors import Actor
from leapp.models import Foo  # deprecated model
from leapp.tags import IPUWorkflowTag, FactsPhaseTag
from leapp.utils.deprecation import suppress_deprecation


@suppress_deprecation(Foo)
class FooProducer(Actor):
    """
    Just produce the right answer to the world.
    """

    name = 'foo_producer'
    consumes = ()
    produces = (Foo,)
    tags = (IPUWorkflowTag, FactsPhaseTag)

    def process(self):
        self.produce(Foo(value='Answer is: 42'))

This is the most simple case. Let‘s do a small change and produce the message inside the private actor‘s library instead. The library looks like this:

from leapp.models import Foo  # deprecated model
from leapp.libraries.stdlib import api

def produce_answer():
    api.produce(Foo(value='Answer is: 42'))

And the updated actor looks like this:

from leapp.actors import Actor
from leapp.libraries.actor import fooproducer_lib
from leapp.models import Foo  # deprecated model
from leapp.tags import IPUWorkflowTag, FactsPhaseTag
from leapp.utils.deprecation import suppress_deprecation


@suppress_deprecation(Foo)
class FooProducer(Actor):
    """
    Just produce the right answer to the world.
    """

    name = 'foo_producer'
    consumes = ()
    produces = (Foo,)
    tags = (IPUWorkflowTag, FactsPhaseTag)

    def process(self):
        fooproducer_lib.produce_answer()

Now, if you execute the actor again you still won‘t get any deprecation message. So the suppress_deprecation decorator works transitively as expected. However, even when the actor is treated well, the current implementation could affect the result of unit tests. To explain the idea of what could be wrong, imagine a unit test like this one:

from leapp.libraries.actor import fooproducer_lib
from leapp.libraries.common.testutils import produce_mocked
from leapp.libraries.stdlib import api
from leapp.models import Foo  # deprecated model

def test_process(monkeypatch):
    produced_msgs = produce_mocked()
    monkeypatch.setattr(api, 'produce', produced_msgs)
    fooproducer_lib.produce_answer()
    assert Foo(value='Answer is: 42') in produced_msgs.model_instance

If you run the test, you will get output like this (shortened):

| 21:48:01 | conftest | INFO | conftest.py | Actor 'foo_producer' context teardown complete

repos/system_upgrade/el7toel8/actors/fooproducer/tests/test_unit_fooproducer.py::test_process PASSED

================================================== warnings summary ==================================================
repos/system_upgrade/el7toel8/actors/fooproducer/tests/test_unit_fooproducer.py::test_process
  /tmp/leapp-repository/repos/system_upgrade/el7toel8/actors/fooproducer/libraries/fooproducer_lib.py:5: _DeprecationWarningContext: Usage of deprecated Model "Foo"
    api.produce(Foo(value='Answer is: 42'))
  /tmp/leapp-repository/repos/system_upgrade/el7toel8/actors/fooproducer/tests/test_unit_fooproducer.py:10: _DeprecationWarningContext: Usage of deprecated Model "Foo"
    assert Foo(value='Answer is: 42') in produced_msgs.model_instances

-- Docs: http://doc.pytest.org/en/latest/warnings.html
======================================== 1 passed, 2 warnings in 0.13 seconds ========================================

As you can see the warning have been generated again. This time on two places, in test_process and produce_answer functions. Unless warning messages affect results of tests, we do not require strictly to handle them. However, it‘s a good practice. But if the warning log could affect the test (e.g. if a test function checks logs) it should be treated by the suppress_deprecation decorator too. In case of the library function, just add the @suppress_deprecation(Foo) line before the definition of the produce_answer function. But if we do the same for the test function, we will get an error (see that we have now just one deprecation warning now):

| 21:59:57 | conftest | INFO | conftest.py | Actor 'foo_producer' context teardown complete

repos/system_upgrade/el7toel8/actors/fooproducer/tests/test_unit_fooproducer.py::test_process FAILED

====================================================== FAILURES ======================================================
____________________________________________________ test_process ____________________________________________________

args = (), kwargs = {'monkeypatch': <_pytest.monkeypatch.MonkeyPatch object at 0x7f21924b24d0>}
suppressed = <class 'leapp.models.foo.Foo'>

    @functools.wraps(target_item)
    def process_wrapper(*args, **kwargs):
        for suppressed in suppressed_items:
            _suppressed_deprecations.add(suppressed)
        try:
            return target_item(*args, **kwargs)
        finally:
            for suppressed in suppressed_items:
>               _suppressed_deprecations.remove(suppressed)
E               KeyError: <class 'leapp.models.foo.Foo'>

tut/lib/python3.7/site-packages/leapp/utils/deprecation.py:35: KeyError
================================================== warnings summary ==================================================
repos/system_upgrade/el7toel8/actors/fooproducer/tests/test_unit_fooproducer.py::test_process
  /tmp/leapp-repository/repos/system_upgrade/el7toel8/actors/fooproducer/tests/test_unit_fooproducer.py:13: _DeprecationWarningContext: Usage of deprecated Model "Foo"
    assert Foo(value='Answer is: 42') in produced_msgs.model_instances

-- Docs: http://doc.pytest.org/en/latest/warnings.html
======================================== 1 failed, 1 warnings in 0.21 seconds ========================================

It‘s because the mechanism of decorators in python and how pytest works. In this case, we need to do a small workaround, like this:

from leapp.libraries.actor import fooproducer_lib
from leapp.libraries.common.testutils import produce_mocked
from leapp.utils.deprecation import suppress_deprecation
from leapp.libraries.stdlib import api
from leapp.models import Foo  # deprecated model


@suppress_deprecation(Foo)
def _foo(value):
    """Small workaround to suppress deprecation messages in tests."""
    return Foo(value=value)

def test_process(monkeypatch):
    produced_msgs = produce_mocked()
    monkeypatch.setattr(api, 'produce', produced_msgs)
    fooproducer_lib.produce_answer()
    assert _foo(value='Answer is: 42') in produced_msgs.model_instances

That‘s the whole solution for the FooProducer actor. Analogically to this, we need to treat the FooConsumer actor. You could notice that all imports of the Foo model are commented. It‘s a good practice as it is more visible to all developers that a deprecated entity is present.

Example of a model replacement

This is analogous to the previous case. Take the same scenario, but extend it with the case in which we want to replace the Foo model by the Bar model. What will be changed in case of deprecation in the model definition? Just the deprecation message and the new model definition:

from leapp.models import Model, fields
from leapp.topics import SomeTopic
from leapp.utils.deprecation import deprecated


@deprecated(since='2020-06-20', message='The model has been replaced by Bar.')
class Foo(Model):
    topic = SomeTopic
    value = fields.String()


class Bar(Model):
    topic = SomeTopic
    value = fields.String()

You can see that in this case, the model has been just renamed, to keep it simple. But it‘s sure that the new model can be different from the original one (e.g. a different name of fields, a different purpose and set of fields,...). The change in the FooProducer will be just extended by handling the new model (it should include update of tests as well; I am skippin the example as the change is trivial):

from leapp.actors import Actor
from leapp.models import Bar
from leapp.models import Foo  # deprecated model
from leapp.tags import IPUWorkflowTag, FactsPhaseTag
from leapp.utils.deprecation import suppress_deprecation


@suppress_deprecation(Foo)
class FooProducer(Actor):
    """
    Just produce the right answer to the world.
    """

    name = 'foo_producer'
    consumes = ()
    produces = (Bar, Foo)
    tags = (IPUWorkflowTag, FactsPhaseTag)

    def process(self):
        self.produce(Foo(value='Answer is: 42'))
        self.produce(Bar(value='Answer is: 42'))

As you can see, the only thing that have been changed is the added production of the Bar message. The original functionality is still present.

Example of a derived model deprecation

Warning: Known issue: The deprecation of a derived model is currently buggy and the documented solution will end probably with traceback right now. The fix will be delivered in the next release of leapp (current one is 0.11.0). As well, we will try to simplify the final solution, so maybe this section will be more simple with the next release.

It‘s a common situation, that some models are derived from others. Typical example is a base model which is actually not produced or consumed by actors, but it is used as a base model for other models. For the sake of simplicity, we‘ll use one of our previous solutions, but update the definition of the Foo model (skipping imports):

@deprecated(since='2020-01-1', message='This model has been deprecated.')
class BaseFoo(Model):
    topic = SystemInfoTopic
    value = fields.String()


class Foo(BaseFoo):
    pass

Previously, the content was handled completely, but with the new change, we will see the warnings again:

================================================== warnings summary ==================================================
repos/system_upgrade/el7toel8/actors/fooproducer/tests/test_unit_fooproducer.py::test_process
  /tmp/leapp-repository/repos/system_upgrade/el7toel8/actors/fooproducer/libraries/fooproducer_lib.py:7: _DeprecationWarningContext: Usage of deprecated Model "BaseFoo"
    api.produce(Foo(value='Answer is: 42'))
  /tmp/leapp-repository/repos/system_upgrade/el7toel8/actors/fooproducer/tests/test_unit_fooproducer.py:10: _DeprecationWarningContext: Usage of deprecated Model "BaseFoo"
    return Foo(value=value)

See that the reported deprecated model is BaseFoo, however only the Foo model is produced. This could be very confusing to users and developers. The deprecation of the BaseFoo model could resolve our troubles, in the meaning that the base class and all other classes are covered already, that the deprecated entity has been used. But it is confusing, that with this solution, you have to update the code in actor, to suppress the BaseFoo model instead of Foo, even when BaseFoo is not used anywhere directly in the code. I mean something like this:

from leapp.models import Foo, BaseFoo
from leapp.libraries.stdlib import api
from leapp.utils.deprecation import suppress_deprecation

@suppress_deprecation(BaseFoo)
def produce_answer():
    api.produce(Foo(value='Answer is: 42'))

Now, I will put here just several ideas what could user do and why these are wrong (if you are interested just about the working correct solution, skip after the list):

1. Deprecate all models (BaseFoo, Foo) - The result will be two deprecation message per one use of Foo. One with the Foo msg, one with BaseFoo. That‘s not good, as we would like to get rid of BaseFoo completely in the messages ideally.
2. Deprecate just the derived models (Foo) - That could resolve the problem, but what if someone else derive a new model from the base one? They will not be notified about the deprecation and removal will break their code instantly.

If you want to ensure that all models (base and derived) are deprecated, the best solution we are able to come up with it‘s little weird and it‘s going slightly against our requirement (we are going to do an exception here), that inside models cannot be defined any method or logic as they are supposed to be used just as the data container. But this one is currently only possible way, to deprecate such models correctly without confusing messages:

from leapp.models import Model, fields
from leapp.topics import SystemInfoTopic
from leapp.utils.deprecation import deprecated
from leapp.utils.deprecation import suppress_deprecation


@deprecated(since='2020-01-01', message='This model has been deprecated.')
class BaseFoo(Model):
    topic = SystemInfoTopic
    value = fields.String()


@deprecated(since='2020-01-01', message='This model has been deprecated.')
class Foo(BaseFoo):
    @suppress_deprecation(BaseFoo)
    def __init__(self, *args, **kwargs):
        super(Foo, self).__init__(*args, **kwargs)

As you see, both models are deprecated. But in the derived one, there is the __init__ method defined, just for the purpose to be able to suppress the deprecation message from the base model. Implementing this, the solution for suppressing the deprecation warning in previous section will be working, without any confusing messages. As well, this is the only possible usecase for a method inside the models in official repositories managed by the OAMG team.

Additional various outputs of snactor and leapp

snactor warning message example

============================================================
                 USE OF DEPRECATED ENTITIES
============================================================

Usage of deprecated function "deprecated_function" @ /Users/vfeenstr/devel/work/leapp/leapp/tests/data/deprecation-tests/actors/deprecationtests/actor.py:19
Near:         deprecated_function()

Reason: This function is no longer supported.
------------------------------------------------------------
Usage of deprecated Model "DeprecatedModel" @ /Users/vfeenstr/devel/work/leapp/leapp/tests/data/deprecation-tests/actors/deprecationtests/actor.py:20
Near:         self.produce(DeprecatedModel())

Reason: This model is deprecated - Please do not use it anymore
------------------------------------------------------------
Usage of deprecated class "DeprecatedNoInit" @ /Users/vfeenstr/devel/work/leapp/leapp/tests/data/deprecation-tests/actors/deprecationtests/actor.py:21
Near:         DeprecatedNoInit()

Reason: Deprecated class without __init__
------------------------------------------------------------
Usage of deprecated class "DeprecatedBaseNoInit" @ /Users/vfeenstr/devel/work/leapp/leapp/tests/data/deprecation-tests/actors/deprecationtests/actor.py:22
Near:         DeprecatedNoInitDerived()

Reason: Deprecated base class without __init__
------------------------------------------------------------
Usage of deprecated class "DeprecatedWithInit" @ /Users/vfeenstr/devel/work/leapp/leapp/tests/data/deprecation-tests/actors/deprecationtests/actor.py:23
Near:         DeprecatedWithInit()

Reason: Deprecated class with __init__
------------------------------------------------------------

============================================================
             END OF USE OF DEPRECATED ENTITIES
============================================================

leapp report example entries

----------------------------------------
Risk Factor: medium
Title: Usage of deprecated class "IsolatedActions" at /usr/share/leapp-repository/repositories/system_upgrade/el7toel8/libraries/repofileutils.py:38
Summary: IsolatedActions are deprecated
Since: 2020-01-02
Location: /usr/share/leapp-repository/repositories/system_upgrade/el7toel8/libraries/repofileutils.py:38
Near: def get_parsed_repofiles(context=mounting.NotIsolatedActions(base_dir='/')):

----------------------------------------
Risk Factor: medium
Title: Usage of deprecated class "IsolatedActions" at /usr/share/leapp-repository/repositories/system_upgrade/el7toel8/actors/scansubscriptionmanagerinfo/libraries/scanrhsm.py:8
Summary: IsolatedActions are deprecated
Since: 2020-01-02
Location: /usr/share/leapp-repository/repositories/system_upgrade/el7toel8/actors/scansubscriptionmanagerinfo/libraries/scanrhsm.py:8
Near:     context = NotIsolatedActions(base_dir='/')

----------------------------------------
Risk Factor: medium
Title: Usage of deprecated function "deprecated_method" at /usr/share/leapp-repository/repositories/system_upgrade/el7toel8/actors/deprecationdemo/actor.py:21
Summary: Deprecated for Demo!
Since: 2020-06-17
Location: /usr/share/leapp-repository/repositories/system_upgrade/el7toel8/actors/deprecationdemo/actor.py:21
Near:         self.deprecated_method()

----------------------------------------

Additional simple usage examples of @deprecated

Functions

...
from leapp.utils.deprecation import deprecated


@deprecated(since='2020-06-20', message='This function has been deprecated!')
def some_deprecated_function(a, b, c):
    pass

Models

...
from leapp.utils.deprecation import deprecated


@deprecated(since='2020-06-20', message='This model has been deprecated!')
class MyModel(Model):
    topic = SomeTopic
    some_field = fields.String()

Classes

...
from leapp.utils.deprecation import deprecated


@deprecated(since='2020-06-20', message='This class has been deprecated!')
class MyClass(object):
    pass


# NOTE: Here we need to offset the stacklevel to get the report of the usage not in the derived class constructor
# but where the derived class has been created.
# How many levels you need, you will have to test, it depends on if you use the builtin __init__ methods or not,
# however it gives you the ability to go up the stack until you reach the position you need to.
@deprecated(since='2020-06-20', message='This class has been deprecated!', stack_level_offset=1)
class ABaseClass(object):
    def __init__(self):
        pass


class ADerivedClass(ABaseClass)
    def __init__(self):
        super(ADerivedClass, self).__init__()

Leapp repositories

Here you can find all the information related to leapp repositories, including the documentation for existing leapp repositories managed by the OS and Application Modernization Group (OAMG)

Repository Directory Layout

 .leapp/                                        # Repository information. Do not edit it manually.
 actors/                                        # All actors are stored here in subdirectories.
    actorname/                                  # An actor directory.
        actor.py                                # The actual actor code. The file name actor.py is required.
        Makefile                                # Optional makefile with target install-deps to install
                                                # actor's dependencies for tests execution.
        tests/                                  # Unit and component tests for the actors are to be stored here.
            unit_test_actorname.py              # should contain the actor name
            component_test_actorname.py         # should contain the actor name
            files/                              # If tests need to use some mocked files, they should be placed here
                                                # and referenced from the tests using path 'tests/files'.
        libraries/                              # Private libraries for the actors only.
            private_actorname.py                # These can be modules. Name should contain actor name
            actorpkg/                           # or packages.
                __init__.py
        tools/                                  # The path of this directory gets injected in the actors PATH
                                                # environment variable before their execution. These tools are private to
                                                # the actor.

        files/                                  # Files that are private to the actor only.

 files/                                         # Files that are shared with all actors (common files).

 libraries/                                     # Libraries that are shared with all actors.
    common.py                                   # These can be modules
    tests/                                      # with tests stored here.
        files/                                  # If tests need to use some mocked files, they should be placed here
                                                # and refenreced from the tests using path 'tests/files'.
        test_common.py
    sharedpkg/                                  # Or they can be packages
        __init__.py
        tests/                                  # with tests stored here.
            test_sharedpkg.py
            files/                              # If tests need to use some mocked files, they should be placed here
                                                # and refenreced from the tests using path 'sharedpkg/tests/files'.

 models/                                        # All models describing the message payload format are stored here.
    model.py

 tags/                                          # All tags for this repository are stored here.
    tag.py

 tools/                                         # The path of this directory gets injected in the PATH environment
                                                # variable before actors execution. These tools are shared with all actors.

 topics/                                        # All topics for this repository are stored here.
    topic.py

 workflows/                                     # Workflows are stored here.
    workflow.py

Leapp repository for RHEL 7 to RHEL 8 upgrade

This is the official upstream documentation for the leapp repository for in-place upgrade (IPU) from RHEL 7 to RHEL 8. The homepage of the project is here.

Environment variables for the el7toel8 repository

Actors in the el7toel8 repository use environment variables specified below. All these envars use the suggested prefixes specified in the best practices document for the leapp project to distinguish their purpose: production or devel use.

If the argument for envars below is not specified, it is expected to set 0 (false) or 1 (true).

LEAPP_GRUB_DEVICE

Overrides the automatically detected storage device with GRUB core (e.g. /dev/sda).

LEAPP_NO_RHSM

If set to 1, Leapp does not use Red Hat Subscription Management for the upgrade. It‘s equivalent to the --no-rhsm leapp option.

LEAPP_OVL_SIZE

For any partition that uses XFS with the ftype option set to 0, Leapp is creating a file of a specific size in order to proceed with the upgrade. By default, the size of that file is 2048 MB. In case the size needs to be increased, Leapp informs you in the pre-upgrade report that the environment variable needs to be specified.

LEAPP_DEBUG

Enables debug logging. Equivalent to --debug, which takes precedence.

LEAPP_VERBOSE

Enables debug logging. Equivalent to --verbose, which takes precedence.

LEAPP_CONFIG

Overrides the default location of leapp.conf. If not specified, leapp/leapp.conf is used when the command is executed inside a leapp repository, otherwise the default /etc/leapp/leapp.conf is used.

LEAPP_LOGGER_CONFIG

Overrides the default location of logger.conf. If not specified, the default /etc/leapp/logger.conf is used.

LEAPP_ENABLE_REPOS

Specify repositories (repoids) split by comma, that should be used during the in-place upgrade to the target system. It‘s overwritten automatically in case the --enablerepo option of the leapp utility is used. It‘s recommended to use the --enablerepo option instead of the envar.

LEAPP_SERVICE_HOST

Overrides the host of the service to which leapp connects to fetch necessary data files in case they are missing. The used protocol (http:// or https://) must be specified. Defaults to https://cert.cloud.redhat.com.

LEAPP_PROXY_HOST

If set, leapp will use this proxy to fetch necessary data files in case they are missing. The used protocol (http:// or https://) must be specified.

LEAPP_TARGET_PRODUCT_CHANNEL

The alternative to the --channel leapp option. As a parameter accepts a channel acronym. E.g. eus or e4s. For more info, see the leapp preupgrade --help. In case the beta channel is required, use the LEAPP_DEVEL_TARGET_PRODUCT_TYPE envar instead.

LEAPP_NO_NETWORK_RENAMING

If set to 1, the actor responsible to handle NICs names ends without doing anything. The actor usually creates UDEV rules to preserve original NICs in case they are changed. However, in some cases it‘s not wanted and it leads in malfunction network configuration (e.g. in case the bonding is configured on the system). It‘s expected that NICs have to be handled manually if needed.

LEAPP_DATABASE_FORCE_SYNC_ON

If set to 1, Leapp will explicitly enable synchronization on the SQLite database. Enabling the synchronization has negative impact on the performance (sometimes very negative). However, it is more reliable in case of extreme situations (e.g. lost power). Note the synchronization is nowadays switched off by default only during the phases executed before the reboot of the system to the upgrade environment, which we consider safe. As a result, we do not expect that someone would want to use this option now.

LEAPP_NO_INSIGHTS_REGISTER

If set to 1, Leapp does not register the system into Red Hat Insights automatically. It‘s equivalent to the --no-insights-register leapp option.

LEAPP_NO_RHSM_FACTS

If set to 1, Leapp does not store migration information using Red Hat Subscription Manager. It‘s equivalent to the --no-rhsm-facts leapp option.

LEAPP_NOGPGCHECK

Set to 1 to disable RPM GPG checks (same as yum/dnf –nogpgckeck option). It‘s equivalent to the --nogpgcheck leapp option.

LEAPP_TARGET_ISO

Set the path to the target OS ISO image that should be used for the IPU. It‘s equivalent to the --iso leapp option.

LEAPP_UNSUPPORTED

Necessary to use in case you use any envar with the LEAPP_DEVEL prefix (see the list below). And in case you use the –whitelist-experimental option for the Leapp tool.

Development environment variables for the el7toel8 repository

LEAPP_DEVEL_RPMS_ALL_SIGNED

Leapp will consider all installed pkgs to be signed by RH - that affects the upgrade process as by default Leapp upgrades only pkgs signed by RH. Leapp takes care of the RPM transaction (and behaviour of applications) related to only pkgs signed by Red Hat. What happens with the non-RH signed RPMs is undefined.

LEAPP_DEVEL_TARGET_RELEASE

Change the default target RHEL 8 minor version.

LEAPP_DEVEL_SKIP_CHECK_OS_RELEASE

Do not check whether the source RHEL 7 version is the supported one. E.g. right now Leapp does not allow you to proceed with the upgrade when you’re not on RHEL 7.9.

LEAPP_DEVEL_DM_DISABLE_UDEV

Setting the environment variable provides a more convenient way of disabling udev support in libdevmapper, dmsetup and LVM2 tools globally without a need to modify any existing configuration settings. This is mostly useful if the system environment does not use udev.

LEAPP_DEVEL_SOURCE_PRODUCT_TYPE

By default the upgrade is processed from the GA (general availability) system using GA repositories. In case you need to do the in-place upgrade from a Beta system, use the variable to tell which of those you would like to use. The value is case insensitive and the default value is ga. Expected values: ga, beta.

LEAPP_DEVEL_TARGET_PRODUCT_TYPE

LEAPP_DEVEL_TARGET_PRODUCT_TYPE is an analogy to LEAPP_DEVEL_SOURCE_PRODUCT_TYPE for the target system and an extension to LEAPP_TARGET_PRODUCT_CHANNEL. If used, it replaces any value set via the --channel option or through the LEAPP_TARGET_PRODUCT_CHANNEL environment variable . It consumes the same set of values as the --channel option, and can be extended with the value beta. This is the only way how to perform the inplace upgrade to a beta version of the target system using the subscription-manager.

LEAPP_DEVEL_USE_PERSISTENT_PACKAGE_CACHE

Caches downloaded packages when equal to 1. This will reduce the time needed by leapp, when executed multiple times, because it will not have to download already downloaded packages. However, this can lead to a random issues in case the data is not fresh or changes of settings and repositories. The environment variable is meant to be used only for the part before the reboot and has no effect or use otherwise.

LEAPP_DEVEL_DATABASE_SYNC_OFF

If set to 1, leapp will disable explicit synchronization on the SQLite database. The positive effect is significant speed up of the leapp execution, however it comes at the cost of risking a corrupted database, so it is currently used for testing / development purposes, only.

LEAPP_DEVEL_INITRAM_NETWORK

You can specify one of the following values: ‘network-manager‘, ‘scripts‘. The ‘scripts‘ value is used for a legacy dracut module when the network is not handled by NetworkManager. Using the option allows experimental upgrades, bringing up the networking inside the upgrade initramfs environment (upgrade phases after the first reboot). It also allows the upgrade e.g. when a network based storage is used on the system. Currently it works only for the most simple configurations (e.g. when only 1 NIC is present, no rdma, no bonding, ...). Network based storage is not handled anyhow during the upgrade, so it‘s possible that the network based storage will not be correctly initialized and usable as expected).

LEAPP_DEVEL_KEEP_DISK_IMGS

If set to 1, leapp will skip removal of disk images created for source OVLs. This is handy for debugging and investigations related to created containers (the scratch one and the target userspace container).

Deprecated functionality in the el7toel8 repository

Deprecated functionality is listed under the first version that the functionality is deprecated in. Note that functionality may be deprecated in later versions but are not listed again. The dates in brackets correspond to the end of the deprecation protection period, after which the related functionality can be removed at any time.

Note The lists cover just the functionality provided inside the el7toel8 repository only. For the functionality deprecated in the leapp framework, see List of deprecated functionality in leapp

current upstream development (till the next release + 6months)

• nothing yet...

v0.19.0 (till March 2024)

• Models
  • InstalledTargetKernelVersion - Deprecated as the new solution has been designed to be able to handle new changes in RHEL 9.3+ system. Use the InstalledTargetKernelInfo message instead.
  • GrubInfo.orig_device_name - The GrubInfo message is still valid, but the orig_device_name field has been deprecated as multiple devices can exist on a system. Use GrubInfo.orig_devices instead.
• Shared libraries
  • leapp.libraries.common.config.version.is_rhel_realtime() - The function has been deprecated as the information cannot be easily determined based on the information inside IPUConfig. Use data in the KernelInfo message instead, the field type.
  • leapp.libraries.common.grub.get_grub_device() - The function has been deprecated as multiple grub devices can exists on a system. Use the leapp.libraries.common.grub.get_grub_devices() function instead.

v0.16.0 (till September 2022)

• Shared libraries
  • leapp.libraries.common.utils.apply_yum_workaround - The apply_yum_workaround function has been deprecated, use DNFWorkaround message as used in the successing RegisterYumAdjustment actor.

v0.15.0 (till April 2022)

• Models
  • RequiredTargetUserspacePackages - Deprecated because the new solution has been designed. Use the TargetUserspacePreupgradeTasks instead (see the install_rpms field).
  • RequiredUpgradeInitramPackages - Deprecated because the new solution around the upgrade initramfs has been designed. Use the TargetUserspaceUpgradeTasks instead (see the install_rpms field).
  • UpgradeDracutModule - Replaced by UpgradeInitramfsTasks (see the include_dracut_modules field).
  • InitrdIncludes - Deprecated because the new solution around the target initramfs (read: initramfs created for the upgraded system) has been designed. Use the TargetInitramfsTasks instead (see the include_files field).

v0.12.0 (till April 2021)

• Models
  • GrubDevice - Deprecated because the current implementation is not reliable. GRUB device detection is now in the shared grub library. Use the leapp.libraries.common.grub.get_grub_device() function instead.
  • UpdateGrub - Deprecated because the current implementation is not reliable. GRUB device detection is now in the shared grub library. Use the leapp.libraries.common.grub.get_grub_device() function instead.
• Shared libraries
  • leapp.libraries.common.testutils.logger_mocked.warn() - The logging.warn method has been deprecated in Python since version 3.3. Use the warning method instead.

v0.11.0 (till April 2021)

• Models
  • TMPTargetRepositoriesFacts - Deprecated because this model was not intended for customer use.

Infrastructure

Here you can find documentation related to our CI/CD, packaging, etc.

How to deal with dependencies

RPM dependencies of Leapp packages can be grouped into two groups:

1. inner dependencies between Leapp-related packages
2. outer dependencies on packages not related to Leapp (e.g. python)

The first group is a standard RPM dependency management. But the outer dependencies are really tricky. When a Leapp-related package needs such a dependency, it has to depend on a special capability (below) and the required dependency has to be set in a special metapackage which provides this capability. At this point, the metapackage installed together with leapp packages is named leapp-deps.

The capability (in case of the leapp tool, leapp framework, and snactor) is called leapp-framework-dependencies. Every time the outer dependencies are changed, the value of the capability has to be incremented by one - so the information about the change is provided to other leapp-related packages.

This metapackage can be simply replaced by a metapackage that would handle outer dependencies of the target system. That means, that the metapackage with dependencies for the target system is not part of this git repository and the actors of the leapp-repository are responsible for providing such package in case it is needed.

Why do I need to have a special way to define a dependency

One possible use of the Leapp framework is to do the in-place upgrade (IPU). In that case, it is necessary to execute Leapp on two different versions of the system - it starts with the source system and then continues with the target (upgraded) system. It is quite common that some packages are renamed between those two systems (or in case of leapp-repository, you would like to use technology on the new system that is not provided on the original one). In such cases, it is hard to satisfy dependencies on both systems - when you want to proceed with the upgrade process with the same Leapp packages.

As a solution, we are using the metapackage that contains those dependencies.

What to do when dependencies needs to be changed

It‘s easy to change outer dependencies for Leapp. Open the packaging/leapp.spec file and change the dependencies as needed under %package deps. The right place is pretty highlighted (you cannot miss it):

##################################################
# Real requirements for the leapp HERE
##################################################
...
Requires: findutils
Requires: your-dependency

And do not forget to increment value of leapp-framework-dependencies:

# IMPORTANT: every time the requirements are changed, increment number by one
# - same for Provides in deps subpackage
Requires: leapp-framework-dependencies = 1

IMPORTANT Do the same thing for all Requires and Provides of the leapp-framework-dependencies capability in the SPEC file. Ensure that value is consistent across the SPEC file.

Some more unusual steps

There is an official Leapp repository that we manage and it is already affected by such change. If you modify the outer dependencies, inform the repository developers about the change (e.g. issue) or even better, continue by following instructions in the doc to reflect the change.

How to deal with dependencies in leapp-repository

First, read this document to better understand the difficulties related to package dependencies in the Leapp project.

When talking about RHEL 7 to RHEL 8 upgrade, the goal is to cover dependencies of all Leapp project-related packages, including the leapp-repository packages, for both RHEL 7 and RHEL 8. Since the situation with dependencies of the leapp packages is similar to the situation with the leapp-repository dependencies, this document focuses on the leapp-repository specifics only.

Currently there are two SPEC files for leapp-repository:

• The leapp-repository.spec file is used to build leapp-repository packages and their dependency metapackage leapp-repository-deps for RHEL 7.
• The leapp-el7toel8-deps.spec file is used to build dependency metapackages leapp-deps-el8 and leapp-repository-deps-el8 for RHEL 8 whose purpose is to replace the RHEL 7 dependency metapackages leapp-deps and leapp-repository-deps during the upgrade.

What to do in leapp-repository when dependencies of leapp change?

Go to the section below the line %package -n %{ldname} in the leapp-el7toel8-deps.spec. This section creates the RHEL 8 leapp-deps-el8 metapackage that replaces the RHEL7 leapp-deps metapackage. So when the leapp package dependencies change in the leapp.spec together with incrementing version of the leapp-framework-dependencies capability, it‘s necessary to:

• provide the same leapp-framework-dependencies capability version by leapp-deps-el8
• decide if this dependency change also applies to RHEL 8 and if so, update the dependencies of the leapp-deps-el8 metapackage accordingly.

There can be another case when we need to modify dependencies of leapp on RHEL 8, e.g. when a RHEL 7 package is renamed or split on RHEL 8. In such case we don‘t need to modify the capability value, just update dependencies of the leapp-deps-el8 metapackage, commenting it properly. Nothing else.

What to do when leapp-repository dependencies need to change?

When you want to modify outer dependencies of leapp-repository packages, do that similarly to instructions related to Leapp packages, following the same rules. Just take care of the leapp-repository-dependencies capability instead of the leapp-framework-dependencies capability. Everything else is the same. Interesting parts of the SPEC files are highlighted in the same way as described in the leapp dependencies document.

Compatibility between leapp and leapp repositories

There is a hidden dragon in the split of leapp framework and leapp repositories projects. The development of features and incompatible changes has different speed in both projects. We want to release our projects using semantic versioning and, at the same time, we do not want to release a new version of the project after each merge of a new feature or incompatible change. We rather prefer releasing new versions with changelogs and everything when we agree it‘s worthwhile.

But we need a mechanism to be able to synchronize with other projects, when we provide new functionality in the upstream (master) branch without the need of immediate release of the new version of leapp. For these purposes the leapp-framework capability is provided in the framework (python[23]-leapp) rpms.

When and how change the capability

The leapp-framework capability has to be changed in case of any change of provided API (e.g. change in the leapp standard library) or functionality that could affect actors in leapp repositories. That includes new features as well as an actor may depend on that in future, and the leapp-repository package should be able to specify correct framework version that is needed.

The leapp-framework capability uses just X.Y versioning:

• In case of a change breaking previous functionality (e.g. changed a name of a function) provided by the framework, bump X and set Y to 0.
• In case of a new feature (e.g. new function in stdlib), which doesn‘t affect already existing actors, bump Y

The value of the capability should be bumped per PR providing all changes. IOW, when a PR implements two compatible/incompatible changes in several commits, the value should be bumped only once!

As well it is recommended to mentioned the value of the capability (when changed) in the commit msg, so it the tracking is easy.

Suggested dependency in leapp-repository rpms

We suggest to set dependency on the leapp-framework capability in any rpm that contains leapp repositories in this way:

Requires:       leapp-framework >= 1.2, leapp-framework < 2

Which means, that you want to install framework of version at least 1.2 but lower than 2 (e.g. 1.99).

Possible issue

There is unfortunately one issue with the above solution. As far as there is just one rpm providing that capability, it‘s ok. The problem occurs on systems where there will be more rpms (of different name) providing the capability. So in case you are able to install e.g. python2-leapp and python3-leapp rpms on the system, you could end up with:

• python2-leapp providing leapp-framework 1.0, and
• python3-leapp providing leapp-framework 3.0

which both are broken for leapp repository and the dependency from the point of rpms is satisfied. This should happen rarely. We suggest you to ensure that you use such repositories where only one of those rpms exists.

Naming schema of Leapp-related packages

All projects under the OAMG should use the same schema for names of RPM builds, to be able to simply and fast find specific builds. The schema itself looks like that (NVR without %{dist}):

• for builds made from the master branch:

<name>-<version>-100.<timestamp>.<short-hash>.<branch>

• for builds made from pull-request:

<name>-<version>-0.<timestamp>.<short-hash>.<branch>.PR<number>

Where:

• name is name of the project
• version is current version of the application
• timestamp in format: +%Y%m%d%H%MZ, e.g. “201812211407Z“
• short-hash is shortened hash of the commit from which the build has been created
• branch is name of the branch from which the builds has been created; in case the name of the branch contains dashes (-), these are automatically replaced by underscore (_) because dash is used as a separator in the name of a (S)RPM file; so it is suggested (not required) to to avoid dashes in names of branches
• number is number of a pull-request (alternatively merge-request) to which builds are related

Best Practices for actor development

Follow the contribution guidelines

See the contribution guidelines.

Avoid running code on a module level

Despite the actors being written in Python, they are loaded beforehand to get all the meta-information from them. To avoid slow downs during this load phase we ask you to not execute any code that is not absolutely necessary on a module level. Simple, fast code with no side-effects is acceptable, for example constant definitions.

Avoid certain global imports

On a module level, try to import only the Python Standard Library modules or modules provided by the Leapp framework. Import any other module within your function or method. This has to do with possible slow down when loading repositories by the leapp tool, and also to avoid unnecessary complications for our tests automation which needs to inspect the actors beforehand.

Use the snactor tool for development

The snactor tool helps you with creating the base layout of a new Leapp repository, and with creating boilerplates for the repository artifacts like actors, models, tags, topics, and workflows. It also helps you with debugging as it is able to execute individual actors.

See the tutorial on basic usage of snactor.

Move generic functionality to libraries

Part of your actor‘s functionality might end up being rather generic or abstract. In that case, consider converting it to a shared library function. You can introduce it in one of these two places:

• The Leapp Standard Library

  The most generic functionality which actors of any workflow can use, e.g. function for exectuting a shell command, should go to the Leapp Standard Library. For that, please submit proposals through issues or pull requests under the leapp GitHub repository.

• Leapp repository-level shared library

  The functionality that may be useful for other actor developers but its scope is limited to the workflow you are writing your actor for, e.g. for an OS in-place upgrade workflow, should go to the <Leapp repository>/libraries folder. In this case, we welcome your proposals under the leapp-repository on GitHub.

  Please note that the name of the library module in this case should be unique and contain the actor name. For example: repos/system_upgrade/el7toel8/actors/vimmigrate/libraries/vimmigrate.py

Discover standard library functions

Before implementing functionality for your actor, browse through the available functionality provided by:

• the Leapp Standard Library,
• the shared library of your Leapp repository (<Leapp repository>/libraries folder).

These libraries contain functions that may satisfy your needs. Using them can save you time, lower code complexity and help avoiding duplicate code. Improvement proposals for the library functions are welcome.

Prefer using stdlib functions over shell commands

Sources of external functionality to be used in your actor in order of preference:

1. the Leapp Standard Library
2. the Python Standard Library
3. shell commands

Examples:

• Prefer os.symlink over ls -s
• Prefer os.remove over rm

There might be a valid reason for calling a shell command instead of a standard library function, e.g. the Python‘s shutil.copyfile is not able to retain all of the file attributes, as opposed to shell‘s cp --preserve.

Utilize messages produced by existing actors

As with the Leapp Standard Library mentioned above, it may be beneficial for you to skim through the actors already in place in the leapp-repository. You might be interested in the messages they produce, for example:

• SystemFactsActor - information about kernel modules, yum repos, sysctl variables, users, firewall, SELinux, etc.
• OSReleaseCollector - system release information
• RpmScanner - list of installed packages
• StorageScanner - storage information

In case you find any message of the existing actors to be incorrect, incomplete or misleading, we encourage you to improve the respective actors.

Write unit testable code

Write all the actor’s logic in the actor’s private library in order to be able to write unit tests for each of the function. It is not currently possible to unit test any method or function in the actor.py. Then, ideally, the actor.process() should contain only calling an entry point to the actor‘s library. To create an actor’s library, create libraries folder in the actor’s folder and in there create an arbitrarily named python file, e.g. {actor_name}.py.

myactor/actor.py:

from leapp.libraries.actor.library import do_the_actor_thingy

class MyActor(Actor):
    # <snip>
    def process(self):
        do_the_actor_thingy(self)

myactor/libraries/myactor.py:

def do_the_actor_thingy(actor):
    actor.log.debug("All the actor’s logic shall be outside actor.py")

For more about unit testing, see the tutorial.

Do not introduce new dependencies

Ideally, actors shouldn‘t require any additional dependency on top of the dependencies already in the leapp and leapp-repository spec files, which are, as of December 2018, just these:

• dnf
• python-six
• python-setuptools
• findutils

Rather than adding a new dependency to the spec file, detect in the actor if the package is installed and if not, have a fallback option or skip the action you wanted to perform and report to the user that they should install the package if they want to experience as smooth upgrade as possible.

If you‘re writing an actor for the RHEL 7 to RHEL 8 in-place upgrade workflow and you really need to have some package installed, then the only acceptable packages to depend on are the the ones available in the minimal installation of RHEL 7 (packages from the @core group + their dependencies).

For more details about dependencies and how to modify them, see the How to deal with dependencies.

Use convenience exceptions to stop the actor‘s execution

If you encounter unexpected input or any other error during execution of an actor and want to stop it, use these exceptions:

• StopActorExecution - raising this exception is a convenient to stop actor‘s execution with no side effect - it has the same effect as returning None from the main process() method in the actor.py
• StopActorExecutionError - raising this exception will stop actor‘s execution and notify the framework that an error has occurred and can influence the result of the workflow execution.

In case of StopActorExecutionError the execution of the workflow will stop or not according to the policy defined in the workflow, there are three possibilities:

• FailImmediately - end the workflow execution right away
• FailPhase - end the workflow execution after finishing the current phase
• ReportOnly - do not end the workflow execution at all and continue with logging the issue only

You can also use the StopActorExecution and StopActorExecutionError exceptions inside a private or shared library.

Use the LEAPP and LEAPP_DEVEL prefixes for new envars

In case you need to change a behaviour of actor(s) for testing or development purposes - e.g. be able to skip a functionality in your actor - use environment variables. Such environment variables should start with prefix LEAPP_DEVEL. Such variables are not possible to use on production systems without special LEAPP_UNSUPPORTED variable. This prevents users to break their systems by a mistake.

If you want to enable user to modify behaviour on production systems as well (e.g. enable user increase size of something the actor is producing), use just LEAPP prefix and ensure you inform user about the variable if needed.

In both cases, such environment variables should be mentioned in related commit messages.

Tests for actors

The Leapp actors are covered by three types of tests - unit, component and e2e.

Unit and component tests

• Both unit and component tests are to be placed in the actor‘s tests folder.
• Unit and component tests modules should have unique names
• Tutorial on How to write unit and component tests

Naming conventions

Test module names should match the following regex:

• test_*.py
• unit_test_*.py
• component_test_*.py

Unit tests

• These tests deal with individual actor‘s functions/methods.
• It‘s not possible to unit test any method/function within the actor.py. You can write unit tests only for functions/methods within the actor‘s libraries.
• Thus, to be able to write unit tests for an actor, ideally the only thing in the actor.py‘s process() method is calling the entry-point function of the actor‘s library python module.
• Example of unit tests

Component tests

• These tests provide fabricated input messages for the actor, check the outputs stated in the actor‘s description.
• These tests should not be written based on the actor‘s code but rather based on the behavior stated in the actor‘s description. They could be written by somebody who didn‘t write the code.
• Example of component tests

End to end (e2e) tests

• The Leapp QA team maintains an internal testing framework facilitating e2e tests.
• [Members of the oamg GitHub organization] can trigger execution of the e2e tests by adding the comment ‘e2e tests‘ under an opened leapp or leapp-repository PR

Bug fixing of actor

Each bug found and fixed in an actor should be covered by tests.

Architecture overview

Architecture overview

There are two tools for working with the framework, the end user application leapp and the development utility snactor. The leapp tool is designed to run specific workflows, while the snactor tool can run arbitrary workflows, but also individual actors.

A workflow describes what work is going to be done and when. Each workflow is made of a sequence of phases, which contain actors split into three stages - before, main, and after. Workflows, actors, and all the parts necessary for the execution are loaded from repositories.

Each actor is executed in a forked child process to prevent the modification of the application state. All messages and logs produced by the actors are stored in the audit database.

For more information about each part of the architecture, check the terminology.

How is this different from Ansible?

Leapp is message-driven. The execution of actors is dependent on the data produced by other actors running before them. This data is passed around in the form of messages. This is in a contrast with Ansible where everything has to be specified up front.

Inplace Upgrade Workflow

In Place Upgrade Workflow

Contributing to the Leapp project

First, thank you for taking your time to contribute to the project.

The following is a set of guidelines for contributing effectively to the Leapp-related repositories hosted under the OS and Application Modernization Group organization <https://github.com/oamg/>_ on GitHub.

Code style guidelines

Your code should follow our Python Coding Guidelines

Best practices in actor development

• Actor development: Best practices for actor development
• Actor directory layout is to be found at Repository directory layout (under actorname/)
• Actor testing: Testing actors

File naming convention

1. New folders and/or Python files shall use lowercase without underscores.
2. The actor‘s main file shall be named actor.py.

Submitting a Pull Request

Before you submit your pull request, consider the following guidelines:

• Fork the repository and clone your fork.

• Make your changes in a new git branch:

  git checkout -b bug/my-fix-branch master

• Include documentation that either describe a change to a behavior or the changed capability to an end user.

• Commit your changes with message conforming to the Git Commit Messages_ guidelines.

• Include tests for the capability you have implemented.

• Make sure your tests pass. We use Jenkins CI for our automated testing.

• Push your branch to GitHub:

  git push --set-upstream origin bug/my-fix-branch

• When opening a pull request, select the master branch as a base.

• Mark your pull request with [WIP] (Work In Progress) to get feedback, but prevent merging (for example, [WIP] Update CONTRIBUTING.rst).

• If you are fixing a GitHub issue, include the issue number you are fixing, e.g. ‘Closes issue #xyz‘.

• Description of the PR should clearly state what is being changed and the rationale for the change.

If we suggest changes, follow these rules:

• Make the required updates.

• Push changes to git (this will update your pull request). For that you can add a new commit or rebase your branch and force push to your GitHub repository like this: ::

  git rebase -i master git push -f origin bug/my-fix-branch

Merge Rules

• Every PR should have at least one code review before merging
• All CI tests should pass

Git Commit Messages

• Write a descriptive commit message
• Use the present tense (“Add feature“ not “Added feature“)
• Use the imperative mood (“Move cursor to...“ not “Moves cursor to...“)
• If you are fixing a GitHub issue, include something like ‘Closes issue #xyz‘
• For more best practices, read How to Write a Git Commit Message <https://chris.beams.io/posts/git-commit/>_

Contact

In case of any question, contact us at #leapp on Libera.Chat IRC network, or write the question as an issue on GitHub.

Coding Guidelines for the Leapp project

While code stylistics (naming, whitespace, structure) is mostly handled by the famous PEP8 - https://www.python.org/dev/peps/pep-0008/ there‘s still much to be defined and codified in terms of code semantics, which is equally important as the overall style. Purpose of these guidelines is to reduce the cognitive load necessary to read through & understand the code base.

1. Avoid Inheriting from Multiple Classes

Python uses something called C3 Linearization (https://en.wikipedia.org/wiki/C3_linearization) to figure out the method resolution order and, as you can see, the logic behind the algorithm is non-trivial - dragging that context in your head all the time is, costly and unnecessary, given the particularly little benefit of using multiple inheritance in the first place.

2. Avoid Operator Overloading

With the exception of mathematics there‘s usually very little reason for succumbing to the mysterious and arcane arts of operator overloading. Why is math OK? Because operator overloading makes it consistent with how everybody assumes the given operator works given how it works in math itself. There‘s little benefit in operator overloading otherwise, because different people have different assumptions and while you think that it‘s pretty clear that invalid state should make the object equal to None, well, pretty much everyone else is going to find this behavior confusing (http://stackoverflow.com/questions/371266/checking-c-sharp-new-initialisation-for-null/371286), because it‘s just not how the rest of the ecosystem works. Going back to math, for the same reason you wouldn‘t like 3 + 3 = 7 because someone decided that the symbols for addition now should return a + b + 1, don‘t put such constructs in code. The situation is much worse because Python isn‘t statically typed, so even if you knew that SomeCrazyClass overloads some particular operator, you might have no idea that some_var even is an instance of SomeCrazyClass.

3. Avoid ‘In-Your-Face‘ error/exception messages

Exceptions or terminal error messages should always give you enough context to either fix the problem on the OS level, if applicable, or know the exact location and reason why the error was raised, given you need to fix it in code. Even some shady corners of Python base library will sometimes simply tell you Permission denied when you‘re trying to access a resource for which you don‘t have sufficient permission, which makes the message exceptionally unhelpful in case your application manages multiple resources in a loop. All you end up doing is catch and re-raise the exception with more helpful message.

3.1. Type Your Exceptions

Raise ValueError or TypeError when validating input parameters. Use custom exception hierarchy with Exception at its root for raising errors related to application logic itself. Callers of your code then have fine grained control over various failure modes, which will always help at making the code more robust.

4. Avoid **kwargs like a plague

Probably apart from function decorators, there‘s no legitimate reason for including **kwargs in your function signatures, ever. Think of function signatures in terms of binding contracts between caller and callee, the caller knows how to invoke the particular function (callee) simply by looking at the signature. In the context of contracts, **kwargs essentially feels like “whatever, dude“, because in the best case scenario the valid kwargs are documented in the function‘s doc string (which is still going to stop code completion tools to a grinding halt), in the worst case you are going on a chase and will need to inspect the code of at least one function body - granted the **kwargs are not passed around like a hot potato, in which case your best bet is to resurrect the long forgotten art of “printf debugging“ and sprinkle the code with a bunch of print calls here and there.

5. Import things the right way

Avoid wildcard * imports. So called “Star“ imports import every module member that‘s name doesn‘t start with _ into the current scope, which can quickly lead to name clashes and funny errors.

Do not make relative imports.

Imported modules should be divided into 3 groups stdlib/third-party/project in the following order separated by a single newline; each group should be sorted alphabetically by full module/object name.

If you need to use plenty of members from some module, use the import module syntax rather than from module import foo. Always use import module syntax for python standard library, specifically instead of from os.path import join use import os.

Sort imported modules as well as all imported functions by name, alphabetically. Our brains are very good at bi-secting names in alphabetical order.

Unless you have a really good reason, don‘t make local imports in function body.

6. Avoid Using Assertions For Control Flow & Mutating Operations

Assertions are, idiomatically, used to verify very basic assumptions about the code. It is assumed that once the code has been stressed enough without triggering any assertion errors the code should be just fine. In C, for example, assertions are not evaluated in non-debug builds. In Python, assertions are ignored if you execute your code with -O flag. This also means that if the right-hand side of the assertion mutates certain state, the action is not carried out.

7. Avoid Map, Reduce and Too Complex Comprehensions

While one might say that writing out for loops makes the code look dumb and uninteresting, it also makes it obvious and easy to understand, which are qualities, not defects. Code that uses map/reduce or several levels of nested dictionary and list comprehensions almost always looks scary and it‘s very hard to deduce the data flow of that particular piece of code.

8. Avoid Deeply Nested Block Scopes

In other words, high cyclomatic complexity is bad (https://en.wikipedia.org/wiki/Cyclomatic_complexity). Reasoning about code with so much noise around is difficult, however, there are several ways how to avoid this situation. Instead of if someCond: followed a by a huge chunk of code, consider the pattern of eager returns and simply return as soon as possible if not someCond: return so that the following block doesn‘t need to be nested. This is of course not applicable for all dark corners of cyclomatic complexity, for nested loops you might want to consider refactoring each loop into it‘s own function or method. While the code will get a little more verbose, and you might face other difficulties like having to pass around variables or even promoting some to instance variables, the resulting code is still much simpler to read then condensed web of loops mutating maybe tens of local variables.

9. Write Docstrings That Work Well With Tools

The preferable way of writing docstrings for functions and methods is to use the first style mentioned at (https://pythonhosted.org/an_example_pypi_project/sphinx.html#function-definitions). Plenty of editors or plugins are able to parse these docstrings and provide targeted code completion and rudimentary validation. For consistency all docstrings should start and end with """.

10. Avoid Shadowing Python Builtins

type, file, id, map, filter, etc. have already been taken, be creative and invent your own object names.

11. String Formatting

11.1 Prefer new style String Formatting To old style

Though there is still a mixture of formatting styles remaining in leapp code base, this rule stands for all new contributions. Please use

new_style = "{}! {} is coming".format("Brace yourself", "Winter")

instead of

old_style_refactor_me = "%s! %s is coming!" % ("Brace yourself", "Winter")
old_style_refactor_me = "%(msg)s! %(obj)s is coming!" % {"msg": "Brace yourself", "obj": "Winter"}

11.2 Use Keyword Arguments To Increase Readability

If you pass more than 2 arguments to format function, please invoke format with keyword arguments.

msg_many_args = "{who} {when} {what}".format(who="A Lanister", when="always", what="pays his debts")

12. Docstrings

Follow PEP 257 - Docstring Conventions

• with the exception, that the summary line of a multi-line docstring shall be on a new line, not on the same line as the opening quotes.

There are some minimal requirements for the information that actor docstrings should provide, to learn more about the specifics please consult the contribution guidelines.

This is example how to write docstrings:

class MyActor(Actor):
    """
    Start with a single-line brief summary of the actor (under the triple quotes).

    Leave a blank line below the summary and then describe the actor's behaviour
    here in detail.
    """
    name = 'my_actor'

    def process(self):
        """This is a simple method."""
        complicated_method(True)

    def complicated_method(switch):
        """
        This is a summary line of a more complicated method.

        Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc porta sed
        urna venenatis faucibus. Phasellus at bibendum ligula, a posuere metus.

        :param switch: Description of the parameter.
        :type switch: Expected type of the parameter.
        :return: Description of what the method returns
        """
        mutliline_string = (
            'I am a multiline string.\n'
            'This is my second line.'
        )
        return mutliline_string

13. Underscore usage

For leapp and leapp-repository the _ and P_ is reserved for localization. Please don‘t use it for anything else like variable-to-be-discarded.

Frequently Asked Questions

• What is Leapp?
• How can I get on board with contributing to Leapp?
• What is an actor and what does it do?
• When and why do I need to write an actor?
• How can I exchange any data between actors?
• What do I have to do in order to execute actor I just wrote?
• What should I do if I need to execute multiple actors? Can I somehow ensure the dependencies between them?
• How can I specify what run time dependencies will my actor have?
• How can I distinguish between actors that I depend on directly (I need to consume their output) and indirectly (I just need them to be executed as part of the upgrade as I don’t handle the upgrade of that specific piece; think PHP vs. Apache - upgrade of Apache is independent of the upgrade of PHP but it needs to be done to enable its upgrade)?
• Once I write an actor that consumes data from some other actors, how can I be sure that the format will not change on the producing side in the future?
• What are the best practices for writing actors?
• What are the requirements for actors to be accepted by upstream?
• How can I debug my actor? Is there a standard/supported way how to log and get logs from actors/channels?
• Are there some technical limitations for an actor? E.g. maximum time execution, size of the input/output, libraries I can use… In case there are, is it possible to specify that the actor needs e.g. longer time for execution?
• Are there some actions that are either forbidden or not recommended to be done in actors?
• I got an error about PES data/ Repositories mapping where I find such files?

What is Leapp?

Leapp project aims to enable users to modernize their existing workloads without disrupting them in three different ways: upgrading them in place, migrating them to a different place or containerize them. Currently, the in-place upgrade functionality is being worked on only.

How can I get on board with contributing to Leapp?

For the Leapp framework we are currently developing the functionality for the in-place upgrade of RHEL 7 to RHEL 8. You can improve the user experience of the upgrade by creating so called actors for the Leapp framework. We‘ve written a quick guide on how to create such actors for the RHEL 7 to RHEL 8 upgrades: How to create a Leapp actor for RHEL 7 to 8 upgrade.

What is an actor and what does it do?

An actor in the realm of the Leapp project is a step that is executed within a workflow. Actors define what kind of data they expect and what kind of data they produce.

One of the use cases for actors is to scan the system and provide the discoveries to other actors through messages. Other actors consume these messages to make decisions, apply changes to the system, or process the information to produce new messages.

When and why do I need to write an actor?

In regards to the upgrades of RHEL 7 to RHEL 8, Leapp should be able to upgrade all the RHEL 7 packages that Red Hat supports to their RHEL 8 equivalents. If it is not possible to upgrade a package, Leapp needs to at least report it to the user. It is a shared responsibility of a) the Leapp developers and b) the development leads of the RHEL 8 subsystems owning RHEL 7 packages. That means that if you are an owner of a package in RHEL 7, you might be approached by your subsystem development lead to analyze the “upgradeability“ of the package from RHEL 7 to RHEL 8. If there is any incompatibility between those packages or any aspect that could negatively impact the upgrade user experience, then that‘s the reason for creating an actor - to make the upgrade experience of the user as smooth as possible. As to when to write such an actor, that is up to discussion among the leads of OAMG, working on Leapp, and your subsystem leads, to set the milestones for the actor development, testing and release.

How can I exchange any data between actors?

All communication between actors in Leapp is carried out using “ messages“. An actor can consume or produce messages. A message may contain any data, but the data needs to be in a specific format defined by a “model“. If an actor wants to consume a message produced by another actor, it needs to specify the specific model of the consumed messages. Leapp will make sure to execute such an actor only after some message of the specified model was produced by another actor. If no message of the specified model was produced in previous phases or in the current phase, the consuming actor will get no messages of that kind. Source: How to create a Leapp actor for RHEL 7 to 8 upgrade

What do I have to do in order to execute actor I just wrote?

If you want to execute just a single actor when developing it, then use the snactor tool. Here’s a tutorial on how to use it. If you want to add your actor to an existing workflow, for example the RHEL 7 to 8 upgrade workflow, then tag your actor with appropriate workflow and phase tags. Source: How to create a Leapp actor for RHEL 7 to 8 upgrade

What should I do if I need to execute multiple actors? Can I somehow ensure the dependencies between them?

To be sure that your ActorA runs before your ActorB, produce a specific message in ActorA and let ActorB consume it. By doing this you create a dependency of ActorB on ActorA. To run just your actors during development, use snactor run –save-output ActorA to save the message of ActorA to the Leapp repository database and then snactor run ActorB. This way, the ActorB will be able to consume the ActorA‘s saved message. Read more about that in the tutorial about messaging.

How can I specify what run time dependencies will my actor have?

See the section about dependencies in the Best practices document

How can I distinguish between actors that I depend on directly (I need to consume their output) and indirectly (I just need them to be executed as part of the upgrade as I don‘t handle the upgrade of that specific piece; think PHP vs. Apache - upgrade of Apache is independent of the upgrade of PHP but it needs to be done to enable its upgrade)?

In the case of actors you depend on directly because you consume their message, you don‘t need to do anything extra, the Leapp framework will make sure that the actors that produce the messages you consume are executed before your actor. In case of the actors you depend on indirectly you may approach it in various ways:

• Talk to the developers of the actors you depend on indirectly and agree on sending a message between their actors and your actor. This will cause a direct dependency described above.
• Talk to the Engineering Lead of the OS and Application Modernization group and tell them to coordinate development, testing and release of your actor and the actors you depend on indirectly, targeting the same milestone.

Once I write an actor that consumes data from some other actors, how can I be sure that the format will not change on the producing side in the future?

The format of a message is specified in a message model. You cannot, however, be sure that the model for the messages you‘re consuming will not change in the future. If that happens, the CI should report errors in the pull request in which the changes to the model are introduced. But for the CI to find out the issue, you as an actor developer need to write thorough unit tests to cover this eventuality.

What are the best practices for writing actors?

Read the Best practices for writing actors.

What are the requirements for actors to be accepted by upstream?

It should follow the Contribution guidelines and the Best practices for writing actors as much as feasible.

How can I debug my actor? Is there a standard/supported way how to log and get logs from actors/channels?

You can run your actor using the snactor tool and printing the output. See the tutorial on how to use snactor. Source: How to create a Leapp actor for RHEL 7 to 8 upgrade

Are there some technical limitations for an actor? E.g. maximum time execution, size of the input/output, libraries I can use... In case there are, is it possible to specify that the actor needs e.g. longer time for execution?

There are no technical limitations but rather conceptual:

• Libraries to use:
  • See the section about actor dependencies in the Best practices document

Execution time:

• Some Red Hat customers do business in fields where time matters a lot. They may have obligations to not allow more than a few minutes of downtime per year. It means that we should make sure that our tooling causes as short downtime as possible.
• It‘s not currently possible to tell the Leapp framework that the actor takes longer time to execute.

Are there some actions that are either forbidden or not recommended to be done in actors?

In regards to the RHEL 7 to RHEL 8 in-place upgrades, the rule is to not alter the system in any way before the point of no return, which is the start of the RPMUpgrade phase running in the initramfs. Any deviation from this rule must be well justified.

I got an error about PES data/ Repositories mapping where I find such files?

These files can not be packaged together in Leapp RPM for license reasons.

For information on how to get these files, please read this article.

Python documentation for the leapp package

Subpackages

leapp.actors package

Module contents

class leapp.actors.Actor(messaging=None, logger=None, config_model=None, skip_dialogs=False)

Bases: object

The Actor class represents the smallest step in the workflow. It defines what kind of data it expects, it consumes (processes) the given data, and it produces data for other actors in the workflow.

class ErrorSeverity

Bases: object

Convenience forward for the leapp.models.error_severity.ErrorSeverity constants.

ALLOWED_VALUES = ('fatal', 'error', 'warning')

ERROR = 'error'

FATAL = 'fatal'

WARNING = 'warning'

classmethod validate(value)

actor_files_paths

Returns the file paths that are bundled with the actor. (Path to the content of the actor’s file directory).

actor_tools_paths

Returns the tool paths that are bundled with the actor. (Path to the content of the actor’s tools directory).

apis = ()

Tuple of leapp.workflow.api.WorkflowAPI derived classes that implement Workflow APIs that are used by an actor. Any models the apis produce or consume will be considered by the framework as if the actor defined them.

common_files_paths

Returns all common repository file paths.

common_tools_paths

Returns all common repository tool paths.

configuration

Returns the config model generated by specific workflow configuration actor

consume(*models)

Retrieve messages specified in the actors consumes attribute, and filter message types by models.

Parameters:models (Variable number of the derived classes from leapp.models.Model) – Models to use as a filter for the messages to return Returns:All messages of the specified model(s) produced by other actors Return type:Iterable with messages or an empty tuple

consumes = ()

Tuple of leapp.models.Model derived classes defined in the repositories that define messages the actor consumes.

current_instance = None

Instance of the currently executed actor. Within a process only exist one Actor instance. This will allow convenience functions for library developers to be available.

description = None

Deprecated since version 0.5.0: Write the actor’s description as a docstring.

dialogs = ()

Tuple of leapp.dialogs.dialog.Dialog derived classes that define questions to ask the user. Dialogs that are added to this list allow for persisting answers the user has given in the answer file storage.

files_paths

Returns all actor file paths related to the actor and common actors file paths.

get_actor_file_path(name)

Finds the first matching file path within actor_files_paths.

Parameters:name (str) – Name of the file Returns:Found file path Return type:str or None

get_actor_folder_path(name)

Finds the first matching folder path within actor_files_paths.

Parameters:name (str) – Name of the folder Returns:Found folder path Return type:str or None

get_actor_tool_path(name)

Finds the first matching executable file path within actor_tools_paths.

Parameters:name (str) – Name of the file Returns:Found file path Return type:str or None

get_answers(dialog)

Gets the answers for a dialog. The dialog needs be predefined in dialogs.

Parameters:dialog – Dialog instance to show Returns:dictionary with the requested answers, None if not a defined dialog

get_common_file_path(name)

Finds the first matching file path within common_files_paths.

Parameters:name (str) – Name of the file Returns:Found file path Return type:str or None

get_common_folder_path(name)

Finds the first matching folder path within common_files_paths.

Parameters:name (str) – Name of the folder Returns:Found folder path Return type:str or None

get_common_tool_path(name)

Finds the first matching executable file path within common_tools_paths.

Parameters:name (str) – Name of the file Returns:Found file path Return type:str or None

get_file_path(name)

Finds the first matching file path within files_paths.

Parameters:name (str) – Name of the file Returns:Found file path Return type:str or None

get_folder_path(name)

Finds the first matching folder path within files_paths.

Parameters:name (str) – Name of the folder Returns:Found folder path Return type:str or None

get_tool_path(name)

Finds the first matching executable file path within tools_paths.

Parameters:name (str) – Name of the file Returns:Found file path Return type:str or None

name = None

Name of the actor that is used to identify data or messages created by the actor.

process(*args, **kwargs)

Main processing method. In inherited actors, the function needs to be defined to be able to be processed.

produce(*models)

By calling produce, model instances are stored as messages. Those messages can be then consumed by other actors.

Parameters:models (Variable number of the derived classes from leapp.models.Model) – Messages to be sent (those model types have to be specified in produces

produces = ()

Tuple of leapp.models.Model derived classes defined in the repositories that define messages the actor produces.

report_error(message, severity='error', details=None)

Reports an execution error

Parameters:

• message (str) – A message to print the possible error
• severity (str with defined values from leapp.messaging.errors.ErrorSeverity.ERROR) – Severity of the error default leapp.messaging.errors.ErrorSeverity.ERROR
• details (dict) – A dictionary where additional context information is passed along with the error

Returns:

None

run(*args)

Runs the actor calling the method process().

serialize()

Returns:Serialized information for the actor

show_message(message)

Display a message in user interterface currently in use (CLI, GUI).

It uses one of the dialog renderers in leapp.dialogs.renderer.

Parameters:message (str) – Message to show

skip_dialogs = None

A configured logger instance for the current actor.

tags = ()

Tuple of leapp.tags.Tag derived classes by which workflow phases select actors for execution.

text_domain = None

Using text domain allows to override the default gettext text domain, for custom localization support. The default locale installation location is used which usually is /usr/share/locale

tools_paths

Returns all actor tools paths related to the actor and common actors tools paths.

leapp.actors.get_actor_metadata(actor)

Creates Actor’s metadata dictionary

Parameters:actor (derived class from leapp.actors.Actor) – Actor whose metadata are needed Returns:Dictionary with the name, tags, consumes, produces, and description of the actor

leapp.actors.get_actors()

Returns:All registered actors with their metadata

leapp.dialogs package

Submodules

leapp.dialogs.components module

class leapp.dialogs.components.BooleanComponent(key=None, label=None, description=None, default=None, reason=None, values=None)

Bases: leapp.dialogs.components.Component

BooleanComponent is used for boolean inputs such as Yes/No questions.

Parameters:

• key – Unique key within a dialog scope. Needs to be in the format: [a-zA-Z_][a-zA-Z0-9_]*
• label – Label for the input to print
• description – Description what this value is used for.
• default – Default value to
• reason – The reason why we need this value.
• values – Values to use as True and False, first is always True and the second is always False (e.g. Yes/No)

choices = ('True', 'False')

default = None

description = None

dispatch(renderer, dialog)

key = None

label = None

reason = None

serialize()

Returns:Serialized component information

value = None

value_type

alias of bool

values = ('Yes', 'No')

class leapp.dialogs.components.ChoiceComponent(choices=None, key=None, label=None, description=None, default=None, reason=None)

Bases: leapp.dialogs.components.Component

ChoiceComponent is used to give a list of options and allows to select one (like a radio button)

Parameters:

• key – Unique key within a dialog scope. Needs to be in the format: [a-zA-Z_][a-zA-Z0-9_]*
• label – Label for the input to print
• description – Description what this value is used for.
• default – Default value to
• reason – The reason why we need this value.
• choices – Choices that are available to the user

choices = ()

default = None

description = None

dispatch(renderer, dialog)

key = None

label = None

multi = False

reason = None

serialize()

Returns:Serialized component information

value = None

value_type

alias of unicode

class leapp.dialogs.components.Component(key=None, label=None, description=None, default=None, reason=None)

Bases: object

Base class for all components

Parameters:

• key – Unique key within a dialog scope. Needs to be in the format: [a-zA-Z_][a-zA-Z0-9_]*
• label – Label for the input to print
• description – Description what this value is used for.
• default – Default value to
• reason – The reason why we need this value.

default = None

description = None

dispatch(renderer, dialog)

key = None

label = None

reason = None

serialize()

Returns:Serialized component information

value = None

value_type = None

class leapp.dialogs.components.MultipleChoiceComponent(choices=None, key=None, label=None, description=None, default=None, reason=None)

Bases: leapp.dialogs.components.ChoiceComponent

MultipleChoiceComponent is used to give a list of options and allows to select more than one (like checkboxes)

Parameters:

• key – Unique key within a dialog scope. Needs to be in the format: [a-zA-Z_][a-zA-Z0-9_]*
• label – Label for the input to print
• description – Description what this value is used for.
• default – Default value to
• reason – The reason why we need this value.
• choices – Choices that are available to the user

choices = ()

default = None

description = None

dispatch(renderer, dialog)

Calls the appropriate rendering implementation on the renderer and passes itself and the dialog instance to it.

Parameters:

• renderer (leapp.dialogs.renderer.DialogRendererBase) – Renderer instance
• dialog –

Returns:

key = None

label = None

multi = True

reason = None

serialize()

Returns:Serialized component information

value = None

value_type

alias of tuple

class leapp.dialogs.components.NumberComponent(key=None, label=None, description=None, default=None, reason=None)

Bases: leapp.dialogs.components.Component

NumberComponent is used for integer inputs.

Parameters:

• key – Unique key within a dialog scope. Needs to be in the format: [a-zA-Z_][a-zA-Z0-9_]*
• label – Label for the input to print
• description – Description what this value is used for.
• default – Default value to
• reason – The reason why we need this value.

default = -1

description = None

dispatch(renderer, dialog)

key = None

label = None

reason = None

serialize()

Returns:Serialized component information

value = None

value_type

alias of int

class leapp.dialogs.components.PasswordComponent(key=None, label=None, description=None, default=None, reason=None)

Bases: leapp.dialogs.components.TextComponent

PasswordComponent is a text input component which will use non echoing input when possible (see getpass).

Parameters:

• key – Unique key within a dialog scope. Needs to be in the format: [a-zA-Z_][a-zA-Z0-9_]*
• label – Label for the input to print
• description – Description what this value is used for.
• default – Default value to
• reason – The reason why we need this value.

default = None

description = None

dispatch(renderer, dialog)

key = None

label = 'Password'

reason = None

serialize()

Returns:Serialized component information

value = None

value_type

alias of str

class leapp.dialogs.components.TextComponent(key=None, label=None, description=None, default=None, reason=None)

Bases: leapp.dialogs.components.Component

TextComponent is a text input component.

Parameters:

• key – Unique key within a dialog scope. Needs to be in the format: [a-zA-Z_][a-zA-Z0-9_]*
• label – Label for the input to print
• description – Description what this value is used for.
• default – Default value to
• reason – The reason why we need this value.

default = None

description = None

dispatch(renderer, dialog)

key = None

label = None

reason = None

serialize()

Returns:Serialized component information

value = None

value_type

alias of str

leapp.dialogs.dialog module

class leapp.dialogs.dialog.Dialog(scope, reason, title=None, components=None, key=None)

Bases: object

The dialog class is used to declare the information passed to the user and retrieved from the user during an interaction.

Parameters:

• scope (str) – Unique scope identifier for the data to be stored in the answer files. Scope + component key is used to address values.
• reason (str) – An explanation for what the data in this dialog is needed for.
• title (str) – Title of the dialog
• components (tuple(leapp.dialogs.components.Component)) – Components to display in the given order in the dialog
• key (str) – Key to appear in the dialog-related report entry

answer(component, value)

Implements storing of answers.

Parameters:

• component – Component for which the answer is set
• value – The answer value

Returns:

None

answerfile_sections

component_by_key(key)

Finds the component with the given key

Parameters:key (str) – Key of the component to return Returns:Component found or None

components = ()

Components to display in the given order in the dialog

get_answers(store)

Checks answerstore if an answer is recorded for the dialog.

Parameters:store – AnswerStore instance Returns:Dictionary with answers once retrieved

min_label_width

Returns:Returns the highest number of characters all labels in the dialog have, to help calculating the minimum width the labels should have.

reason = None

An explanation for what the data in this dialog is needed for.

request_answers(store, renderer)

Same as get_answers but with interactive retrieval of the answer in case no recorded answer found in answerstore.

Parameters:

• store – AnswerStore instance
• renderer – Target renderer instance

Returns:

Dictionary with answers once retrieved

scope = None

Unique scope identifier for the data to be stored in the answer files. Scope + component key is used to address values.

serialize()

Returns:Dictionary with the serialized representation of a component

title = None

Title of the dialog

leapp.dialogs.renderer module

class leapp.dialogs.renderer.CommandlineRenderer

Bases: leapp.dialogs.renderer.DialogRendererBase

CommandlineRenderer implements the handling for commandline user interactions.

render(dialog)

Renders the given dialog

Parameters:dialog (leapp.dialogs.dialog.Dialog) – The dialog to render Returns:None

render_bool_component(component, dialog)

Renders the boolean component for displaying Yes/No questions

Parameters:

• component (leapp.dialogs.components.BooleanComponent) – The boolean component to render
• dialog (leapp.dialogs.dialog.Dialog) – The dialog to render

Returns:

None

render_choice_component(component, dialog)

Renders the choices component

Parameters:

• component (leapp.dialogs.components.ChoicesComponent) – The choices component to render
• dialog (leapp.dialogs.dialog.Dialog) – The dialog to render

Returns:

None

render_multiple_choice_component(component, dialog)

Renders the multiple choices component

Parameters:

• component (leapp.dialogs.components.MultipleChoiceComponent) – The multiple choices component to render
• dialog (leapp.dialogs.dialog.Dialog) – The dialog to render

Returns:

None

render_number_component(component, dialog)

Renders the number component

Parameters:

• component (leapp.dialogs.components.NumberComponent) – The number component to render
• dialog (leapp.dialogs.dialog.Dialog) – The dialog to render

Returns:

None

render_password_component(component, dialog)

Renders the password component

Parameters:

• component (leapp.dialogs.components.PasswordComponent) – The password component to render
• dialog (leapp.dialogs.dialog.Dialog) – The dialog to render

Returns:

None

render_text_component(component, dialog)

Renders the text component

Parameters:

• component (leapp.dialogs.components.TextComponent) – The text component to render
• dialog (leapp.dialogs.dialog.Dialog) – The dialog to render

Returns:

None

class leapp.dialogs.renderer.DialogRendererBase

Bases: object

Base class for all renderer implementations

render(dialog)

Renders the given dialog

Parameters:dialog (leapp.dialogs.dialog.Dialog) – The dialog to render Returns:None

render_bool_component(component, dialog)

Renders the boolean component for displaying Yes/No questions

Parameters:

• component (leapp.dialogs.components.BooleanComponent) – The boolean component to render
• dialog (leapp.dialogs.dialog.Dialog) – The dialog to render

Returns:

None

render_choice_component(component, dialog)

Renders the choices component

Parameters:

• component (leapp.dialogs.components.ChoicesComponent) – The choices component to render
• dialog (leapp.dialogs.dialog.Dialog) – The dialog to render

Returns:

None

render_multiple_choice_component(component, dialog)

Renders the multiple choices component

Parameters:

• component (leapp.dialogs.components.MultipleChoiceComponent) – The multiple choices component to render
• dialog (leapp.dialogs.dialog.Dialog) – The dialog to render

Returns:

None

render_number_component(component, dialog)

Renders the number component

Parameters:

• component (leapp.dialogs.components.NumberComponent) – The number component to render
• dialog (leapp.dialogs.dialog.Dialog) – The dialog to render

Returns:

None

render_password_component(component, dialog)

Renders the password component

Parameters:

• component (leapp.dialogs.components.PasswordComponent) – The password component to render
• dialog (leapp.dialogs.dialog.Dialog) – The dialog to render

Returns:

None

render_text_component(component, dialog)

Renders the text component

Parameters:

• component (leapp.dialogs.components.TextComponent) – The text component to render
• dialog (leapp.dialogs.dialog.Dialog) – The dialog to render

Returns:

None

Module contents

class leapp.dialogs.RawMessageDialog(message)

Bases: leapp.dialogs.dialog.Dialog

Reusable message dialog. - This sends a message to the user only without any special formatting.

answer(component, value)

Implements storing of answers.

Parameters:

• component – Component for which the answer is set
• value – The answer value

Returns:

None

answerfile_sections

component_by_key(key)

Finds the component with the given key

Parameters:key (str) – Key of the component to return Returns:Component found or None

components = ()

get_answers(store)

Checks answerstore if an answer is recorded for the dialog.

Parameters:store – AnswerStore instance Returns:Dictionary with answers once retrieved

min_label_width

Returns:Returns the highest number of characters all labels in the dialog have, to help calculating the minimum width the labels should have.

reason = None

request_answers(store, renderer)

Parameters:

• store – AnswerStore instance
• renderer – Target renderer instance

Returns:

Dictionary with answers once retrieved

scope = None

serialize()

Returns:Dictionary with the serialized representation of a component

title = ''

class leapp.dialogs.UsernamePasswordDialog(scope, reason, title=None, components=None, key=None)

Bases: leapp.dialogs.dialog.Dialog

Reusable username and password request dialog.

Parameters:

• scope (str) – Unique scope identifier for the data to be stored in the answer files. Scope + component key is used to address values.
• reason (str) – An explanation for what the data in this dialog is needed for.
• title (str) – Title of the dialog
• components (tuple(leapp.dialogs.components.Component)) – Components to display in the given order in the dialog
• key (str) – Key to appear in the dialog-related report entry

answer(component, value)

Implements storing of answers.

Parameters:

• component – Component for which the answer is set
• value – The answer value

Returns:

None

answerfile_sections

component_by_key(key)

Finds the component with the given key

Parameters:key (str) – Key of the component to return Returns:Component found or None

components = (<leapp.dialogs.components.TextComponent object>, <leapp.dialogs.components.PasswordComponent object>)

get_answers(store)

Checks answerstore if an answer is recorded for the dialog.

Parameters:store – AnswerStore instance Returns:Dictionary with answers once retrieved

min_label_width

Returns:Returns the highest number of characters all labels in the dialog have, to help calculating the minimum width the labels should have.

reason = None

request_answers(store, renderer)

Same as get_answers but with interactive retrieval of the answer in case no recorded answer found in answerstore.

Parameters:

• store – AnswerStore instance
• renderer – Target renderer instance

Returns:

Dictionary with answers once retrieved

scope = None

serialize()

Returns:Dictionary with the serialized representation of a component

title = 'Please enter username and password'

leapp.libraries package

Subpackages

leapp.libraries.actor package

Module contents

leapp.libraries.actor represents the import location for private actor libraries that are placed in the actor’s libraries folder.

Example:

If you actor has a libraries folder with a module.py python module, import it from the actor like this:

from leapp.libraries.actor import module

leapp.libraries.common package

Module contents

leapp.libraries.common represents an import location for shared libraries that are placed in the repository’s libraries folder.

Example:

If any of the repositories has a libraries folder with a module.py python module, import it from the actor like this:

from leapp.libraries.common import module

leapp.libraries.common.LEAPP_BUILTIN_COMMON_INITIALIZED = False

Internal variable to the framework to inform if the common libraries have been already initialized

leapp.libraries.stdlib package

Module contents

leapp.libraries.stdlib represents a location for functions that otherwise would be defined multiple times across leapp actors and at the same time, they are really useful for other actors.

exception leapp.libraries.stdlib.CalledProcessError(message, command, result)

Bases: leapp.exceptions.LeappError

Leapp Call Process Exception Error.

Raised when the result of a called process is of a none zero return code.

Initialize CalledProcessError Exception. :param message: An CalledProcessError exception message. :param command: The command that has been executed, with its arguments. :param result: A non-zero whatever result that the command returned.

args

exit_code

Retrieve the exit code. :return: An exit code.

message

pid

Retrieve the pid of the finished process. :return: The pid of the process.

signal

Retrieve the signal which the process was signalled by. :return: A signal that the process received.

stderr

Retrieve the stderr. :return: Standard Error.

stdout

Retrieve the stdout. :return: Standard Output.

leapp.libraries.stdlib.run(args, split=False, callback_raw=<function _console_logging_handler>, callback_linebuffered=<function _logfile_logging_handler>, env=None, checked=True, stdin=None, encoding='utf-8')

Run a command and return its result as a dict.

The execution of the program and it’s results are captured by the audit.

Parameters:

• args (list or tuple) – Command to execute
• split (bool) – Split the output on newlines
• callback_raw ((fd: int, buffer: bytes) -> None) – Optional custom callback executed on raw data to print in console
• env (dict) – Environment variables to use for execution of the command
• checked (bool) – Raise an exception on a non-zero exit code, default True
• stdin (int, str) – String or a file descriptor that will be written to stdin of the child process

Returns:

{‘stdout’ : stdout, ‘stderr’: stderr, ‘signal’: signal, ‘exit_code’: exit_code, ‘pid’: pid}

Return type:

dict

Raises:

OSError if an executable is missing or has wrong permissions

Raises:

CalledProcessError if the cmd has non-zero exit code and checked is False

Raises:

TypeError if any input parameters have an invalid type

Raises:

valueError if any of input parameters have an invalid value

This module implements a convenience API for actions that are accessible to actors.

Any code that wants use this convenience library has to be called from within the actors context. This is true for actors, actor private libraries and repository libraries.

leapp.libraries.stdlib.api.actor_files_paths()

Returns the file paths that are bundled with the actor. (Path to the content of the actor’s file directory).

leapp.libraries.stdlib.api.actor_tools_paths()

Returns the tool paths that are bundled with the actor. (Path to the content of the actor’s tools directory).

leapp.libraries.stdlib.api.common_files_paths()

Returns all common repository file paths.

leapp.libraries.stdlib.api.common_tools_paths()

Returns all common repository tool paths.

leapp.libraries.stdlib.api.consume(*models)

Retrieve messages specified in the actors consumes attribute, and filter message types by models.

Parameters:models (Variable number of the derived classes from leapp.models.Model) – Models to use as a filter for the messages to return

leapp.libraries.stdlib.api.current_actor()

Retrieve the Actor class instance of the current active actor. :return: Instance of the currently instantiated actor. :rtype: leapp.actors.Actor

leapp.libraries.stdlib.api.current_logger()

Retrieve the logger of the current active actor. :return: Logger instance for the current actor. :rtype: logging.Logger

leapp.libraries.stdlib.api.files_paths()

Returns all actor file paths related to the actor and common actors file paths.

leapp.libraries.stdlib.api.get_actor_file_path(name)

Finds the first matching file path within files_paths.

Parameters:name (str) – Name of the file Returns:Found file path Return type:str or None

leapp.libraries.stdlib.api.get_actor_folder_path(name)

Finds the first matching folder path within files_paths.

Parameters:name (str) – Name of the folder Returns:Found folder path Return type:str or None

leapp.libraries.stdlib.api.get_actor_tool_path(name)

Finds the first matching executable file path within actor_tools_paths.

Parameters:name (str) – Name of the file Returns:Found file path Return type:str or None

leapp.libraries.stdlib.api.get_answers(dialog)

Get the answers for a dialog from answerfile. The dialog needs be predefined in dialogs of the actor.

Parameters:dialog – Dialog instance to show Returns:dictionary with the requested answers, None if not a defined dialog

leapp.libraries.stdlib.api.get_common_file_path(name)

Finds the first matching file path within files_paths.

Parameters:name (str) – Name of the file Returns:Found file path Return type:str or None

leapp.libraries.stdlib.api.get_common_folder_path(name)

Finds the first matching folder path within files_paths.

Parameters:name (str) – Name of the folder Returns:Found folder path Return type:str or None

leapp.libraries.stdlib.api.get_common_tool_path(name)

Finds the first matching executable file path within common_tools_paths.

Parameters:name (str) – Name of the file Returns:Found file path Return type:str or None

leapp.libraries.stdlib.api.get_file_path(name)

Finds the first matching file path within files_paths.

Parameters:name (str) – Name of the file Returns:Found file path Return type:str or None

leapp.libraries.stdlib.api.get_folder_path(name)

Finds the first matching folder path within files_paths.

Parameters:name (str) – Name of the folder Returns:Found folder path Return type:str or None

leapp.libraries.stdlib.api.get_tool_path(name)

Finds the first matching executable file path within tools_paths.

Parameters:name (str) – Name of the file Returns:Found file path Return type:str or None

leapp.libraries.stdlib.api.produce(*model_instances)

By calling produce, model instances are stored as messages. Those messages can be then consumed by other actors.

Parameters:model_instances (Variable number of instances of derived classes from leapp.models.Model) – Messages to be sent (those model types have to be specified in produces

leapp.libraries.stdlib.api.report_error(message, severity='error', details=None)

Reports an execution error

Parameters:

• message (str) – A message to print the possible error
• severity (str with defined values from leapp.messaging.errors.ErrorSeverity.ERROR) – Severity of the error default leapp.messaging.errors.ErrorSeverity.ERROR
• details (dict) – A dictionary where additional context information is passed along with the error

Returns:

None

leapp.libraries.stdlib.api.show_message(message)

Display a message in user interterface currently in use (CLI, GUI).

It uses one of the dialog renderers in leapp.dialogs.renderer.

Parameters:message (str) – Message to show

leapp.libraries.stdlib.api.tools_paths()

Returns all actor tools paths related to the actor and common actors tools paths.

Module contents

leapp.logger package

Module contents

class leapp.logger.LeappAuditHandler(*args, **kwargs)

Bases: logging.Handler

acquire()

Acquire the I/O thread lock.

addFilter(filter)

Add the specified filter to this handler.

close()

Tidy up any resources used by the handler.

This version removes the handler from an internal map of handlers, _handlers, which is used for handler lookup by name. Subclasses should ensure that this gets called from overridden close() methods.

createLock()

Acquire a thread lock for serializing access to the underlying I/O.

emit(record)

filter(record)

Determine if a record is loggable by consulting all the filters.

The default is to allow the record to be logged; any filter can veto this and the record is then dropped. Returns a zero value if a record is to be dropped, else non-zero.

flush()

Ensure all logging output has been flushed.

This version does nothing and is intended to be implemented by subclasses.

format(record)

Format the specified record.

If a formatter is set, use it. Otherwise, use the default formatter for the module.

get_name()

handle(record)

Conditionally emit the specified logging record.

Emission depends on filters which may have been added to the handler. Wrap the actual emission of the record with acquisition/release of the I/O thread lock. Returns whether the filter passed the record for emission.

handleError(record)

Handle errors which occur during an emit() call.

This method should be called from handlers when an exception is encountered during an emit() call. If raiseExceptions is false, exceptions get silently ignored. This is what is mostly wanted for a logging system - most users will not care about errors in the logging system, they are more interested in application errors. You could, however, replace this with a custom handler if you wish. The record which was being processed is passed in to this method.

name

release()

Release the I/O thread lock.

removeFilter(filter)

Remove the specified filter from this handler.

setFormatter(fmt)

Set the formatter for this handler.

setLevel(level)

Set the logging level of this handler.

set_name(name)

leapp.logger.configure_logger(log_file=None)

leapp.messaging package

Submodules

leapp.messaging.inprocess module

class leapp.messaging.inprocess.InProcessMessaging(stored=True, config_model=None, answer_store=None)

Bases: leapp.messaging.BaseMessaging

This class implements the direct database access for the messaging.

command(command)

Called to send a command to the workflow execution

Parameters:command (Instance of leapp.messaging.commands.WorkflowCommand) – A command to send to the workflow execution. Returns:None

commands

Returns:List of commands that have been sent to the workflow execution.

consume(actor, *types)

Returns all consumable messages and filters them by types

Parameters:

• types – Variable number of leapp.models.Model derived types to filter messages to be consumed
• actor – Actor that consumes the data

Returns:

Iterable with messages matching the criteria

dialogs()

Gets all dialogs actually encountered during workflow run

Returns:List of encountered dialogs

errors()

Gets all produced errors.

Returns:List of newly produced errors

feed(model, actor)

Called to pre-fill sent messages and make them available for other actors.

Parameters:

• model (leapp.models.Model) – Model to send as message payload
• actor (leapp.actors.Actor) – Actor that sends the message

Returns:

the updated message dict

Return type:

dict

get_answers(dialog)

load(consumes)

Loads all messages that are requested from the consumes attribute of leapp.actors.Actor

Parameters:consumes – Tuple or list of leapp.models.Model types to preload Returns:None Raises:leapp.exceptions.CannotConsumeErrorMessages – When trying to consume ErrorModel

load_answers(answer_file, workflow)

Loads answers from a given answer file

Parameters:

• answer_file (str) – Path to file to load as answer file
• workflow (leapp.workflows.Workflow) – leapp.workflows.Workflow instance to load the answers for.

Returns:

None

messages()

Gets all newly produced messages.

Returns:List of newly processed messages

produce(model, actor)

Called to send a message available for other actors.

Parameters:

• model (leapp.models.Model) – Model to send as message payload
• actor (leapp.actors.Actor) – Actor that sends the message

Returns:

the updated message dict

Return type:

dict

register_dialog(dialog, actor)

report_error(message, severity, actor, details)

Reports an execution error

Parameters:

• message (str) – Message to print the error
• severity (leapp.models.error_severity.ErrorSeverity) – Severity of the error
• actor (leapp.actors.Actor) – Actor name that produced the message
• details (dict) – A dictionary where additional context information can be passed along with the error

Returns:

None

report_stacktrace(message, trace, actorname)

request_answers(dialog)

request_stop_after_phase()

If called, it will cause the workflow to stop the execution after the current phase ends.

show_message(message)

Display a message in user interterface currently in use (CLI, GUI).

It uses one of the dialog renderers in leapp.dialogs.renderer.

Parameters:message (str) – Message to show

stop_after_phase

Returns True if execution stop was requested after the current

Returns:True if the executed was requested to be stopped.

stored

Returns:If the messages are stored immediately, this function returns True, otherwise False.

Module contents

class leapp.messaging.BaseMessaging(stored=True, config_model=None, answer_store=None)

Bases: object

BaseMessaging is the Base class for all messaging implementations. It provides the basic interface that is supported within the framework. These are called the produce and consume methods.

command(command)

Called to send a command to the workflow execution

Parameters:command (Instance of leapp.messaging.commands.WorkflowCommand) – A command to send to the workflow execution. Returns:None

commands

Returns:List of commands that have been sent to the workflow execution.

consume(actor, *types)

Returns all consumable messages and filters them by types

Parameters:

• types – Variable number of leapp.models.Model derived types to filter messages to be consumed
• actor – Actor that consumes the data

Returns:

Iterable with messages matching the criteria

dialogs()

Gets all dialogs actually encountered during workflow run

Returns:List of encountered dialogs

errors()

Gets all produced errors.

Returns:List of newly produced errors

feed(model, actor)

Called to pre-fill sent messages and make them available for other actors.

Parameters:

• model (leapp.models.Model) – Model to send as message payload
• actor (leapp.actors.Actor) – Actor that sends the message

Returns:

the updated message dict

Return type:

dict

get_answers(dialog)

load(consumes)

Loads all messages that are requested from the consumes attribute of leapp.actors.Actor

Parameters:consumes – Tuple or list of leapp.models.Model types to preload Returns:None Raises:leapp.exceptions.CannotConsumeErrorMessages – When trying to consume ErrorModel

load_answers(answer_file, workflow)

Loads answers from a given answer file

Parameters:

• answer_file (str) – Path to file to load as answer file
• workflow (leapp.workflows.Workflow) – leapp.workflows.Workflow instance to load the answers for.

Returns:

None

messages()

Gets all newly produced messages.

Returns:List of newly processed messages

produce(model, actor)

Called to send a message available for other actors.

Parameters:

• model (leapp.models.Model) – Model to send as message payload
• actor (leapp.actors.Actor) – Actor that sends the message

Returns:

the updated message dict

Return type:

dict

register_dialog(dialog, actor)

report_error(message, severity, actor, details)

Reports an execution error

Parameters:

• message (str) – Message to print the error
• severity (leapp.models.error_severity.ErrorSeverity) – Severity of the error
• actor (leapp.actors.Actor) – Actor name that produced the message
• details (dict) – A dictionary where additional context information can be passed along with the error

Returns:

None

report_stacktrace(message, trace, actorname)

request_answers(dialog)

request_stop_after_phase()

If called, it will cause the workflow to stop the execution after the current phase ends.

show_message(message)

Display a message in user interterface currently in use (CLI, GUI).

It uses one of the dialog renderers in leapp.dialogs.renderer.

Parameters:message (str) – Message to show

stop_after_phase

Returns True if execution stop was requested after the current

Returns:True if the executed was requested to be stopped.

stored

Returns:If the messages are stored immediately, this function returns True, otherwise False.

leapp.models package

Subpackages

leapp.models.fields package

Module contents

class leapp.models.fields.Blob(default=None, help=None)

Bases: leapp.models.fields.BuiltinField

Blob field

Parameters:

• default – Default value to be used if the field is not set
• help (str) – Documentation string for generating model documentation

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

Returns:Serialized form of the workflow

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

class leapp.models.fields.Boolean(default=None, help=None)

Bases: leapp.models.fields.BuiltinField

Boolean field

Parameters:

• default – Default value to be used if the field is not set
• help (str) – Documentation string for generating model documentation

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

Returns:Serialized form of the workflow

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

class leapp.models.fields.BuiltinField(default=None, help=None)

Bases: leapp.models.fields.Field

Base class for all builtin types to act as pass-through with an additional validation

Parameters:

• default – Default value to be used if the field is not set
• help (str) – Documentation string for generating model documentation

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

Returns:Serialized form of the workflow

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

class leapp.models.fields.DateTime(default=None, help=None)

Bases: leapp.models.fields.BuiltinField

DateTime field to handle datetime objects which are converted to the ISO format and parsed back from there

Parameters:

• default – Default value to be used if the field is not set
• help (str) – Documentation string for generating model documentation

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

Returns:Serialized form of the workflow

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

class leapp.models.fields.EnumMixin(choices, **kwargs)

Bases: leapp.models.fields.Field

EnumMixin adds the ability to use the field as an Enum type of the field

Parameters:choices (List or tuple of allowed values) – List of values that are allowed for this field

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

class leapp.models.fields.Field(default=None, help=None)

Bases: object

Field is the base of all supported fields.

Parameters:

• default – Default value to be used if the field is not set
• help (str) – Documentation string for generating model documentation

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

Returns:Serialized form of the workflow

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

class leapp.models.fields.Float(default=None, help=None)

Bases: leapp.models.fields.BuiltinField

Float field

Parameters:

• default – Default value to be used if the field is not set
• help (str) – Documentation string for generating model documentation

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

Returns:Serialized form of the workflow

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

class leapp.models.fields.FloatEnum(choices, **kwargs)

Bases: leapp.models.fields.EnumMixin, leapp.models.fields.Float

Field that represents an enumeration of Floats

Parameters:choices (List or tuple of allowed values) – List of values that are allowed for this field

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

class leapp.models.fields.Integer(default=None, help=None)

Bases: leapp.models.fields.BuiltinField

Integer field (int, long in python 2, int in python 3)

Parameters:

• default – Default value to be used if the field is not set
• help (str) – Documentation string for generating model documentation

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

Returns:Serialized form of the workflow

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

class leapp.models.fields.IntegerEnum(choices, **kwargs)

Bases: leapp.models.fields.EnumMixin, leapp.models.fields.Integer

Field that represents an enumeration of Integers

Parameters:choices (List or tuple of allowed values) – List of values that are allowed for this field

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

class leapp.models.fields.JSON(default=None, help=None)

Bases: leapp.models.fields.String

The JSON field allows to use json encodable python types as a value.

The value will be internally encoded to a JSON string and converted back into, whatever the result of json.loads is for that value passed.

Note: The value None, however follows the same rules as for all fields and requires the field to be nullable,

to allow this value. Within nested values such as lists or dicts, a None value is perfectly valid.

Parameters:

• default – Default value to be used if the field is not set
• help (str) – Documentation string for generating model documentation

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

Returns:Serialized form of the workflow

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

class leapp.models.fields.List(elem_field, minimum=None, maximum=None, **kwargs)

Bases: leapp.models.fields.Field

List represents lists of elem_field values

Parameters:

• elem_field –
• minimum (int or None) – Minimal number of elements
• maximum (int or None) – Maximum number of elements
• default (A list of elements with the value type as specified in elem_field) – Default value to use if the field is not set
• help (str) – Documentation string for generating model documentation

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

class leapp.models.fields.Model(model_type, **kwargs)

Bases: leapp.models.fields.Field

Model is used to use other Models as fields

Parameters:

• model_type (leapp.model.Model derived class) – A leapp.model.Model derived class
• help (str) – Documentation string for generating the model documentation

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

exception leapp.models.fields.ModelMisuseError

Bases: exceptions.Exception

ModelMisuseError is raised if the Model definition is not valid.

args

message

exception leapp.models.fields.ModelViolationError

Bases: exceptions.Exception

ModelViolationError is raised if the data in the instances is not matching its definition.

args

message

leapp.models.fields.Nullable(elem_field)

Helper function to make a field nullable

class leapp.models.fields.Number(default=None, help=None)

Bases: leapp.models.fields.BuiltinField

Combined Integer and Float field

Parameters:

• default – Default value to be used if the field is not set
• help (str) – Documentation string for generating model documentation

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

Returns:Serialized form of the workflow

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

class leapp.models.fields.NumberEnum(choices, **kwargs)

Bases: leapp.models.fields.EnumMixin, leapp.models.fields.Number

Field that represents an enumeration of Numbers

Parameters:choices (List or tuple of allowed values) – List of values that are allowed for this field

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

class leapp.models.fields.String(default=None, help=None)

Bases: leapp.models.fields.BuiltinField

String field

Parameters:

• default – Default value to be used if the field is not set
• help (str) – Documentation string for generating model documentation

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

Returns:Serialized form of the workflow

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

class leapp.models.fields.StringEnum(choices, **kwargs)

Bases: leapp.models.fields.EnumMixin, leapp.models.fields.String

Field that represents an enumeration of Strings

Parameters:choices (List or tuple of allowed values) – List of values that are allowed for this field

as_nullable()

Set object` “_nullable_” field to True and return the object back

from_initialization(source, name, target)

Assigns the value to the target model passed through during the model initialization

Parameters:

• source (dict) – Dictionary to extract the value from (usually kwargs)
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

help

Returns:Documentation help string defining what the field is about.

serialize()

to_builtin(source, name, target)

Converts the value with the given name to the builtin representation and assigns the field

Parameters:

• source (Instance of a Model derived class) – Source model to get the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (dict) – Dictionary to set the value to

Returns:

None

to_model(source, name, target)

Converts the value with the given name to the model representation and assigns the attribute

Parameters:

• source (dict) – Dictionary to extract the value from
• name (str) – Name of the field (used for a better error reporting only)
• target (Instance of a Model derived class) – Target model instance

Returns:

None

Submodules

leapp.models.error_severity module

class leapp.models.error_severity.ErrorSeverity

Bases: object

ALLOWED_VALUES = ('fatal', 'error', 'warning')

ERROR = 'error'

FATAL = 'fatal'

WARNING = 'warning'

classmethod validate(value)

Module contents

leapp.reporting package

Module contents

class leapp.reporting.Audience(value=None)

Bases: leapp.reporting.BasePrimitive

Target audience of the report

apply(report)

Add report entry to the report dict based on provided path

name = 'audience'

path

Return report field destination path (nested dict description)

to_dict()

value

Return report field value

class leapp.reporting.BaseListPrimitive(value=None)

Bases: leapp.reporting.BasePrimitive

Report primitive (field) base class for list targets (we append value to a list)

apply(report)

Add report entry to the report dict based on provided path

name = ''

path

Return report field destination path (nested dict description)

to_dict()

value

Return report field value

class leapp.reporting.BasePrimitive(value=None)

Bases: object

Report primitive (field) base class for dict targets (implies unique path)

apply(report)

Add report entry to the report dict based on provided path

name = ''

path

Return report field destination path (nested dict description)

to_dict()

value

Return report field value

class leapp.reporting.BaseRemediation(value=None)

Bases: leapp.reporting.BaseListPrimitive

Remediation base class

apply(report)

Add report entry to the report dict based on provided path

name = 'remediations'

path

to_dict()

value

Return report field value

class leapp.reporting.ExternalLink(url=None, title=None)

Bases: leapp.reporting.BaseListPrimitive

External link report detail field

apply(report)

Add report entry to the report dict based on provided path

name = 'external'

path

to_dict()

value

Return report field value

class leapp.reporting.Flags(*args, **kwargs)

Bases: leapp.reporting.Groups

ACCESSIBILITY = 'accessibility'

AUTHENTICATION = 'authentication'

BOOT = 'boot'

COMMUNICATION = 'communication'

DESKTOP = 'desktop environment'

DRIVERS = 'drivers'

EMAIL = 'email'

ENCRYPTION = 'encryption'

FAILURE = 'failure'

FILESYSTEM = 'filesystem'

FIREWALL = 'firewall'

HIGH_AVAILABILITY = 'high availability'

INHIBITOR = 'inhibitor'

KERNEL = 'kernel'

MONITORING = 'monitoring'

NETWORK = 'network'

OS_FACTS = 'OS facts'

POST = 'post'

PUBLIC_CLOUD = 'public cloud'

PYTHON = 'python'

REPOSITORY = 'repository'

RHUI = 'rhui'

SANITY = 'sanity'

SECURITY = 'security'

SELINUX = 'selinux'

SERVICES = 'services'

TIME_MANAGEMENT = 'time management'

TOOLS = 'tools'

UPGRADE_PROCESS = 'upgrade process'

apply(report)

Add report entry to the report dict based on provided path

name = 'groups'

path

Return report field destination path (nested dict description)

to_dict()

value

Return report field value

class leapp.reporting.Groups(value=None)

Bases: leapp.reporting.BaseListPrimitive

Report groups.

ACCESSIBILITY = 'accessibility'

AUTHENTICATION = 'authentication'

BOOT = 'boot'

COMMUNICATION = 'communication'

DESKTOP = 'desktop environment'

DRIVERS = 'drivers'

EMAIL = 'email'

ENCRYPTION = 'encryption'

FAILURE = 'failure'

FILESYSTEM = 'filesystem'

FIREWALL = 'firewall'

HIGH_AVAILABILITY = 'high availability'

INHIBITOR = 'inhibitor'

KERNEL = 'kernel'

MONITORING = 'monitoring'

NETWORK = 'network'

OS_FACTS = 'OS facts'

POST = 'post'

PUBLIC_CLOUD = 'public cloud'

PYTHON = 'python'

REPOSITORY = 'repository'

RHUI = 'rhui'

SANITY = 'sanity'

SECURITY = 'security'

SELINUX = 'selinux'

SERVICES = 'services'

TIME_MANAGEMENT = 'time management'

TOOLS = 'tools'

UPGRADE_PROCESS = 'upgrade process'

apply(report)

Add report entry to the report dict based on provided path

name = 'groups'

path

Return report field destination path (nested dict description)

to_dict()

value

Return report field value

class leapp.reporting.Key(uuid)

Bases: leapp.reporting.BasePrimitive

Stable identifier for report entries.

apply(report)

Add report entry to the report dict based on provided path

name = 'key'

path

Return report field destination path (nested dict description)

to_dict()

value

Return report field value

class leapp.reporting.RelatedResource(scheme=None, identifier=None)

Bases: leapp.reporting.BaseListPrimitive

Report detail field for related resources (e.g. affected packages/files)

apply(report)

Add report entry to the report dict based on provided path

name = 'related_resources'

path

to_dict()

value

Return report field value

class leapp.reporting.Remediation(playbook=None, commands=None, hint=None)

Bases: object

apply(report)

classmethod from_dict(data)

to_dict()

class leapp.reporting.RemediationCommand(value=None)

Bases: leapp.reporting.BaseRemediation

apply(report)

Add report entry to the report dict based on provided path

name = 'remediations'

path

to_dict()

value

Return report field value

class leapp.reporting.RemediationHint(value=None)

Bases: leapp.reporting.BaseRemediation

apply(report)

Add report entry to the report dict based on provided path

name = 'remediations'

path

to_dict()

value

Return report field value

class leapp.reporting.RemediationPlaybook(value=None)

Bases: leapp.reporting.BaseRemediation

apply(report)

Add report entry to the report dict based on provided path

name = 'remediations'

path

to_dict()

value

Return report field value

class leapp.reporting.Report(init_method='from_initialization', **kwargs)

Bases: leapp.models.Model

Framework model used for reporting

create(data)

Create an instance of this class and use the data to initialize the fields within.

Parameters:data (dict) – Data to initialize the Model from deserialized data Returns:Instance of this class

dump()

Dumps the data in the dictionary form that is safe to serialize to JSON.

Returns:dict with a builtin representation of the data that can be safely serialized to JSON

fields = {'report': <leapp.models.fields.JSON object>}

report = <leapp.models.fields.JSON object>

serialize()

Returns serialized data of the model

topic

alias of ReportTopic

class leapp.reporting.Severity(value=None)

Bases: leapp.reporting.BasePrimitive

Report severity

HIGH = 'high'

INFO = 'info'

LOW = 'low'

MEDIUM = 'medium'

apply(report)

Add report entry to the report dict based on provided path

name = 'severity'

path

Return report field destination path (nested dict description)

to_dict()

value

Return report field value

class leapp.reporting.Summary(value=None)

Bases: leapp.reporting.BasePrimitive

Report summary

apply(report)

Add report entry to the report dict based on provided path

name = 'summary'

path

Return report field destination path (nested dict description)

to_dict()

value

Return report field value

class leapp.reporting.Tags(*args, **kwargs)

Bases: leapp.reporting.Groups

ACCESSIBILITY = 'accessibility'

AUTHENTICATION = 'authentication'

BOOT = 'boot'

COMMUNICATION = 'communication'

DESKTOP = 'desktop environment'

DRIVERS = 'drivers'

EMAIL = 'email'

ENCRYPTION = 'encryption'

FAILURE = 'failure'

FILESYSTEM = 'filesystem'

FIREWALL = 'firewall'

HIGH_AVAILABILITY = 'high availability'

INHIBITOR = 'inhibitor'

KERNEL = 'kernel'

MONITORING = 'monitoring'

NETWORK = 'network'

OS_FACTS = 'OS facts'

POST = 'post'

PUBLIC_CLOUD = 'public cloud'

PYTHON = 'python'

REPOSITORY = 'repository'

RHUI = 'rhui'

SANITY = 'sanity'

SECURITY = 'security'

SELINUX = 'selinux'

SERVICES = 'services'

TIME_MANAGEMENT = 'time management'

TOOLS = 'tools'

UPGRADE_PROCESS = 'upgrade process'

apply(report)

Add report entry to the report dict based on provided path

name = 'groups'

path

Return report field destination path (nested dict description)

to_dict()

value

Return report field value

class leapp.reporting.Title(value=None)

Bases: leapp.reporting.BasePrimitive

Report title

apply(report)

Add report entry to the report dict based on provided path

name = 'title'

path

Return report field destination path (nested dict description)

to_dict()

value

Return report field value

leapp.reporting.create_report(entries)

Produce final report message

leapp.reporting.create_report_from_deprecation(deprecation)

Convert deprecation json to report json

leapp.reporting.create_report_from_error(error_dict)

Convert error json to report json

leapp.repository package

Submodules

leapp.repository.actor_definition module

class leapp.repository.actor_definition.ActorCallContext(definition, logger, messaging, config_model, skip_dialogs)

Bases: object

Wraps the actor execution into child process.

Parameters:

• definition (leapp.repository.actor_definition.ActorDefinition) – Actor definition
• logger (logging.Logger) – Logger
• messaging (leapp.messaging.BaseMessaging) – Leapp Messaging
• config_model (leapp.models.Model derived class) – Workflow provided configuration model

run(*args, **kwargs)

Performs the actor execution in the child process.

class leapp.repository.actor_definition.ActorDefinition(directory, repo_dir, log=None)

Bases: object

Defines actor resources.

Parameters:

• log (logging.Logger) – Logger
• directory (str) – Actor directory
• repo_dir (str) – Repository directory

add(kind, path)

Adds any kind of actor resource to the Definition

Parameters:

• kind (str) – kind of resource added
• path (str) – path to the added resource

apis

Returns:names of APIs used by this actor

class_name

Returns:Actor class name

consumes

Returns:Tuple of consumed models

description

Returns:Actor description

dialogs

Returns:Tuple of defined dialogs

directory

Returns:The folder path of the actor

discover()

Performs introspection through a subprocess.

Returns:Dictionary with discovered items.

files

Returns:Tuple with path to the files folder of the actor, empty tuple if none

full_path

injected_context(*args, **kwds)

Prepares the actor environment for running the actor. This includes injecting actor private libraries into leapp.libraries.actor and setting environment variables for private tools and files.

Note:Use with caution.

libraries

Returns:Tuple with path to the libraries folder of the actor, empty tuple if none

load()

Loads the actor module to be introspectable.

name

Returns:Actor internal name

produces

Returns:Tuple of produced models

serialize()

Returns:dump of actor resources (path, name, tools, files, libraries, tests)

tags

Returns:Tuple of tags assigned to the actor

tests

Returns:Tuple with path to the tests folder of the actor, empty tuple if none

tools

Returns:Tuple with path to the tools folder of the actor, empty tuple if none

leapp.repository.actor_definition.inspect_actor(definition, result_queue)

Retrieves the actor information in a child process and returns the results back through result_queue.

Parameters:

• definition (ActorDefinition) – the actor definition to load
• result_queue (multiprocessing.Queue) – queue to pass results back to the calling process

leapp.repository.definition module

class leapp.repository.definition.DefinitionKind

Bases: object

Represents all known repository resources in Leapp.

ACTOR = <leapp.repository.definition._Kind object>

ACTOR_WHITELIST = (<leapp.repository.definition._Kind object>, <leapp.repository.definition._Kind object>, <leapp.repository.definition._Kind object>, <leapp.repository.definition._Kind object>)

API = <leapp.repository.definition._Kind object>

FILES = <leapp.repository.definition._Kind object>

LIBRARIES = <leapp.repository.definition._Kind object>

MODEL = <leapp.repository.definition._Kind object>

REPO_WHITELIST = (<leapp.repository.definition._Kind object>, <leapp.repository.definition._Kind object>, <leapp.repository.definition._Kind object>, <leapp.repository.definition._Kind object>, <leapp.repository.definition._Kind object>, <leapp.repository.definition._Kind object>, <leapp.repository.definition._Kind object>, <leapp.repository.definition._Kind object>, <leapp.repository.definition._Kind object>)

TAG = <leapp.repository.definition._Kind object>

TESTS = <leapp.repository.definition._Kind object>

TOOLS = <leapp.repository.definition._Kind object>

TOPIC = <leapp.repository.definition._Kind object>

WORKFLOW = <leapp.repository.definition._Kind object>

leapp.repository.manager module

class leapp.repository.manager.RepositoryManager

Bases: object

Handles multiple loaded repositories

actors

Returns:Tuple of leapp.repository.actor_definition.ActorDefinition instances representing actors from all repositories

add_repo(repo)

Add new repository to manager.

Parameters:repo (leapp.repository.Repository) – Repository to be added (registered)

files

Returns:Tuple of paths to “files” folders from all repositories

get_missing_repo_links()

Gather all missing repository ids linked by the added repositories.

Returns:Set of missing repository ids.

libraries

Returns:Tuple of paths to “libraries” folders from all repositories

load(resolve=True, skip_actors_discovery=False)

Load all known repositories.

Parameters:

• resolve (bool) – Whether or not to perform the resolving of model references
• skip_actors_discovery (bool) – specifies whether to skip discovery process of the actors When we testing actors, we’re directly injecting the actor context, so we don’t need to inject it during the repo loading. This option helps to solve this problem.

lookup_actor(name)

Find actor in all loaded repositories

Parameters:name (str) – Name of the actor Returns:None or Actor

lookup_workflow(name)

Find workflow in all loaded repositories

Parameters:name (str) – Name of the workflow Returns:None or Workflow

models

Returns:Tuple of paths to model-defining python modules from all repositories

repo_by_id(repo_id)

Look up a repository by id

Parameters:repo_id – Repository id Returns:Repository or None

repos

Returns:A tuple of all repository instances

serialize()

Returns:List of resources in all known repositories

tags

Returns:Tuple of paths to tag-defining python modules from all repositories

tools

Returns:Tuple of paths to “tools” folders from all repositories

topics

Returns:Tuple of paths to topic-defining python modules from all repositories

workflows

Returns:Tuple of paths to workflow-defining python modules from all repositories

leapp.repository.scan module

leapp.repository.scan.find_and_scan_repositories(path, manager=None, include_locals=False)

Finds and scans all repositories found in the path and it will also resolve linked repositories. Using include_locals=True will additionally include user local repositories to be considered for resolving linked repositories.

Parameters:

• path – Path to scan for repositories
• manager – Optional repository manager to add found repos too
• include_locals – Should repositories linked be searched from the user local registry

Returns:

repository manager instance (either passed through or a new instance if none was passed)

leapp.repository.scan.scan(repository, path)

Scans all repository resources

Parameters:

• repository (leapp.repository.Repository) –
• path (str) – path to the repository

Returns:

instance of leapp.repository.Repository

leapp.repository.scan.scan_actors(repo, path, repo_path)

Scans actors and adds them to the repository.

Parameters:

• repo (leapp.repository.Repository) – Instance of the repository
• path (str) – path to the actors
• repo_path (str) – path to the repository

leapp.repository.scan.scan_apis(repo, path, repo_path)

Scans apis and adds them to the repository.

Parameters:

• repo (leapp.repository.Repository) – Instance of the repository
• path (str) – path to the apis
• repo_path (str) – path to the repository

leapp.repository.scan.scan_files(repo, path, repo_path)

Scans files and adds them to the repository.

Parameters:

• repo (leapp.repository.Repository) – Instance of the repository
• path (str) – path to the files
• repo_path (str) – path to the repository

leapp.repository.scan.scan_libraries(repo, path, repo_path)

Scans libraries and adds them to the repository.

Parameters:

• repo (leapp.repository.Repository) – Instance of the repository
• path (str) – path to the libraries
• repo_path (str) – path to the repository

leapp.repository.scan.scan_models(repo, path, repo_path)

Scans models and adds them to the repository.

Parameters:

• repo (leapp.repository.Repository) – Instance of the repository
• path (str) – path to the models
• repo_path (str) – path to the repository

leapp.repository.scan.scan_repo(path)

Scans all related repository resources

Parameters:path (str) – Returns:repository

leapp.repository.scan.scan_tags(repo, path, repo_path)

Scans tags and adds them to the repository.

Parameters:

• repo (leapp.repository.Repository) – Instance of the repository
• path (str) – path to the tags
• repo_path (str) – path to the repository

leapp.repository.scan.scan_tests(repo, path, repo_path)

Scans tests and adds them to the repository.

Parameters:

• repo (leapp.repository.Repository) – Instance of the repository
• path (str) – path to the tests
• repo_path (str) – path to the repository

leapp.repository.scan.scan_tools(repo, path, repo_path)

Scans tools and adds them to the repository.

Parameters:

• repo (leapp.repository.Repository) – Instance of the repository
• path (str) – path to the tools
• repo_path (str) – path to the repository

leapp.repository.scan.scan_topics(repo, path, repo_path)

Scans topics and adds them to the repository.

Parameters:

• repo (leapp.repository.Repository) – Instance of the repository
• path (str) – path to the topics
• repo_path (str) – path to the repository

leapp.repository.scan.scan_workflows(repo, path, repo_path)

Scans workflows and adds them to the repository.

Parameters:

• repo (leapp.repository.Repository) – Instance of the repository
• path (str) – path to the workflows
• repo_path (str) – path to the repository

Module contents

class leapp.repository.Repository(directory)

Bases: object

The Repository class represents a place where all resources (actors, models, tags, etc.) are defined. See the Repository Directory Layout.

Parameters:directory (str) – Path to the repository folder

actors

Returns:Tuple of actors in the repository

add(kind, item)

Adds any supported kind of a resource to the repository

Parameters:

• kind (leapp.repository.definition.DefinitionKind) – specific kind of the repository resource
• item (leapp.repository.actor_definition.ActorDefiniton or str) – Item that will be added

apis

Returns:Tuple of apis in the repository

files

Returns:Tuple of files in the repository

libraries

Returns:Tuple of libraries in the repository

load(resolve=True, stage=None, skip_actors_discovery=False)

Loads the repository resources

Parameters:

• resolve (bool) – Decides whether or not to perform the resolving of model references
• stage (_LoadStage value) – Stage to load - Required for repository managers

lookup_actor(name)

Finds an actor in the repository

Parameters:name (str) – Name of the actor Returns:None or Actor

static lookup_workflow(name)

Finds a workflow in the repository

Parameters:name (str) – Name of the workflow class name, Workflow.name, or Workflow.short_name Returns:None or Workflow

models

Returns:Tuple of models in the repository

relative_paths(paths)

Returns:Tuple of repository relative paths

repo_dir

repo_id

repo_links

serialize()

Returns:Dictionary of all repository resources

tags

Returns:Tuple of tags in the repository

tools

Returns:Tuple of tools in the repository

topics

Returns:Tuple of topics in the repository

workflows

Returns:Tuple of workflows in the repository

leapp.tags package

Module contents

class leapp.tags.DisabledTag

Bases: leapp.tags.Tag

actors = ()

name = 'disabled'

serialize()

class leapp.tags.ExperimentalTag

Bases: leapp.tags.Tag

actors = ()

name = 'experimental'

serialize()

class leapp.tags.Tag

Bases: leapp.tags.with_meta_base_object_TagMeta

Tag is the base class for all Tags. Tags are used as filtering mechanism for actors to be loaded during workflow executions. Phases do use tags to filter actors according to their tags.

Special Tag class attributes:

Tag here refers to the derived class of Tag

Tag.Common:

Dynamically created class type that designates actors to be executed in the main stage during workflow phases. Using common includes this actor in any workflow, which means the that any workflow tag filter will be ignored if this tag matches.

Tag.Before:

Dynamically created class type that designates actors to be executed in the before stage during workflow phases.

Tag.Before.Common:

Dynamically created class type that designates actors to be executed in the before stage during workflow phases. Using common includes this actor in any workflow, which means the that any workflow tag filter will be ignored if this tag matches.

Tag.After:

Dynamically created class type that designates actors to be executed in the after stage during workflow phases.

Tag.After.Common:

Dynamically created class type that designates actors to be executed in the after stage during workflow phases. Using common includes this actor in any workflow, which means the that any workflow tag filter will be ignored if this tag matches.

actors = ()

Tuple of all registered actors using this tag

name = None

Name of the Tag in snake case

classmethod serialize()

class leapp.tags.TagMeta

Bases: type

Meta class for the registration of tags

This meta class adds dynamically Common, Before, Before.Common, After and After.Common attributes to the tag class. For more information see leapp.tags.Tag

leapp.tags.get_tags()

Returns:All registered leapp.tags.Tag derived classes

leapp.topics package

Module contents

class leapp.topics.DialogTopic

Bases: leapp.topics.Topic

A special topic for dialogs shown to user during workflow execution.

messages = (<class 'leapp.models.DialogModel'>,)

name = 'dialog_topic'

serialize()

class leapp.topics.ErrorTopic

Bases: leapp.topics.Topic

A special topic for errors during the execution.

messages = (<class 'leapp.models.ErrorModel'>,)

name = 'errors'

serialize()

class leapp.topics.ReportTopic

Bases: leapp.topics.Topic

A special topic for reporting purposes.

messages = (<class 'leapp.reporting.Report'>,)

name = 'report_topic'

serialize()

class leapp.topics.Topic

Bases: leapp.topics.with_meta_base_object_TopicMeta

Base class for all topics

messages = ()

Tuple of leapp.models.Model derived classes that are using this topic are automatically added to this variable.

name = None

Name of the topic in snake case

classmethod serialize()

class leapp.topics.TopicMeta

Bases: type

Meta class for the registration of topics

leapp.topics.get_topics()

Returns:All registered leapp.topics.Topic derived classes

leapp.utils package

Subpackages

leapp.utils.audit package

Module contents

class leapp.utils.audit.Audit(event=None, stamp=None, message=None, data=None, actor=None, phase=None, hostname=None, context=None)

Bases: leapp.utils.audit.DataSource

Parameters:

• event (str) – Type of this event e.g. new-message or log-message but can be anything
• stamp (str) – Timestamp string of the event creation in iso format
• message (leapp.utils.audit.Message or None) – A message object, if this audit entry represents a message otherwise None
• data (str or None) – If message is None this has to be the data for the audit entry
• actor (str) – Name of the actor that triggered the entry
• phase (str) – In which phase of the workflow execution the data entry was created
• context (str) – The execution context
• hostname (str) – Hostname of the system that produced the entry

audit_id

data_source_id

Returns the id of the entry, which is only set when already stored. :return: Integer id or None

do_store(connection)

host_id

Returns the id of the entry, which is only set when already stored. :return: Integer id or None

store(db=None)

Stores the data within a transaction :param db: Database object (optional) :return: None

class leapp.utils.audit.DataSource(actor=None, phase=None, context=None, hostname=None)

Bases: leapp.utils.audit.Host

Parameters:

• actor (str) – Name of the actor that triggered the entry
• phase (str) – In which phase of the workflow execution the data entry was created
• context (str) – The execution context
• hostname (str) – Hostname of the system that produced the entry

data_source_id

Returns the id of the entry, which is only set when already stored. :return: Integer id or None

do_store(connection)

host_id

Returns the id of the entry, which is only set when already stored. :return: Integer id or None

store(db=None)

Stores the data within a transaction :param db: Database object (optional) :return: None

class leapp.utils.audit.Execution(context=None, kind=None, configuration=None, stamp=None)

Bases: leapp.utils.audit.Storable

Stores information about the current execution

Parameters:

• context (str) – Execution context identifier
• kind (str) – Execution kind - Can be any string and used for filtering
• configuration (str, dict, list or tuple) –
• stamp (str) – Timestamp string of the execution start in iso format

do_store(connection)

execution_id

Returns the id of the entry, which is only set when already stored. :return: Integer id or None

store(db=None)

Stores the data within a transaction :param db: Database object (optional) :return: None

class leapp.utils.audit.Host(context=None, hostname=None)

Bases: leapp.utils.audit.Storable

Host information

do_store(connection)

host_id

Returns the id of the entry, which is only set when already stored. :return: Integer id or None

store(db=None)

Stores the data within a transaction :param db: Database object (optional) :return: None

class leapp.utils.audit.Message(stamp=None, msg_type=None, topic=None, data=None, actor=None, phase=None, hostname=None, context=None)

Bases: leapp.utils.audit.DataSource

Parameters:

• stamp (str) – Timestamp string of the message creation in iso format
• msg_type (str) – Name of the model that represents the message payload
• topic (str) – Topic for this message
• data (leapp.utils.audit.MessageData) – Payload data
• actor (str) – Name of the actor that triggered the entry
• phase (str) – In which phase of the workflow execution the message was created
• context (str) – The execution context
• hostname (str) – Hostname of the system that produced the message

data_source_id

Returns the id of the entry, which is only set when already stored. :return: Integer id or None

do_store(connection)

host_id

Returns the id of the entry, which is only set when already stored. :return: Integer id or None

message_id

Returns the id of the entry, which is only set when already stored. :return: Integer id or None

store(db=None)

Stores the data within a transaction :param db: Database object (optional) :return: None

class leapp.utils.audit.MessageData(data=None, hash_id=None)

Bases: leapp.utils.audit.Storable

Message data

Parameters:

• data (str) – Message payload
• hash_id (str) – SHA256 hash in hexadecimal representation of data

do_store(connection)

store(db=None)

Stores the data within a transaction :param db: Database object (optional) :return: None

class leapp.utils.audit.Storable

Bases: object

Base class for database storables

do_store(connection)

Performs the actual storing of the data

Parameters:connection – Database connection to use (Can be a transaction cursor) Returns:None

store(db=None)

Stores the data within a transaction :param db: Database object (optional) :return: None

leapp.utils.audit.checkpoint(actor, phase, context, hostname)

Creates a checkpoint audit entry

Parameters:

• actor (str) – Name of the actor that triggered the entry
• phase (str) – In which phase of the workflow execution the data entry was created
• context (str) – The execution context
• hostname (str) – Hostname of the system that produced the entry

Returns:

None

leapp.utils.audit.create_audit_entry(event, data, message=None)

Create an audit entry

Parameters:

• event – Event type identifier
• data – Data related to Type of the event, e.g. a command and its arguments
• message – An optional message.

Returns:

leapp.utils.audit.create_connection(path)

Creates a database connection to the path and ensures it’s initialized and up to date.

Parameters:path – Path to the database Returns:Connection object

leapp.utils.audit.dict_factory(cursor, row)

leapp.utils.audit.get_audit_entry(event, context)

Retrieve audit entries stored in the database for the given context

Parameters:

• context (str) – The execution context
• event (str) – Event type identifier

Returns:

list of dicts with id, time stamp, actor and phase fields

leapp.utils.audit.get_checkpoints(context)

Retrieve all checkpoints stored in the database for the given context

Parameters:context (str) – The execution context Returns:list of dicts with id, timestamp, actor and phase fields

leapp.utils.audit.get_connection(db)

Get the database connection or passes it through if it is already set

Parameters:db – Database connection to be passed through in case it exists already Returns:database object initialized and migrated to the latest schema version

leapp.utils.audit.get_errors(context)

Queries all error messages from the database for the given context

Parameters:context (str) – The execution context Returns:List of error messages

leapp.utils.audit.get_messages(names, context, connection=None)

Queries all messages from the database for the given context and the list of model names :param names: List of names that should be messages returned for :type names: list or tuple of str :param context: Execution id the message should be queried from. :param connection: Database connection to use instead of the default one. :return: Iterable with messages :rtype: iterable

Submodules

leapp.utils.actorapi module

exception leapp.utils.actorapi.RequestException(*args, **kwargs)

Bases: exceptions.IOError

Exceptions that are raised through the actor API, which is retrieved from get_actor_api()

args

errno

exception errno

filename

exception filename

message

strerror

exception strerror

leapp.utils.actorapi.get_actor_api()

Returns:An instance of the Leapp actor API session that is using requests.Session over a UNIX domain socket

leapp.utils.clicmd module

class leapp.utils.clicmd.Command(name, target=None, help='', description=None)

Bases: object

Command implements a convenient command-based argument parsing the framework.

Parameters:

• name (str) – Name of the sub command
• target (Callable) – Function called when the command is invoked
• help (str) – Shows a help message
• description (str) – Extended description of the command (the default is help)

add_argument(name, value_type=None, help='', wrapped=None)

Parameters:

• name –
• value_type –
• help –
• wrapped –

Returns:

add_option(name, short_name='', help='', is_flag=False, inherit=False, value_type=<type 'str'>, wrapped=None, action=None, metavar=None, choices=None, default=None)

Add an option

Parameters:

• name (str) – Name of the option
• short_name (str) – Short name of the option (one letter)
• help (str) – Help string for the option
• is_flag (bool) – Decides if it is a flag
• inherit (bool) – Decides if this option should be inherited by sub commands
• value_type – Type of the value by default string
• wrapped (Callable) – Function that is wrapped (aka the target)
• action (str) – ArgumentParser actions to take (e.g. store)
• metavar (str) – Changes the display name of arguments in generated help messages. It has no influence on the attribute name from the generated arguments namespace.
• choices (str) – range of values that the argument is allowed to take
• choices – default value of the argument if nothing is specified

Returns:

self

add_sub(cmd)

Adds a sub command to this command

Parameters:cmd (leapp.utils.clicmd.Command) – The sub command object Returns:self

apply_parser(sparser, parent=None, parser=None)

Parameters:

• sparser (_SubParserActionOverride) – ArgumentParser.add_subparsers
• parent (_Command) – Instance of _Command
• parser (argparse.ArgumentParser) – ArgumentParser instance usually received from sparser.add_parser

Returns:

None

called(args)

The actual call is dispatched through this method. It ensures that the parent is also called to allow generic handling of some flags (especially inherited flags).

Parameters:args (argparse.Namespace) – Arguments object that is a result of the argparse commandline parser Returns:None

execute(version)

Entry point to the command execution. It is used for the main entry function of an application.

Parameters:version (str) – Version string to display for –version calls Returns:None

get_inheritable_options()

Returns:Returns all options that are marked as ‘inherit’

leapp.utils.clicmd.command(name, help='', description=None, parent=None)

Decorator to mark a function as a sub command

Parameters:

• name (str) – Sub command name
• help (str) – Help string for the sub command
• description (str) – Extended description for the sub command defaults to help if it is not set
• parent (Command) – Instance to the parent command if it is a sub-command

leapp.utils.clicmd.command_arg(name, value_type=None, help='')

Decorator wrapping functions to add command line arguments to the sub command to be invoked

Parameters:

• name – Name of the argument
• value_type – Type of the argument
• help – Help string for the argument

leapp.utils.clicmd.command_aware_wraps(f)

Decorator passing the command attribute of the wrapped function to the wrapper

This needs to be used by decorators that are trying to wrap clicmd decorated command functions.

leapp.utils.clicmd.command_opt(name, **kwargs)

Decorator wrapping functions to add command line options to the sub command to be invoked

Parameters:

• name – Name of the option
• kwargs – parameters as specified in leapp.utils.clicmd.Command.add_option()

leapp.utils.meta module

leapp.utils.meta.get_flattened_subclasses(cls)

Returns all the given subclasses and their subclasses recursively for the given class :param cls: Class to check :type cls: Type :return: Flattened list of subclasses and their subclasses

leapp.utils.meta.with_metaclass(meta_class, base_class=<type 'object'>)

Parameters:

• meta_class – The desired metaclass to use
• base_class (Type) – The desired base class to use, the default one is object

Returns:

Metaclass type to inherit from

Example:

class MyMetaClass(type):
    def __new__(mcs, name, bases, attrs):
        klass = super(MyMetaClass, mcs).__new__(mcs, name, bases, attrs)
        klass.added = "Added field"
        return klass

class MyClass(with_metaclass(MyMetaClass)):
    pass

# This is equivalent to python 2:
class MyClass(object):
    __metaclass__ = MyMetaClass

# Or python 3
class MyClass(object, metaclass=MyMetaClass):
    pass

leapp.utils.repository module

leapp.utils.repository.add_repository_link(path, repo_id)

Add a link from another repository to the current repository.

Parameters:

• path – Path within the leapp repository to modify
• repo_id – UUIDv4 string identifier for the repository to link
• repo_id – str

Returns:

None

leapp.utils.repository.find_repos(path)

Finds repositories within the given path.

Parameters:path – Path to search for repositories. Returns:List of strings with found repository paths.

leapp.utils.repository.find_repository_basedir(path)

Tries to find the .leapp directory recursively ascending until it hits the root directory

Parameters:path – Path to start from (can be relative) Returns:None if the base directory was not found, otherwise the absolute path to the base directory

leapp.utils.repository.get_global_repositories_data()

Returns the data of all system wide available repositories.

Returns:Repository information

leapp.utils.repository.get_repository_id(path)

Retrieves the repository name from the repository metadata from within the given path. (it can be anywhere within the repository it will use find_repository_dir() to find the repository base directory) :param path: Path within the leapp repository :return: ID of the repository :raises: KeyError if no name was found (e.g. not a valid repository path)

leapp.utils.repository.get_repository_links(path)

Retrieves a list of repository ids that are linked to given repository.

Parameters:path – Path within the leapp repository Returns:List of repository ids this repository is linked to

leapp.utils.repository.get_repository_metadata(path)

Gets the parsed metadata file as a dictionary

Parameters:path – Path to start loading the metadata from (it can be anywhere within the repository it will use find_repository_dir() to find the repository base directory) Returns:Dictionary with the metadata or an empty dictionary

leapp.utils.repository.get_repository_name(path)

Retrieves the repository name from the repository metadata from within the given path. (it can be anywhere within the repository it will use find_repositoryt_dir() to find the repository base directory) :param path: Path within the leapp repository :return: Name of the repository :raises: KeyError if no name was found (e.g. not a valid repository path)

leapp.utils.repository.get_user_config_path()

Returns the path to the user configuration directory and creates it if it does not exist already.

Returns:Path to the configuration directory of leapp.

leapp.utils.repository.get_user_config_repo_data()

Returns the user config repository data.

Returns:Data for user configurations.

leapp.utils.repository.get_user_config_repos()

Returns the path to the user config file for repositories.

Returns:Path to the repos.json configuration file.

leapp.utils.repository.make_class_name(name)

Converts a snake_case_name to an UpperCaseName

Parameters:name – Name to convert Returns:Converted class name

leapp.utils.repository.make_name(name)

Converts a given name to a lower snake case

Parameters:name – Name to convert Returns:Lower snake case

leapp.utils.repository.requires_repository(f)

Decorator for snactor commands that require to be run in a repository directory.

leapp.utils.repository.to_snake_case(name)

Converts an UpperCaseName to a snake_case_name

Parameters:name – Name to convert Returns:converted snake case name

Module contents

leapp.utils.get_api_models(actor, what)

Used to retrieve the full list of models including the ones defined by WorkflowAPIs used by the actor.

Parameters:

• what (str) – A string which either is ‘consumes’ or ‘produces’
• actor (Actor or ActorDefinition) – Actor type/instance or ActorDefinition instance to retrieve the information from

Returns:

Tuple of all produced or consumed models as specified by actor and APIs used by the actor.

leapp.utils.reboot_system()

leapp.workflows package

Submodules

leapp.workflows.flags module

class leapp.workflows.flags.Flags(request_restart_after_phase=False, restart_after_phase=False, is_checkpoint=False)

Bases: object

is_checkpoint = False

request_restart_after_phase = False

restart_after_phase = False

serialize()

Returns:Serialized data of phase flags

leapp.workflows.phaseactors module

class leapp.workflows.phaseactors.PhaseActors(actors, stage)

Bases: object

actors

consumes

initial

produces

leapp.workflows.phases module

class leapp.workflows.phases.Phase

Bases: leapp.workflows.phases.with_meta_base_object_PhaseMeta

filter = None

flags = <leapp.workflows.flags.Flags object>

classmethod get_index()

name = None

policies = <leapp.workflows.policies.Policies object>

classmethod serialize()

Returns:Dictionary with the serialized representation of the phase

class leapp.workflows.phases.PhaseMeta

Bases: type

classes = [<class 'leapp.workflows.phases.with_meta_base_object_PhaseMeta'>, <class 'leapp.workflows.phases.Phase'>, <class 'leapp.workflows._ConfigPhase'>]

leapp.workflows.policies module

class leapp.workflows.policies.Policies(error=<class 'leapp.workflows.policies.ReportOnly'>, retry=<class 'leapp.workflows.policies.Disabled'>)

Bases: object

class Errors

Bases: object

class FailImmediately

Bases: object

class FailPhase

Bases: object

class ReportOnly

Bases: object

class Retry

Bases: object

class Actor

Bases: object

class Disabled

Bases: object

class Phase

Bases: object

serialize()

leapp.workflows.tagfilters module

class leapp.workflows.tagfilters.TagFilter(phase, *tags)

Bases: object

get()

get_after()

get_before()

serialize()

Returns:Serialized data for tag filter

Module contents

class leapp.workflows.Workflow(logger=None, auto_reboot=False)

Bases: leapp.workflows.with_meta_base_object_WorkflowMeta

Workflow is the base class for all workflow definitions.

Parameters:logger (Instance of logging.Logger) – Optional logger to be used instead of leapp.workflow

answer_store

: return: AnswerStore instance used for messaging

configuration = None

Model to be used as workflow configuration

consumes

All consumed messages

description = ''

Documentation for the workflow

dialogs

All encountered dialogs

errors

Returns:All reported errors

experimental_whitelist

Whitelist of actors that may be executed even that they are marked experimental

failure

initial

Initial messages required

is_valid_phase(phase=None)

load_answers(answerfile_path, userchoices_path)

name = None

Name of the workflow

phase_actors

Return all actors for the phase

phases = ()

produces

All produced messages

run(context=None, until_phase=None, until_actor=None, skip_phases_until=None, skip_dialogs=False, only_with_tags=None)

Executes the workflow

Parameters:

• context (str) – Custom execution ID to be used instead of a randomly generated UUIDv4
• until_phase (str) – Specify until including which phase the execution should run - phase.stage can be used to control it even more granularly. phase is any phase name where stage refers to main, before or after. If no stage is defined, after is assumed to be the default value. The execution ends when this phase (and stage, if specified) has been executed.
• until_actor (str) – The execution finishes when this actor has been executed.
• skip_phases_until (str or None) – Skips all phases until including the phase specified, and then continues the execution.
• skip_dialogs (bool) – Inform actors about the mode of dialogs processing. If skip_dialogs is set to True it means that dialogs can’t be processed in the current workflow run interactively and every attempted call of get_answers api method will be non-blocking, returning an empty dict if no user choice was found in answerfile or a selected option otherwise. If skip_dialogs is set to False then in case of absent recorded answer the dialog will be rendered in a blocking user input requiring way. The value of skip_dialogs will be passed to the actors that can theoretically use it for their purposes.
• only_with_tags (List[str]) – Executes only actors with the given tag, any other actor is going to get skipped.

save_answers(answerfile_path, userchoices_path)

Generates an answer file for the dialogs of the workflow and saves it to answerfile_path. Updates a .userchoices file at userchoices_path with new answers encountered in answerfile.

Parameters:

• answerfile_path – The path where to store the answer file.
• userchoices_path – The path where to store the .userchoices file.

Returns:

None

classmethod serialize()

Returns:Serialized form of the workflow

short_name = None

Short name of the workflow

tag = None

Workflow Tag

whitelist_experimental_actor(actor)

Adds an actor to the experimental whitelist and allows them to be executed.

Parameters:actor (class derived from py:class:leapp.actors.Actor) – Actor to be whitelisted Returns:None

class leapp.workflows.WorkflowMeta

Bases: type

Meta class for the registration of workflows

leapp.workflows.actor_names(actor=None)

leapp.workflows.contains_tag(needle_tags, actor_tags)

leapp.workflows.get_workflows()

Returns:all registered workflows

leapp.workflows.phase_names(phase=None)

leapp.workflows.tag_names(tag=None)

Submodules

leapp.compat module

leapp.compat.unicode_type

alias of unicode

leapp.compat.raise_with_traceback(exc, tb)

This is a helper function to raise exceptions with a traceback.

This is function is required to workaround the syntax changes between Python 2 and 3 Python 3.4 introduced a with_traceback method to Exception classes and Python 3 removed the syntax which used to be used in Python 2.

Parameters:

• exc – Exception to raise
• tb – Traceback to use

Returns:

Nothing

leapp.config module

class leapp.config.BetterConfigParser(defaults=None, dict_type=<class 'collections.OrderedDict'>, allow_no_value=False)

Bases: ConfigParser.ConfigParser

OPTCRE = <_sre.SRE_Pattern object>

OPTCRE_NV = <_sre.SRE_Pattern object>

SECTCRE = <_sre.SRE_Pattern object>

add_section(section)

Create a new section in the configuration.

Raise DuplicateSectionError if a section by the specified name already exists. Raise ValueError if name is DEFAULT or any of it’s case-insensitive variants.

defaults()

get(section, *args, **kwargs)

getboolean(section, option)

getfloat(section, option)

getint(section, option)

has_option(section, option)

Check for the existence of a given option in a given section.

has_section(section)

Indicate whether the named section is present in the configuration.

The DEFAULT section is not acknowledged.

items(section, raw=False, vars=None)

Return a list of tuples with (name, value) for each option in the section.

All % interpolations are expanded in the return values, based on the defaults passed into the constructor, unless the optional argument `raw’ is true. Additional substitutions may be provided using the `vars’ argument, which must be a dictionary whose contents overrides any pre-existing defaults.

The section DEFAULT is special.

options(section)

Return a list of option names for the given section name.

optionxform(optionstr)

read(filenames)

Read and parse a filename or a list of filenames.

Files that cannot be opened are silently ignored; this is designed so that you can specify a list of potential configuration file locations (e.g. current directory, user’s home directory, systemwide directory), and all existing configuration files in the list will be read. A single filename may also be given.

Return list of successfully read files.

readfp(fp, filename=None)

Like read() but the argument must be a file-like object.

The `fp’ argument must have a `readline’ method. Optional second argument is the `filename’, which if not given, is taken from fp.name. If fp has no `name’ attribute, `<???>’ is used.

remove_option(section, option)

Remove an option.

remove_section(section)

Remove a file section.

sections()

Return a list of section names, excluding [DEFAULT]

set(section, option, value=None)

Set an option.

write(fp)

Write an .ini-format representation of the configuration state.

leapp.config.get_config()

leapp.exceptions module

exception leapp.exceptions.ActorDiscoveryExecutionError(message)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.ActorInspectionFailedError(message)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.CannotConsumeErrorMessages

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.CommandDefinitionError(message)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.CommandError(message)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.CyclingDependenciesError(message)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.InvalidTagDefinitionError(message)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.InvalidTopicDefinitionError(message)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.InvalidTopicItemError(message)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.LeappError(message)

Bases: exceptions.Exception

args

message

exception leapp.exceptions.LeappRuntimeError(message, exception_info=None)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.MissingActorAttributeError(message)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.ModelDefinitionError(message)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.ModuleNameAlreadyExistsError(message)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.MultipleActorsError(path)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.MultipleConfigActorsError(config_actors)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.RepoItemPathDoesNotExistError(kind, rel_path, full_path)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.RepositoryConfigurationError(message)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.RequestStopAfterPhase

Bases: leapp.exceptions.LeappError

This exception is used to gracefully stop the current actor and request the stop of the workflow execution after the current phase.

args

message

exception leapp.exceptions.StopActorExecution

Bases: exceptions.Exception

This exception is used to gracefully stop execution of actor, but allows the workflow to continue.

args

message

exception leapp.exceptions.StopActorExecutionError(message, severity='error', details=None)

Bases: leapp.exceptions.LeappError

This exception is used to gracefully stop execution of actor and it will call leapp.actors.Actor.report_error().

Parameters:

• message (str) – A message to print the possible error
• severity (str with defined values from leapp.messaging.errors.ErrorSeverity.ERROR) – Severity of the error default leapp.messaging.errors.ErrorSeverity.ERROR
• details (dict) – A dictionary where additional context information is passed along with the error

class ErrorSeverity

Bases: object

ALLOWED_VALUES = ('fatal', 'error', 'warning')

ERROR = 'error'

FATAL = 'fatal'

WARNING = 'warning'

classmethod validate(value)

args

message

exception leapp.exceptions.TagFilterUsageError(message)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.UnknownCommandError(command)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.UnsupportedDefinitionKindError(message)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.UsageError(message)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.WorkflowConfigNotAvailable(actor)

Bases: leapp.exceptions.LeappError

args

message

exception leapp.exceptions.WrongAttributeTypeError(message)

Bases: leapp.exceptions.LeappError

args

message

leapp.snactor.fixture module

class leapp.snactor.fixture.ActorContext(actor=None)

Bases: object

ActorContext is a helper class for testing actors. It helps to eliminate the boilerplate for executing actors. It provides a set of methods that allow specifying input messages for the actor, executing the actor and to retrieve messages sent by the actor.

apis = ()

consume(*models)

Retrieve messages produced by the actor execution and specified in the actors produces attribute, and filter message types by models.

Parameters:models (Variable number of the derived classes from leapp.models.Model) – Models to use as a filter for the messages to return Returns:

feed(*models)

Feed the messaging model with messages to be available to consume.

Parameters:models (Variable number of instances of classes derived from leapp.models.Model) – Data in form of model instances to be available for the actor to consume. Returns:None

messages()

Returns raw messages produced by the actor.

Returns:list of raw message data dictionaries.

run(config_model=None)

Execute the current actor.

Parameters:config_model (Config model instance derived from leapp.models.Model) – Config model for the actor to consume. Returns:None

set_actor(actor)

Internally used method to set the current actor specification object to setup the current actor for the test function.

Parameters:actor – ActorSpecification instance to use. Returns:None

leapp.snactor.fixture.current_actor_context(*args, **kwargs)

This fixture will prepare an environment for the actor the test belongs to, to be safely executable.

current_actor_context Is an instance of leapp.snactor.fixture.ActorContext and gives access to its methods for feeding an actor with input data, running the actor, and retrieving messages produced by the actor during its execution.

Example:

from leapp.snactor.fixture import current_actor_context
from leapp.models import ConsumedExampleModel, ProducedExampleModel

def test_actor_lib_some_function(current_actor_context):
    # Feed with messages to be consumable by the actor that is going to be executed.
    current_actor_context.feed(ConsumedExampleModel(value='Some random data'))

    # Execute the actor
    current_actor_context.run()

    # Ensure that at least one message is produced
    assert current_actor_context.consume(ProducedExampleModel)

    # Ensure the value is what we expect
    assert current_actor_context.consume(ProducedExampleModel)[0].value == 42

leapp.snactor.fixture.current_actor_libraries(*args, **kwargs)

This fixture will make libraries that are private to the actor only available only for the scope of the test function that uses this fixture.

Example:

from leapp.snactor.fixture import current_actor_libraries

def test_actor_lib_some_function(current_actor_libraries):
    from leapp.libraries.actor import private
    assert private.some_function(1) == 42

leapp.snactor.fixture.leapp_forked(*args, **kwargs)

leapp.snactor.fixture.loaded_leapp_repository(*args, **kwargs)

This fixture will ensure that the repository for the current test run is loaded with all its links etc.

This enables running actors and using models, tags, topics, workflows etc.

Additionally loaded_leapp_repository gives you access to a leapp.repository.manager.RepositoryManager instance.

Example:

from leapp.snactor.fixture import loaded_leapp_repository
from leapp.models import ExampleModel, ProcessedExampleModel

def my_repository_library_test(loaded_leapp_repository):
    from leapp.libraries.common import global
    e = ExampleModel(value='Some string')
    result = global.process_function(e)
    assert type(result) is ProcessedExampleModel

leapp.snactor.fixture.pytest_pyfunc_call(pyfuncitem)

This function is a hook for pytest implementing the ability to run the actors in tests safely.

It will call leapp.snactor.fixture._execute_test() in a child process if the current test uses the current_actor_context() fixture. If it doesn’t use the current_actor_context() fixture, it will default to the default pytest_pyfunc_call implementation.

Module contents