deb-murano/doc/source/admin/appdev-guide/step-by-step/part4.rst

Part 4: Refactoring code to use the Application Framework

Up until this point we wrote the Plone application in a manner that was common to all applications that were written before the application framework was introduced.

In this last tutorial step we are going to refactor the Plone code in order to take advantage of the framework.

Application framework was written in order to simplify the application development and encapsulate common deployment workflows. This gives things primitives for application scaling and high availability without the need to develop them over and over again for each application.

When using the frameworks, an application developer only has to inherit the class that best suits him and provide it only with the code that is specific to the application, while leaving the rest to the framework. This typically includes:

• instructions on how to provision the software on each node (server)
• instructions on how to configure the provisioned software
• server group onto which the software should be installed. This may be a fixed server list, a shared server pool, or a scalable server group that creates servers using the given instance template, or one of the several other implementations provided by the framework

The framework is located in a separate library package io.murano.applications that is shipped with Murano. We are going to use the apps namespace prefix to refer to this namespace through the code.

Step 1: Add dependency on the App Framework

In order to use one Murano Package from another, the former must be explicitly specified as a requirement for the latter. This is done by filling the Require section in the package's manifest file.

Open the Plone's manifest.yaml file and append the following lines:

Require:
  io.murano.applications:

Requirements are specified as a mapping from package name to the desired version of that package (or version range). The missing value indicates the dependency on the latest 0.*.* version of the package which is exactly what we need since the current version of the app framework library is 0.

Step 2: Get rid of the instance

Since we are going to have a multi-sever Plone application there won't be a single instance belonging to the application. Instead, we are going to provide it with the server group that abstracts the server management from the application.

So instead of

Properties:
  instance:
    Contract: $.class(res:Instance)

we are going to have

Properties:
  servers:
    Contract: $.class(apps:ServerGroup).notNull()

Step 3: Change the base classes

Another change that we are going to make to the main application class is to change its base classes. Regular applications inherit from the std:Application which only has the method deploy that does all the work.

Application framework provides us with its own implementation of that class and method. Instead of one monolithic method that does everything, with the framework, the application provides only the code needed to provision and configure the software on each server.

So instead of std:Application class we are going to inherit two of the framework classes:

Extends:
  - apps:MultiServerApplicationWithScaling
  - apps:OpenStackSecurityConfigurable

The first class tells us that we are going to have an application that runs on multiple servers. In the following section we are going to split out deploy method into two smaller methods that are going to be invoked by the framework to install the software on each of the servers. By inheriting the apps:MultiServerApplicationWithScaling, the application automatically gets all the UI buttons to scale it out and in.

The second class is a mix-in class that tells the framework that we are going to provide the OpenStack-specific security group configuration for the application.

Step 4: Split the deployment logic

In this step we are going to split the installation into two phases: provisioning and configuration.

Provisioning is implemented by overriding the onInstallServer method, which is called every time a new server is added to the server group. In this method we are going to install the Plone software bits onto the server (which is provided as a method parameter).

Configuration is done through the onConfigureServer, which is called upon the first installation on the server, and every time any of the application settings change, and onCompleteConfiguration which is executed on each server after everything was configured so that we can perform post-configuration steps like starting application daemons and reporting messages to the user.

Thus we are going to split the install-plone.sh script into two scripts: installPlone.sh and configureServer.sh and execute each one in their corresponding methods:

onInstallServer:
  Arguments:
    - server:
        Contract: $.class(res:Instance).notNull()
    - serverGroup:
        Contract: $.class(apps:ServerGroup).notNull()
  Body:
    - $file: sys:Resources.string('installPlone.sh').replace({
          "$1" => $this.deploymentPath,
          "$2" => $this.adminPassword
        })
    - conf:Linux.runCommand($server.agent, $file)

onConfigureServer:
  Arguments:
    - server:
        Contract: $.class(res:Instance).notNull()
    - serverGroup:
        Contract: $.class(apps:ServerGroup).notNull()
  Body:
    - $primaryServer: $serverGroup.getServers().first()
    - If: $server = $primaryServer
      Then:
        - $file: sys:Resources.string('configureServer.sh').replace({
              "$1" => $this.deploymentPath,
              "$2" => $primaryServer.ipAddresses[0]
            })
      Else:
        - $file: sys:Resources.string('configureClient.sh').replace({
            "$1" => $this.deploymentPath,
            "$2" => $this.servers.primaryServer.ipAddresses[0],
            "$3" => $this.listeningPort})
    - conf:Linux.runCommand($server.agent, $file)


  onCompleteConfiguration:
    Arguments:
      - servers:
          Contract:
            - $.class(res:Instance).notNull()
      - serverGroup:
          Contract: $.class(apps:ServerGroup).notNull()
      - failedServers:
          Contract:
            - $.class(res:Instance).notNull()
    Body:
      - $startCommand: format('{0}/zeocluster/bin/plonectl start', $this.deploymentPath)
      - $primaryServer: $serverGroup.getServers().first()
      - If: $primaryServer in $servers
        Then:
          - $this.report('Starting DB node')
          - conf:Linux.runCommand($primaryServer.agent, $startCommand)
          - conf:Linux.runCommand($primaryServer.agent, 'sleep 10')

      - $otherServers: $servers.where($ != $primaryServer)
      - If: $otherServers.any()
        Then:
          - $this.report('Starting Client nodes')
          # run command on all other nodes in parallel with pselect
          - $otherServers.pselect(conf:Linux.runCommand($.agent, $startCommand))

      # build an address string with IPs of all our servers
      - $addresses: $serverGroup.getServers().
          select(
            switch($.assignFloatingIp => $.floatingIpAddress,
                   true => $.ipAddresses[0])
            + ':' + str($this.listeningPort)
          ).join(', ')
      - $this.report('Plone listeners are running at ' + str($addresses))

During configuration phase we distinguish the first server in the server group from the rest of the servers. The first server is going to be the primary node and treated differently from the others.

Step 5: Configuring OpenStack security group

The last change to the main class is to set up the security group rules. We are going to do this by overriding the getSecurityRules method that we inherited from the apps:OpenStackSecurityConfigurable class:

getSecurityRules:
  Body:
    - Return:
        - FromPort: $this.listeningPort
          ToPort: $this.listeningPort
          IpProtocol: tcp
          External: true
        - FromPort: 8100
          ToPort: 8100
          IpProtocol: tcp
          External: false

The code is very similar to that of the old deploy method with the only difference being that it returns the rules rather than sets them on its own.

Step 6: Provide the server group instance

Do you remember, that previously we replaced the instance property with servers of type apps:ServerGroup? Since the object is coming from the UI definition, we must change the latter in order to provide the class with the apps:ServerReplicationGroup instance rather than resources:Instance.

To do this we are going to replace the instance property in the Application template with the following snippet:

servers:
  ?:
    type: io.murano.applications.ServerReplicationGroup
  numItems: $.ploneConfiguration.numNodes
  provider:
    ?:
      type: io.murano.applications.TemplateServerProvider
    template:
      ?:
        type: io.murano.resources.LinuxMuranoInstance
      flavor: $.instanceConfiguration.flavor
      image: $.instanceConfiguration.osImage
      assignFloatingIp: $.instanceConfiguration.assignFloatingIP
    serverNamePattern: $.instanceConfiguration.unitNamingPattern

If you take a closer look at the code above you will find out that the new declaration is very similar to the old one. But now instead of providing the Instance property values directly, we are providing them as a template for the TemplateServerProvider server provider. ServerReplicationGroup is going to use the provider each time it requires another server. In turn, the provider is going to use the familiar template for the new instances.

Besides the instance template we also specify the initial number of Plone nodes using the numItems property and the name pattern for the servers. Thus we must also add it to the list of our controls:

Forms:
  - instanceConfiguration:
      fields:
        ...
        - name: unitNamingPattern
          type: string
          label: Instance Naming Pattern
          required: false
          maxLength: 64
          initial: 'plone-{0}'
          description: >-
            Specify a string, that will be used in instance hostname.
            Just A-Z, a-z, 0-9, dash and underline are allowed.

  - ploneConfiguration:
      fields:
        ...
        - name: numNodes
          type: integer
          label: Initial number of Client Nodes
          initial: 1
          minValue: 1
          required: true
          description: >-
            Select the initial number of Plone Client Nodes

Step 6: Using server group composition

By this step we should already have a working Plone application. But let's go one step further and enhance our sample application.

Since we are running the database on the first server group server only, we might want it to have different properties. For example we might want to give it a bigger flavor or just a special name. This is a perfect opportunity for us to demonstrate how to construct complex server groups. All we need to do is to just use another implementation of apps:ServerGroup. Instead of apps:ServerReplicationGroup we are going to use the apps:CompositeServerGroup class, which allows us to compose several server groups together. One of them is going to be a single-server server group consisting of our primary server, and the second is going to be the scalable server group that we used to create in the previous step.

So again, we change the Application section of our UI definition file with even a more advanced servers property definition:

servers:
  ?:
    type: io.murano.applications.CompositeServerGroup
  serverGroups:
    - ?:
        type: io.murano.applications.SingleServerGroup
      server:
        ?:
          type: io.murano.resources.LinuxMuranoInstance
        name: format($.instanceConfiguration.unitNamingPattern, 'db')
        image: $.instanceConfiguration.image
        flavor: $.instanceConfiguration.flavor
        assignFloatingIp: $.instanceConfiguration.assignFloatingIp
    - ?:
        type: io.murano.applications.ServerReplicationGroup
      numItems: $.ploneConfiguration.numNodes
      provider:
        ?:
          type: io.murano.applications.TemplateServerProvider
        template:
          ?:
            type: io.murano.resources.LinuxMuranoInstance
          flavor: $.instanceConfiguration.flavor
          image: $.instanceConfiguration.osImage
          assignFloatingIp: $.instanceConfiguration.assignFloatingIP
        serverNamePattern: $.instanceConfiguration.unitNamingPattern

Here the instance definition for the SingleServerGroup (our primary server) differs from the servers in the ServerReplicationGroup by its name only. However the same technique might be used to customize other properties as well as to create even more sophisticated server group topologies. For example, we could implement region bursting by composing several scalable server groups that allocate servers in different regions. And all of that without making any changes to the application code itself!