From c68f9e2b087d6ac1ca98329086336d13bad40cc5 Mon Sep 17 00:00:00 2001 From: Alex Kavanagh <alex.kavanagh@canonical.com> Date: Thu, 9 Dec 2021 11:55:56 +0000 Subject: [PATCH] Add the charmhub-migration spec This spec proposes a new, better, but alternative means for maintaining and releasing OpenStack charms in a world with a new charm store (charmhub.io) which provides richer mechanisms for charm delivery. Change-Id: If5719ad54c52d46dc001da4df91caa4b00af256f --- doc/source/index.rst | 1 + specs/yoga/implemented/charmhub-migration.rst | 733 ++++++++++++++++++ 2 files changed, 734 insertions(+) create mode 100644 specs/yoga/implemented/charmhub-migration.rst diff --git a/doc/source/index.rst b/doc/source/index.rst index d2d2829..36f9ce5 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -14,6 +14,7 @@ Here you can find the specs, and spec template, for each release: specs/antelope/index specs/zed/index specs/yoga/index + specs/yoga/implemented/* specs/xena/index specs/wallaby/index specs/victoria/index diff --git a/specs/yoga/implemented/charmhub-migration.rst b/specs/yoga/implemented/charmhub-migration.rst new file mode 100644 index 0000000..c9f343e --- /dev/null +++ b/specs/yoga/implemented/charmhub-migration.rst @@ -0,0 +1,733 @@ +.. + Copyright 2021, Canonical UK + + This work is licensed under a Creative Commons Attribution 3.0 + Unported License. + http://creativecommons.org/licenses/by/3.0/legalcode + +.. + This template should be in ReSTructured text. Please do not delete + any of the sections in this template. If you have nothing to say + for a whole section, just write: "None". For help with syntax, see + http://sphinx-doc.org/rest.html To test out your formatting, see + http://www.tele3.cz/jbar/rest/rest.html + +================================ +Charmstore to Charmhub Migration +================================ + +This spec proposes a new, better, but alternative means for maintaining and +releasing OpenStack charms in a world with a new charm store (charmhub.io) +which provides richer mechanisms for charm delivery. + +A new charm store, charmhub, has been introduced that shares features and +capabilities with snaps. The legacy charm store, jaas.ai, is planned to be +retired and all projects delivering charms should migrate to charmhub. One of +the compelling features of charmhub is the ability to offer multiple tracks for +releases. + +The charms themselves are written to support multiple releases of OpenStack +from pre-Mitaka to Xena. This is convenient and straightforward for having +development working only on one branch at a time for a charm, but requires that +code and dependencies work towards the lowest common supported denominator. +This is problematic as these charms depend on python libraries that are moving +forward and dropping support for legacy python versions such as Python 2, 3.5 +and beyond, which causes maintenance challenges for current charms. + +Problem Description +=================== + +Currently, OpenStack Charms are delivered in two charmstore namespaces - +``openstack-charmers`` and ``openstack-charmers-next``. The use of two +namespaces predates the existence of channels within the jaas.ai charm store. +These namespaces are equivalent to the stable and edge channels respectively. + +The git repositories hosting the source code for the OpenStack charms generally +live on opendev.org git servers. The repositories contain branches of each +stable charm release and an unstable development branch (master). A post-merge +job in the CI system publishes an updated version of the charm to the jaas.ai +charm store after any change is merged. Changes landing in the master +development branch are published in the openstack-charmers-next namespace and +changes landing in the most recent stable branch are published in the +openstack-charmers namespace. + +One of the challenges with the current approach is that the two branches must +focus on the lowest common denominator for dependencies; meaning that most +python libraries leveraged and included in the charms must be limited to +versions that support older python releases that are still included on older +OpenStack releases. As the world of Python library development moves on, +support for older python versions are being deprecated and removed causing a +variety of maintenance headaches for the current charms. + +Another challenge with this approach is that each new feature introduced adds +potential risk to older install bases. In order to mitigate this, the gate +testing of the OpenStack Charms runs a broad sampling of tests across a +multitude of OpenStack and base distro releases. New contributors are often +running into challenges with the CI system and needing to debug older releases +even when their patches are not relevant to these older versions. + +One of the positive sides to this arrangement is that development primarily +only needs to consider 2 branches at a time - the master development branch +and the most recent stable branch. All patches are primarily targeted at the +master development branch and relevant bug fixes are backported to a single +release. + +Proposed Change +=============== + +OpenStack charms will adopt a model of release and delivery which takes +advantage of the various delivery channels offered by charmhub which closely +matches channels in the `Snap store`_. Note, that not all charms will support +all releases/versions of a specific service and charms will only adopt branches +and tracks for those releases which are relevant to the service itself. + +Tracks +------ + +Tracks will be set up to track git branches in the upstream repositories and +will result in a track per release of the charm payload (e.g. the software +service it delivers). When the primary software payload of the charm uses +well-known release names, the release name will be used in lieu of version +numbers for the track name. When the primary software payload of the charm does +not use well-known release names, the version of the software will be used. + +For example, the nova-compute charm delivers nova hypervisor services and the +charm tracks will correspond to the OpenStack release of the nova services +installed (e.g. ussuri, victoria, etc.). + +As another example, the OVN Central charm delivers OVN as the payload and OVN +uses release names/versions of YY.MM. In this scenario, the OVN charm Central +charm will provide a track corresponding to the YY.MM naming schema (e.g. +21.03, 21.09, etc). + +Risk Levels +----------- + +The charmhub store also has the ability to allow the developers and users to +provide and consume different risk levels. This feature was also available in +the legacy charmstore, however it was not leveraged by the release process +until very recently. There are 4 risk levels available within any track: edge, +beta, candidate, and stable. The CI system will automatically publish a new +charm for any change which lands in the corresponding git branch to the edge +risk level. + +Charms will be promoted through the edge -> beta -> candidate -> stable risk +levels by a human actor, not through an automated script. Tooling may be +developed to assist in this process, but automatic promotion is out of scope at +this time. Future enhancements may include triggering a promotion of a build +upon the introduction of git tags, but that is considered out of scope for this +work. + +Branches +-------- + +The charmhub store also provides for a further subdivision of tracks and risk +levels into branches, which are meant to provide temporary, short-lived, +releases for bug fixing purposes. There are various possibilities to consider +regarding how branch flows should be worked into the development flow of +OpenStack charms. + +For the purposes of this work, branches are explicitly out of scope and should +be considered separately. + +Git Branches +------------ + +Git branches will need to be created for the various tracks that are delivered +in Charmhub. The relationship between git branches and charmhub tracks are +one-to-many, meaning that a change landing in one branch may result in a charm +being published into multiple tracks. This is useful when the tracks provide +logically different targets, while the branch serves the same functionality. +For example, the latest master branch of development will logically point to +the latest OpenStack release as well as the latest/edge release. + +Additionally, the charms in each of the git branches and tracks are intended to +support the targeted release N and the previous release N-1 for upgrade +purposes. + +OpenStack Charms +---------------- + +For those charms specifically delivering OpenStack services a new git branch +will be created for each OpenStack release the charm supports. This branch will +be named in the form of ``stable/<openstack-release>`` for all stable versions +of OpenStack. Branches will only be created back to the Queens release of +OpenStack. Each of these branches will be created from the tip of the current +stable charms release (stable/21.10) as a base. These branches will differ from +each other regarding the primary version of software that they are targeting as +well as the default configuration values for the track they are deployed from. + ++-----------------+-------------------+ +| Git Branch | Charmhub Track(s) | ++=================+===================+ +| stable/queens | train | +| stable/rocky | | +| stable/stein | | +| stable/train | | ++-----------------+-------------------+ +| stable/ussuri | ussuri | ++-----------------+-------------------+ +| stable/victoria | victoria | ++-----------------+-------------------+ +| stable/wallaby | wallaby | ++-----------------+-------------------+ +| stable/xena | xena | ++-----------------+-------------------+ +| stable/yoga | yoga | ++-----------------+-------------------+ +| master | zed, latest/edge | ++-----------------+-------------------+ + +Ceph Charms +----------- + +Currently, Ceph updates are delivered through the Ubuntu Cloud Archive which +ties the ceph charms to OpenStack versions, causing some challenges in defining +tracks per Ceph release. In scenarios where services are co-located (e.g. +nova-compute and ceph-osd), updating the source repository for the ceph-osd +charm will have unintended consequences on the nova-compute services and their +versions. Rather than introduce complex pinning or a new package archive, the +charms used to deliver Ceph will track the OpenStack release names in addition +to the Ceph release names, up to the Octopus release. This is expected to make +the migration easier for users. + +The development git branches will be created corresponding to the appropriate +Ceph version where the product was delivered. In order to make it easier for +users to migrate, Ceph charms may have a single branch delivered to multiple +Charmhub tracks. + ++-----------------+-------------------------+ +| Git Branch | Charmhub Track(s) | ++=================+=========================+ +| stable/luminous | luminous, queens, rocky | ++-----------------+-------------------------+ +| stable/mimic | mimic, stein | ++-----------------+-------------------------+ +| stable/nautilus | nautilus, train | ++-----------------+-------------------------+ +| stable/octopus | octopus | ++-----------------+-------------------------+ +| stable/pacific | pacific | ++-----------------+-------------------------+ +| master | quincy, latest/edge | ++-----------------+-------------------------+ + +On the newer branches, a mapping can be made to safely convert a Ceph release +name into a cloud-archive pocket in a way that doesn’t unintentionally upgrade +either one, with the assumption that a valid mapping between OpenStack and Ceph +versions is clearly documented. What that mapping should look like: + ++-------------------+---------------------------------------------+ +| Ceph Release | OpenStack Release | ++===================+=============================================+ +| Luminous | Queens | ++-------------------+---------------------------------------------+ +| Mimic | Stein | ++-------------------+---------------------------------------------+ +| Nautilus | Train | ++-------------------+---------------------------------------------+ +| Octopus | Ussuri | ++-------------------+---------------------------------------------+ +| Pacific | Wallaby (mid-cycle Openstack, more support) | ++-------------------+---------------------------------------------+ +| Unstable / Quincy | Yoga | ++-------------------+---------------------------------------------+ + +The above works because we can include distro (bionic-queens) with Rocky +safely, and we can include Pacific’s focal-wallaby with Xena safely, as the +Xena packages have higher versions. + +The Ceph charms include the following charms: + +* ceph-fs +* ceph-iscsi +* ceph-mon +* ceph-osd +* ceph-proxy +* ceph-radosgw +* ceph-rbd-mirror +* ceph-dashboard + +Provider-specific Subordinate Charms +------------------------------------ + +Some services such as Cinder and Neutron allow for provider specific +subordinate charms to be used in order to allow for a specific SDN, storage +plugin, etc. These provider-specific subordinate charms are considered +supporting charms rather than OpenStack specific charms, however they are often +enabling specific functionality for a specific storage backend. As such, they +will follow the OpenStack Charms track schema. + +These charms will include the following subordinate charms, and as of this +writing include: + +* cinder-ceph +* cinder-lvm +* cinder-netapp +* cinder-purestorage +* neutron-openvswitch +* neutron-api-plugin-arista +* neutron-api-plugin-ovn +* keystone-saml-mellon + +OVN Charms +---------- + +OVN is a networking SDN which is used as the primary SDN for Charmed OpenStack +deployments. OVN has been provided through the Ubuntu Cloud Archive as well as +the Ubuntu archives. The OVN charms are a bit of a mix when it comes to +providing source versions. Standalone charms such as OVN Dedicated Chassis and +OVN Central have source options which allow for configuring the archive to use +for package installation. As a subordinate charm, OVN Chassis will adopt the +version that is installed alongside the principal charm. + +As OVN is a general SDN and not implemented specifically for OpenStack, the +tracks in charmhub should closely follow the versions of the software delivered +rather than the OpenStack release. This may be a bit confusing at first for +users migrating, but this should be addressed with clear documentation as well +as tooling to help the end-users select the appropriate track to upgrade their +charms to. Additionally, tracks will be put in place labelled +openstack-{release-name} to ease the migration burden. + +OVN charms will have the following sets of branches and tracks: + ++--------------+----------------------------+ +| Branch | Track(s) | ++==============+============================+ +| stable/20.03 | 20.03, ussuri, victoria | ++--------------+----------------------------+ +| stable/20.12 | 20.12, wallaby | ++--------------+----------------------------+ +| stable/21.09 | 21.09, xena | ++--------------+----------------------------+ +| stable/22.03 | 22.03 | ++--------------+----------------------------+ +| master | latest/edge | ++--------------+----------------------------+ + +Supporting Charms +----------------- + +In the OpenStack ecosystem, there are other core services which are required to +produce an OpenStack cloud. These services include messaging, database, and +certificate management services. While they are critical to the functionality +of an OpenStack cloud, they are not tied to specific releases of OpenStack and +these charms should result in tracks which are appropriate for their specific +behavior. + +These charms consist of the following services, and will result in tracks that +are based on the versions of software provided. These services are generally +delivered from the Ubuntu archives (not the Ubuntu Cloud Archive) and will +track the versions of their versions in Ubuntu. + ++----------------------+---------------+-------------------------+ +| Charm | Branch | Track(s) | ++======================+===============+=========================+ +| hacluster | stable/bionic | 1.1.x | ++----------------------+---------------+-------------------------+ +| hacluster | stable/focal | 2.0.3, latest/stable | ++----------------------+---------------+-------------------------+ +| hacluster | master | 2.0.5, latest/edge | ++----------------------+---------------+-------------------------+ +| pacemaker-remote | stable/bionic | 1.1.x | ++----------------------+---------------+-------------------------+ +| pacemaker-remote | stable/focal | 2.0.3, latest/stable | ++----------------------+---------------+-------------------------+ +| pacemaker-remote | master | 2.0.5, latest/edge | ++----------------------+---------------+-------------------------+ +| rabbitmq-server | stable/bionic | 3.6 | ++----------------------+---------------+-------------------------+ +| rabbitmq-server | stable/focal | 3.8, latest/stable | ++----------------------+---------------+-------------------------+ +| rabbitmq-server | master | 3.9, latest/edge | ++----------------------+---------------+-------------------------+ +| vault | stable/1.5 | 1.5 | ++----------------------+---------------+-------------------------+ +| vault | stable/1.6 | 1.6 | ++----------------------+---------------+-------------------------+ +| vault | stable/1.7 | 1.7 | ++----------------------+---------------+-------------------------+ +| vault | master | latest/edge | ++----------------------+---------------+-------------------------+ +| percona-cluster | stable/bionic | 5.7, latest/stable | ++----------------------+---------------+-------------------------+ +| percona-cluster | master | latest/edge | ++----------------------+---------------+-------------------------+ +| mysql-innodb-cluster | stable/jammy | 8.0, latest/stable | ++----------------------+---------------+-------------------------+ +| mysql-innodb-cluster | master | latest/edge | ++----------------------+---------------+-------------------------+ + +Trilio Charms +------------- + +Trilio Charms will also need to be migrated to the charmhub delivery mechanism. +While implemented for OpenStack, the Trilio software is versioned separately +from the OpenStack releases and should use Trilio specific tracks. + ++-----------------------+------------+----------------------+ +| trilio-data-mover | stable/4.0 | 4.0 | ++=======================+============+======================+ +| trilio-data-mover | stable/4.1 | 4.1, latest/stable | ++-----------------------+------------+----------------------+ +| trilio-data-mover | master | 4.2 (?), latest/edge | ++-----------------------+------------+----------------------+ +| trilio-dm-api | stable/4.0 | 4.0 | ++-----------------------+------------+----------------------+ +| trilio-dm-api | stable/4.1 | 4.1, latest/stable | ++-----------------------+------------+----------------------+ +| trilio-dm-api | master | 4.2 (?), latest/edge | ++-----------------------+------------+----------------------+ +| trilio-horizon-plugin | stable/4.0 | 4.0 | ++-----------------------+------------+----------------------+ +| trilio-horizon-plugin | stable/4.1 | 4.1, latest/stable | ++-----------------------+------------+----------------------+ +| trilio-horizon-plugin | master | 4.2 (?), latest/edge | ++-----------------------+------------+----------------------+ +| trilio-wlm | stable/4.0 | 4.0 | ++-----------------------+------------+----------------------+ +| trilio-wlm | stable/4.1 | 4.1, latest/stable | ++-----------------------+------------+----------------------+ +| trilio-wlm | master | 4.2 (?), latest/edge | ++-----------------------+------------+----------------------+ + + +Installation Sources +-------------------- + +Most charms in the OpenStack charm collection provide a config option that +allows the user to set the archive or repository that debian packages/snaps are +installed from. This is generally provided via either the ``openstack-origin`` +or ``source`` charm config option, which defaults to install using the local +installations default distribution repositories. + +In order to ensure that the tracks are installing the appropriate versions of +packages, the default value for this config option will be set to the +appropriate installation source for the track. If a track spans multiple distro +releases (e.g. the cloud-archive at the LTS release crossover), the charm will +need to be able to determine whether this should be sourced from the +cloud-archive or the distribution, whichever is appropriate. + +Primarily, this will affect the tracks that use the Ussuri Ubuntu Cloud Archive +and the Yoga Ubuntu Cloud Archive as an installation source. + +Latest Track +------------ + +The ‘latest’ track in charmhub is its own track, and like other tracks - charms +can be pushed to the track and the various risk levels within the track. While +it is a full blown track, to end users it is a means to get the "latest" +version of a piece of software. However, the risk level must be taken into +consideration for which version should be installed. + +The latest track will be used into two ways: + +* The ``latest/stable`` track will be used to park the stable 21.10 charms + release until (at least) the 22.10 release. +* At 22.10, the ``latest/stable`` track will follow the current stable release, + and for 22.10 this will be the ``zed`` track. + +The reasoning is that the 21.10 charms' metadata advertise focal support, +against ``all`` architecturees and so it difficult to replace until a jammy, +and greater, charm can be released to the channel. However, the zed charms will +only support jammy and kinetic, and so they can be released alongside the +existing 21.10 charms. Then existing 21.10 charms won't upgrade as the new +charms will not support focal. Therefore, it is safe to re-use the +``latest/stable`` track at that point. + +Risks +----- + +The purpose of each risk level for each track is as follows: + +Edge +~~~~ + +For each charm, the latest/edge channel will always map the current master +development branch. + +Currently, for stable branches, the edge track will not be used, as the +Launchpad builders will push directly to the stable track. This is in keeping +with the existing policy of requiring 2 x +2 reviews on gerrit for stable +branches. At some future point, automation/policy will be introduced to enable +pushing to edge to start with and then having testing/policy to promote to more +stable risks. + +Beta +~~~~ + +The beta risk level will be used for milestone releases during the charm +development cycle. + +Candidate +~~~~~~~~~ + +The ``<track>/candidate`` channel will be used to track the next software +release as it enters the final weeks of development and stabilizes in +preparation for a release. Note that this is on the next stable release track. + +Stable +~~~~~~ + +The stable track will track the stable, tested, released version of the charm. + +Technology Preview Charms +------------------------- + +Charms are initially released in the "Technology Preview" state. Charms which +are in the technology preview state should not be promoted into the stable +channel until it is determined it is stable for the corresponding payload +release. Once it has graduated, all releases which have qualified as graduated +from technology preview will then be promoted to the stable channel. It is +possible that not all tracks of a charm will have a stable risk level. Charms +which are still in technology preview will only promote those charms to the +beta channel until it is determined that the charm has reached maturity, at +which point it can be promoted to the candidate and/or stable channels as +appropriate. + +For example, a charm such as ovn-chassis may be delivered as technology preview +in the Stein release track but may not reach maturity until the Train release +track. The Train release track of the charm will be promoted to the stable +channel but the Stein release track will only have the charm in the beta +channel. + +Special consideration is needed for those charms considered Technology Preview +at the time of this transition. Technology Preview charms are not promulgated +in the charm store and are available under the ``openstack-charmers`` +namespace. Charmhub does not have namespaces in the same way and these charms +exist but are prefixed with the ‘openstack-charmers’ name, e.g. +openstack-charmers-ironic-api. These charms will be created in charmhub and +promoted to the beta channel. + +Continuous Integration +---------------------- + +Charmhub will deliver charms that are built for specific architectures. +OpenStack charms to date, are built using python modules on amd64 architectures +and distributed as such. This is generally acceptable as they are primarily +source charms which have deliveries, but can pose some interesting challenges +for non-amd64 architectures with native extensions. + +The OpenStack CI system is currently composed of a Zuul-CI system which is used +to trigger a variety of pep8 (pycodestyle), unit and functional tests for each +patch proposed to a charm. Post-merge hooks are registered which trigger a +Jenkins task to publish the charm to the legacy charm store. + +Publishing Charms +----------------- + +For newer charms, delivered through charmhub, they are expected to have +platform specific charms so that native extensions or tooling required by the +charm can be built for the appropriate target architecture. Launchpad now +offers charm recipes, which can be used to build and publish charms in a +similar manner to snap recipes. Changes merged into a repository will trigger a +build of the charm on all supported architectures and publish the resulting +charm to Charmhub. Leveraging this service removes a step from the OpenStack +Charmers maintained CI system for publishing charms to the charm store and +results in completely eliminating the need for jenkins in the CI +infrastructure. + +The Launchpad charm recipes require that the git trees are known and stored in +Launchpad itself. In order to continue to use the upstream OpenStack gerrit +review system, which is standard for OpenStack services and has been standard +for the charms for years, each charm project in Launchpad will be configured to +mirror the source from the upstream opendev git system. + +A charm recipe will be created for each branch of each charm such that upon a +patch being merged in the upstream gerrit repository, the Launchpad service +will update the mirrored (a.k.a. imported) repository, detect that new changes +are present, initiate a charm build and publish these results to the +appropriate channel(s) in charmhub. + +Note that charm recipes in Launchpad will only build by default once per hour. +While there is normally a slight delay in publishing charms to the charm store, +there is one publish per change to the charm. In the Launchpad build model, +multiple changes can be queued up and included in a single build unless builds +are explicitly requested within the hour time window. This is considered +acceptable for charm development and publishing purposes. + +Testing Charms +-------------- + +With the introduction of branches and tracks per OpenStack release, the need to +test a multitude of OpenStack releases for each change is no longer required. +Instead, targeted tests for the branch will be run. For example, a change +proposed to the Keystone charm in the stable/xena branch will run only tests +relevant to the Xena release of OpenStack. + +Reducing the releases that are running in the gate allows for the charm to be +focused on the specific payload release and need not worry about spurious +failures on older releases. + +The branches may also include tests that exercise the N-1 version of the +release software, in order to validate that the upgrade of N-1 to N are +functioning properly. + +Charmcraft +---------- + +In order to leverage the charm recipes in Launchpad, all of the charms will +need to be built using the charmcraft tool. Charmcraft, however, was designed +to build and publish operator framework charms, which excludes the majority of +the charms in the OpenStack charms collection. + +Charmcraft has recently grown parity with the snapcraft toolset, which allows a +variety of plugins to be used instead of the default charm plugin. The ‘dump’ +plugin is appropriate to use for classic charms, however it is unsuitable for +reactive charms. In the long term, a new plugin should be created for building +reactive charms using charmcraft. In the short term, the various steps for +producing a charm can be overridden using the ‘override-build’, +‘override-stage’, and ‘override-prime’ steps which will use the existing tox +infrastructure to build, stage, and prime the reactive charm. + +Documentation +------------- + +Documentation for the OpenStack Charms and Ceph Charms are generally available +as a solution level, with charm README files serving as the primary source of +documentation within the Charm store and the published OpenStack Charm Guide, +OpenStack Charms Deployment Guide and Ubuntu Ceph Documentation providing +solution level documentation. + +The release notes need to clearly identify the change to use and leverage +Charmhub as well as referencing migration documentation. Migration +documentation needs to be clearly outlined to make it simple for users to +migrate from using the Charm Store to Charmhub. + +Developer documentation should also be provided in the OpenStack Charm Guide to +provide details for developers wishing to contribute new features, bug fixes, +etc. + + +Alternatives +------------ + +There aren't really any good alternatives. When the charmstore to charmhub +cut-over is implemented, the existing ``charm`` command will stop working, +which means that the entire build and publishing system in Jenkins will simply +stop working too. So the only real alternative is to re-work the Jenkins +software to use charmcraft and then use that to push to the charm store. + +This means continuing to maintain Jenkins and not take advantage of the +Launchpad builders to build architecture aware charms. + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + Alex Kavanagh + +Gerrit Topic +------------ + +N/A + +Work Items +---------- + +Produce 'charmcraft' recipes for classic and reactive charms +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Launchpad can be used to build charms when branches in git repositories are +changed. These builds are controlled by recipes. Part of the work will be to +ensure that these recipes work for all of the charms. They will be centrally +controlled and 'synced' into charms as part of the development cycle. + +* Write classic recipe +* Write reactive charm recipe using charm-tools ``build`` command. +* Add recipes to ``release-tools`` repository. + +Produce helper tools/library to manage launchpad projects/recipes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In order to manage the charms, recipes and mapping of git repository branches +to charmhub tracks/channels, a suite of tools and configuration will be +produced to manage the complexity of the number of charms, branches and tracks. + +Essential functionality will be: + +* display gap between config and actual in git repositories and launchpad + recipes. +* Show desired tracks vs actual tracks for charms (in charmhub) to generate + 'requests' for new tracks, or for tracks to be deleted. +* update existing recipes against the configuration. +* Create new recipes as required against the config. + + +Repositories +------------ + +Initially, the existing ``release-tools`` repository will be the collecting +point for the tools, but it has become a bit of a dumping ground for related +tools and needs a major re-organistion. In the fullness of time, these tools +may be broken out into their own repository where changes to the config +automatically updates the recipes. + +Additionaly, the manage the recipes a separate ``charmhub-lp-tools`` tool will +be created to automate recipe creation, updates, deletes and changed, and to +authorize uploads to the charmhub. + +.. code-block:: bash + + https://github.com/openstack-charmers/release-tools + https://github.com/openstack-charmers/charmhub-lp-tools + +Documentation +------------- + +Documentation for the tools, configuration schemas and how the system works +will be inside the repository in the form of Markdown files. This is to ensure +that the documentation 'lives' with the code. Where possible documentations +for the schemas (config files) will be in the config files themselves. + +Security +-------- + +No additional security concerns. + +The user that runs the tool must have a launchpad account in the +``openstack-charmers`` team, and a charmhub account in the +``openstack-charmers`` team. + +Testing +------- + +The possibility to break the charmstore and charmhub is pretty high, so the +tools and process will be developed incrementally against 'test' charms and low +priority 'real' charms. + +Stage 1 +~~~~~~~ + +Initially copies of the existing charms will be registered into the charmhub +(under new names), to test the recipes and ability to build charms. This is to +verify that the charm build recipes work for classic and reactive charms. + +Stage 2 +~~~~~~~ + +Selecting two charms (notionally, ``neutron-dynamic-routing`` and +``manila-generic``), break the link to the charmstore and ensure that tracks, +branches and recipes can be created such that updates to the charms git +repositories result in built charms in the correct tracks. + +This will allow the correct changes to the charms to be enumerated and captured +in the release-tools repository. + +Dependencies +============ + +None + +References +---------- + +* `Charmhub unofficial docs <https://github.com/canonical/charmhub-unofficial-docs>`__ +* `Snapcraft channels <https://snapcraft.io/docs/channels>`__ +* `Mirroring repositories from other sites <https://help.launchpad.net/Code/Git#Mirroring_repositories_from_other_sites>`__ + +.. targets +.. _`Snap store`: https://snapcraft.io/docs/channels