diff --git a/specs/9.0/upload-deployment-facts-to-configdb.rst b/specs/9.0/upload-deployment-facts-to-configdb.rst new file mode 100644 index 00000000..8ee7954a --- /dev/null +++ b/specs/9.0/upload-deployment-facts-to-configdb.rst @@ -0,0 +1,505 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +========================================================== +Upload serialized deployment facts to ConfigDB service API +========================================================== + +https://blueprints.launchpad.net/fuel/+spec/upload-deployment-facts-to-configdb + +There are multiple levels of hierarchy in Hiera used by deployment tasks on +nodes. Some of those data exist only in YaML files on a node and can't be +accessed by 3rd party components. + +With configuration database service, we can store serialized deployment data +for later use. It requires though that we can upload the data from nodes to +the service as a part of deployment process. + +We propose to develop a deployment task to +upload the facts to external HTTP-based API +and add it to a plugin that enables +3rd-party LCM application with Fuel [1]_. + +-------------------- +Problem description +-------------------- + +The store for serialized deployment information (e.g. ConfigDB API +extension in Nailgun [2]_) allows 3rd party applications to access +it. It also allows alternative deployment/lifecycle management +solutions to synchronize their configrations with Fuel installer. + +However, it doesn't solve the problem of getting the information +into the service, since the extension itself is a more or less +passive store. + +The solution is needed to perform actual upload of required information +into the configuration database. It also must keep the data up to date +by synchronizing them upon every change applied to the environment. + +Synchronization required in the following cases: + +#. Deployment settings changed in Nailgun via UI/CLI/API. + In this case, Nailgun DB will have the latest changes, and Nailgun API + will respond with properly updated serialized deployment data [2]_. + This data can be imported into ConfigDB directly by requesting + the Nailgun API and sending result to ConfigDB API. They are + accessible by Hiera as ``astute`` data source [3]_. + +#. Deployment data changed due to changes made to the node (e.g. hardware + updated, versions of packages updated, etc) outside the Fuel context. + These changes are reflected in the serialized data generated and stored + on the node itself, in YaML files: + + * ``/etc/hiera/globals.yaml`` - global configurations calculated by + deployment task ``globals``. + + * ``/etc/hiera/override/plugins.yaml`` - plugin-specific overrides + of parameters defined in data sources on higher levels (i.e. + ``astute`` and ``globals``). + + * ``/etc/hiera/override/configuration.yaml`` - specific overrides + for OpenStack configuration parameters which are not exposed + by Nailgun directly. + + * ``/etc/hiera/override/.yaml`` - node-level configurations + that override the basic parameters from other sources. + +#. Deployment data changed in 3rd party deployment/lifecycle management + application (e.g. in Puppet Master's top-level manifests or in External + Nodes Classifier application for the Puppet Master). Here we need + to import data from the 3rd party application. This case is out of + scope of the current proposal. + +In this specification, we will focus on the use cases #1 and #2. + +---------------- +Proposed changes +---------------- + +The current deployment process implemented in Fuel installer assumes +that all actual deployment data is available to Puppet agent on a target +node locally as a set of YaML files in ``/etc/hiera`` directory. + +By the time Puppet agent starts to execute actual deployment tasks, +all the configuration settings must be up to date. It means that we +can import the actual set of the deployment configuration data from +those files as a part of deployment process. + +The following changes are proposed in scope of this specification: + +* Create a deployment task that uploads the serialized + deployment data from files in ``/etc/hiera`` of a target node to + the corresponding resources in ConfigDB API endpoint (e.g. + ``/environment/<:id>/node/<:id>/resource/<:datasource>/values``). + See Orchestration_ section for details. + +* Integrate the a task into the deployment graph using plugins + mechanisms of Fuel. The task must run in the end of the deployment + process to make sure that all the tasks that impact + the deployment settings files are already executed, including: + + * ``globals`` + + * ``override_configuration`` + + * any plugin task that updates/overrides basic deployment settings + +* Implement a pass of auth information for ConfigDB API + extension of Nailgun API to the deployment task in question + in a secure way. + +**Deployment task details** + +The deployment task shall have the following ID: + +:: + + upload_data_to_configdb + + +The task shall depend on the following tasks: + +:: + + globals + upload_configuration + pre_deployment_end + +The task shall run at the Fuel Master node. Auth credentials for the +ConfigDB API shall be made available for it upon LCM plugin installation +from the following file: + +:: + + /var/www/nailgun/plugins//environment_config.yaml + +That file will be created by the plugin builder and will contain metadata +of the LCM application, including auth credentials in question. + +The task shall perform the following operations using ConfigDB API: + +* Verify if the environment ``env_id`` exists in the ConfigDB API. + + * If not, create an environment with ``POST`` request. It should + contain a list of data sources to create for the environment. See details + in ConfigDB API specification [2]_. + + * The list of data sources is fetched from ``astute.yaml`` file, + from parameter ``data_sources`` of Puppet Master LCM plugin's metadata. + +* Read data from files in ``/etc/hiera`` directory via ``mcollective`` + client into internal variable of the deployment task. + +* Upload data to ConfigDB API's data sources based on the filenames from which + the data was read. + +* Read the *effective data* from ConfigDB API (i.e. values calculated by + merging uploaded data with configured overrides at all levels or hierarchy) + and write the data to files in ``/etc/hiera`` directory via ``mcollective`` + client. + +The latter function ensures that the deployment process accounts for data +overriden in ConfigDB. This is required for 2nd day operations based on +the Fuel workflow, specifically for scale-out opertaions (adding a node +to the cluster) and replacement of failed nodes. + +**Auth mechanism details** + +ConfigDB API extension as a part of Nailgun API +uses Keystone to verify and authorize users. + +Before installing an environment with the 3rd-party LCM plugin, user must add +a service account for the plugin in that environment with Keystone CLI. For +example: + +:: + + $ keystone user-create --name=lcm-plugin --tenant=admin + +While configuring environment, the user enables the LCM plugin and configures +the service account credentials for the plugin via Fuel UI or API. + +Plugin uses these access creates credentials to configure deployment task +``upload_data_to_configdb`` and the custom Hiera backend. + +**Example workflow** + +The following example illustrates the workflow of +the solution: + +* Assume that the User intends to use 3rd-party + application to perform some tasks, for example, + LCM operations, on an OpenStack environment deployed + by Fuel. + + * User installs the Fuel Master node with the + ConfigDB extension. The extension is installed + as an RPM on top of the existing system. + + * User installs a plugin for LCM operations that + should include components to upload deployment + data to ConfigDB API (e.g. deployment task + ``upload_data_to_configdb``) and to + perform lookup for certain parameters in ConfigDB + API (e.g. custom Hiera backend). + +* User configures OpenStack environment using Fuel UI. + Nailgun creates metadata for the environment + and individual nodes. + + * The deployment data for the + environment and nodes is accessible via Nailgun + API by URIs ``/cluster/<:id>/orchestrator/deployment`` + and ``/cluster/<:id>/orchestrator/deployment/default``. + + * Deployment data for specific node are exposed + via the same URIs with addition of parameter + ``?node=<:node_id>`` to the URI path. + +* User deploys the environment as usual via Fuel + UI or CLI. + + * Deployment task ``upload_data_to_configdb`` + runs on every node in the environment and + uploads serialized deployment data from + YaML files in ``/etc/hiera/`` directory to + ConfigDB API. + + * Another deployment task configures the node + to work with 3rd party LCM tool. This might + or might not include disable of the ordinary + Fuel means of deployment. + +* Afterwards the User makes changes to + the environment configuration using 3rd-party + LCM tool. + + * User changes or extends the deployment + settings by assigning values to parameters via + ConfigDB API, for example, changes ``keystone_url`` + parameter in ``globals`` data source. + + * ConfigDB saves the override data to an override for the + data source ``globals``. + + * User triggers 3rd party application which reads the *effective data* + (i.e. raw uploaded values with applied overrides) from ConfigDB API + and applies changes to all affected nodes. + +* In future, the User adds another node to the + environment and deploys it using standard Fuel + methods. + + * Deployment data for the new node provided by + Nailgun's standard serializers. + + * When the deployment is initiated, the task ``upload_data_to_configdb`` + synchronizes ``astute.yaml`` file created at the node by Astute with + data overrides created in ConfigDB API: it uploads the contents of + the file to ConfigDB API and then downloads *effective data* for the + ``asuste`` data source from there. + + * After the pre-deployment finishes, the prepared deployment + data are synchronized to the ConfigDB API by task + ``upload_data_to_configdb``. This ensures that override settings + from all data sources in the ConfigDB API are applied to all files + in ``/etc/hiera`` at the node. + + * Deployment is done by Fuel standard deployment tasks, but + with settings adjusted with overrides from ConfigDB. + + * After successful deployment, the LCM plugin reconfigures the node to + work with 3rd-party LCM tool. + +Web UI +====== + +None. + +Nailgun +======= + +None. + +Data model +---------- + +None. + +REST API +-------- + +None. + +Orchestration +============= + +A new deployment task shall be added to ensure +that all changes to files in ``/etc/hiera`` directory +are synchronized with the ConfigDB. + +The task shall send a series of requests to the URI of the +resource in ConfigDB based on the parameters +of the deployment: + +:: + + <:service_uri>/environments/<:env_id>/nodes/<:node_id>/resources/<:datasource>/values + +* ``service_uri`` is a endpoint from Keystone Service Catalog, + defaults to ``/api/v1/config``. + +* ``env_id`` is an identifier of cluster the node belongs to. + The ID of environment shall be fetched + from deployment fact ``deployment_id``. + +* ``node_id`` is an identifier of the node. It should be matched in + Nailgun API by the node's ``FQDN`` scope recieved from Puppet Master. + +* ``datasource`` is a name of the data source. + +The task will: +#. upload data to ``/values`` of the data source's resource and + +#. download *effective data* from ``/values?effective`` and write it to files. + +See detailed description of the API in corresponding +specification. [2]_ + +RPC Protocol +------------ + +None. + +Fuel Client +=========== + +None. + +Fuel Library +============ + +None. + +------------ +Alternatives +------------ + +The alternative way to keep deployment data from nodes in +sync with ConfigDB is to upload data to API from deployment tasks. + +While it is possible to adjust ``globals`` and ``openstack_config`` +tasks to upload configuration data to external service, it is +generally impossible to do with all supported plugins. + +A plugin can override default values in ``astute.yaml`` +generated by the Nailgun-provided serialized data. However, +this overrides are configured by plugin tasks +on a per-node basis. Override information is not available +to Nailgun or even Astute directly. So, to ensure sync +of plugins' override data we need to modify each and every plugin, +which apparently is not an option. + +Another way to keep data in sync is to upload it from some +bottom-level catch-all Astute post-deployment task. This +would allow to keep Nailgun/ConfigDB credentials limited to +the Master node and not expose them to target nodes +in the deployment. + +On the other hand, there was a work done on Astute to +convert its tasks into standard deployment tasks in +``fuel-library``. Thus, we should net add new tasks +to Astute in this proposal. + +-------------- +Upgrade impact +-------------- + +None. + +--------------- +Security impact +--------------- + +Sensitive configuration data, such as passwords and access credentials, +shall be uploaded to the ConfigDB API using proposed functions. +It is recommended to use encrypted HTTP protocol to +transfer these data. + +The approach to authentication of the plugin's application with Nailgun API +assumes that the user is responsible for configuring access credentials +for the plugin applications in Keystone. The user is also responsible for +configuring proper credentials for the plugin when installing an environment +with 3rd-party LCM application. + +-------------------- +Notifications impact +-------------------- + +None. + +--------------- +End user impact +--------------- + +None. + +------------------ +Performance impact +------------------ + +The deployment task proposed in this spec will take +some time to upload all data to the ConfigDB API. +Moreover, if many nodes trying to write to the same +API endpoint at the same time, it might significantly +affect the overall duration of deployment. + +----------------- +Deployment impact +----------------- + +None. + +---------------- +Developer impact +---------------- + +None. + +--------------------- +Infrastructure impact +--------------------- + +The deployment task is packaged as a part of 3rd-party LCM plugin. + +-------------------- +Documentation impact +-------------------- + +None. + +-------------- +Implementation +-------------- + +Assignee(s) +=========== + +Primary assignee: + Oleg S. Gelbukh + +Other contributors: + Sergey Ryabin + +Mandatory design review: + Evgeniy Li + Igor Kalnitsky + Vladimir Sharshov + Vladimir Kuklin + +Work Items +========== + +* Develop deployment task as a part of Puppet Master LCM + plugin code base [1]_. + +* Develop unit tests for the deployment task in the + plugin's code base. + +* Develop automated integration tests for the plugin in + ``openstack/fuel-qa`` repository. + +Dependencies +============ + +#. ConfigDB API implementation as Nailgun extension [2]_ + +------------ +Testing, QA +------------ + +* The feature shall be tested in conjunction with + ConfigDB API feature [2]_ + +* Tests shall verify that contents of data sources + are consistent with contents of files in ``/etc/hiera`` + at nodes after the deployment finishes. + +Acceptance criteria +=================== + +* Deployment data from nodes uploaded to corresponding + data sources in ConfigDB API upon successful + deployment of the OpenStack environment. + +---------- +References +---------- + +.. [1] Puppet Master LCM plugin specification TBD +.. [2] Nailgun API extension for serialized deployment facts https://review.openstack.org/#/c/284109/ +.. [3] Nailgun API for Deployment Information https://github.com/openstack/fuel-web/blob/master/nailgun/nailgun/api/v1/handlers/orchestrator.py#L190