Blueprint: replace-obs
Blueprint describes OBS replacement with Perestroika Change-Id: I030d65dce327718b0b34ec6a63b3066c8a4a677c Implements: replace-obs Implements: enable-gpg-check-and-sign
This commit is contained in:
parent
06c0889426
commit
d7205f5679
341
specs/7.0/replace-obs.rst
Normal file
341
specs/7.0/replace-obs.rst
Normal file
@ -0,0 +1,341 @@
|
||||
..
|
||||
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||
License.
|
||||
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
==========================================
|
||||
Replace OBS with another build system
|
||||
==========================================
|
||||
|
||||
https://blueprints.launchpad.net/fuel/+spec/replace-obs
|
||||
|
||||
https://blueprints.launchpad.net/fuel/+spec/enable-gpg-check-and-sign
|
||||
|
||||
Problem description
|
||||
===================
|
||||
|
||||
* As a CI engineer, I need a package build system that doesn't introduce
|
||||
package management limitations compared to upstream Linux distributions used
|
||||
in MOS
|
||||
|
||||
* As the Cloud Administrator I want to validate integrity of all Mirantis
|
||||
OpenStack packages with end-to-end signatures on either individual rpm
|
||||
packages or whole deb package repositories
|
||||
|
||||
We've found following fundamental limitations of OBS:
|
||||
|
||||
* OBS builds packages in its own way which is different from upstream. It
|
||||
causes an issues when upstream package can't be rebuilt without changing
|
||||
packages sources.
|
||||
|
||||
* OBS rebuilds package when its build dependency changed (and doesn't update
|
||||
revision number of such package)
|
||||
|
||||
* OBS uses base upstream packages for target in the building stage. Every
|
||||
change in the target causes rebuilding of each package which was built with
|
||||
this target.
|
||||
|
||||
* OBS doesn't support publishing udeb binary packages. This is due to the fact
|
||||
that it uses plain debian repository structure. But deb and udeb packages
|
||||
should not be published in one repository.
|
||||
|
||||
* Our current OBS version (2.4) doesn't support debian python:any
|
||||
dependencies. That's why we decided to create new OBS (2.6) instance. We
|
||||
can't update current version because it totally breaks supporting previously
|
||||
shipped releases.
|
||||
|
||||
* OBS doesn't support signing with predefined key. Only OBS autogenerated keys
|
||||
can be used. Every OBS project has it's own key. Such keys can't be exported
|
||||
from OBS.
|
||||
|
||||
* It's quite hard to reproduce our CI due to OBS. Every MOS OBS project based
|
||||
on previously shipped project (e.g. 6.1 and 6.0.1 based on 6.0 release, 6.0
|
||||
based on 5.1 and so on). So if you need to reproduce our CI for 6.1 release,
|
||||
you need to rebuild all packages for all shipped releases since 3.2.
|
||||
|
||||
* OBS server side natively supported on openSUSE
|
||||
and SUSE Linux Enterprise Server.
|
||||
|
||||
* We cannot support OBS as well as distribute it for our customers.
|
||||
|
||||
Proposed change
|
||||
===============
|
||||
|
||||
This specification introduces replacement for existing OBS infrastructure
|
||||
using new build system Perestroika.
|
||||
|
||||
Solution will use standard upstream Linux distribution tools to build packages
|
||||
(sbuild/mock), publish packages to repositories and manage package repositories
|
||||
(reprepro/createrepo).
|
||||
|
||||
Every package will be built in a clean and up-to-date buildroot.
|
||||
Packages, their dependencies and build dependencies will be fully
|
||||
self-contained for each MOS release.
|
||||
Any package included in any release can be rebuilt at any point in time using
|
||||
only the packages from that release.
|
||||
|
||||
Package build CI will be reproducible and can be recreated from scratch in a
|
||||
repeatable way.
|
||||
|
||||
New build system is based on Docker, which provides easy distribution. There
|
||||
will be created proper Docker Images for each supported Linux distribution with
|
||||
necessary tools and scripts.
|
||||
|
||||
Puppet will be used for configuration of those images.
|
||||
|
||||
We can wrap host side scripts of interaction in a package for easy deployment.
|
||||
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
* Open Build Service (OBS) (see `Problem description`):
|
||||
Supports rpm/debian based distributions
|
||||
http://openbuildservice.org/
|
||||
|
||||
* Koji:
|
||||
Supports rpm based distributions only
|
||||
https://fedoraproject.org/wiki/Koji
|
||||
|
||||
* Automated build farm (ABF):
|
||||
Supports rpm based distributions only
|
||||
http://www.rosalab.ru/products/rosa_abf
|
||||
https://abf.io/
|
||||
|
||||
* buildd + wanna-build:
|
||||
Supports debian based distributions only
|
||||
https://wiki.debian.org/DebianWannaBuildInfrastructure
|
||||
|
||||
* Delorean
|
||||
Supports rpm based distributions only
|
||||
Supports python packages only
|
||||
Requires separate docker image for each supported distribution
|
||||
https://github.com/openstack-packages/delorean
|
||||
|
||||
* openstack-pkg-tools
|
||||
Supports debian based distributions only
|
||||
Supports python packages only
|
||||
https://packages.debian.org/sid/openstack-pkg-tools
|
||||
|
||||
* docker-rpm-builder
|
||||
Supports rpm based distributions only
|
||||
Requires separate docker image for each supported distribution
|
||||
https://github.com/alanfranz/docker-rpm-builder
|
||||
|
||||
Data model impact
|
||||
-----------------
|
||||
|
||||
None
|
||||
|
||||
REST API impact
|
||||
---------------
|
||||
|
||||
None
|
||||
|
||||
Upgrade impact
|
||||
--------------
|
||||
|
||||
None
|
||||
|
||||
Security impact
|
||||
---------------
|
||||
|
||||
None
|
||||
|
||||
Notifications impact
|
||||
--------------------
|
||||
|
||||
None
|
||||
|
||||
Other end user impact
|
||||
---------------------
|
||||
|
||||
* System will be able to perform package/repository signing
|
||||
|
||||
* Packaging CI infrastructure will be reproducible.
|
||||
|
||||
Performance Impact
|
||||
------------------
|
||||
|
||||
* Unnecessary rebuild of packages and their dependencies will be avoided.
|
||||
|
||||
Other deployer impact
|
||||
------------------------
|
||||
|
||||
None
|
||||
|
||||
Developer impact
|
||||
----------------
|
||||
|
||||
None
|
||||
|
||||
Infrastructure impact
|
||||
---------------------
|
||||
|
||||
* Current workflow of building packages will be the same in general.
|
||||
* We should think about using Docker Hub as main repository for Docker Images
|
||||
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
New build system will contain following parts:
|
||||
|
||||
* Code storage
|
||||
We use gerrit code review system as code storage.
|
||||
|
||||
Gerrit projects structure:
|
||||
|
||||
- MOS+master-node Openstack packages
|
||||
code projects:
|
||||
|
||||
`[customer-name]/openstack/{package name}`
|
||||
|
||||
spec projects:
|
||||
|
||||
`[customer-name]/openstack-build/{package name}`
|
||||
|
||||
- MOS linux packages
|
||||
code+spec projects:
|
||||
|
||||
`[customer-name]/packages/{distribution}/{packagename}`
|
||||
|
||||
- Master-node linux packages (separated from MOS linux in 7.0)
|
||||
code+spec projects:
|
||||
|
||||
`[customer-name]/packages/fuel/{distribution}/{package name}`
|
||||
|
||||
- Versioning scheme will be supported by project branches
|
||||
openstack:
|
||||
|
||||
`openstack-ci/fuel-{fuel version}/{openstack version}`
|
||||
|
||||
MOS linux/master-node:
|
||||
|
||||
`{fuel version}`
|
||||
|
||||
* Scheduler
|
||||
This part is based on Jenkins CI tool. All jobs will be configured via
|
||||
jenkins-job-builder
|
||||
Jenkins has a separate set of jobs for each [customer name]+[fuel version]
|
||||
case.
|
||||
Gerrit-trigger configured to track events from `{version}` branch of all
|
||||
[customer-name] gerrit projects.
|
||||
|
||||
Each set of jobs will contain:
|
||||
|
||||
- Jobs for openstack packages for cluster (rpm/deb)
|
||||
- Jobs for MOS linux packages for cluster (rpm/deb)
|
||||
- Jobs for openstack packages for master-node (optional in case of using
|
||||
cluster packages) (rpm)
|
||||
- Jobs for non-openstack master-node packages (rpm)
|
||||
- Jobs for fuel packages (rpm/deb)
|
||||
- Job for package publishing
|
||||
|
||||
* Build workers
|
||||
Hardware nodes with preconfigured build tools for all supported distribution.
|
||||
Will be configured as Jenkins slave.
|
||||
|
||||
Each worker will contain:
|
||||
|
||||
- preconfigured docker images with native build tools for each distro type:
|
||||
mockbuild: will build packages by mock (`centos6/7` target distributions)
|
||||
sbuild: will build packages by sbuild (`trusty` target distribution)
|
||||
- prepared minimal build chroots for all supported distribution
|
||||
These chroots will be updated on daily basis in order to be up-to-date
|
||||
against upstream state.i
|
||||
- precofigured packages caching system (optional)
|
||||
All packaged downloaded from upstream repositories should be cached on
|
||||
build host in order to be reused by build stages. This part will reduce
|
||||
building time.
|
||||
Could be done with squid/polipo/approx
|
||||
|
||||
Build system will use short-lived docker containers to perform package
|
||||
building. Docker images contain preconfigured build tools only. No chroots
|
||||
inside images. Build chroots will be mounted to docker container on start in
|
||||
read-only mode. Additionally tmpfs partition will be mounted over read-only
|
||||
chroot folder with AUFS overlays inside docker container. Docker container
|
||||
will be destroyed after build stage is done.
|
||||
|
||||
Goals of this scheme:
|
||||
- Could run a number of containers with the only chroot simultaneously on
|
||||
the same build host
|
||||
- No need to perform cleanup operations after build (all changes matters
|
||||
inside container only and will be purged after container is destroyed)
|
||||
- tmpfs works much faster than disk fs/lvm snapshots
|
||||
|
||||
All worker nodes will be joined by jenkins slave label
|
||||
|
||||
* Publisher
|
||||
Publisher node will contain all repositories for all customer projects.
|
||||
Will be configured as Jenkins slave.
|
||||
Repositories will be maintained by native tools of respective distribution
|
||||
(reprepro/createrepo). Publisher node will be fully private and available
|
||||
from Jenkins master node only because of containing secret GPG key.
|
||||
All packages and repositories will be signed in terms of respective
|
||||
distribution by GPG key, stored on Publisher node.
|
||||
|
||||
* Mirror node
|
||||
All repositories should be available via http/rsync protocols. All
|
||||
repositories will be synced by Publisher to Mirror host
|
||||
|
||||
|
||||
Backward compatibility
|
||||
----------------------
|
||||
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Primary assignee:
|
||||
`dburmistrov <https://launchpad.net/~dburmistrov>`_
|
||||
|
||||
Other contributors:
|
||||
`dkaiharodsev <https://launchpad.net/~dkaiharodsev>`_
|
||||
|
||||
`dszeluga <https://launchpad.net/~dszeluga>`_
|
||||
|
||||
|
||||
Work Items
|
||||
-------------
|
||||
|
||||
* Write a scripts for interaction with native build tools inside
|
||||
Docker Images and pack them into DEB package.
|
||||
|
||||
* Create Docker Images with packaging tools (sbuild and mockbuild)
|
||||
|
||||
* Create a Jenkins job for building packages by using
|
||||
Docker based packaging system.
|
||||
|
||||
* Create Puppet manifests for configuring build hosts
|
||||
|
||||
* Create Puppet manifests for configuring publisher host
|
||||
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
None
|
||||
|
||||
Documentation Impact
|
||||
====================
|
||||
|
||||
In case of using new build system we should change workflow documentation where
|
||||
OBS mentioned.
|
||||
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
* OBS https://build.opensuse.org/
|
||||
* Docker https://www.docker.com/
|
||||
* Docker Hub https://hub.docker.com/
|
||||
* sbuild https://wiki.debian.org/sbuild
|
||||
* mock https://fedoraproject.org/wiki/Projects/Mock).
|
||||
* Puppet https://puppetlabs.com/
|
||||
|
||||
Testing
|
||||
=======
|
||||
|
||||
All of the scripts and Jenkins jobs should be tested in a sandbox environment
|
||||
for building packages.
|
Loading…
Reference in New Issue
Block a user