Blueprint: replace-obs
Blueprint describes OBS replacement with Perestroika Change-Id: I030d65dce327718b0b34ec6a63b3066c8a4a677c Implements: replace-obs Implements: enable-gpg-check-and-sign
This commit is contained in:
Damian Szeluga
Dmitry Burmistrov
parent
06c0889426
commit
d7205f5679
341
specs/7.0/replace-obs.rst
Normal file
341
specs/7.0/replace-obs.rst
Normal file
@ -0,0 +1,341 @@
|
|||||||
|
..
|
||||||
|
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||||
|
License.
|
||||||
|
|
||||||
|
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||||
|
|
||||||
|
==========================================
|
||||||
|
Replace OBS with another build system
|
||||||
|
==========================================
|
||||||
|
|
||||||
|
https://blueprints.launchpad.net/fuel/+spec/replace-obs
|
||||||
|
|
||||||
|
https://blueprints.launchpad.net/fuel/+spec/enable-gpg-check-and-sign
|
||||||
|
|
||||||
|
Problem description
|
||||||
|
===================
|
||||||
|
|
||||||
|
* As a CI engineer, I need a package build system that doesn't introduce
|
||||||
|
package management limitations compared to upstream Linux distributions used
|
||||||
|
in MOS
|
||||||
|
|
||||||
|
* As the Cloud Administrator I want to validate integrity of all Mirantis
|
||||||
|
OpenStack packages with end-to-end signatures on either individual rpm
|
||||||
|
packages or whole deb package repositories
|
||||||
|
|
||||||
|
We've found following fundamental limitations of OBS:
|
||||||
|
|
||||||
|
* OBS builds packages in its own way which is different from upstream. It
|
||||||
|
causes an issues when upstream package can't be rebuilt without changing
|
||||||
|
packages sources.
|
||||||
|
|
||||||
|
* OBS rebuilds package when its build dependency changed (and doesn't update
|
||||||
|
revision number of such package)
|
||||||
|
|
||||||
|
* OBS uses base upstream packages for target in the building stage. Every
|
||||||
|
change in the target causes rebuilding of each package which was built with
|
||||||
|
this target.
|
||||||
|
|
||||||
|
* OBS doesn't support publishing udeb binary packages. This is due to the fact
|
||||||
|
that it uses plain debian repository structure. But deb and udeb packages
|
||||||
|
should not be published in one repository.
|
||||||
|
|
||||||
|
* Our current OBS version (2.4) doesn't support debian python:any
|
||||||
|
dependencies. That's why we decided to create new OBS (2.6) instance. We
|
||||||
|
can't update current version because it totally breaks supporting previously
|
||||||
|
shipped releases.
|
||||||
|
|
||||||
|
* OBS doesn't support signing with predefined key. Only OBS autogenerated keys
|
||||||
|
can be used. Every OBS project has it's own key. Such keys can't be exported
|
||||||
|
from OBS.
|
||||||
|
|
||||||
|
* It's quite hard to reproduce our CI due to OBS. Every MOS OBS project based
|
||||||
|
on previously shipped project (e.g. 6.1 and 6.0.1 based on 6.0 release, 6.0
|
||||||
|
based on 5.1 and so on). So if you need to reproduce our CI for 6.1 release,
|
||||||
|
you need to rebuild all packages for all shipped releases since 3.2.
|
||||||
|
|
||||||
|
* OBS server side natively supported on openSUSE
|
||||||
|
and SUSE Linux Enterprise Server.
|
||||||
|
|
||||||
|
* We cannot support OBS as well as distribute it for our customers.
|
||||||
|
|
||||||
|
Proposed change
|
||||||
|
===============
|
||||||
|
|
||||||
|
This specification introduces replacement for existing OBS infrastructure
|
||||||
|
using new build system Perestroika.
|
||||||
|
|
||||||
|
Solution will use standard upstream Linux distribution tools to build packages
|
||||||
|
(sbuild/mock), publish packages to repositories and manage package repositories
|
||||||
|
(reprepro/createrepo).
|
||||||
|
|
||||||
|
Every package will be built in a clean and up-to-date buildroot.
|
||||||
|
Packages, their dependencies and build dependencies will be fully
|
||||||
|
self-contained for each MOS release.
|
||||||
|
Any package included in any release can be rebuilt at any point in time using
|
||||||
|
only the packages from that release.
|
||||||
|
|
||||||
|
Package build CI will be reproducible and can be recreated from scratch in a
|
||||||
|
repeatable way.
|
||||||
|
|
||||||
|
New build system is based on Docker, which provides easy distribution. There
|
||||||
|
will be created proper Docker Images for each supported Linux distribution with
|
||||||
|
necessary tools and scripts.
|
||||||
|
|
||||||
|
Puppet will be used for configuration of those images.
|
||||||
|
|
||||||
|
We can wrap host side scripts of interaction in a package for easy deployment.
|
||||||
|
|
||||||
|
|
||||||
|
Alternatives
|
||||||
|
------------
|
||||||
|
* Open Build Service (OBS) (see `Problem description`):
|
||||||
|
Supports rpm/debian based distributions
|
||||||
|
http://openbuildservice.org/
|
||||||
|
|
||||||
|
* Koji:
|
||||||
|
Supports rpm based distributions only
|
||||||
|
https://fedoraproject.org/wiki/Koji
|
||||||
|
|
||||||
|
* Automated build farm (ABF):
|
||||||
|
Supports rpm based distributions only
|
||||||
|
http://www.rosalab.ru/products/rosa_abf
|
||||||
|
https://abf.io/
|
||||||
|
|
||||||
|
* buildd + wanna-build:
|
||||||
|
Supports debian based distributions only
|
||||||
|
https://wiki.debian.org/DebianWannaBuildInfrastructure
|
||||||
|
|
||||||
|
* Delorean
|
||||||
|
Supports rpm based distributions only
|
||||||
|
Supports python packages only
|
||||||
|
Requires separate docker image for each supported distribution
|
||||||
|
https://github.com/openstack-packages/delorean
|
||||||
|
|
||||||
|
* openstack-pkg-tools
|
||||||
|
Supports debian based distributions only
|
||||||
|
Supports python packages only
|
||||||
|
https://packages.debian.org/sid/openstack-pkg-tools
|
||||||
|
|
||||||
|
* docker-rpm-builder
|
||||||
|
Supports rpm based distributions only
|
||||||
|
Requires separate docker image for each supported distribution
|
||||||
|
https://github.com/alanfranz/docker-rpm-builder
|
||||||
|
|
||||||
|
Data model impact
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
REST API impact
|
||||||
|
---------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Upgrade impact
|
||||||
|
--------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Security impact
|
||||||
|
---------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Notifications impact
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Other end user impact
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
* System will be able to perform package/repository signing
|
||||||
|
|
||||||
|
* Packaging CI infrastructure will be reproducible.
|
||||||
|
|
||||||
|
Performance Impact
|
||||||
|
------------------
|
||||||
|
|
||||||
|
* Unnecessary rebuild of packages and their dependencies will be avoided.
|
||||||
|
|
||||||
|
Other deployer impact
|
||||||
|
------------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Developer impact
|
||||||
|
----------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Infrastructure impact
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
* Current workflow of building packages will be the same in general.
|
||||||
|
* We should think about using Docker Hub as main repository for Docker Images
|
||||||
|
|
||||||
|
|
||||||
|
Implementation
|
||||||
|
==============
|
||||||
|
|
||||||
|
New build system will contain following parts:
|
||||||
|
|
||||||
|
* Code storage
|
||||||
|
We use gerrit code review system as code storage.
|
||||||
|
|
||||||
|
Gerrit projects structure:
|
||||||
|
|
||||||
|
- MOS+master-node Openstack packages
|
||||||
|
code projects:
|
||||||
|
|
||||||
|
`[customer-name]/openstack/{package name}`
|
||||||
|
|
||||||
|
spec projects:
|
||||||
|
|
||||||
|
`[customer-name]/openstack-build/{package name}`
|
||||||
|
|
||||||
|
- MOS linux packages
|
||||||
|
code+spec projects:
|
||||||
|
|
||||||
|
`[customer-name]/packages/{distribution}/{packagename}`
|
||||||
|
|
||||||
|
- Master-node linux packages (separated from MOS linux in 7.0)
|
||||||
|
code+spec projects:
|
||||||
|
|
||||||
|
`[customer-name]/packages/fuel/{distribution}/{package name}`
|
||||||
|
|
||||||
|
- Versioning scheme will be supported by project branches
|
||||||
|
openstack:
|
||||||
|
|
||||||
|
`openstack-ci/fuel-{fuel version}/{openstack version}`
|
||||||
|
|
||||||
|
MOS linux/master-node:
|
||||||
|
|
||||||
|
`{fuel version}`
|
||||||
|
|
||||||
|
* Scheduler
|
||||||
|
This part is based on Jenkins CI tool. All jobs will be configured via
|
||||||
|
jenkins-job-builder
|
||||||
|
Jenkins has a separate set of jobs for each [customer name]+[fuel version]
|
||||||
|
case.
|
||||||
|
Gerrit-trigger configured to track events from `{version}` branch of all
|
||||||
|
[customer-name] gerrit projects.
|
||||||
|
|
||||||
|
Each set of jobs will contain:
|
||||||
|
|
||||||
|
- Jobs for openstack packages for cluster (rpm/deb)
|
||||||
|
- Jobs for MOS linux packages for cluster (rpm/deb)
|
||||||
|
- Jobs for openstack packages for master-node (optional in case of using
|
||||||
|
cluster packages) (rpm)
|
||||||
|
- Jobs for non-openstack master-node packages (rpm)
|
||||||
|
- Jobs for fuel packages (rpm/deb)
|
||||||
|
- Job for package publishing
|
||||||
|
|
||||||
|
* Build workers
|
||||||
|
Hardware nodes with preconfigured build tools for all supported distribution.
|
||||||
|
Will be configured as Jenkins slave.
|
||||||
|
|
||||||
|
Each worker will contain:
|
||||||
|
|
||||||
|
- preconfigured docker images with native build tools for each distro type:
|
||||||
|
mockbuild: will build packages by mock (`centos6/7` target distributions)
|
||||||
|
sbuild: will build packages by sbuild (`trusty` target distribution)
|
||||||
|
- prepared minimal build chroots for all supported distribution
|
||||||
|
These chroots will be updated on daily basis in order to be up-to-date
|
||||||
|
against upstream state.i
|
||||||
|
- precofigured packages caching system (optional)
|
||||||
|
All packaged downloaded from upstream repositories should be cached on
|
||||||
|
build host in order to be reused by build stages. This part will reduce
|
||||||
|
building time.
|
||||||
|
Could be done with squid/polipo/approx
|
||||||
|
|
||||||
|
Build system will use short-lived docker containers to perform package
|
||||||
|
building. Docker images contain preconfigured build tools only. No chroots
|
||||||
|
inside images. Build chroots will be mounted to docker container on start in
|
||||||
|
read-only mode. Additionally tmpfs partition will be mounted over read-only
|
||||||
|
chroot folder with AUFS overlays inside docker container. Docker container
|
||||||
|
will be destroyed after build stage is done.
|
||||||
|
|
||||||
|
Goals of this scheme:
|
||||||
|
- Could run a number of containers with the only chroot simultaneously on
|
||||||
|
the same build host
|
||||||
|
- No need to perform cleanup operations after build (all changes matters
|
||||||
|
inside container only and will be purged after container is destroyed)
|
||||||
|
- tmpfs works much faster than disk fs/lvm snapshots
|
||||||
|
|
||||||
|
All worker nodes will be joined by jenkins slave label
|
||||||
|
|
||||||
|
* Publisher
|
||||||
|
Publisher node will contain all repositories for all customer projects.
|
||||||
|
Will be configured as Jenkins slave.
|
||||||
|
Repositories will be maintained by native tools of respective distribution
|
||||||
|
(reprepro/createrepo). Publisher node will be fully private and available
|
||||||
|
from Jenkins master node only because of containing secret GPG key.
|
||||||
|
All packages and repositories will be signed in terms of respective
|
||||||
|
distribution by GPG key, stored on Publisher node.
|
||||||
|
|
||||||
|
* Mirror node
|
||||||
|
All repositories should be available via http/rsync protocols. All
|
||||||
|
repositories will be synced by Publisher to Mirror host
|
||||||
|
|
||||||
|
|
||||||
|
Backward compatibility
|
||||||
|
----------------------
|
||||||
|
|
||||||
|
|
||||||
|
Assignee(s)
|
||||||
|
-----------
|
||||||
|
|
||||||
|
Primary assignee:
|
||||||
|
`dburmistrov <https://launchpad.net/~dburmistrov>`_
|
||||||
|
|
||||||
|
Other contributors:
|
||||||
|
`dkaiharodsev <https://launchpad.net/~dkaiharodsev>`_
|
||||||
|
|
||||||
|
`dszeluga <https://launchpad.net/~dszeluga>`_
|
||||||
|
|
||||||
|
|
||||||
|
Work Items
|
||||||
|
-------------
|
||||||
|
|
||||||
|
* Write a scripts for interaction with native build tools inside
|
||||||
|
Docker Images and pack them into DEB package.
|
||||||
|
|
||||||
|
* Create Docker Images with packaging tools (sbuild and mockbuild)
|
||||||
|
|
||||||
|
* Create a Jenkins job for building packages by using
|
||||||
|
Docker based packaging system.
|
||||||
|
|
||||||
|
* Create Puppet manifests for configuring build hosts
|
||||||
|
|
||||||
|
* Create Puppet manifests for configuring publisher host
|
||||||
|
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
============
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Documentation Impact
|
||||||
|
====================
|
||||||
|
|
||||||
|
In case of using new build system we should change workflow documentation where
|
||||||
|
OBS mentioned.
|
||||||
|
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
|
||||||
|
* OBS https://build.opensuse.org/
|
||||||
|
* Docker https://www.docker.com/
|
||||||
|
* Docker Hub https://hub.docker.com/
|
||||||
|
* sbuild https://wiki.debian.org/sbuild
|
||||||
|
* mock https://fedoraproject.org/wiki/Projects/Mock).
|
||||||
|
* Puppet https://puppetlabs.com/
|
||||||
|
|
||||||
|
Testing
|
||||||
|
=======
|
||||||
|
|
||||||
|
All of the scripts and Jenkins jobs should be tested in a sandbox environment
|
||||||
|
for building packages.
|
||||||
Loading…
Reference in New Issue
Block a user