212 Commits

Author SHA1 Message Date
Mark Goddard
cdcd5610e9 Fix lower-constraints and kolla CI jobs
Multiple fixes to fix multiple problems.

1. Bump setuptools requirement to 21.0.0

The 16.0.0 requirement breaks the lower-constraints job on Ubuntu
bionic. We see the following error:

AttributeError: module 'importlib._bootstrap' has no attribute 'SourceFileLoader'

2. ubuntu-binary horizon image needs to use python3 for django
manage.py

3. ubuntu-binary nova-placement-api image needs to use python3 for
apache mod wsgi.

4. nova-libvirt needs exact qemu dependencies due to packaging change

5. OracleLinux repos definition was changed in official image.

6. Fetch InfluxDB gpg key for Ubuntu base image from influx server.

Change-Id: Ic797d41f53a5f146d112619f1683cbfbf3a85dc2
2019-03-15 12:01:25 +01:00
Michal Nasiadka
a490eef0d5 Fix py36 gate job
python 3.6 pycache also includes links - so those also
need to be removed

Depends-On: https://review.openstack.org/636750
Change-Id: I0f995df45bf203acc79cc7bd52b9ac51577468da
2019-02-14 09:13:32 +01:00
confi-surya
8acba6ff56 Update hacking version
Use latest release 1.1.0 and compatible changes w.r.t pep8

Change-Id: Ia3d1166a2746a46591e4e5a04ea91e03ddc28d93
2018-12-27 12:02:17 +00:00
Zuul
e215e51fc1 Merge "Use infra mirrors for ODL and percona repos" 2018-12-19 13:48:02 +00:00
Zuul
1bcd0617b6 Merge "No need to use dict_object.keys()" 2018-12-18 10:41:33 +00:00
Eduardo Gonzalez
0c4238da7c Use infra mirrors for ODL and percona repos
Change-Id: I212a53b235f2785d93d474660c87a8521f3cc8b2
2018-12-18 09:13:52 +00:00
Eduardo Gonzalez
65296e27e6 Fix Oraclelinux build, disable collectd in gate
Oraclelinux repositories has been upgraded and
previous file removed, this change update repos
to new URLs.


Disable collectd in gate as is currently broken and
blocks mergingin voting jobs.

Change-Id: I402cff1ac86a2ba108f2f5378cc98db5f6ecc673
2018-12-05 08:46:29 +00:00
confi-surya
8504a20609 No need to use dict_object.keys()
for in dict_object.keys() can be replaced with
for in dict_object, so just a short cleaup

Change-Id: Id934eae6db05298b85b800531b9dfdcb28782cc5
2018-12-03 10:21:27 +05:30
confi-surya
24c993a059 Fix the parameter order of assertEqual
On assertEqual, the order of parameters should be (expected, observed).
But, some part of kolla tests was written with invalid order.
This patch fix the problem.

Change-Id: I9fcfb3923716243ddb1886223e932ec12829fb2e
Related-Bug: #1277104
2018-11-29 16:20:25 +05:30
Eduardo Gonzalez
9c9a7f4303 Suppress log copy output in gates
Copying collectd logs is half of the job ouput.
This change is to remove the verbosity of the syncronize
tasks by applying --quiet to only show errors

Change-Id: I39b47b844aab87d043a2cd745103d6589f3632e3
2018-11-20 10:09:16 +01:00
Zuul
53000a956e Merge "Fix epel mirror replacement in gates" 2018-09-24 12:46:19 +00:00
Mark Goddard
71d8fb478a Disable debug logs for image builds
Logs emitted when building via the tox environments can exceed the
maximum length supported by subunit [1], causing the following message:

Length too long: <length>

This change disables debug logging in these tox environments to reduce
the length of the output.

[1]
8cdd703980/python/subunit/v2.py (L208)

Change-Id: Icb5793a8ea58edafe2f3a249eb1da7d4ae2e395f
2018-09-21 14:55:58 +01:00
Eduardo Gonzalez
3275274616 Fix epel mirror replacement in gates
Change-Id: I7b74497931de5e2a89fc1a2b827a1cf6d64ab7e9
2018-09-19 11:15:35 +00:00
Martin André
3a644f7764 Add CI script to retrieve info from build logs
This should serve as a base for retrieving useful information from the
kolla build logs.

For now, the script warns about obsolete package names and helps track
packages that are installed too many times in different images.

It only supports rpm-based distros for now.

Change-Id: I160b6ede1336bd0c202621a48ca0218f72fc1270
2018-07-10 10:14:36 +02:00
Martin André
7e770472b1 Show disk space used by docker images in CI
The `docker system df` command can show the total size of the docker
cache. If you pass it the `-v` option, it will break it down per image,
replacing the use of `docker ps -a` and `docker images`.

This should be very useful in CI to see the variation in the images
size.

Change-Id: I407434d88f59df65693974265551c624420d4aaf
2018-07-10 10:12:48 +02:00
Martin André
c4eb866ad5 Stop creating an empty system_logs dir in CI
The system logs from the building node are copied to the system_log
directory. There is no need to create an empty system_logs directory.
It was certainly a typo.

Change-Id: I1f0eea485550bd7b41c9954a6b7077251518b31d
2018-07-09 21:23:58 +02:00
Zuul
5caa5e30de Merge "Bump Ubuntu base image to 18.04" 2018-06-04 10:39:19 +00:00
Jeffrey Zhang
14374e715d Bump Ubuntu base image to 18.04
Ubuntu 18.04 has been released. We need upgrade to it this cycle.

Changes made in this patch

- Bump ubuntu repo from xenial to bionic. Some repos do not have bionic
  packages, so still using xenial repos.
- Drop mariadb, percona and nfs-ganesha repo, because these packages are
  already provided by Ubuntu official repo.
- Since qdrouterd does not have bionic repo and xenial packages does not
  work on Ubuntu bionic, just add it into ignore image list.
- Use python-rtslib-fb and targetcli-fb rather than python-rtslib
  and targetcli
- use xen-utils package name rather than with package version one.
- Seems unsigned repo won't work on Ubuntu bionic, stop using infra
  mirrored unsigned repo in gate building job.

Needed-By: https://review.openstack.org/568728
Implement blueprint: ubuntu-bionic
Change-Id: I4e3b0ca669cfbf6316bf591f2d8428fa1a0d6182
2018-05-26 00:24:30 +08:00
Martin André
58742c212e Create symlinks for failed builds logs
This allows to know which image failed to build without having to parse
the huge job-output.txt file.

This commit creates a convenient 000_FAILED_image_name.log symlink to
the relevant log files when kolla is configured to output to a log
directory. This creates symlinks only for the parent image with error,
i.e. it skips images that failed to build because of parent error.

Change-Id: I05954e2335dc3fac469276809c91f9c87e3ecab2
2018-05-08 09:08:05 +02:00
Zuul
69a4b7e049 Merge "No use of mox/mox3" 2018-04-04 02:40:05 +00:00
Zuul
6a3c5c58fe Merge "Revert "Seem there are some wrong in infra ubuntu repo"" 2018-04-03 10:13:07 +00:00
Surya Prakash Singh
edb4368fee No use of mox/mox3
Now onwards no support of mox/mox3, please check storyboard link
for reference

https://storyboard.openstack.org/#!/story/2001546

Change-Id: Ic1fa65951f1e43e1279d8c72fcfc5e09609faa70
2018-03-30 03:06:07 +00:00
Jeffrey Zhang
abc61c012c Support push kolla tag image to hub.docker.com site
Change-Id: I8a21f60a37ba038e46c29d0bcff9ffbd2457a257
2018-03-21 16:53:15 +08:00
Eduardo Gonzalez
01a5388169 Verify YAML syntax in gates
Test yaml syntax is correct on new changes.
Fixes current warnings and errors

Change-Id: I5888f8e4a9d27a08506036df2c564b9f2081ccee
2018-02-27 21:45:57 +00:00
Jeffrey Zhang
ad8666f5a0 Revert "Seem there are some wrong in infra ubuntu repo"
This reverts commit 7b562990b5b717bb83d56673b368de9b355bb315.

Change-Id: Id5f0e9532a7714a18125832d4f38bbe7bb849c84
2018-02-26 07:37:55 +00:00
Jeffrey Zhang
7b562990b5 Seem there are some wrong in infra ubuntu repo
Revert this after infra ubuntu repo is OK.

Change-Id: I8bbf8257d5a932b7e9f2e9a64d62cd5892e3f1bb
2018-02-12 21:04:54 +08:00
Zuul
4a423d387b Merge "Add collectd in ci to collect system information" 2018-01-16 08:44:59 +00:00
Jeffrey Zhang
77612f7eed Add collectd in ci to collect system information
Change-Id: Ic5cbdfd4ebca1d907ada5c39f27dc4a7889daa21
2017-12-18 13:08:10 +08:00
Jeffrey Zhang
e6036f52ed Use separate publish jobs in periodic pipeline
The secrets can not be inherited.

Change-Id: Ie0f99f470a419c57865eb8b0a14e465ae228c799
2017-12-06 06:03:54 +00:00
Jeffrey Zhang
5feb113033 Change more repo url to infra mirror
Change-Id: I7e9fd2760ae3202f3a84833bb65edfdcffa67fe4
2017-12-04 17:23:31 +08:00
Jeffrey Zhang
9846a27a6b Improve gate post jobs
- Reuse build jobs to publish images
- Add binary jobs in periodic pipeline
- Push all image to dockerhub
- Use branch name for image tag

Change-Id: Ifefb96bcbcc9c6840cf865f7786b43a43fae4c17
2017-11-28 04:45:30 +00:00
Michal (inc0) Jastrzebski
9674e2da1c Fixup small bug with dockerhub publisher
Jinja2 vs gotpl...

Change-Id: I27a54eb9202ac5f3e9b3ef9c307e81475e8833f5
2017-11-08 15:00:13 -08:00
Michal (inc0) Jastrzebski
c506e626e7 Dockerhub publisher job
This patchset includes pushing mechanism for daily periodic jobs.

Change-Id: Ic4179ffd3a6e640abfb181dbd000bc75cc64b750
2017-11-06 09:18:22 -08:00
Jeffrey Zhang
b64198cce0 Moving jobs to kolla repo
This is zuulv3 migration patch. This patch only implement build jobs.
Deploy jobs will be pushed in another one.

Partial-Bug: #1720601
Change-Id: Ib9157ed132820d752ef1dbd7a53353ebd68b1934
2017-10-19 10:34:03 +08:00
Jeffrey Zhang
b1b7492e65 Skip known failed kolla images
Change-Id: Ide8478c248e77b9c85201981e8086f3dbf06d131
2017-09-23 16:06:51 +08:00
Eduardo Gonzalez
7ba7fdd855 Enable mistral in test_build centos source
For some reason, mistral was disabled in test_build since
long time ago. I've been using mistral with no problem.
Also current gates build mistral images properly.

Change-Id: I89e79c22f0e15b8165b23fa8cd98df309efdaf45
2017-08-29 11:27:55 +02:00
Eduardo Gonzalez
7b2dea5506 Implement tacker-conductor image, rework all tacker
Tacker is broken in pike without conductor images.
Conductor is needed to communicate with mistral for
VIM monitoring
This change rework tacker image to allow tacker-conductor
service image.
Split tacker global image into tacker-server and tacker-conductor.

Closes-Bug: #1710874
Change-Id: I52778e86e4f2c297ead8d4b09983e5e38ca88c70
2017-08-20 09:58:27 +00:00
Jenkins
6e2130fdad Merge "ODL installed for source images" 2017-08-01 10:14:35 +00:00
Eduardo Gonzalez
7129203bdb ODL installed for source images
ODL is an infrastructure service, in kolla we do not
support source installs for non OpenStack projects.
This change install odl from binaries to all
install_types, installing source code is still supported
from template-overrides

Change-Id: Ie60d35d1df6b7acd62cd529b0208d1c18bd7ee5a
2017-07-19 17:45:53 +02:00
Jeffrey Zhang
68826b64e2 implement panko api for ubuntu binary
Change-Id: I34ea4097be491c70e5fa183ee7d505885d7d7c3c
2017-07-18 08:37:18 +00:00
Eduardo Gonzalez
7b1e550c3f Implement blazar Docker images
Blazar is a resource reservation service for OpenStack,
allows to reserve compute host for tenants.
Really usefull for HPC, NFV and Scientific environments.

Implements: blueprint blazar-images
Change-Id: If42087a791a7f84105c867509e4bbc7b423cb41e
2017-07-07 08:02:33 +00:00
Martin André
f54f340630 Make set_configs.py work with symlinks
Closes-Bug: #1702305

Change-Id: I92a404b8d301937dc1847d42576e30db43951990
2017-07-06 11:18:53 +02:00
Jenkins
0d1983ca96 Merge "introduces support for the OVS DPDK dataplane" 2017-07-05 21:08:00 +00:00
Sean Mooney
4f5bac1089 introduces support for the OVS DPDK dataplane
- DPDK is a  userspace library for high-performance networking.
- This change enable support of Open vSwitch with the DPDK
  accelerated netdev datapath.
- This change provided binary and source support via
  a template-override.
- This change provides an example template override file
  and corresponding documentation.

Co-Authored-By: Mauricio Lima <mauriciolimab@gmail.com>
Partial-Implements: blueprint ovs-dpdk
Change-Id: I7c3a5dddeca1886fe1c7681ce8d5bebde08f3102
2017-07-05 11:34:51 +00:00
Jenkins
86a15be86f Merge "Add images for novajoin services" 2017-07-04 13:52:36 +00:00
Juan Antonio Osorio Robles
1ae703e4a4 Add images for novajoin services
novajoin is a nova vendordata service that enables the enrollment of
nova instances into freeIPA.

Change-Id: Ia1c0f896ef45b8bb00d0354e7343c7eb4e060fd1
2017-07-03 09:38:53 -04:00
Chulmin Kang
ea3d140cab Fix argument to handle_permissions
Fix the format of argument passed to handle_permissions function, so
that handle_permissions can handle permissions and ownerships of
config_files.

Change-Id: Ic07975a16973df22cdaa6685fc326620812f8f65
Closes-Bug: #1701876
2017-07-02 19:05:36 +09:00
Jenkins
98e4a4bf56 Merge "Introduce merge and preserve_properties for config_files" 2017-06-28 20:50:46 +00:00
Jenkins
b500866531 Merge "Allow setting copied files group more precisely" 2017-06-28 18:34:04 +00:00
Chuck Short
718d66f677 Use packages when installing neutron-sfc-agent
All linux distros supported by kolla has a python-networking-sfc
package. So install said package.

Change-Id: I1ce40ffd36ac72ea4f052b3aec8fa874530dc41e
Signed-off-by: Chuck Short <charles.short@ericsson.com>
2017-06-27 12:53:47 -05:00