Commit Graph

95 Commits

Author SHA1 Message Date
KAVVA, JAGAN MOHAN REDDY (jk330k)
dc58ef9ddd Move Tiller version to 2.16.9
Update Helm chart for Armada to use Tiller version 2.16.9.

Change-Id: I6556a6e2a1fad3946a92da5e8dd97e0da5803c62
2020-09-02 08:05:00 -05:00
Andrii Ostapenko
817efe966c Pass parameter to skip helm tests for component in the same script
Pass parameter from job allowing to parallelize helm tests using
separate scripts.

Change-Id: I3e06c5590d51c75448dc5ff5978dc7fc90daca6f
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-08-06 00:51:15 +00:00
Andrii Ostapenko
fdcbd03784 Allow parallelization in gate runner
Change-Id: I393a22cfcaecb00d14d8ac643bd4b7ffbba03b12
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-08-06 00:51:11 +00:00
Andrii Ostapenko
cf90f32e8b
Fix overrides diff
Check proper path.

Change-Id: Icd3d0711fb530b77d049227b09904c433e26dc78
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-31 21:52:59 -05:00
Andrii Ostapenko
68097edf36 Improve overrides diff script
Change-Id: I4af33e57ee31c0d4f52afb3e2ff248039333f702
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-23 02:25:36 +00:00
Andrii Ostapenko
0ff44fc155
Support images overrides
Adds ability to override images in osh and osh-i repositories
if docker_images dict is provided with tags following zuul-jobs
upload-docker-image naming convention.

This allows to inherit osh job from osh-images project providing
required vars.

Change-Id: I5e7acb0dd0edd6838b6f7ddb5111db0fa34912d4
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-12 11:41:49 -05:00
Andrii Ostapenko
32a860072a
Explicitly specify a script directory for jobs run
Required to support reusing of a job in another project.

Change-Id: I1c5968ea3d785c4902c7ab011f7538877b10ce24
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-11 01:00:18 -05:00
Andrii Ostapenko
2b4cf6a2d9 Completely switch to python3 for developers installation
This addresses an issue with using py2 as interpreter while
installing required dependencies with py3.

Also switch kubeadm-aio image to bionic.

Change-Id: I5a9e6678c45fad8288aa6971f57988b46001c665
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-03 09:47:43 +00:00
Gage Hugo
548356e86c Update keyword to become instead of sudo
Zuul updated ansible to 2.9 and broke one of the playbooks
that had the old sudo keyword, which is no longer valid in 2.9.

This change updates the offending file to use "become" instead,
which is the valid keyword instead of sudo.

Change-Id: I2057de7470d65a60c4c984cb99d0715c9d43b3a8
2020-06-16 21:52:00 +00:00
Andrii Ostapenko
8f24a74bc7 Introduces templates linting
This commit rewrites lint job to make template linting available.
Currently yamllint is run in warning mode against all templates
rendered with default values. Duplicates detected and issues will be
addressed in subsequent commits.

Also all y*ml files are added for linting and corresponding code changes
are made. For non-templates warning rules are disabled to improve
readability. Chart and requirements yamls are also modified in the name
of consistency.

Change-Id: Ife6727c5721a00c65902340d95b7edb0a9c77365
2020-06-11 23:29:42 -05:00
jacky06
655f0f4db5 Use force OPTIONS to install the jq
when the force is yes, get_url moudel will download the file
every time and replace the file if the contents change, so it's
not necessary to remove the jq before get it.

Change-Id: I7337afecd1f9d7c66da46bff433016a39fd9ef7a
2020-05-19 18:46:48 +00:00
Gage Hugo
d14d826b26 Remove OSH Authors copyright
The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.

This change removes all references to this copyright by the
non-existent group and any blank lines underneath.

Change-Id: I1882738cf9757c5350a8533876fd37b5920b5235
2020-05-07 02:11:15 +00:00
Steven Fitzpatrick
9d2e08f1a4 Fluentd: Switch to Native Metrics Plugin
This change updates the fluentd chart to use the native
fluent-plugin-prometheus for metric production. This plugin
provides more detailed metrics about fluentd's operations,
specifically regarding input and output statistics.

https://github.com/fluent/fluent-plugin-prometheus

Using the plugin, each fluentd pod produces metrics, so
scape annotations have been added to the pod spec. The
zuul check on metric producers has been updated to account
for this.

Depends-On: https://review.opendev.org/714167
Change-Id: I809356d92b0cff1e31cb2062102bbedefd4843fd
2020-03-30 00:44:58 -05:00
Steven Fitzpatrick
510f55a48b Update gather-prom-metrics service selection
This change updates how the gather-prom-metrics playbook role
chooses which services and ports to scrape when gathering metrics
at the end of a zuul build.

We can hit more metric endpoints by finding services with a "metrics"
port, as not all applications have a service labeled "component=exporter"

Change-Id: Ib8db7ea2e7034063eefadad74828d0396407275b
2020-03-23 22:19:55 -05:00
Phil Sphicas
927c018d29 Fix ansible docker support task
Removes become: and become_user: when including another role (that
already defines become: true and become_user: root)

Fixes an error occurring in the gates:
ERROR! 'become_user' is not a valid attribute for a IncludeRole

Change-Id: I362eefbe5b09ad64e97b3b541d07db2e6b990613
2020-01-03 08:37:11 -08:00
Itxaka
cace08c016
Fix openstack release in gates
Change I7def8df68371deda0b75a685363c8a73b818dd45 removed one
line by mistake passing the openstack release var down to the zuul
jobs, so all jobs are currently running under ocata.

This patch restores the missing line, thus fixing the opensuse jobs
and making sure the other jobs run under the correct release.

Change-Id: Ia7a488928e521de1afb821f141d77d2b0268ff0a
2019-12-12 10:51:19 +01:00
Steve Wilkerson
3a6df3b544 Grafana: Remove default dashboards from chart
This removes the default dashboards from the Grafana chart and
instead places them in the values_overrides directory, similar to
what was done for the Prometheus rules. As Grafana dashboards
will likely be heavily dependent upon end-user needs, the old
default dashboard configs should only be used as a reference
instead of opinionated defaults that are difficult to override.
The previous defaults made using specialized labels for dashboard
variables difficult, as they were making dangerous assumptions
about deployed namespaces and host fqdns. By removing the defaults
entirely, end users can define their own dashboards to meet their
specialized needs

Change-Id: I7def8df68371deda0b75a685363c8a73b818dd45
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-12-09 13:39:13 +00:00
Steve Wilkerson
ef4cbb3b08 Add ceph metrics to postrun metrics gathering role
This updates the gather-prom-metrics role to include gathering
metrics from the active ceph-mgr endpoint

Change-Id: Icb5d27b6a070e9065f6276725bf06dec7d2cbc0d
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-11-20 21:42:49 +00:00
Tin Lam
7b332076d7 Fix K8s version
Trivial fix to make all kubernetes version consistently 1.16.2.

Change-Id: I51d567c57604150cba2274c153817b4401a8e707
2019-11-17 06:20:33 +00:00
Steven Fitzpatrick
13f99c1cfa Update Zuul Selenium to use Python3
This change updates the deploy-selenium playbook
to install Selenium using pip 3.

The shell scripts which call our selenium test python
files have been updated to use python 3.

Change-Id: I7fc82ecd830d460639c718614ce1cca9fd4d1953
2019-10-21 18:27:41 +00:00
Steve Wilkerson
c9acad238c Update Kubernetes version to 1.16.2
This updates the kubeadm and minikube Kubernetes deployments to
deploy version 1.16.2

Change-Id: I324f9665a24c9383c59376fb77cdb853facd0f18
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-10-16 21:54:53 +00:00
Tin Lam
1d903948ae Add label to namespaces
This patch set adds the ability to add a label to the following namespaces:

- kube-public
- kube-system
- default

This was previously done in kubeadm in patch [0] but appears to be
removed as the script moved around. Also, this PS adds this to the
minikube deployment.

[0] https://review.opendev.org/#/c/540131/

Change-Id: I0f06e8ae0cd7742313b447dc2d563c7d92318fb0
Signed-off-by: Tin Lam <tin@irrational.io>
2019-09-13 13:20:07 -05:00
Arun Kant
7a8bb7058b Removing deprecated option usage in gatther pod logs logic
As per PR, https://github.com/kubernetes/kubernetes/pull/60210,
in kubectl get show-all option is deprecated and no longer needed.
Presumably now that's the default behavior.

Also in current logs gathering logic, we are interested in capturing
only pod names, so removing that option is harmless.

We are seeing related failures in local CI when kubectl version is
1.15.x. So removing this option.

Change-Id: I3886c792fe28bc8b80504d8c91e9524039131b15
2019-07-30 08:19:38 -07:00
Steve Wilkerson
d83cc610b0 Disable systemd-resolved service in nameserver role
This updates the task in the disable-local-nameserver role to
include disabling the systemd-resolved service, as this causes
the entries we update in /etc/resolv.conf to not be honored as
systemd-resolved will use a different set of files for configuring
the nameservers it uses.

See: https://www.freedesktop.org/software/systemd/man/systemd-resolved.service.html

Change-Id: I68a623b7bcb32037b9eeff2d76c7f2cb317cb7d8
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-07-10 20:54:04 +00:00
Steve Wilkerson
09366598b5 Remove docker.io version pin
Change-Id: Id45132d8476ca931042dddc4544074e26135f9fc
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-07-08 20:39:58 +00:00
Zuul
ce2c0e08d2 Merge "Update helm version to 2.14.1" 2019-07-04 18:36:57 +00:00
Kostiantyn Kalynovskyi
1ef750c8bb ignore errors if set zuul_site_mirror_fqdn fails
It can be that zuul_site_mirror_fqdn env variable will not be set,
in this case the whole job will fail, instead of simply not configuring
mirrors during image build. With this patch, if set_fact fails, mirrors
simply will not be configured during image build, as planned in lines 62
and 88 in this playbook

Change-Id: I049c696c7fb0d7cadb527a9f17dd01a42a671baa
2019-07-01 16:15:37 +03:00
Zuul
d5655e95ea Merge "[Kibana] Add improvements to Selenium tests for Kibana" 2019-06-27 07:07:14 +00:00
Kabanov, Dmitrii (dk370c)
1db1ddf0ba [Kibana] Add improvements to Selenium tests for Kibana
This PS adds several fixes to Selenium tests (for Kibana) and adds
role which allows to collect the results.

Change-Id: If9fb5f50e395379fdd3ccc46e945a93606dcbabe
2019-06-25 20:51:01 -07:00
Steve Wilkerson
b117b14c3a Update helm version to 2.14.1
This updates the helm version from 2.13.1 to 2.14.1

Change-Id: I619351d846253bf17caa922ad7f7b0ff19c778a2
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-25 21:33:20 +00:00
Pete Birley
e80c7f8aac Gate: Store helm values for release
This PS stores the applied helm values for releases in the gate.

Change-Id: I6563104ded6631b63d9fced775b9b9dba7fd00ef
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-25 18:54:09 +00:00
Pete Birley
2bb0dcb5b0 Gate: Simpler multiple distro/over-ride support
This PS indroduces a simpler way to incorp over-rides into gate
runs, and also ensures that they are scoped to a single chart, rather
than all of the charts deployed within a gate run.

Change-Id: Iba80f645f33c6d5847fbbb28ce66ee3d23e4fce8
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-24 14:16:17 -05:00
Tin Lam
752bbf48c4 Migrate ubuntu jobs from Xenial to Bionic
This patch set is based on [0] that also fixed up a handful of Bionic-
specific changes required for the gates to pass.

[0] https://review.openstack.org/#/c/649698/

Co-Authored-By: ghanshyam <gmann@ghanshyammann.com>

Change-Id: I217a27c53eec2a51ddbea7226a23042558c5946b
2019-05-24 13:52:08 +00:00
Steve Wilkerson
84c6931c98 Fix prometheus metrics gathering in postrun job
This adds '|| true' to the curl command for gathering metrics from
prometheus exporters in the postrun job. After the move to
minikube for single node jobs, the headless services for the
kubernetes components no longer work as intended. The addition of
'|| true' allows the post run job to continue through the list
of services tied to the prometheus exporters without the task
failing outright

Change-Id: I56f0f56b799c3df9b2bd66a2c2044d71473606e3
2019-04-22 12:02:11 +00:00
Steve Wilkerson
cc8c78dd6c Gate: Gather previous pod logs for failed containers
This updates the post-run pod logs task to gather logs from any
failed containers, allowing for identifying issues associated with
pods that fail to start in the gate jobs

Change-Id: I9195f319a064f84f62d2aa558df05f8f81b9abea
2019-04-20 18:59:12 +00:00
OpenDev Sysadmins
29b124e3dc OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:34:11 +00:00
Jean-Philippe Evrard
811a1df1ef Allow value overrides in CI
This allows to pass a new env var into shell scripts, for value
overrides, with Zuul's help (value_overrides can be part of the
job definition).

Change-Id: Ia5dcecb73f4b872fd8fb65d3cd0bf69c19addf07
2019-04-10 11:46:37 +00:00
Steve Wilkerson
9f0ccaf186 Use Helm v2.13.1
This updates Helm from v2.13.0 to v2.13.1

Change-Id: If9a6284fca770f706261e7a2127485a23e04f5e5
2019-03-25 10:59:02 -05:00
Steve Wilkerson
e247b6faf1 Update kubeadm kubernetes version to 1.13.4
This updates the kubernetes version used when deploying via
kubeadm and minikube to v1.13.4

This required updating the apiVersion in the kubeadm configuration
file template, as well as removing the --cadvisor-port flag from
the kubelet args, as this has been removed entirely

Change-Id: I3088b65ece0a5c9c5ef2669247ac293d6a6f66ed
2019-03-20 00:34:26 +00:00
Steve Wilkerson
544805f99c Pin Docker version to 18.06.1
This pins the version of docker installed to a validated version
of docker for kubernetes 1.12.2.

Change-Id: If543dc4b84a6ea1224d4a8bb3af71cc9c20f28e4
2019-03-12 12:49:22 -05:00
Roman Gorshunov
31e3469d28 Update Helm to version 2.13.0
This updates Helm from version v2.12.3 to v2.13.0.
The `merge` function has changed behavior, and is now called
`mergeOverwrite`.

Change-Id: Ie3364256c8abb714b748b3bf3658bd6cd1e3ce35
2019-03-11 05:53:08 +00:00
Steve Wilkerson
be5ac2d4cc Revert "Update kubeadm kubernetes version to 1.12.2"
This reverts commit 8e8193f9e1.

Change-Id: I69c4d8716389d17dc806bd56f6afc14233f003af
2019-02-15 19:19:30 +00:00
Steve Wilkerson
8e8193f9e1 Update kubeadm kubernetes version to 1.12.2
This updates the kubernetes version used when deploying via
kubeadm to v1.12.2, which matches what is deployed via minikube
for the single node jobs

This required updating the apiVersion in the kubeadm configuration
file template, as well as removing the --cadvisor-port flag from
the kubelet args, as this has been removed entirely

Change-Id: I26573de35529ce44e91e6d4d4530f608b8cee476
2019-02-15 10:42:57 -06:00
Steve Wilkerson
ea0b94a052 Helm: Update version to 2.12.3
This updates Helm from version 2.12.1 to 2.12.3

Change-Id: Ie85e4ae3b55ce2e9f67a5e67af1e785540a6637e
2019-01-24 09:09:00 -06:00
Steve Wilkerson
9bb603ed0c Update Helm to version 2.12.1
This updates Helm from version 2.11.0 to 2.12.1

Change-Id: I9bc37c330b068388df9840eb84dfb12e2536c173
2019-01-23 16:43:36 -06:00
Zuul
1c6d48f4a5 Merge "Selenium Tests for OSH Infra" 2019-01-08 18:03:17 +00:00
chengli3
17108b7d95 Correct the test for proxy.http
proxy.http | trim returns "None" instead of "". To test if proxy should
be used, when: proxy.http should be good enough.
We don't have to test if proxy.http is defined or not. Because it is
alredy defined in defaults/main.yaml.

Change-Id: Ia6330907d22c3f46586aec16db3eefab1a5bbac9
Task: 28755
Story: 2004717
2019-01-08 12:33:55 +08:00
Meg Heisler
c3bef9e88f Selenium Tests for OSH Infra
This adds scripts using Selenium Webdriver to verify
the dashboards for Gafana, Nagios, and Prometheus are
reachable and functioning as expected. The scripts
create screenshots of each dashboard as well as
pages that can be navigated to.

It also adds the scripts to the gates for the single
and multinode deployments.

Change-Id: I1699e0ba8ff82ce8f59342cc71aad10cff7d2516
2019-01-07 15:59:42 -06:00
weiyj
f5d1d6938c spelling error
Change-Id: Idd9e6bc6a1459e74703599b94f0b410fdb9f94ef
2019-01-05 04:39:23 +00:00
Aaron Sheffield
8b201ea0eb Adding AppArmor profile to Calico v3
- Adds AppArmor profile to the privileged pod
  using kubernetes_manadatory_access_control_annotation.
- Added apparmor install to the gate jobs.

Change-Id: I8b53e0b8ddc2695fa278481edf5688efa23ab06b
2018-12-10 16:23:35 +00:00