38 Commits

Author SHA1 Message Date
Tin Lam
92e68d33ea Add network policy toolkit function
This patch set implements the helm toolkit function to generate a
kubernetes network policy manifest based on overrideable values.
This also adds a chart that shuts down all the ingress and egress
traffics in the namespace. This can be used to ensure the
whitelisted network policy works as intended.

Additionally, implementation is done for some infrastructure charts.

Change-Id: I78e87ef3276e948ae4dd2eb462b4b8012251c8c8
Co-Authored-By: Mike Pham <tp6510@att.com>
Signed-off-by: Tin Lam <tin@irrational.io>
2018-10-15 13:50:50 +00:00
Steve Wilkerson
ff116a26fd Kibana: Add session affinity to ingress
This adds session affinity to Kibana's ingress. This allows for
the use of cookies for Kibana's session affinity

Change-Id: I0863493ba7051a08350971da9c6e4d59cc2d8fa5
2018-09-25 15:38:25 -05:00
Pete Birley
bb3ff98d53 Add release uuid to pods and rc objects
This PS adds the ability to attach a release uuid to pods and rc
objects as desired. A follow up ps will add the ability to add arbitary
annotations to the same objects.

Change-Id: Iceedba457a03387f6fc44eb763a00fd57f9d84a5
Signed-off-by: Pete Birley <pete@port.direct>
2018-09-13 05:35:35 +00:00
Steve Wilkerson
9a311475ba Charts: Use secrets for configs in chart
This updates the osh-infra charts to use a secret for their
configuration files instead of a configmap, allowing for the
storage of sensitive information

Change-Id: Ia32587162288df0b297c45fd43b55cef381cb064
2018-08-24 15:56:53 -05:00
Seungkyu Ahn
a430533e6a Quoting node_select_value in Ingress Controller
In most cases, the ingress controller's nodeSelector key and value
are "node-role.kubernetes.io/ingress" and "true".
Using quote to treat the nodeSelector value as a string.

Change-Id: Ie1745629b90795e4d888d85f35565e6d6350e09b
2018-08-01 02:39:05 +00:00
Steve Wilkerson
6f6c6b8b99 Nagios/Kibana: Update configmap annotations
This changes the ordering of the configmap annotations for kibana,
as older versions of helm require the configmap with the values
template definition for the apache proxy to be listed last. This
was addressed in the elasticsearch-client template but missed in
kibana.

This also adds the configmap hash annotations to the nagios chart
as they were previously missing. It also places them in the
correct order as above

Change-Id: I13befe8684d975f310f2723c5172b8a0f9f365d6
2018-07-30 12:33:17 -05:00
Steve Wilkerson
4f78e1f6fc Drive apache proxy configuration via values templates
This proposes defining the apache proxy hosts entirely via values
templates. While complicated on its face, this gives flexibility
by allowing the ability to define the desired authentication
mechanism via values templates. These options can range from
using http basic auth for development purposes to defining more
complex ldap configurations without a need to modify the chart
directly

Change-Id: Ief1b6890444ff90cc9c0ca872087af74836c0771
Signed-off-by: Pete Birley <pete@port.direct>
2018-07-30 07:52:26 -05:00
Steve Wilkerson
c26a1b53f6 Update TLS secret templates, remove nagios readiness probe
This updates the TLS secret templates to include the backend
service in the dict supplied to the manifest template, as it is
required for the TLS secret to render correctly.

This also removes the readiness probe from the nagios container in
the deployment for the nagios chart, as it wasn't functioning as
intended due to the port not being available for the probe

Change-Id: Iabcfd40c74938e0497d08ffeeebc98ab722fa660
2018-06-27 18:56:45 -05:00
Steve Wilkerson
b823954787 Ingress: Add initial TLS Support for osh-infra public endpoints
Adds support for TLS on overriden fqdns for public endpoints for
the services that have them in openstack-helm-infra. Currently this
implementation is limited, in that it does not provide support for
dynamically loading CAs into the containers, or specifying them manually
via configuration. As a result only well known or CA's added manually
to containers will be recognised.

Change-Id: I4ab4bbe24b6544b64cd365467e8efb2a421ac3f4
2018-06-26 14:47:19 -05:00
Steve Wilkerson
cb7bf2c0b3 Add missing readiness probes to openstack-helm-infra charts
This adds missing readiness probes to the following charts in
openstack-helm-infra: elasticsearch, fluent-logging, kibana,
nagios, prometheus-kube-state-metrics, prometheus-node-exporter,
and prometheus-openstack-exporter

Change-Id: I6a2635b08667c31eadb1b05ba848c658935a17e5
2018-06-26 12:25:36 +00:00
Pete Birley
fa629cdbbd Daemonsets: Use current kubernetes daemonset api version
This PS moves to use the current ga version for kubernetes daemonsets,
additionally any remaining deployments that were using the
`extensions/v1beta1` have been updated to `apps/v1`.

Story: 2002205
Task: 21735

Change-Id: If9703162dc472af1e6096bf2b9062802fd5ce8ab
Signed-off-by: Pete Birley <pete@port.direct>
2018-06-13 21:53:18 +00:00
Steve Wilkerson
9e11fc11af Update resource tree for elasticsearch/kibana
This adds the entry for resources for the apache proxy running in
the elasticsearch client and kibana pods. This also fixes an
incorrect enabled flag for resources in the kibana chart

Change-Id: Ifcd33a680167d7debfae2c4d71bdcb693632fce9
2018-05-15 20:55:24 +00:00
Steve Wilkerson
3c692abd6e Add ldap support in elasticsearch/kibana apache proxies
This adds required configuration for enabling LDAP through
the apache proxy in the elasticsearch and kibana charts by
default

Change-Id: Iaff8f328ff50944ddad94ec86b1134ca73750176
2018-05-14 13:53:30 +00:00
Sean Eagan
f402171e42 Move to v0.3.1 of kubernetes-entrypoint
Move to v0.3.1 of kubernetes-entrypoint which has 2 breaking changes to
pod dependencies, and also adds support for depending on jobs via
labels.

Change-Id: I2bafc2153ddd46b3833b253a2e7950bccbccf8ed
2018-04-25 12:38:44 -05:00
Steve Wilkerson
e166432a98 Add manifest for image_repo_sync job
This ps proposes adding a common template for the image_repo_sync
jobs for consumption by the charts

Change-Id: I48476d1e4fd94bd1b08b13b46983e3d999f8d8ca
2018-04-19 14:10:08 +00:00
Zuul
d93649da5f Merge "Update kibana authentication values in endpoints and deployment" 2018-04-19 00:59:04 +00:00
Steve Wilkerson
a31afb2f85 Update kibana authentication values in endpoints and deployment
The Kibana username and password needs to match the Elasticsearch
username and password, as Kibana requires an authorized elasticsearch
user to make queries against the elasticsearch backend to display
its dashboards and set up the initial .kibana index. This changes
the apache proxy running in front of kibana to consume the
elasticsearch username and password via the elasticsearch secret in
the chart to ensure kibana has proper access

Change-Id: Ife3fd916e8d9a3f8877d01a9048a892f92e412d8
2018-04-18 21:55:38 +00:00
Zuul
49e9084679 Merge "OSH-Infra: Update labels for chart components" 2018-04-18 18:47:08 +00:00
Zuul
626b94e0c8 Merge "Helm-Toolkit: Kubernetes Entrypoint, simplify image dependencies" 2018-04-17 15:11:00 +00:00
Steve Wilkerson
7757400edc OSH-infra: move charts to use ingress manifest in htk
This moves all relevant charts in osh-infra to use the htk manifest
template for ingresses, bringing them in line with the charts in
openstack-helm

Change-Id: Ic9c3cc6f0051fa66b6f88ec2b2725698b36ce824
2018-04-13 15:41:12 -05:00
Steve Wilkerson
aaffc4caf0 OSH-Infra: Update labels for chart components
This ps adds more granular node selectors for the charts in osh
infra to match what is currently done in osh

Change-Id: I8957a95053b9fb3ea329fd37ff049cd223a7695d
2018-04-13 08:44:33 -05:00
Pete Birley
b9336ca613 Helm-Toolkit: Kubernetes Entrypoint, simplify image dependencies
This PS simplify the logic for dyanmicly merging the image management
depenencies into pod deps when active.

Change-Id: I0cf6c93173bc5fbce697ac15be8697d3b1326d0a
2018-04-13 08:42:37 -05:00
Sean Eagan
db15b5e30b Support pod dependencies
Adds support for a new feature of kubernetes-entrypoint, pod
dependencies, that was added in v0.3.0.

Change-Id: I78d9e0545ca3b837cd2386783386a253f7f5a2d6
2018-03-20 10:53:53 -05:00
Zuul
33cb0e8433 Merge "Revert Elasticsearch/Kibana image change" 2018-03-12 19:13:34 +00:00
Steve Wilkerson
8e4da9da55 Revert Elasticsearch/Kibana image change
This reverts the changes made to Elasticsearch, Kibana and fluent
logging charts in https://review.openstack.org/#/c/550229/7.

Specifically, this moves the images back to previous used versions
and makes the required changes to the fluent-logging elasticsearch
template job to include the correct mapping directives for the
elasticsearch template.

This change was made to give more time for evaluating a more
robust solution for switching to the official upstream images that
will not cause intermittent gate failures as seen since 550229 was
merged

Change-Id: I9f70b3412a8edc5cb1d80937b158aa2fe7b1ec82
2018-03-12 10:27:35 -05:00
Zuul
eb3cbf0f95 Merge "yaml cleanup: trim multiline strings" 2018-03-10 07:01:35 +00:00
Chris Wedgwood
3a8c00764c yaml cleanup: trim multiline strings
Change-Id: I7e8f423be2efb84f3116258beca805265ca388f7
2018-03-08 20:18:53 +00:00
Steve Wilkerson
4f67560c5d Kibana: use endpoints section and lookups to set port
This PS moves kibana to use the endpoints section and lookups to
set the port it serves on.

Change-Id: I710428f92e80faf6ac5bb444f938447248e99217
2018-03-08 20:01:12 +00:00
Steve Wilkerson
d681396412 Address errors with Elasticsearch and Kibana
This moves Elasticsearch and Kibana to use the latest version
(6.2.2), as the images we were using are no longer supported with
the 6.x release.  There was a change in the doc reference in the
log entries that prevented the previous ES version from indexing
those entries, resulting in a busted gate.  Moving Kibana to 6.2.2
was required to match major/minor versions with Elasticsearch

The Elasticsearch version change also required changing config file
locations, changing the entrypoint used for launching the service,
changing the running user for the elasticsearch service, and
updated the ES tests as some of the API responses changed between
versions

This also required updating the elasticsearch template job as the
mapping definition entries changed between versions

Change-Id: Ia4cd9a66851754a1bb8f225c7e24513c43568e93
2018-03-08 10:27:06 -06:00
Pete Birley
3c101a6324 dependencies: move dynamic common deps under a 'dynamic.common' key
This PS moves existing dynamic common dependencies under a
'dynamic.common' key to simplify the yaml tree.

Change-Id: I4332bcfdf11197488e7bd5d8cf4c25565ea1c7b6
2018-02-24 17:42:10 -05:00
Pete Birley
e0c688d7ee dependencies: move static dependencies under a 'static' key
This PS moves static dependencies unser a 'static' key to allow
expansion to cover dynamic dependencies.

Change-Id: Ia0e853564955e0fbbe5a9e91a8b8924c703b1b02
2018-02-24 17:39:55 -05:00
Sean Eagan
641c79c902 Add deep merge utility to helm-toolkit
Adds "helm-toolkit.utils.merge" which is a replacement for the
upstream sprig "merge" function which didn't quite do what we
wanted, specifically it didn't merge slices, it just overrode
one with the other.  This PS also updates existing callsites
of the sprig merge with "helm-toolkit.utils.merge".

Change-Id: I456349558d4cf941d1bcb07fc76d0688b0a10782
2018-02-13 10:08:50 -06:00
Steve Wilkerson
e04be06ee3 Update kibana image to 5.6.4 tag
Updates the kibana version to match the elasticsearch version

Change-Id: I4ba2410ebf00ce2b269806f46f2c0a14652b71dc
2018-02-01 14:26:48 -06:00
Steve Wilkerson
3ec7f5f0ff Gate fix: httpd image, elasticsearch, openstack-exporter
There was a change in the upstream reference httpd image for
apache that changed how modules were built for apache.
This change adds the required fix to accomodate the change.
See isssue here https://github.com/docker-library/httpd/pull/87

The Elasticsearch image tag was updated to accomodate the kernel
versions used in the gate as part of the kernel update playbook
See https://github.com/elastic/elasticsearch/issues/28349#issuecomment-360233779

The openstack-exporter binary was changed to reflect changes made
to the openstack-exporter image

Change-Id: I1deb9e7cde794421dd33fade566c2a9fdb5007e6
2018-01-28 15:07:24 -06:00
Steve Wilkerson
b63afdd10c Run kibana behind apache
Run kibana behind apache as a reverse proxy to supply basic auth
for kibana, as xpack requires a suscription to support security
for kibana

Change-Id: I82168fc47fad29e26bcb02964709a04200dac467
2018-01-16 08:15:06 -06:00
Steve Wilkerson
d197c4f9a2 Run elasticsearch behind apache
Run elasticsearch behind apache as a reverse proxy to supply basic
auth for elasticsearch, as xpack requires a suscription to support
security for elasticsearch

Change-Id: I72d06ed9cd2179ead86ddc67db33c68a1e40c437
2018-01-16 08:14:47 -06:00
Steve Wilkerson
4ed181bc37 Include ingress entries in manifests key for kibana
Kibana was missing entries for enabling the ingress and ingress
service. This adds the entries in the manifests key for kibana

Change-Id: I12bdf0f2f82f7e666c8c058aacb798dbd22c3ff7
2018-01-03 12:22:33 -06:00
Steve Wilkerson
b45e8ddcbd Move kibana to OSH infra
This moves the Kibana chart to OSH infra, which finalizes moving
the logging components to OSH infra

Change-Id: Iacbfde8d5d7099fcb4dde8a437e030c2d4936de6
2017-12-26 18:26:58 -06:00