Commit Graph

61 Commits

Author SHA1 Message Date
KHIYANI, RAHUL (rk0850)
a43f479e6c Fix application name for grafana session sync
Implement helm-toolkit snippet for grafana add-home-dashboard
which adds security context template at pod/container

Change-Id: I12a5fd6c5043079f830eb36043f5b0ca495a3e93
2020-07-07 16:50:41 -05:00
Andrii Ostapenko
824f168efc Undo octal-values restriction together with corresponding code
Unrestrict octal values rule since benefits of file modes readability
exceed possible issues with yaml 1.2 adoption in future k8s versions.
These issues will be addressed when/if they occur.

Also ensure osh-infra is a required project for lint job, that matters
when running job against another project.

Change-Id: Ic5e327cf40c4b09c90738baff56419a6cef132da
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-07 15:42:53 +00:00
Andrii Ostapenko
83e27e600c Enable key-duplicates and octal-values yamllint checks
With corresponding code changes.

Change-Id: I11cde8971b3effbb6eb2b69a7d31ecf12140434e
2020-06-17 13:14:30 -05:00
diwakar thyagaraj
163c5aa780 Enable Apparmor to all osh-infra test pods
Also Changed container names to static.

Change-Id: I51f53b480d18aaa38a9707429f01052ee122e7e9
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-05-19 15:36:07 +00:00
Gage Hugo
d14d826b26 Remove OSH Authors copyright
The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.

This change removes all references to this copyright by the
non-existent group and any blank lines underneath.

Change-Id: I1882738cf9757c5350a8533876fd37b5920b5235
2020-05-07 02:11:15 +00:00
diwakar thyagaraj
aaeb0b1abb Enable Apparmor to Grafana Completed pods
This also adds init containers.

Change-Id: Ia70db208a1583b9a44a32d9a3d485ca7dc8a3ce2
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-05-05 15:59:22 +00:00
Radhika Pai
20aad64409 [Update] Grafana: add home dashboard script
Adding a loop to wait for the grafana dashboard to be up and contain the
OSH Home dashboard before running the script.
This should resolve the job completing before the OSH Home dashboard is
in the grafana pod.

Change-Id: I7ab20fad3ce7f7216e2b2679d863f02f97ef1ff4
2020-03-12 14:08:35 -05:00
Radhika Pai
dc9e435abb Grafana: Change to import the dashboards in json raw format
This code change is to enable grafana to use the raw json format
dashboards . This is to avoid the conversion of dashboards from json to
yaml and back to json during which the format is encountering issue.
Also this will help in adding new dashboards and maintaining the old
ones.
All the exisiting dashboards under values_override folder are updated to
use raw json format.

Change-Id: I48a7db1514857e082cecbb3b57deff9174509601
2020-03-04 09:52:36 -06:00
Radhika Pai
b0bb378a3c Grafana: Provision to add customized HomePage
This code will help to add any customized dashboard as a Home Page for
Grafana. The add_home_dashboard script will be executed after the
Grafana is deployed which sets a new Dashboard(OSH Home) as a landing
Page for a specific Organization.

Change-Id: I32b6b9cad4eaefe7d153cae797d3b3143be5c49b
2020-02-26 10:00:21 -06:00
dt241s@att.com
cc392aaa85 Add Apparmor to Grafana
Added apparmor feature gate and Zuul Gate Job

Change-Id: I9ce522f77447b1cb3f189ab7023c5c711e577618
2020-02-25 19:25:12 +00:00
Steve Wilkerson
3a6df3b544 Grafana: Remove default dashboards from chart
This removes the default dashboards from the Grafana chart and
instead places them in the values_overrides directory, similar to
what was done for the Prometheus rules. As Grafana dashboards
will likely be heavily dependent upon end-user needs, the old
default dashboard configs should only be used as a reference
instead of opinionated defaults that are difficult to override.
The previous defaults made using specialized labels for dashboard
variables difficult, as they were making dangerous assumptions
about deployed namespaces and host fqdns. By removing the defaults
entirely, end users can define their own dashboards to meet their
specialized needs

Change-Id: I7def8df68371deda0b75a685363c8a73b818dd45
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-12-09 13:39:13 +00:00
Steve Wilkerson
97e029e606 Grafana: Support multiple datasources
This updates the Grafana chart to support the definition of
multiple datasources. This moves to defining a template in the
chart's values.yaml file that allows for inline gotpl for
defining an arbitrary number of datasources. This also updates the
grafana dashboards to include a selector for the Prometheus
datasource to use via a drop down selector. This is vetted out in
the federated monitoring job

Change-Id: I55171fed5c2b343130d135d0b42bc96ff11c4712
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-11-22 14:45:04 +00:00
Steve Wilkerson
a4816feda2 Grafana: Add support for arbitrary environment variables
This updates the Grafana chart to support the definition of
arbitrary environment variables to support scenarios where
additional information may be required at runtime for things like
datasource and dashboard provisioning

Change-Id: I95e4abe9030116a440c6d78a1d14dbcaaf743b40
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-11-21 12:40:04 +00:00
Steve Wilkerson
1bfa091203 Grafana: Update version
This updates the Grafana version deployed by default from 5.0.0 to
6.2.0

Change-Id: I39b5405cc3f3fe7754ed6544a8388ff912a4ef58
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-11-18 08:49:38 -06:00
Steven Fitzpatrick
998885c330 Update Grafana Helm test to use python3
This change updates the selenium_tests container image
to one which installs python3.

The selenium-test.py template file has been refactored
to match the structure of the selenium tests in
/tools/gate/selenium

Depends on: https://review.opendev.org/688436
Change-Id: I4ece5c71df18c21f0cdff536140f63881ff24e30
2019-10-17 11:39:26 -05:00
Steve Wilkerson
d52fd14373 Use internal endpoint lookups for selenium helm tests
This updates the grafana and nagios helm test pod templates to
use the internal endpoints for their selenium tests instead of the
public endpoints when defined

Change-Id: I1138cb29a808894d3339bc1b07c3a60804b9546f
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-08-28 15:06:55 -05:00
Sreejith Punnapuzha
7314edc57d Fix Grafana helm test exception error
Grafana helm test is failing with the below error
"NameError: name 'exception' is not defined"
This is because exception is defined in smaller case. changing
exception to Exception fixes this issue

Change-Id: I533ae822babb4f063242fee1cd42b5b821519b5f
Signed-off-by: Sreejith Punnapuzha <Sreejith.Punnapuzha@outlook.com>
2019-07-08 14:02:42 +00:00
Steve Wilkerson
c7290b7ffe Grafana: Remove tests that query API
This removes the tests that query the Grafana API for checking
whether the prometheus datasource has been provisioned and for
checking the number of active dashboards against the number of
expected dashboards determined via the chart's values.yaml.

The reason for removing these is that Grafana can be configured
to use data source types beyond just Prometheus and additional
dashboards can be added to Grafana via the Grafana UI.  In cases
where dashboards are added via the Grafana UI, they are persisted
in the grafana database which will cause helm test failures during
upgrade scenarios.  Now that we have selenium tests executed as
part of the Grafana helm tests that validate Grafana is
functional, these API tests add little value

Change-Id: I9f20ca28e9c840fb3f4fa0707a43c9419fafa2c1
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-26 09:25:47 -05:00
Steve Wilkerson
b4b1dd9528 Add missing affinity keys to chart pod specs
This adds the affinity key to the pod spec for the grafana,
nagios, kube-state-metrics, and openstack-exporter charts as it
was previously missed

Change-Id: Ifefa88d7f33607b4d595effa5fbf72f3387e5081
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-13 19:15:42 +00:00
Steve Wilkerson
25e105f26a Grafana: Add Selenium tests to helm test pod
This adds selenium tests for the grafana chart to the helm test
pod to help ensure the Grafana deployment is functional and
accessible

Change-Id: Idc8d97e5111628d1ed4f25145086d54c5e0136e7
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-06-05 16:09:04 +00:00
Tin Lam
6bae1b020d Fix python template
This patch set removes an unused import that is not python3 compatible.

Change-Id: I360989c8eb23065d8e655d4583eb97338244412d
Signed-off-by: Tin Lam <tin@irrational.io>
2019-05-26 21:43:19 +00:00
Zuul
9388c2ba5a Merge "Grafana: Add security context to chart and read-only-fs" 2019-05-13 21:45:01 +00:00
Rahul Khiyani
d8ca55c685 Logs format standardization for LMA component
Added file name, line number and function name to logging message format
for troubleshooting purpose
    - This change is related to Grafana's session-db-sync job

Change-Id: Iaadbedfda0fd9cd7fe4b5c09fc05cb6181c400d1
2019-05-06 21:13:27 +00:00
RAHUL KHIYANI
e1c9a35230 Grafana: Add security context to chart and read-only-fs
This PS adds the security context macros to the grafana chart,
and moves the default to read-only-rootfs for all containers

Change-Id: Ie79e3bfc6af07b16cd53eddae17eceac3d9f8613
2019-04-23 03:22:21 +00:00
Pete Birley
2abf62ff4d OSH-Infra: Add emptydirs for tmp
This PS adds emptydirs backing the /tmp directory in pods, which
is required in most cases for full operation when using a read only
filesystem backing the container.

Additionally some yaml indent issues are resolved.

Change-Id: I8b7f1614da059783254aa6efc09facf23fca3cad
Signed-off-by: Pete Birley <pete@port.direct>
2019-04-20 20:50:59 +00:00
Steve Wilkerson
84f30ec103 Add release-annotation to pod spec, add missing annotations
This adds the release-annotation to the pod spec for the charts in
openstack-helm-infra. This also adds missing configmap annotations
to charts in openstack-helm-infra

Change-Id: Ie23f0c16a7a21d3929e98928db2bbcef69ae6490
2019-03-21 09:10:48 -05:00
Rahul Khiyani
33897b9a01 Grafana securityContext
securityContext with readOnlyRootFilesystem is implemented at container
level and leveraged the helm-toolkit snippet

Change-Id: I98ca4211e0e236beb3dfe0e11cf5bb10a91b16a6
2019-03-18 14:07:52 +00:00
Rahul Khiyani
28e0493955 readOnlyRootFilesystem: true for Grafana chart
Fix for adding readOnlyRootFilesystem flag at pod
level

Change-Id: Icf0244ca0e5c5eb1b96b17e8b7a64819d1093f0d
2019-03-07 23:05:25 +00:00
Steve Wilkerson
65ce9c73d7 Grafana: Add job to update admin password
This change adds a job to the Grafana chart that  allows for the
changing of the grafana admin user password if required, as
Grafana only allows the changing of this password via the
grafana-admin CLI or via an http call that requires both the old
and new password

Change-Id: I59a5d26edc4aa4da16e80c5454ecdebbae3a1d15
2019-02-12 09:59:45 -06:00
Steve Wilkerson
bf5840fa7a Grafana: Add container security context
This adds the container security context to grafana, which
explicitly sets allowPrivilegeEscalation to false

Change-Id: I3723a0c96699b9a517dafa2df08bf8cc916bf117
2019-01-03 16:19:03 -06:00
Steve Wilkerson
680f920312 Grafana: Add pod security context for grafana user
This updates the Grafana chart to include the pod security context
on the grafana pod. This changes the pod's user from root to the
grafana user instead

Change-Id: Id64853640f1941001b83566865defe93227b4291
2019-01-03 12:42:52 -06:00
Tin Lam
92e68d33ea Add network policy toolkit function
This patch set implements the helm toolkit function to generate a
kubernetes network policy manifest based on overrideable values.
This also adds a chart that shuts down all the ingress and egress
traffics in the namespace. This can be used to ensure the
whitelisted network policy works as intended.

Additionally, implementation is done for some infrastructure charts.

Change-Id: I78e87ef3276e948ae4dd2eb462b4b8012251c8c8
Co-Authored-By: Mike Pham <tp6510@att.com>
Signed-off-by: Tin Lam <tin@irrational.io>
2018-10-15 13:50:50 +00:00
Steve Wilkerson
bfa237d347 Charts: Update helm test pod templates
This updates the helm test pod templates in the charts with helm
tests defined. This change includes the addition of:

- Generate test pod cluster roles and role bindings
- Generate service accounts for test pods
- Add node selectors to the test pods
- Add service accounts to the test pods
- Addition of entrypoint container to the test pods
- Indentation fix for rabbitmq test pod template

Change-Id: I9a0dd8a1a87bfe5eaf1362e92b37bc004f9c2cdb
2018-10-09 21:00:00 +00:00
Pete Birley
bb3ff98d53 Add release uuid to pods and rc objects
This PS adds the ability to attach a release uuid to pods and rc
objects as desired. A follow up ps will add the ability to add arbitary
annotations to the same objects.

Change-Id: Iceedba457a03387f6fc44eb763a00fd57f9d84a5
Signed-off-by: Pete Birley <pete@port.direct>
2018-09-13 05:35:35 +00:00
Steve Wilkerson
9a311475ba Charts: Use secrets for configs in chart
This updates the osh-infra charts to use a secret for their
configuration files instead of a configmap, allowing for the
storage of sensitive information

Change-Id: Ia32587162288df0b297c45fd43b55cef381cb064
2018-08-24 15:56:53 -05:00
Steve Wilkerson
8652e14acb Add auth for prometheus
This adds authentication to Prometheus with an apache reverse
proxy, similar to elasticsearch, kibana and nagios. This adds an
admin user and password via htpasswd along with adding ldap
support.

This required modifying the grafana chart to configure the
prometheus datasource's basic auth credentials in the data sources
provisioning configuration file by checking whether basic auth is
enabled and injecting the username/password defined in the
corresponding endpoint definition.

This also modifies the nagios chart to use the authenticated
endpoint for prometheus, which is required for nagios to
successfully query the prometheus endpoint for its service
checking mechanism

Change-Id: Ia4ccc3c44a89b2c56594be1f4cc28ac07169bf8c
2018-08-08 18:49:45 +00:00
Seungkyu Ahn
a430533e6a Quoting node_select_value in Ingress Controller
In most cases, the ingress controller's nodeSelector key and value
are "node-role.kubernetes.io/ingress" and "true".
Using quote to treat the nodeSelector value as a string.

Change-Id: Ie1745629b90795e4d888d85f35565e6d6350e09b
2018-08-01 02:39:05 +00:00
Steve Wilkerson
dc16a897d7 Add missing labels to helm test pods
This adds missing labels to the helm test pods in osh-infra

Change-Id: I618d9089bfde2d847411f5f876f0ff6afd9cce7f
2018-07-10 08:55:40 -05:00
Steve Wilkerson
c26a1b53f6 Update TLS secret templates, remove nagios readiness probe
This updates the TLS secret templates to include the backend
service in the dict supplied to the manifest template, as it is
required for the TLS secret to render correctly.

This also removes the readiness probe from the nagios container in
the deployment for the nagios chart, as it wasn't functioning as
intended due to the port not being available for the probe

Change-Id: Iabcfd40c74938e0497d08ffeeebc98ab722fa660
2018-06-27 18:56:45 -05:00
Zuul
714bc3e6da Merge "Ingress: Add initial TLS Support for osh-infra public endpoints" 2018-06-26 23:07:28 +00:00
Steve Wilkerson
b823954787 Ingress: Add initial TLS Support for osh-infra public endpoints
Adds support for TLS on overriden fqdns for public endpoints for
the services that have them in openstack-helm-infra. Currently this
implementation is limited, in that it does not provide support for
dynamically loading CAs into the containers, or specifying them manually
via configuration. As a result only well known or CA's added manually
to containers will be recognised.

Change-Id: I4ab4bbe24b6544b64cd365467e8efb2a421ac3f4
2018-06-26 14:47:19 -05:00
Steve Wilkerson
68fa1d6fbe Grafana: Provision data sources via dynamic template in values
This moves to define the datasources provisioned by grafana via
a template defined in the values.yaml. This allows us to define
multiple datasource types that can be mapped directly to the
corresponding entries in endpoints, which enables us to generate
the data source urls via endpoint lookups rather than hardcoding
this. This is the first step to support multiple data sources in
a singular grafana deployment

Change-Id: Iac7f4b1e07aaf83ae4d2a0c923cd06817f0d8c0d
2018-06-26 13:57:46 -05:00
Steve Wilkerson
497959371d Grafana: Update LDAP configuration, update volume mounts
This updates the LDAP configuration for grafana, using a template
defined in the values.yaml file. Using the template allows us to
dynamically define LDAP configuration values, such as the bind dn,
search base and group search base paths, the password, and the
LDAP fqdn.  This also updates the volume mount for the
provisioning directory to be defined by the configuration value in
the values.yaml file

Change-Id: I1e4866d1189cf40b08b3443dc725646a1b76094c
2018-06-26 07:36:15 -05:00
Pete Birley
abb00e97fd Gotpl: remove quote and trunc to suppress output
This PS removes the use of the `quote and truncate` approach to
suppress output from gotpl actions in templates and replaces it
with the recommended practice of defining `$_` instead.

Change-Id: I5fedc3471dcbecef37d2fe1302bf9760b3163467
Signed-off-by: Pete Birley <pete@port.direct>
2018-06-16 16:37:08 -05:00
Pete Birley
fa629cdbbd Daemonsets: Use current kubernetes daemonset api version
This PS moves to use the current ga version for kubernetes daemonsets,
additionally any remaining deployments that were using the
`extensions/v1beta1` have been updated to `apps/v1`.

Story: 2002205
Task: 21735

Change-Id: If9703162dc472af1e6096bf2b9062802fd5ce8ab
Signed-off-by: Pete Birley <pete@port.direct>
2018-06-13 21:53:18 +00:00
Steve Wilkerson
de9c46bcfa Charts: Tidy up openstack-helm-infra charts
This moves the charts in openstack-helm-infra closer towards a
standard structure. It addresses multiple deviations, including:
missing resources for init containers, incorrect indents for
disabled resources in some charts, incorrect indents for volumes
and volumemounts added via values, missing resources for some
helm test templates, missing helm-toolkit image functions, and
moving the resource template declarations to be under the image
template declarations

Change-Id: I4834a5d476ef7fc69c5583caacc0229050f20a76
2018-05-21 12:58:22 -07:00
Steve Wilkerson
e081c19fe8 Add ldap support to grafana, update version, add helm tests
This adds ldap support to the grafana chart. This required updating
the version of Grafana to 5.0, as this version allows for using
configuration files to bootstrap the datasources and dashboards
instead of using the grafana http api. This was a necessary change
as using ldap for grafana presented issues trying to create the
datasource via the http api

This also adds a basic helm test for grafana. This test simply
verifies whether the prometheus datasource configured exists and
whether the number of dashboards reported by the admin api matches
the number of dashboards expected

Change-Id: I2e987cb425adba9f909722ffdb25b83f82710c4d
2018-05-15 01:42:04 +00:00
Steve Wilkerson
e166432a98 Add manifest for image_repo_sync job
This ps proposes adding a common template for the image_repo_sync
jobs for consumption by the charts

Change-Id: I48476d1e4fd94bd1b08b13b46983e3d999f8d8ca
2018-04-19 14:10:08 +00:00
Zuul
49e9084679 Merge "OSH-Infra: Update labels for chart components" 2018-04-18 18:47:08 +00:00
Zuul
626b94e0c8 Merge "Helm-Toolkit: Kubernetes Entrypoint, simplify image dependencies" 2018-04-17 15:11:00 +00:00