236 Commits

Author SHA1 Message Date
Tin Lam
d239c70152 fix(yaml): correct bad annotations
In a recent apparmor security patch [0], additional annotations were
added to the cronjobs that were incorrectly indented. While helm v2
seems fairly tolerant and ignores these errors, running this usig helm
v3 seems to cause rendering problems as we are placing incorrect key
and value pair into the spec: field. This patch set corrects this.

[0] https://review.opendev.org/#/c/725727/8

Change-Id: I9aae94bc0a68318b2c16fedbc973f7a0a2a3729e
Signed-off-by: Tin Lam <tin@irrational.io>
2020-05-26 11:18:59 +00:00
diwakar thyagaraj
a61050c2b3 Enable Apparmor to init containers for Heat Components
Also added Ingress apparmor Fix.

Change-Id: I6f4a1e6778b16c855072c0d6583e61af86f252a6
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-05-07 08:30:28 -05:00
Gage Hugo
db79e79788 Remove OSH Authors copyright
The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.

This change removes all references to this copyright by the
non-existent group and any blank lines underneath.

Change-Id: Ia035037e000f1bf95202fc07b8cd1ad0fc019094
2020-04-03 20:53:32 +00:00
sg774j
f76a3f5e0a Heat: Add ingress network policy overrides
This patch set adds in default heat ingress overrides.

Change-Id: I4e1f2b6687a05f2bf3ca91c941c2cf11abe68f07
2020-03-19 09:08:14 +00:00
dt241s@att.com
bcb6f71894 Enable Apparmor to Heat Components
Change-Id: I706b7503520b80bf5a4f0f34cb1be3298e2ad4ca
2020-03-02 22:59:19 +00:00
Tin Lam
2aa32665b4 Add train release support
This patch set adds in job to test the OpenStack train releases.

Depends-On: https://review.opendev.org/#/c/706456/
Change-Id: I89fef1264f68dab7e921a9e5503c29d6a051f342
Signed-off-by: Tin Lam <tin@irrational.io>
2020-02-28 20:19:58 +00:00
Gage Hugo
f9dbba7043 Revert "Revert "Keystone Authtoken Cache: allow universal secret key to be set""
This reverts commit 90d070390db08abf9da42a2bac54397112bbcd48.

Change-Id: I017c6e9676b872e1aab21f9dc8aa2f93db58d49f
2020-02-21 11:16:55 -06:00
Vasyl Saienko
90d070390d Revert "Keystone Authtoken Cache: allow universal secret key to be set"
This reverts commit 1c85fdc390e05eb578874e77fad9d4ec942da791.

Do not use randomly generated strings in configmaps as this leads to
whole helm release redeployment even no values are changed. The random
items have to be generated outside of helm chart and provided via
values.
Also previous behaviour didn't allow to use cache during rolling upgrade
as new pods were spawned with new key.

Change-Id: I423611b18fca0d65e2e721a9c6a0c3d8df0813d2
2020-02-12 11:18:06 +00:00
Tin Lam
12bee1bb97 Migrate default release to Stein
This patch set updates the default job to use OpenStack Stein release.
The previously default Ocata release will be place in separate job.

Change-Id: I489324f762a179a2cab5499a6d8e57e97c81297f
Signed-off-by: Tin Lam <tin@irrational.io>
2020-01-09 10:00:31 -06:00
Tin Lam
5057052c70 Fix feature gate envvar overriding
Currently using envsubst to perform substitution of value overrides in
the feature gate caused conflicts as gotpl gets templated into those
overrides. This adds in '%%%REPLACE_${var}%%%' and uses sed to perform
the substitution instead to address the issue.

This is to achieve parity with OSH-infra patch in [0].

[0] https://review.opendev.org/#/c/697749/

Depends-On: https://review.opendev.org/#/c/697749

Change-Id: I3ed504c65900e7b84728019f3acdf706a40c0427
Signed-off-by: Tin Lam <tlam@omegaprime.dev>
2019-12-17 09:49:38 +00:00
Tin Lam
a25eccb7cb Implements egress network policy
This patch set adds in the egress policy for core OpenStack Services.

Depends-On: https://review.opendev.org/#/c/679853/

Change-Id: I585ddabcbd640db784520c913af8eddecaee3843
Signed-off-by: Tin Lam <tlam@omegaprime.dev>
2019-11-22 01:16:49 +00:00
Steve Wilkerson
9736f5f544 Update kubernetes-entrypoint image reference
This updates the kubernetes-entrypoint image reference to consume
the publicly available kubernetes-entrypoint image that is built
and maintained under the airshipit namespace, as the stackanetes
image is no longer actively maintainedy

Depends-On: https://review.opendev.org/688435

Change-Id: I8e76cdcc9d4db8975b330e97169754a2a407341f
Signed-off-by: Steve Wilkerson <sw5822@att.com>
2019-10-21 13:58:22 +00:00
Gage Hugo
c3e085b800 Add network policy nonvoting checks
This change adds two network policy zuul checks, one for the compute-kit,
and one for cinder/ceph, to test network policy for each OpenStack
service. These checks will be non-voting initially.

The network policy rules for each service will initially allow all
traffic. These ingress/egress rules will be defined in future changes
to only explicitly allow traffic between services that are explicitly
allowed to communicate, other traffic will be denied.

Depends-On: https://review.opendev.org/#/c/685130/

Change-Id: Ide2998ebb2af2832f24ca7abc398a82e4a6d70e3
2019-09-26 11:57:15 -05:00
Pavlo Shchelokovskyy
688ded096d Depend on identity service in heat boostrap job
this job is actually trying to create a role in Keystone,
and thus needs it running to complete.
Without such dependency depending on cluster performance it may fail
by exhausting retry attempts.

Also depend on this bootstrap job in all Heat services so that they
are actually able to work and create arbitrary stacks once finished deploying.

Change-Id: I94ce96591b1f02d64d15c38686e9bc8bae31ddbb
2019-09-03 11:23:07 +03:00
Pete Birley
59a017d834 RabbitMQ: Dont mirror reply queues
This PS updates the default RMQ policy to not mirror reply queues
as they cause signifigant blocking when resorting a rabbit node to
a cluster, with no advantage.

Change-Id: I6f8d4eaa482fcdf3e877bd38caa9b24358ea5be0
Signed-off-by: Pete Birley <pete@port.direct>
2019-07-30 09:54:16 -05:00
Pete Birley
74e4474ec6 OpenStack: Check Stein release on Ubuntu Bionic
This PS adds checks for the Stein Release of OpenStack in Ubuntu Bionic
containers.

Depends-On: https://review.opendev.org/667726

Change-Id: Icfad3434ca496a841993b95adaf5d853728d920f
Signed-off-by: Pete Birley <pete@port.direct>
2019-07-03 00:26:30 +00:00
Pete Birley
e5f8fcf728 Container Distro: Add checks for OS (rocky) in Ubuntu Bionic
This PS adds checks for running the Rocky release of Openstack under
Python3 in Ubuntu Bionic containers.

Change-Id: I269cef9f8f157e22f6b857822df9a8960dac6ea8
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-26 10:36:26 -05:00
Pete Birley
6606c8bc2e OpenStack: Check Rocky release on Ubuntu Xenial
This PS adds checks for the Rocky Release of OpenStack in Ubuntu Xenial
containers.

Change-Id: Ieed4a6a3afa6e3ebd9b2f72ba227aac891d65214
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-25 21:14:28 +00:00
Pete Birley
ffb24e337c OpenStack: Check Queens release on Ubuntu Xenial
This PS adds checks for the Queens Release of OpenStack in Ubuntu Xenial
containers.

Change-Id: I0d4d427e43f06fa955dfd275859939d0adca113c
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-25 21:10:26 +00:00
Pete Birley
184b3e4326 OpenStack: Check Pike release on Ubuntu Xenial
This PS adds checks for the Pike Release of OpenStack in Ubuntu Xenial
containers.

Change-Id: I402584bbcdd53a4a6bc21f370586b3498142bf81
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-22 11:20:55 +00:00
Pete Birley
9bcf0df94c Messaging: use htk function to directly hit RabbitMQ servers
This PS updates the charts to use the htk function recently introduced
to allow oslo.messaging clients ans servers to directly hit their
backends rather than using either DNS or K8S svc based routing.

Depends-On: I5150a64bd29fa062e30496c1f2127de138322863

Change-Id: I458b4313c57fc50c8181cedeca9919670487926a
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-18 21:47:45 +00:00
Zuul
cd460f12c2 Merge "Rafactoring volume mount variables in db sync job" 2019-06-18 18:24:18 +00:00
Pete Birley
d0b135cd77 AMPQ: update ha policy regex
Change-Id: I2f023c2e41a52b5753cdb77e93c9e876bc60a87d
Signed-off-by: Pete Birley <pete@port.direct>
2019-06-11 12:17:22 -05:00
Gage Hugo
976cab856c Create separate users for helm test
Currently each service uses the same name for their helm test user,
"test". While this works when services are ran sequentially, when
multiple services are deployed and tested at the same time, it can
lead to a race condition where one service deletes the user before
the other is done testing, causing a failure.

This change makes it so that each service defines its own test user
in the form of [service]-test.

Change-Id: Idd7ad3bef78a039f23fb0dd79391e3588e94b73c
2019-06-03 11:26:18 -05:00
John Haan
0ea9be7ade Rafactoring volume mount variables in db sync job
This patch make the db sync job template follows the same pattern
that other templates utilize the variables to make in a predictable
pattern.

Change-Id: Idbedd046c6b4fd001cf63004ffac792173a5778b
Story: 2005754
Task: 33457
2019-05-22 17:47:03 +09:00
Zuul
f8adab245b Merge "Point to OSH-images images" 2019-05-18 19:12:58 +00:00
Jean-Philippe Evrard
1d335146fa Point to OSH-images images
We now have a process for OSH-images image building,
using Zuul, so we should point the images by default to those
images, instead of pointing to stale images.

Without this, the osh-images build process is completely not
in use, and updating the osh-images process or patching its
code has no impact on OSH.

This should fix it.

Change-Id: I672b8755bf9e182b15eff067479b662529a13477
2019-05-13 10:58:02 +02:00
Roy Tang (rt7380)
5df6fa3789 Expose Anti-Affinity Weight Setting.
Add weight default setting to anti-affinity.

Depends-on: Id8eb303674764ef8b0664f62040723aaf77e0a54
Change-Id: I09f96522cddf3a77dae73daca4557877eda5df50
2019-05-10 22:05:24 -05:00
Pete Birley
4550ea9376 Heat: Fix cadf values key
This PS fixes the heat cadf values key, to use the correct value.

Change-Id: I3efeccc2ba9bbebd7dc4b175244f00173c39d1ef
Signed-off-by: Pete Birley <pete@port.direct>
2019-05-10 14:11:28 -05:00
Zuul
7f95467e3d Merge "Replace git.openstack.org URLs with opendev.org URLs" 2019-05-01 16:11:28 +00:00
Zuul
5361c3282a Merge "Add OpenSUSE Leap15 testing" 2019-04-26 16:44:16 +00:00
caoyuan
cb77d3adff Replace git.openstack.org URLs with opendev.org URLs
Change-Id: I9a7bcee8727cb127d57ccb4dce1183895a4130cd
2019-04-25 00:37:57 +08:00
Pete Birley
623c131292 OSH: Add emptydirs for tmp
This PS adds emptydirs backing the /tmp directory in pods, which
is required in most cases for full operation when using a read only
filesystem backing the container.

Additionally some yaml indent issues are resolved.

Change-Id: I9df8f70e913b911ff755600fa2f669d9c5dcb928
Signed-off-by: Pete Birley <pete@port.direct>
2019-04-20 08:55:44 -05:00
Jean-Philippe Evrard
a828d38316 Add OpenSUSE Leap15 testing
There is currently no testing of the Leap 15 images in OSH.

This addresses it by:
- Using the values_overrides folder according to the multi-os
  spec, creating value override files there for changes that
  needs to happen on Leap 15 images.
- Point to the right images using the previously created folder,
  to allow using those in CI easily.
- Change CI to use previously created overrides.

Depends-On: https://review.openstack.org/#/c/651501
Change-Id: I520d3676195c62b253a19397c86b0d0fbabee710
2019-04-15 11:15:35 +02:00
Zuul
a187084bdd Merge "Heat: Make it optional to use log_config_append option" 2019-04-14 18:34:46 +00:00
Pete Birley
3f83d6978c Heat: Make it optional to use log_config_append option
This PS enables the use of simple logging options if desired.

Change-Id: I0278cefeaa46a39a893ba1fdc9f4c4b633a8866b
Signed-off-by: Pete Birley <pete@port.direct>
2019-04-12 15:37:36 -05:00
Gage Hugo
9c13e80492 Enable audit pipeline for heat
This change adds the keystonemiddleware audit paste filter[0]
and enables it for the heat-api, heat-cfn, and heat-cloudwatch
services. This provides the ability to audit API requests
for heat.

[0] https://docs.openstack.org/keystonemiddleware/latest/audit.html

Change-Id: Ib5a7dfd882416553ff6f43aa009e3e67871d7f4c
2019-04-11 12:38:06 -05:00
Zuul
280c5f192f Merge "Fix globbing" 2019-04-03 13:42:20 +00:00
Pete Birley
ca104ef11e Heat: make rabbitmq queues ha
This PS tells heat to make rabbitmq queues ha when available.

Change-Id: I40222a4e6a52527a1336ab2a6e58144d62a30269
Signed-off-by: Pete Birley <pete@port.direct>
2019-03-29 20:04:57 +00:00
Tin Lam
3a4c1665eb Fix globbing
Trivial fix to follow up on comment from [0].

[0] https://review.openstack.org/#/c/638249/4

Change-Id: Ia53dd43a5a237f466909649e69889b41064db933
Signed-off-by: Tin Lam <tin@irrational.io>
2019-03-27 22:45:31 -05:00
David Sullivan
0107af60bf Purge deleted items from heat database
Long running operations (for example autoscale stacks) can lead to heat
database growth over time. This will remove entries that have been soft
deleted from the heat database.

This adds a cron job that will call heat-manage purge_deleted every 24h.

Change-Id: I3b7c174cc7ed147a8f5700135d3da2e63696008d
Story: 2005020
Task: 29499
2019-03-25 13:47:15 -04:00
pd2839
e7b06a4076 Implement Security Context for Heat
Implement container security context for the following Heat resources:
 - Heat server deployment

Change-Id: I4c1efb0807c56f750cc8d74155b96f06735e5051
2019-03-22 21:30:57 +00:00
Jiří Suchomel
a2a5dda47c Added volume related keys to pod mounts to ease the overriding
If user wants to add an extra volumeMounts/volume to a pod,
amd uses override values e.g. like this

pod:
  mounts:
    nova_placement:
      init_container: null
      nova_placement:
        volumeMounts:
          - name: nova-etc
          ...

helm template parser complains with

Warning: The destination item 'nova_placement' is a table and ignoring the source 'nova_placement' as it has a non-table value of: <nil>

So when we create empty values for such keys in values.yaml, the source
will be present and warning does not need to be shown.

Change-Id: Ib8dc53c3a54e12014025de8fafe16fbe9721c0da
2019-03-15 16:29:19 +00:00
pd2839
294866a81c readOnlyFilesystem: true for heat chart
fix for adding readOnlyFilesystem flag at pod level

Change-Id: I014cf0f9c6c19e900d3c210a7f52b4e941bc46e7
2019-03-07 05:09:28 +00:00
Zuul
1ad5467252 Merge "Increase default logging" 2019-03-06 04:09:58 +00:00
Jean-Philippe Evrard
5890ebf4f8 Increase default logging
The current helm chart defaults drops logs of any warnings
(and above) for any logger outside of the namespace
of the deployed chart.

This is a problem, as logging could reveal important information for
operators. While this could be done with a value override, there
is no reason to hide warning, errors, or critical information that
are happening in the cycle of the operation of the software
deployed with the helm charts. For example, nothing would get
logged in oslo_service, which is a very important part of running
OpenStack.

This fixes it by logging to stdout all the warnings (and above)
for OpenStack apps.

Change-Id: I16f77f4cc64caf21b21c8519e6da34eaf5d31498
2019-02-28 09:53:01 +00:00
Zuul
66ba0be81c Merge "Explicitly set datefmt for logging" 2019-02-27 01:31:09 +00:00
Zuul
c869b4ef4a Merge "Add startingDeadlineSeconds field to cronJobs" 2019-02-23 21:18:26 +00:00
sg774j
93658c75ec enable_proxy_headers_parsing to various services.
Adding this parameter to Cinder, Heat, Glance,and Neutron
config. Adding this parameter allows proper handling to resource
links in response using API services behind https proxy.

Change-Id: Ib99a16b6252b15d9f138417485731ec401cb8f81
2019-02-21 15:01:45 +00:00
Pavlo Shchelokovskyy
55645c7e73 Explicitly set datefmt for logging
the defaults in Python [0] and oslo.log [1] are such that when using
separate config file for logging configuration (log-config-append)
the log fomat of dates containes miliseconds twice (as in sec,ms.ms)
which is exactly what is currently seen in logs of OpenStack services
deployed by openstack-helm.

When not provided with datefmt log formatter option, Python effectively
uses '%Y-%m-%d %H:%M:%S,%f' [0] as a default time formatting string to
render `%(asctime)s`, but the defaults in oslo.log add another `.%f`
to it [1].

Since `log-date-format` oslo.log option has no effect when using
log-config-append, we need to explicitly set date format to avoid double
miliseconds rendering in date of log entries.

[0] 6ee41793d2/Lib/logging/__init__.py (L427-L428)
[1] http://git.openstack.org/cgit/openstack/oslo.log/tree/oslo_log/_options.py?id=7c5f8362b26313217b6c248e77be3dc8e2ef74a5#n148

Change-Id: I47aa7ce96770d94b905b56d6fe4abad428f01047
2019-02-21 08:28:35 +00:00