708 Commits

Author SHA1 Message Date
Zuul
466858cc5d Merge "docker: Run mistral-db-manage populate at step 5" 2017-11-05 15:50:17 +00:00
Zuul
0b1c2b516e Merge "Enable keystone db sync to log to stdout" 2017-11-04 06:09:16 +00:00
Zuul
c458b7c2ff Merge "Set ipc=host for services attaching encrypted volumes" 2017-11-04 03:10:57 +00:00
Zuul
3b17b3fccd Merge "Only mount Haproxy TLS if contents are set" 2017-11-03 08:25:34 +00:00
Zuul
9bd858ebdf Merge "HAProxy: Mount the CA certificate if TLS everywhere is enabled" 2017-11-03 02:53:25 +00:00
Zuul
274a38d15f Merge "Fix iptables rules override bug in clustercheck docker service" 2017-11-02 19:08:02 +00:00
Juan Antonio Osorio Robles
56ce14134d Only mount Haproxy TLS if contents are set
For non-pacemaker deployments, this mounts the TLS certificate only if
it's actually going to be used.

Change-Id: Id8ba09902d25689e642f922c43e71649977bf248
2017-11-02 08:13:17 +02:00
Juan Antonio Osorio Robles
3de6bc039c HAProxy: Mount the CA certificate if TLS everywhere is enabled
In non-pacemaker containerized setups, the CA certificate wasn't mounted
on HAProxy. It's needed.

Change-Id: If289ad457465a674a232ec6207f69f201ba78251
2017-11-02 08:04:56 +02:00
Eric Harney
05b6147246 Set ipc=host for services attaching encrypted volumes
Without ipc=host set, cryptsetup/devicemapper will never
see devices created when running "cryptsetup luksOpen",
causing the command to hang.

This is required for attaching encrypted Cinder volumes.

Closes-Bug: #1729419
Change-Id: Ic7184b1fbbafea266f8ec1e7974d0a4a2cf4d750
2017-11-02 05:33:47 +00:00
Zuul
d6a2160b82 Merge "Force memcached container log to file" 2017-11-01 04:58:21 +00:00
Zuul
1c0909939b Merge "Add --detailed-exitcodes when running puppet via ansible" 2017-11-01 02:17:07 +00:00
Michele Baldessari
3df6a4204a Fix iptables rules override bug in clustercheck docker service
When deploying a composable HA overcloud with a database role split off
to separate nodes we could observe a deployment failure due to galera
never starting up properly.

The reason for this was that instead of having the firewall rules for
the galera bundle applied (i.e. those with the extra control-port for
the bundle), we would see the firewall rules for the BM galera service.
E.g. we would see the following on the host:

tripleo.mysql.firewall_rules: {
  104 mysql galera: {
    dport: [ 873, 3306, 4444, 4567, 4568, 9200 ]

Instead of the correct mysq bundle firewall rules:
tripleo.mysql.firewall_rules:
  104 mysql galera-bundle:
    dport: [ 873, 3123, 3306, 4444, 4567, 4568, 9200 ]

The reason for this is the following piece of code in
https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/pacemaker/clustercheck.yaml#L62:
...
  MysqlPuppetBase:
    type: ../../../puppet/services/pacemaker/database/mysql.yaml
    properties:
      EndpointMap: {get_param: EndpointMap}
      ServiceData: {get_param: ServiceData}
      ServiceNetMap: {get_param: ServiceNetMap}
      DefaultPasswords: {get_param: DefaultPasswords}
      RoleName: {get_param: RoleName}
      RoleParameters: {get_param: RoleParameters}

outputs:
  role_data:
    description: Containerized service clustercheck using composable services.
    value:
      service_name: clustercheck
      config_settings: {get_attr: [MysqlPuppetBase, role_data, config_settings]}
logging_source: {get_attr: [MysqlPuppetBase, role_data, logging_source]}
...

Depending on the ordering of the clustercheck service within the role
(before or after the mysql service), the above code will override the
tripleo.mysql.firewall_rules with the wrong rules because we derive from
puppet/services/... which contain the BM firewall rules.

Let's just switch to derive from the docker service so we do not risk
getting the wrong firewall rules during the map_merge.

Tested this change successfully on a composable HA with split-off DB
nodes.

Change-Id: Ie87b327fe7981d905f8762d3944a0e950dbd0bfa
Closes-Bug: #1728918
2017-10-31 13:23:17 +01:00
Juan Antonio Osorio Robles
543f8463e6 Enable keystone db sync to log to stdout
This tells the db sync to use stdout instead of a specific log file
when stdout logging is enabled

bp logging-stdout-rsyslog

Depends-On: Id9e8c641a6b00725d2f5c9623b05854a1b4e2af2
Change-Id: I25d15aac6adfab1dfd11d558404930736aace977
2017-10-31 09:19:03 +00:00
Juan Antonio Osorio Robles
ca1fc58486 Force memcached container log to file
We were relying on the sysconfig options to set the memcached log file,
however, this is not happening, as the redirection is being taken as an
option and ends up being ignored by the memcached command. So instead,
we set the redirection in the container template.

Change-Id: Ic94e3fd7884d518eb9558c53acdc6b294823cd0a
Closes-Bug: #1720183
2017-10-31 09:08:40 +02:00
Dan Prince
a78166e1ea docker: Run mistral-db-manage populate at step 5
This resolves issues in loading some of Mistral openstack
actions which appears to require endpoints to be running first
before fake clients can be initialized properly.

Change-Id: Ia588e5ec8880da1df95a33696b5b8c9e72ac49e2
Closes-bug: #1728682
2017-10-30 14:34:04 -04:00
Zuul
a0991433fa Merge "Move keystone init log container to files logging template" 2017-10-29 12:28:46 +00:00
Zuul
8254d0b396 Merge "Enable Cinder as a backend for Glance" 2017-10-29 12:28:44 +00:00
Zuul
a584e3107a Merge "persist memcached logs in /var/log/containers/memcached/memcached.log" 2017-10-28 09:22:34 +00:00
Zuul
84b67abd73 Merge "Remove log-dir option from neutron-dhcp execution" 2017-10-28 05:41:35 +00:00
Zuul
b13a5e9461 Merge "Add option for nova containers to log to stdout/stderr" 2017-10-28 05:41:33 +00:00
Zuul
ed874058be Merge "glance logging: Fix description and use yaml anchor" 2017-10-28 05:41:32 +00:00
Zuul
a073c02d73 Merge "Add option for Glance API containers to log to stdout/stderr" 2017-10-28 05:29:23 +00:00
Zuul
9be29684bd Merge "Update CephPools format in the docker templates to fit ceph-ansible" 2017-10-27 11:15:17 +00:00
Juan Antonio Osorio Robles
5020f38301 persist memcached logs in /var/log/containers/memcached/memcached.log
We used to bind-mount /var/log/memcached.log, but this resulted in the
file being createdin the memcached container as a directory, since this
file didn't exist.

This commit takes the approach of other containers and gets the logs to
a memcached directory in /var/log/containers.

Change-Id: I926b65fa557ad56b4faa2be34452b58f7b01247a
Closes-Bug: #1720183
2017-10-27 10:33:01 +03:00
Juan Antonio Osorio Robles
85c86069d4 Remove log-dir option from neutron-dhcp execution
oslo.log already sets a file name based on the executable. Removing
this makes it easier for us to configure logging to stdout.

bp logging-stdout-rsyslog

Change-Id: Ib754162b01a3c22a826e549a97af0d59dfd8895b
2017-10-27 05:58:34 +00:00
Juan Antonio Osorio Robles
1e854d613d Add option for nova containers to log to stdout/stderr
This adds the option to get the nova containers to log to stdout.
The option is disabled by default.

If enabled, for nova-api and placement it also adds a sidecar
container that reads the apache access logs.

bp logging-stdout-rsyslog

Depends-On: I59d02fe8731c20c64ca389000f12c78cfc1f12be
Change-Id: I8137d61f2d4352d4d8055e93a30511cf1aeaa6b0
2017-10-27 05:58:28 +00:00
Juan Antonio Osorio Robles
dcc415e4f9 glance logging: Fix description and use yaml anchor
This cleans up the file logging template by fixing up some descriptions
that were left written wrongly and using an anchor to specify the
volumes to avoid repetition.

bp logging-stdout-rsyslog

Change-Id: If0daf6a0fd47cc1e9bb8c7bef6f8f702096e8152
2017-10-27 08:26:53 +03:00
Juan Antonio Osorio Robles
56140c57f0 Move keystone init log container to files logging template
It shouldn't really be needed for stdout logging.

bp logging-stdout-rsyslog

Change-Id: I685c6fc57a734bfc1068946d769888ccf7950a0d
2017-10-27 08:18:59 +03:00
Zuul
6754ba1435 Merge "Add option for keystone containers to log to stdout/stderr" 2017-10-26 21:06:45 +00:00
Michele Baldessari
11e599d116 Add --detailed-exitcodes when running puppet via ansible
puppet run on never fails, even when it should, since we moved
to the ansible way of applying it. The reason is the current following code:

    - name: Run puppet host configuration for step {{step}}
      command: >-
        puppet apply
        --modulepath=/etc/puppet/modules:/opt/stack/puppet-modules:/usr/share/openstack-puppet/modules
        --logdest syslog --logdest console --color=false
        /var/lib/tripleo-config/puppet_step_config.pp

The above is missing the --detailed-exitcodes switch and so puppet will never
really error out on us and the deployment will keep on running all the
steps even though a previous puppet manifest might have failed. This
cause extra hard-to-debug failures.

Initially the issue was observed on the puppet host runs, but this
parameter is missing also from docker-puppet.py, so let's add it there
as well as it makes sense to return proper error codes whenever we call
puppet.

Besides this being a good idea in general, we actually *have* to do it
because puppet does not fail correctly without this option due to the
following puppet bug:
https://tickets.puppetlabs.com/browse/PUP-2754

Depends-On: I607927c2ee5c29b605e18e9294b0f91d37337680
Co-Authored-By: Damien Ciabrini <dciabrin@redhat.com>

Change-Id: Ie9df4f520645404560a9635fb66e3af42b966f54
Closes-Bug: #1723163
2017-10-26 20:03:11 +00:00
Zuul
a24f1d436c Merge "Run containerized mistral-api eventlet" 2017-10-26 09:02:20 +00:00
Juan Antonio Osorio Robles
f037da0f36 Add option for Glance API containers to log to stdout/stderr
This adds the option to get the glance containers to log to stdout.
The option is disabled by default.

bp logging-stdout-rsyslog

Depends-On: I3fa4a38d21f0f7e447157ab7814a547c10a4b7d3
Change-Id: I81101d9ef4df2e6ece7c17025e2265489864a6f6
2017-10-25 07:17:23 +00:00
Juan Antonio Osorio Robles
e5f0113f6f Add option for keystone containers to log to stdout/stderr
This adds the option to get the keystone containers to log to stdout.
The option is disabled by default.

If enabled, It also adds a sidecar container that reads the apache
access logs.

bp logging-stdout-rsyslog

Depends-On: I4250ebce75933c8fb3f85b9efdb3e2ade392a60c
Change-Id: Ibb633731a60a16d595d10d38a79ec284da18c5c2
2017-10-25 07:17:08 +00:00
Giulio Fidente
c10aa7a043 Update CephPools format in the docker templates to fit ceph-ansible
The format which ceph-ansible uses to describe the list of pools
to be created in the cluster is different from the one which
puppet-ceph uses; this commit updates the description and the
the docker templates accordingly.

Change-Id: I1e5b2c3cbf6ae02c19a2275ca119fed6e173319d
Closes-Bug: #1720373
2017-10-25 09:00:44 +02:00
Zuul
2a7efbada6 Merge "Remove rsync from xinetd when upgrading to containerized deployment" 2017-10-25 03:14:27 +00:00
Martin André
e158acb14c Run containerized mistral-api eventlet
The mistral-api container image we use doesn't have the necessary
packages to run via wsgi and this cause puppet to error with:

  "Notice: /Stage[main]/Mistral::Wsgi::Apache/Openstacklib::Wsgi::Apache[mistral_wsgi]/File[mistral_wsgi]: Dependency File[/var/www/cgi-bin/mistral] has failures: true",

Fallback to eventlet mistral-api for the time being until we get
a usable mistral-api image.

Change-Id: Ic10c579aa3b6d0d6a01f120669be3b5dcc5efcda
Depends-On: I54627f1c5a8867738a55bee42075bb6087830c61
Related-Bug: #1724607
2017-10-24 19:30:13 +00:00
Michele Baldessari
b5eeeab73e Disable xinetd class when creating swift-storage puppet configuration
Due to missing puppet invocation with --detailed-exitcodes we ignored
a large amount of puppet errors during deploy. Swift storage fails
during the puppet_config step with the following error:

Debug: /Stage[main]/Swift::Storage::Object/Swift::Storage::Generic[object]/Package[swift-object]: Not tagged with file, file_line, concat, augeas, cron, swif t_proxy_config, swift_config, swift_container_config, swift_container_sync_realms_config, swift_account_config, swift_object_config, swift_object_expirer_con fig, rsync::server
Debug: /Stage[main]/Swift::Storage::Object/Swift::Storage::Generic[object]/Package[swift-object]: Resource is being skipped, unscheduling all events
Debug: Executing: '/usr/bin/systemctl is-active xinetd'
Debug: Executing: '/usr/bin/systemctl is-enabled xinetd'
Debug: Executing: '/usr/bin/systemctl unmask xinetd'
Debug: Executing: '/usr/bin/systemctl start xinetd'
Debug: Runing journalctl command to get logs for systemd start failure: journalctl -n 50 --since '5 minutes ago' -u xinetd --no-pager
Debug: Executing: 'journalctl -n 50 --since '5 minutes ago' -u xinetd --no-pager'
Error: Systemd start for xinetd failed!

The problem is that by using the rsync::server tag we end up including
the xinetd class automatically which will try to start a service inside
a container. By nooping the xinetd class, we're able avoid systemctl
calls and have a successfuly deployment. The resulting swift_rsync
container seems to work correctly:

[root@overcloud-controller-0 ~]# docker exec -it swift_rsync /bin/bash -c "ps -axuwf"
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root        10  0.0  0.0  47444  1624 pts/1    Rs+  18:16   0:00 ps -axuwf
root         1  0.0  0.0    188     4 ?        Ss   17:27   0:00 /usr/local/bin/dumb-init /bin/bash /usr/local/bin/kolla_start
root         6  0.0  0.0  11036   924 ?        Ss   17:27   0:00 /usr/bin/rsync --daemon --no-detach --config=/etc/rsyncd.conf

[root@overcloud-controller-0 ~]# docker logs swift_rsync 2>&1|tail -n4
INFO:__main__:Deleting /etc/rsyncd.conf
INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/rsyncd.conf to /etc/rsyncd.conf
INFO:__main__:Writing out command to execute
Running command: '/usr/bin/rsync --daemon --no-detach --config=/etc/rsyncd.conf'

Change-Id: I5e43e8fd61e002d2acc56a7de52e6aae64ab60be
Closes-Bug: #1723463
2017-10-17 17:10:54 +02:00
Zuul
998e1885ed Merge "Fix /etc/openstack-dashboard/ permissions for access to *policy.json" 2017-10-16 11:18:21 +00:00
Jenkins
71781d5e93 Merge "Remove monitor_interface from ceph-ansible parameters" 2017-10-14 07:50:50 +00:00
Jenkins
efd86fb1a8 Merge "Add a Skydive composable service" 2017-10-13 20:37:59 +00:00
Jenkins
3f6db21211 Merge "Add Mistral event engine" 2017-10-13 12:50:20 +00:00
Jenkins
466da79636 Merge "Take all mounted config_volumes into account" 2017-10-13 10:10:01 +00:00
Giulio Fidente
3e90ae3df5 Remove monitor_interface from ceph-ansible parameters
We should not pass any hardcoded value for monitor_interface and
rely on monitor_address_block only instead.

Also removes journal_collocation which is not consumed by
newer (and stable) builds of ceph-ansible.

Change-Id: Idf213a1f43a66506f76d07102f122839b5096948
Closes-Bug: #1715246
2017-10-12 17:33:41 +02:00
Rhys Oxenham
fd657aa4e6 Fix /etc/openstack-dashboard/ permissions for access to *policy.json
The Kolla Dockerfile sets the permissions for /etc/openstack-dashboard/
to horizon:horizon. We need this to be readable by the apache user
as the horizon user is not the user in which httpd runs with. We may
want to consider fixing this in the upstream Dockerfile instead, e.g.
checking if we're using centos/rhel and changing the permissions that
way. I'm not sure why it's set to horizon:horizon upstream, and I'm keen
not to break any existing functionality that relies on the horizon based
permissions.

Closes-Bug: #1723125
Change-Id: If5feebae38f7fdfffa60bfaedc4521f676006484
2017-10-12 13:54:05 +00:00
Jenkins
3e7ea2202f Merge "Also match config volumes for /var/lib/config-data/puppet-generated/" 2017-10-12 13:22:47 +00:00
Alan Bishop
e828e8c7bb Enable Cinder as a backend for Glance
Enable Cinder as a backend for Glance by adding 'cinder' to the list of
allowed choices for the GlanceBackend heat parameter.

Update the glance-api docker configuration to allow the feature to work.
This is necessary because the feature uses iSCSI, which requires additional
privileges.

Depends-On: I850047e32f3608b3ce490e52e2e540695cb1a4ff
Change-Id: I42241747de931103a04aa5ee2ed18fd46197d183
2017-10-12 08:11:52 -04:00
Christian Schwede
7c6e0a43c5 Remove rsync from xinetd when upgrading to containerized deployment
The rsync service needs to be removed from the xinetd service, otherwise
the swift_rsync container will permanently restart because the rsync
port (873) is still in use after upgrading.

Closes-Bug: 1718403
Change-Id: I283919891d00731c96ef963b0c4137d10144ccaf
2017-10-10 14:25:31 +02:00
Martin André
8aef24de81 Take all mounted config_volumes into account
We need to account for all the mounted config volumes when generating
the TRIPLEO_CONFIG_HASH in order for paunch to know to restart the
container when any one of the config_volume gets updated.

Change-Id: I473a71f49bd446694da48bb5b7b0a49126df7845
Closes-Bug: #1721306
2017-10-09 18:04:31 +01:00
Jenkins
09a6b939ae Merge "Create mysql user for non-ha deployments" 2017-10-09 12:14:58 +00:00
Jenkins
59c1a57b20 Merge "Remove package if service stopped and disabled" 2017-10-09 11:36:38 +00:00