179 Commits

Author SHA1 Message Date
Jenkins
d26e55b800 Merge "Add missing SCRIPTDIR variable" 2015-04-23 04:01:45 +00:00
Rabi Mishra
c12ba5c603 Add missing SCRIPTDIR variable
Add missing SCRIPTDIR in 50-heat-config-kubelet

Change-Id: I3ba5e21e3bc7f4af84ab1008cdefac41dd4876e9
2015-04-22 20:21:50 +05:30
Rabi Mishra
fe41a72ef3 Some minor changes to container boot-config
Some minor changes and cleanup.

Change-Id: Ic98a0138744d031308e437f14207e03be67a0019
2015-04-22 09:48:15 +05:30
Jenkins
a3f38b393e Merge "Boot config to deploy containers" 2015-04-17 07:48:18 +00:00
Rabi Mishra
73f8bd3944 Add some useful outputs to the asg example template
This change adds some outputs based on path based nested attribute
access using `outputs_list` and `outputs`.

Change-Id: I3ab0c0ec1156fbb1a7ebbf51f520d584faa726b7
2015-04-15 14:56:00 +05:30
Rabi Mishra
24e45841cb Boot config to deploy containers
This defines environment files which declare a resource type
Heat::InstallConfigAgent.

This can be used by server user_data when booting a pristine
image to install parts to deploy a container with docker-compose
or atomic hook.

This container would then help deploy all other 'docker-compose'
and 'atomic' SoftwareConfig resources.

Example template can be run with downloadable Fedora Atomic Host
image or RHEL Atomic Host, specifically tailored for use
with containers.

https://getfedora.org/cloud/download

Change-Id: Ib5bdcef1111ec9a0bcf588ece277b9328f6e6def
2015-04-13 07:35:09 +05:30
Jenkins
eb3b82b7d1 Merge "Add env_file support for docker-compose hook" 2015-04-07 21:48:07 +00:00
Rabi Mishra
96898e3824 Add env_file support for docker-compose hook
Added support for `env_file` which can be single file or a list
of files. Use special input 'env_files' for the env_files. These
files will be created by the hook relative to the compose project
directory.

if 'env_file' entries in 'docker-compose.yml' do not exist in the
`input_values` supplied, docker-compose will throw an error as it
can't find these files.

Change-Id: I0dbe80e8c8c56cc680f8ee86b4f34cf3d27533c4
2015-04-07 22:46:47 +05:30
Jenkins
24671d4e64 Merge "heat-config-kubelet hook" 2015-04-07 12:25:12 +00:00
Steve Baker
841ee38a35 heat-config-kubelet hook
This hook uses the kubelet agent from the kubernetes project
to provision containers. The StructuredConfig resource data
represents a pod of containers to be provisioned.

The files have the following purpose:
- extra-data.d/50-docker-images allows an archive file of docker images to
  be included in the dib image
- install.d/50-heat-config-kubelet installs kubernetes for redhat based distros
  during dib image build, along with the required systemd and config files required
  to enable a working kubelet service on the host
- install.d/hook-kubelet.py polls docker images and containers until the expected
  kubelet-provisioned containers are running (or a timeout occurs)
- os-refresh-config/configure.d/50-heat-config-kubelet runs before 55-heat-config
  (and the kubelet hook it triggers). This orc script writes out all pod definition files
  for the pods that should currently be running. Kubelet is configured to monitor
  the directory containing these files, so the current running containers will
  change when kubelet acts on these config changes

Author: Steve Baker <sbaker@redhat.com>
Co-Authored-By: Rabi Mishra <ramishra@redhat.com>

Change-Id: Iabb844c7d8a6f916093c364ad8caa0dbe50df3dd
2015-04-02 08:41:38 +05:30
Jenkins
0b1d937a0e Merge "Add sample multi region template" 2015-04-01 09:43:34 +00:00
Jenkins
b7b0eb0951 Merge "Fix puppet template "file patch parse error"" 2015-03-30 10:19:18 +00:00
Jenkins
600924e5a3 Merge "Fix hiera deployment JSON generation" 2015-03-29 22:49:24 +00:00
Jiri Stransky
f00cf8fbb8 Fix hiera deployment JSON generation
We were hitting strange intermittent errors in TripleO where the hiera
JSON file generated by Heat was not even valid JSON, ending with bits of
text which seem out of place, and of varying length, e.g.

"CREATE"}"}

or

"CREATE"}EATE"}

The cause could be that the generation of the file runs multiple times,
and if the first generated JSON is longer than the subsequent one, the
ending of the previous JSON is still present in the file, because the
generation opens the file with O_CREAT (create if not exists) and
O_WRONLY (write only) flags, but not with O_TRUNC (truncate file before
writing). This patch adds the truncation to fix the issue.

Change-Id: Icf184f973decec2eb7de1dee9959b60774d83eb6
Closes-Bug: #1434187
2015-03-27 14:30:09 +01:00
Daniel Gonzalez
927f08cdb8 Add sample multi region template
Add a simple multi region hot template defining
two remote stacks based on the 'hello_world' template.

Change-Id: Id3e4dc5cff284d906b9ed86f96b4047501a242c7
2015-03-27 13:11:10 +01:00
Haiwei Xu
2fca7ef1e7 Fix puppet template "file patch parse error"
The parameter in software-config puppet template
can't be parsed by using "'", should change it to '"'.

Change-Id: I83e49ae538a04684ab836f7848e8861b20975be3
Closes-Bug: #1436817
2015-03-27 10:16:10 +09:00
Haiwei Xu
faaf253b2f Fix AttributeError caused by hook-ansible
Currently a python miss in hook-ansible causes AttributeError,
which will stop the ansible config being deployed.

Change-Id: I33918424d664dfbe1d9471f0ac80b2a3c974f4d0
Closes-Bug: #1436740
2015-03-26 19:20:01 +09:00
Miguel Grinberg
fc775a18d6 Added required tool property to resource
The tool sub-property is required in each of the sections of the configs
property of OS::Heat::SoftwareComponent. This change adds it to the
sections that are missing it.

Change-Id: I8fe986df43cec381b4ba04243a30e222e8ac1460
2015-03-24 11:35:07 -07:00
Jenkins
8bfade8908 Merge "pip_mirror param for test image boot config" 2015-03-24 03:20:47 +00:00
Jenkins
fa19627e0a Merge "Update example template for 'docker-compose' hook" 2015-03-19 00:43:55 +00:00
Jenkins
eb7c318239 Merge "Remove experimental 'heat-config-docker' hook" 2015-03-18 02:23:38 +00:00
Steve Baker
73639fe9b0 pip_mirror param for test image boot config
This allows a local pip mirror to be used to install the heat
agent tools (such as the mirror built by devstack when
HEAT_BUILD_PIP_MIRROR=True).

Change-Id: Ib48d652fba8dabe1e94203d44278b2bae56b356c
2015-03-18 14:02:34 +13:00
Rabi Mishra
b8290c1942 Update example template for 'docker-compose' hook
Update example template to provide external access
to the deployed nginx container.

http://<server_floating_ip>:8000

Change-Id: I0c74692f15cdeb74d4e3e84d3bc5a9b5ace1022e
2015-03-17 17:33:16 +05:30
Rabi Mishra
717246d99a Remove experimental 'heat-config-docker' hook
This removes the experimental docker hook in favour of docker-compose
hook.

Change-Id: I5a9bac4746df4682fb2165a4ad70f668366ffab5
2015-03-17 15:27:43 +05:30
Jenkins
eb482527c0 Merge "Build selinux-permissive test images" 2015-03-16 21:23:08 +00:00
Steve Baker
8fb72a205b Build selinux-permissive test images
With selinux enabled, the dib-built fedora image needs to do a full
relabel on boot, which is so slow in the gate environment that tests
are timing out.

This change makes wrapper elements for all the test distros which
allows for adding extra per-distro elements to the image like
selinux-permissive.

This change also goes back to image file names which have no timestamp.
This is because the publishing method is going to be to swift instead
of tarballs.o.o, and swift supports atomic uploads.

Change-Id: I797588a90f2aea2f60042739f35bd43b8b0c284c
2015-03-17 08:53:59 +13:00
Jenkins
4c1a0408e9 Merge "heat-config-docker-compose hook" 2015-03-16 03:20:36 +00:00
Rabi Mishra
5cbf3ba4a5 heat-config-docker-compose hook
This hook which uses 'docker-compose' to deploy containers.
The StructuredConfig resource data represents a 'docker-compose'
project that can include more than one services/containers.

Change-Id: I63074c51f59893ba4c6500eae2f42fa5bd7ed061
Closes-Bug: #1426223
2015-03-12 15:46:27 +05:30
Steve Baker
ef5311ea75 Datestamp the generated test image
Copying to tarballs.openstack.org is not atomic, so the image
name needs to include a datestamp and the publish mechanism needs
to symlink to the lastest image.

Change-Id: I7ffba492812beca39cdb773d9e284750a3a80878
2015-03-11 10:02:00 +13:00
Jenkins
c44365e618 Merge "Add network parameter to templates" 2015-03-10 00:34:29 +00:00
JUNJIE NAN
581dfb3fa1 Add network parameter to templates
Or nova will complain do not know how to select network

Change-Id: I628d6c2dbc4bfe03577488fa464b8d3083d27dd3
2015-03-10 11:22:53 +13:00
Jenkins
6e3380501d Merge "Boot config environment for test images" 2015-03-04 02:01:55 +00:00
Jenkins
146b22c8de Merge "Elements for building a heat test image" 2015-03-04 02:00:51 +00:00
Jenkins
5badcc0967 Merge "Boot config for installing software-config agent" 2015-03-04 01:59:02 +00:00
Steve Baker
78f4a32c6e Boot config environment for test images
These environments assume an image which is ready to pip install
the heat agent projects without needing to install any distro packages
first.

The aim is to enable faster boot times during functional test runs by
starting with a standard base image, and pip installing agent code to
be tested during boot.

Change-Id: Id117fd649b62ca5f1ef49fea5cbbc51d03e09486
2015-03-04 10:21:03 +13:00
Steve Baker
0175da2ee6 Elements for building a heat test image
The heat functional test job needs to boot full images containing the
heat agent code (os-collect-config etc) so that it can test the
interaction between heat and the agent.

Images built with the elements in this change contain the necessary
distro packages so that only pip packages need to be installed on
server boot. This is very fast compared to doing a disk-image-create
on every gate run or doing distro package installs during boot.

Initially pip packages would be installed from upstream pypi during
boot, but devstack already has the code required [1] to build pip
repos from local git checkouts so it will be possible to check and
gate on agent code changes, which is not currently possible.

The intent would be for these images to replace the vanilla
Fedora-x86_64-20-20140618-sda which devstack currently uses during gate
jobs. The Fedora-x86_64-20-20140618-sda qcow is cached on the nodepool
images but it might be better for these images to be hosted on a
server like tarballs.o.o and downloaded on every gate run. Initially
I'd like to request that an image be manually built and uploaded
somewhere so we can start using it immediately and drop
Fedora-x86_64-20-20140618-sda.

Building these images does not need to be fully automated and likely
only needs to be triggered when there are dependency changes in the
heat agent tools.

The script build-heat-test-image.sh will trigger a image build
defaulting to fedora. Ubuntu and CentOS7 are also fully supported by
these elements.

[1] https://github.com/openstack-dev/devstack/blob/master/lib/dib#L52

Change-Id: I3f10f9137cc4391de884fddcab5b4fcb802a8dd8
2015-03-04 10:21:03 +13:00
Steve Baker
426b924c41 Boot config for installing software-config agent
This defines environment files which each declare a resource type
Heat::InstallConfigAgent.

This can be used by server user_data when booting a pristine image
to install the agent required use software deployment resources in
templates.

This template can be run with downloadable Fedora images from
http://fedoraproject.org/en/get-fedora#clouds

or Ubuntu images from https://cloud-images.ubuntu.com/

Tested with Fedora-x86_64-20-20140618-sda (yum and pip)
and trusty-server-cloudimg-amd64-disk1 (pip)

Change-Id: I654561719b88c7e2a548ff6db406dd0caf3e8489
2015-03-04 10:21:03 +13:00
Jenkins
1851aeec42 Merge "Start of docker hook unit tests" 2015-03-03 21:13:29 +00:00
Steve Baker
c055d5bdea Use relative path for cfn-init
/opt/aws/bin is a non standard path, so just assume that cfn-init
is in PATH, which it always is, no matter how it is installed.

Change-Id: I9f261fa885da4b9747e7be204b19127f3aad2ecf
2015-02-27 12:40:35 +13:00
Steve Baker
08d00b0c41 Start of docker hook unit tests
Other hooks process json then execute commands. The docker hook
makes socket API calls via the docker-py module. For this reason
a different approach has been taken with testing this hook:
- symlink hook-docker.py to the tests package as hook_docker.py
- in the test, mock out the docker import and run main() directly

Change-Id: Ia069b2cc0db71952732dea310a8986b73981b656
2015-02-26 13:19:45 +05:30
Haiwei Xu
67301b6ff4 Remove quotation marks of resource type
Change-Id: I672703e212d27a676a95b67efa5f2e9d403185f2
2015-02-24 16:24:45 +09:00
Jenkins
e6111f8822 Merge "Added Puppet Agent HOT" 2015-02-23 20:48:50 +00:00
Steve Baker
f27ef03211 heat-config-notify support deploy_signal_verb
The URL for deploy_signal_id is called with POST by default, however some
signals need to be called with PUT (specifically signals backed by swift
objects).

This change uses the value of deploy_signal_verb for POST or PUT, using
POST by default.

Change-Id: I8c83fa34cad9b1e91abcfc7e26e04b21cdc9bde4
Partial-Blueprint: software-config-swift-signal
2015-02-18 11:27:55 +13:00
Steven Hardy
3a4b1953a9 Fix ntp parameters in os-apply-config example
Ref https://review.openstack.org/#/c/145737/, we inherited
a bad config from the tripleo template I copied from.

Change-Id: Ib3865c498d7fe62aba5efd42dcf76e7bccf0fc08
2015-01-16 14:28:01 +00:00
Steven Hardy
84cc387921 Add an example of combined deployment and user-data
Adds an extended example of the os-apply-config example, where we
also add a user-data config to influence the cloud-init configuration.

This can be useful if, for example, you want to override some
cloud-init configuration on deployment, before the StructuredDeployment
is applied to the box.

Change-Id: Ifdb027dfe08ee618455ad9df44c53ed223fbb376
2015-01-16 13:57:40 +00:00
Steven Hardy
405c270c21 Add example of SoftwareConfig driving os-apply-config
Adds a minimal example of driving os-apply-config directly, e.g
with no heat-config hooks.  This is essentially how TripleO
currently uses SoftwareConfig resources, so it's useful for
reporoducing TripleO related problems.

Note, this needs rebuilding your image with the addition of the
"ntp" element, and currently the signalling won't work
https://review.openstack.org/#/c/145764/ is the update to
os-refresh-config which aims to fix that.

Change-Id: Id7a84655632da9fa6c4079954178f4abda787e4a
Related-Bug: #1389178
2015-01-08 12:29:13 +00:00
Jenkins
06e4dab03e Merge "Correct unresolved reference 'true' and 'false'" 2015-01-08 08:04:29 +00:00
huangtianhua
4da7ae6597 Correct unresolved reference 'true' and 'false'
Correct unresolved references, set boolean value
to True/False instead of true/false.

Change-Id: I7bf54c32fda7a9da862f93a491ceca1d3743e964
2015-01-07 15:41:33 +08:00
Steve Baker
5c188939b6 Write docker operations log to deploy_stdout/stderr
Previously deploy_stdout contained a dictionary of container info,
which is not a stdout stream, nor does it give the user any feedback
on what docker operations were performed.

The container info dict can be added back as a special output.

Change-Id: I856268053eb755f585183c6a6047eea6c569f9c2
2014-12-29 17:32:32 +13:00
Jenkins
f35e89b5c3 Merge "Remove underscore in autoscaling group name" 2014-12-25 21:45:04 +00:00