36 Commits

Author SHA1 Message Date
Jenkins
82ce1d5229 Merge "Add '-H -E' options to pass proxy env for sudo" 2015-12-02 19:42:16 +00:00
Julia Kreger
807c17c396 Switch to Ansible 2.0
Once Ansible 2.0 is released, we should use it. Updated branch
to be stable-2.0.

An additional driver behind this is the introduction of additional
filters which allow users to combine dictonary values.

Change-Id: I7527ad084e67b5707c4aeeb09fdc9446889d25b6
2015-12-01 07:51:44 -05:00
Haomeng, Wang
74aeb022cf Add '-H -E' options to pass proxy env for sudo
'sudo python /tmp/get-pip.py' command call not working with proxy
env, so add '-H -E' sudo options to pass proxy env to get-pip.py.

Change-Id: If60bacc130d311f7694dba108bf922b393c1017c
Closes-bug: #1519596
2015-12-01 02:58:58 +00:00
Robert Collins
4148a5a621 Use sudo -H -E when running pip
-E will result in pip writing to the current users disk cache, and
that prevents future non-root pip calls using it properly.

Change-Id: Ib70b3d59a20967bc7e08c70387eb7dd6f7f1b5c5
2015-11-07 00:25:45 -05:00
Jenkins
7bbd56ba70 Merge "Stop using distro setuptools and easy_install" 2015-10-06 17:39:07 +00:00
Jenkins
ac866d0135 Merge "Ansible 2.0 compatibility" 2015-10-06 17:04:14 +00:00
Monty Taylor
c4636fe436 Stop using distro setuptools and easy_install
distro setuptools is old and should never be touched if you're going
to install things from pip. easy_install is insecure and open to
MITM attacks.

Downloading get-pip.py over https and running it will install modern
pip and setuptools on the system and is the preferred method for
installing pip in all cases.

Change-Id: I1bf88f018806142ca639f3c183598b318636e61d
2015-10-04 13:07:47 -04:00
Jenkins
c505c409d2 Merge "Fix default log file location" 2015-10-02 19:04:24 +00:00
Julia Kreger
15e1c3f22e Ansible 2.0 compatibility
These changes allow the various playbooks to be executed by
Ansible 1.9.x and 2.0. This was necessary as 2.0 has a number
of changes to the YAML processor, and certain ways of doing
things have naturally had to change.

- Removed pass-through variable when a role is explicitly defined
  with variables passed through to it as updated yaml parser fails
  to handle appropriately.
- Defined a default dib_packages variable of an empty string.
- Changed stray boolean compare to use an is defined check as
  the the parser in Ansible 2.0 does not equate undefined to
  false.
- Explicitly defined the testing_user for the syntax check
  as the check fails without it.
- Revised MySQL module syntax usage to be compatible between
  Ansible 1.9.x and 2.0 development branches.
- Updated some conditionals making string compares with lookups
  which is apparently problematic in Ansible 2.0.

Depends-On: I9d5fa719793896cce00a69dafba738755b45b068
Depends-On: I23e902c8637e142fba23d71467225d48ee265253
Change-Id: I4fec7f44dd9d591388f345b3f449cb44b8e50744
Closes-Bug: 1469862
2015-10-02 12:58:11 +00:00
Julia Kreger
940280ceac Fix default log file location
The default log file location was not remedied by change
I76a2a23e2a75022bae3511700c81145b5cbeae77.  This fix changes
the LOG_LOCATION such that the logs folder is always defined
instead of using the workspace root which is not compatible
with the test jobs presently defined for bifrost.

Depends-On: I23e902c8637e142fba23d71467225d48ee265253
Change-Id: I9329d9485c7d5ab3d51831446993df47b70d881e
Closes-Bug: 1490532
2015-09-27 13:10:51 +00:00
stephane
1f2c7e9b27 gcc can also be missing on deb distros
Add a check for gcc when apt-get exists
so we can install it if it's missing.

Change-Id: Ib7fbe63bf4c71aa569110d6f63e26b6bff4e89ff
2015-09-23 16:23:54 -07:00
Jenkins
b9e9446681 Merge "Allow Test VM RAM setting to be tunable" 2015-09-09 15:02:04 +00:00
Julia Kreger
75ac413026 Allow Test VM RAM setting to be tunable
Due to the debian + simple-init image size, which is not a small
image, we need to support having slightly larger guest VMs in order
to allow for the image to be downloaded to the guest VM.

Add tunable setting, and default the image build test job to create
a VM with 4096 MB of RAM instead of 3072 MB.

Change-Id: Ifdccb32035fa91fe8d8200c3090973c34d453801
2015-09-08 17:18:57 -04:00
Julia Kreger
37b0c9ce76 Correct log file storage location
Due to zuul jenkins-job-builder changes, the base folder the CI
job was being executed changed.  This is okay, however the log
collection script only looks in the root of the WORKSPACE folder
if present, so we need to identify and default to that folder if
defined.

Change-Id: I76a2a23e2a75022bae3511700c81145b5cbeae77
Closes-Bug: 1490532
2015-09-08 16:01:08 -04:00
Julia Kreger
e929254f93 Test script to drive image testing
Support to enable building an IPA image for testing
and script to leverage testing image building.

Change-Id: I31263909d2cb275491e57d7ed8c3f08a86462699
2015-08-14 17:54:56 -04:00
Julia Kreger
d6b7701250 Move set +e to ensure VM creation causes failure
Change-Id: I0f9575eec79b3b81626ee0a1d9cc362f807fb6cd
2015-07-29 15:09:21 -04:00
Julia Kreger
7f3ece10f8 Removal of legacy roles and related updates
Removed legacy roles that utilize the baremetal.csv and looping
to achieve the desired results which was extremely limiting and the
predicessor to the dynamic inventory module and roles which
allow greater flexibility.

Additionally updated the deploy and enroll yaml files to redirect
users to the newer process and updated the README to remove
references to the older process.

Change-Id: Ia394652686608c012a848c0329195fba9abd45d1
2015-07-20 14:51:25 -04:00
Julia Kreger
abbd7ca060 Follow-up doc patch for 200834
Updating the README.rst file to be more clear and added an
additional note in the test script to provide better documentation
and context to a user reviewing those areas.

Change-Id: I59718b83928f2cccb198dcc993967be4d809ad70
2015-07-15 20:15:18 +00:00
Julia Kreger
edd5b67afc Switch to simple-init for network configuration
Bifrost defining and attempting to bake in configuration files
into configuration drives to allow initial configuration of
connectivity is a crazy idea.

Since the Openstack-infra folks have glean and simple-init, we
will utilize their tooling to provide greater flexibility across
operating systems.

Additionally, the ability to write out a simple debian style
interfaces config file has been retained, however off by default
as it is necessary for cirros testing.  In addition to this, the
ability for inventory based data to override the contents of the
network_info.json file.

Change-Id: Id85b18277b4d78acf418f63f046e39e889590719
Closes-Bug: 1463191
2015-07-14 08:31:34 -04:00
Peter Martini
94e2f9a4e3 Allow env-setup.sh to use alternate url/branches
For an offline install, env-setup.sh needs to be able to clone
from locations other than GitHub.  Additionally, since
the ansible repo will have to have its .gitmodules file patched
to point submodules to some other location as well, its convenient
to allow for alternate branches.

As a side-effect, this moves the documentation of what branch
we're using to the top of the script for easy reference and removes
the duplicated branch name.

Change-Id: I7432531915c03638aaf52f836cda607d424351ba
2015-07-02 07:00:15 -04:00
Jenkins
39e1a06403 Merge "Change Bifrost over to using dynamic inventory" 2015-06-26 19:48:59 +00:00
Julia Kreger
5f14c9b834 Change Bifrost over to using dynamic inventory
Changing the test path to leverage the dynamic inventory roles.

Change-Id: Ib4fe4c6af47cfe475f85b45c6956c40015bddf72
2015-06-25 13:07:43 +00:00
stephane
5a180949a2 Refactor ironic-install playbook to support CentOS & RHEL
Refactor the ironic-install role so that it can be used on
Centos and RHEL. Also add a new template for systemd configuration
so we can install and run the ironic services.

Document the additional steps that RHEL and CentOS users will
need to take to get bifrost up and running.

Install gcc in env-setup.sh as it is needed to install pip and
we can't take it for granted.

Temporarily remove amttool from RedHat family dependencies since
it's not currently packaged for RHEL or CentOS 7. We'll handle this
in a separate patch.

Change libxslt-dev to libxslt1-dev as it is the correct name for the
package.

Closes-Bug: 1460930
Closes-Bug: 1456896
Partial-Bug: 1463871

Change-Id: I72a5b329d3c55350644a777b5a77d61947c921bc
2015-06-24 07:58:48 -04:00
Jenkins
cb5c4ed735 Merge "Check for and install easy_install" 2015-06-23 18:47:44 +00:00
Peter Martini
9c110d176f Check for and install easy_install
On Ubuntu 14.04 at least, easy_install is not present by default.
We need to check for it, and if its not there, add it so we can
install pip.  Added a comment block noting the bug and rationale
for using the package manager to get a baseline setuptools.

Change-Id: Icf5434e828ce6c15968d1f46c19a9e11b33f4ed6
2015-06-20 00:12:47 -04:00
Julia Kreger
47e8b00c10 Change job logging such that successful jobs save logs
Change job logging so we always save logs along with
changing the file ownership on the copied log files such
that the CI system can save them.

Change-Id: I7a2c02316ec82a4bed1af7b8064f4cfb21011a0f
2015-06-18 16:17:23 +00:00
stephane
f239fe358f Refactor pip check/install & add dependency for CentOS
Use easy_install to install pip to avoid additional dependencies
on CentOS. Refactor the check for an installed pip since it will
now be the same in both OS families.

env-setup.sh was missing a package required if the user is
starting from a very basic CentOS install.

Also fix a minor typo in the script.

Change-Id: I1ee783862b2bc5ad8b5df442ed7ca25732d8f706
2015-06-16 12:32:04 -07:00
Julia Kreger
d2d08fb2cc Change pip install method and add six to requirements
Minor changes to make Bifrost's test/installation sequence more
compatible with Centos7.  Attempt easy_install based install
for pip if it is missing and install six from pypi as it
is required by Ansible.

Change-Id: Ia583bcaa15239753b7f50afbae001aff0904efd3
Partial-Bug: 1463871
2015-06-11 16:47:07 -04:00
Chris Krelle
0609604ec8 Update test-birfrost to limit log lines
test-birfrost notes that only the last 1000 lines of some log files
will be output, but uses cat to dump the entire file. This patch limits
output to 1000 lines.

Change-Id: I02fce16edd9d1494a7bee3c534db4c264923a12d
2015-06-03 11:49:38 +00:00
Julia Kreger
2994425544 Preserve logs in logs/ folder
Add retention of logs from a test run in a delineated folder
so OpenStack CI can be configured to store the logs after a
completed test.

Change-Id: I8d686918649a69b8aab9e3c5d7faf3a79a42a4e8
Partial-Bug: 1459803
2015-06-02 20:31:10 -04:00
Julia Kreger
48b4d6abfe Lock Bifrost to Ansible stable-1.9
Due to a change landing in Ansible devel branch, we need to lock
the version to stable-1.9.

Update modules to explicitly define auth_type for compatability with
Ansible 1.9.

This change superceeds: I6dc3842bda19234921cd10fc5836ef8cfdcabb82

Ansible bug: https://github.com/ansible/ansible/issues/11119

Change-Id: Id4002fd95c09cb765ef9268c8ee35f1d9ce90107
Partial-Bug: #1461025
2015-06-02 20:28:05 -04:00
Julia Kreger
6b9581d908 Use Cirros for CI testing
Utilizing Cirros for the guest operating system when performing a
basic test saves a significant amount of time and resources for the
test as it prevents the creation of a disk image.

Additionally, the native support for configuration drives allows
the amount of time we're waiting on the guest OS to be greatly
reduced.

Change-Id: I5a4c497d3502c975db060d3ee08ea2056399a062
Closes-Bug: #1460052
2015-06-01 15:27:27 -04:00
Julia Kreger
2c1c54d7b7 Fix playbooks for OpenStack CI
The OpenStack CI systems are preconfigured with various components
such as databases, message queues, caching dns resolvers.  The
CI systems also lack things like public/private key pairs, and
known_hosts files.

Due to this, a number of minor changes are required in Bifrost's test
sequence to account for and route around these differences in order
to allow test jobs to run until completion. This also demonstrated
that some steps have been operating under the assumption that the user
likely already had certain things installed, so service restart
tweeks have also been necessary.

Added a step to restart libvirt-bin in order to allow it to pickup
new capabilities due to the install of the qemu emulator.

Added additional steps to collect kernel log and `virsh capabilities`
output.

Removed MySQL step nolog statements as the password is automatically
suppressed and it suppresses information that could be required for
troubleshooting.

Added in specific username/password logic for MySQL steps if ZUUL
is detected in order to use the OpenStack CI defaults.

Changed VM test node creation to occur upon every attempt.

Extended the startup wait for a test node to be 900 seconds from 600
seconds.

Added collection of process list and listening sockets if ZUUL is
detected.

Added report of VM console log and various commands to assist in
troubleshooting a failing job.

Added auto-generation of a user ssh keypair if absent when operating
in OpenStack CI.

Added explicit adjustment of iptables to permit connections on the
network interface that the hosts are being provisioned via on
8080/tcp and 6385/tcp.

Change-Id: I3c37c5c21af0aefb5007c5775043f7f837389c2d
2015-05-16 06:18:48 +00:00
Julia Kreger
c3bcd20969 Change to Ansible core modules devel branch
Switching to the devel branch until the submodule
pointer gets updated so bifrost is operational again.

This was due to the Ansible mysql_user module breaking when
no configuration file was explicitly set.  A fix has been landed
however the pointer for the main Ansible repo naturally lags
behind the current development state of the modules repo.
2015-04-30 13:08:19 -04:00
Julia Kreger
cd48a75419 Script improvements for greater compatability/testing
Changed the scripts to support yum as well as apt, and added a
step to execute the ansible check-syntax sequence in order to fail
fast in the event of a syntax issue.

Additionally, the installation of jinja2 has been switched to use
a requirements.txt file placed in the root of the repository.
2015-04-21 11:55:00 -04:00
Julia Kreger
7615939bf1 Move env-setup to /scripts and add test script
Created and moved scripts into a scripts folder to keep things tidy.

Added a basic test script which invokes env-setup.sh, loads the
Ansible environment, and then initiates the test playbook.
2015-04-16 15:57:49 -04:00