22 Commits

Author SHA1 Message Date
Emilien Macchi
b1b9e3253d copy_puppet_logs: copy audit.log if it does exist
/var/log/audit/audit.log contains SElinux logs that we sometimes need
for debug, since Puppet OpenStack CI is running SElinux in enforced
mode.

This file will be useful for us so we would like to copy it from nodes.

Change-Id: Ic554dcad8d28b9cbc612019002b19982c53e74f7
2016-04-11 19:31:19 -04:00
Emilien Macchi
98b5dfff0a copy_puppet_logs: gzip only files
At the end of jenkins/scripts/copy_puppet_logs.sh script,
make sure we only try to compress files and not directories.

It will avoid this kind of error:
gzip: ./ssl.txt is a directory -- ignored

Change-Id: Iab6273b16e503d1b5f81267a14f0a3d144f730a3
2016-04-01 20:39:32 -04:00
Emilien Macchi
5e6d1f863d copy_puppet_logs: do not keep symlinks before upload
The problem with symlinks is that source files can have restrictive
permissions that would prevent jenkins user to scp logs/config to log
servers.

For history, it was causing an issue with /etc/neutron/plugin.ini on centos7 jobs.

Change-Id: I55c29d18794b19b16931e9abd08fa9a96a4e8a4d
2016-03-18 18:44:47 -04:00
Emilien Macchi
70837e7f58 copy_puppet_logs: use SUDO to rename config/log files
In a previous commit [1], we stopped changing the owner to manually set
file mode bits. The problem is that 644 is not enough for files, because
they need to be renamed so write permissions are needed.

This patch aims to make sure files can be renamed by any user, so we
use SUDO prefix for each 'mv' command.

[1] I1c93a841b82514fb146d827332eecea98d9dbd3b

Change-Id: I2ea4b090c99af33dd0095cbe664cd4187b114b54
2016-01-26 08:41:15 -05:00
David Moreau Simard
e95e5f0e5d Use file permissions instead of ownership to grant access to files
This means the script can run even if the jenkins user is not
present.

Change-Id: I1c93a841b82514fb146d827332eecea98d9dbd3b
2016-01-12 14:41:54 -05:00
Emilien Macchi
93b383b5f9 copy_puppet_logs: manage testrepository.subunit
Puppet CI wants to have testrepository.subunit file in the logdir so we
can monitor the jobs on http://status.openstack.org/openstack-health/
like other projects.
If the file exists, copy it in the right place.

Change-Id: I89cdd9c3aa9add79b77e1bde35b758b4ee64ead1
Depends-On: I70ecc95dad93b79dbf73f6853ff0ef883f07c9fb
2015-12-04 17:57:10 -05:00
Emilien Macchi
770e86b081 copy_puppet_logs: use sudo to rename horizon config
Use sudo to rename horizon config file because the default permissions do not allow it.

Without sudo, the mv fails because of "Permission denied".

Change-Id: Iec7f557c8565c5f70c490a32e1ad1d0ebd39868a
2015-09-30 14:15:21 +00:00
David Moreau Simard
41c7bc8479 Also look for stackforge puppet modules when copying logs
This will make it so ceph-related configuration and logs
will be picked up too since puppet-ceph is still in
stackforge for the time being.

Change-Id: Ib6e6cb72eb7f9188155cc5d393771aa5105d6937
2015-09-29 16:28:18 -04:00
Emilien Macchi
9b09f43344 copy_puppet_logs.sh: copy horizon config when possible
If /etc/openstack-dashboard is present, copy the Horizon config file(s)
in LOG_DIR/etc/openstack-dashboard and make sure they can be read by the
browser (txt files).

Change-Id: I55e8aac6a066820d3e24967ac68bc46ab95e67b6
2015-09-25 10:39:43 -04:00
Emilien Macchi
f51fc8f0fb copy_puppet_logs.sh: rename apache config
Rename .conf file to .txt, so we can use the browser to view the files
directly. Currently it's a *.conf.gz file so we need to download the
file, and extract it to see it.

Renaming to .txt.gz will be consistent with other files and will
simplify debug.

Change-Id: I8c6e029124b98190baf94f7d8e7dc179f92bc80d
2015-09-18 13:35:29 -04:00
Emilien Macchi
fb4fabb334 jenkins/scripts/copy_puppet_logs: tempest.log support
If tempest.log file is here (generated by puppet-openstack-integration),
we will archive it in logdir beside other logs.

Change-Id: Id1813c83e5c8d6d2ab8d562082d038825788e95b
2015-09-04 14:23:16 -04:00
Jenkins
d4167f728d Merge "copy_puppet_logs: copy tempest.conf in txt file" 2015-09-01 01:54:00 +00:00
Emilien Macchi
edc1903e65 copy_puppet_logs: copy dstat logs if exist
In Puppet OpenStack Integration CI jobs, we are using dstat
(like devstack does) to monitor the system and help us to debug
Puppet & Tempest runs in case of failures.

This patch aims to copy the log file only if it exists.

Change-Id: I79c5b9521265e97595d3323149ba6e7d1e8e972b
2015-08-31 10:50:10 +00:00
Emilien Macchi
7777df54fc copy_puppet_logs: copy tempest.conf in txt file
To be able to browse the file from a Web Browser without downloading the
file on the local system, rename tempest.conf into tempest.conf.txt so
it will be compressed to be at the end tempest.conf.txt.gz.
It will still take less space but also will be readable easily from the
Web Browser.

Change-Id: I0d8d016d3cb1a328c2852cb746b3d33f6ae332a9
2015-08-28 08:34:10 -04:00
Emilien Macchi
e9d431ddfb copy_puppet_logs: if possible, copy tempest config
For instance, /tmp/openstack/tempest is the directory where we run tempest.
We want to have access to tempest config after job runs, so let's copy
the file.

Change-Id: I3a38c9ca41706970d4033765833379632eced875
2015-08-26 15:59:46 -04:00
Emilien Macchi
a36840931f copy_puppet_logs: get more informations
* to know RAM status, use 'free -m' and create free.txt
* to know CPU informations, use 'cat /proc/cpuinfo' and create
  cpuinfo

This will be useful when debugging VMs running OpenStack, and eventually
detect services that consume too much CPU & RAM during Puppet OpenStack
jobs.

Change-Id: I033e8d42b430ee002e6ec3c3a589b23739b5b57c
2015-08-25 23:35:09 -04:00
Emilien Macchi
e30670c1b2 jenkins/copy_puppet_logs.sh: prepare apache_config dir
This patch aims to manage apache_config directory before copying
apache configuration from /etc.
Otherwise, the job will fail since apache_config dir does not
exist.

Change-Id: Iafd27e9edf8865e462d4e33a2514afc8c466e290
2015-08-17 13:57:27 +00:00
Emilien Macchi
b887bd9cd1 jenkins/scripts/copy_puppet_logs: fix typo
Fix a typo in OpenStack logs location.

Change-Id: I69d67c5d1f93ee6630bfff3095b08de82540b413
2015-08-13 19:24:25 -04:00
Emilien Macchi
704d558427 jenkins/scripts/copy_puppet_logs: copy apache logs only if exist
Copy Apache logs only if the directory exits.
Otherwise, if Apache is not installed, the script will fail.

Change-Id: I7f76a99e7cf2023bfdf6f59c98adf20da14d92bb
2015-08-13 07:40:19 -04:00
Emilien Macchi
af1f854dc4 copy_puppet_logs.sh: fix bash conditionnal to test apache directory
This bash error makes the publisher failing:
cp: cannot stat ‘/var/log/apache2’: No such file or directory

So the script stops here and logs can't be scp because the rest of the
script can't be executed so logs have wrong permissions.

Change-Id: I25fc65ee4bb5b6a96af6cb9a62839d805ad40c26
2015-08-09 00:23:54 -04:00
Sebastien Badia
c086daedcb scripts/copy_puppet_logs: Create LOG_DIR
Ensure $LOG_DIR is created before use

Refs: http://goo.gl/WQq6a0
Refs: I454cf0af8708869a511ac1ccbeee29fe47f0d5d9
Change-Id: I2fe59385b31bbf80e0330678c38b4d916b4e3901
2015-08-07 08:37:51 +10:00
Emilien Macchi
ffd4d4d55f jenkins: implement copy-puppet-logs publisher
* Create a new jenkins slave script: copy_puppet_logs.sh
  This script starts with devstack-gate cleanup_host() but strips it
  down to be suitable for a more generic copy.

  We make a guess of the project config/logs to copy based on the
  installed puppet modules; a project can add to this via PROJECTS

* Create a new jenkins publisher: copy-puppet-logs

* Execute this new publisher on puppet-beaker-upgrade jobs
  Because this job is non-voting, let's try this new
  publisher in this job so we make sure to not break Puppet OpenStack
  CI.

Change-Id: I454cf0af8708869a511ac1ccbeee29fe47f0d5d9
2015-08-05 03:07:58 +00:00