Puppet does not like cron list syntax to be made as a normal string.
Instead you must give puppet cron resources a puppet list that it will
convert to a cron list...
Change-Id: Ia0839be9b7d8ae772065f782adf61f831e75b92e
Decrease the time between e-r batch jobs to half an hour between
consecutive runs of the same job to reduce the latency on the updates of
these reports. Offset each of the three batch jobs from each other by
ten minutes. The run times look like this now:
minute: 0 10 20 30 40 50
job: 1 2 3 1 2 3
Change-Id: Ic4176b019e6da71e8c1e33ca89f53b6700309a91
Currently elastic-recheck is hammering the ElasticSearch cluster every
15 minutes, basically bringing everything else to a halt while these
queries are processed. Spread elastic-recheck periodic queries out, run
each once an hour and 20 minutes after the previous set of queries.
Also, remove the auto triggering when files change, instead we wait for
the hour to roll over and give us new data.
Change-Id: Ibf5dad1071f65f8d676e8b9a0ec4331bada9943f
We want to display the hits for all queues and for just the gate queue.
A parameter was added to elastic-recheck-graph to support this, and a
second patch updates the web page to give the gate queue only option.
This patch updates the usage of elastic-recheck-gate.
Related elastic-recheck patchs:
* I9217a2ceedf86ffe04851084df78238384fccd51
* I23ed8d945c9517221622e1ca215c80539bb07bbd
Change-Id: Iac51aeaf152f50f8b9a064cd249b0920bef824df
puppet exec doesn't run under a real shell, so running cd doesn't
work because it's not a command. Instead use cwd properly to
actually run these commands.
Also actually call the correct command to generate the uncategorized
jobs page.
Change-Id: I84ed19bbdc16d185316e3967efbd798e06a521d1
The elastic-recheck change: Ic3dd76bad94213c7152c29a99c00ed23a2c01a31
will add a new section to the recheckwatchbot yaml file to specify
which bug targets get reported to which channels. This commit preempts
that change by adding the new section to the -qa channel to preserve
the existing functionality.
Change-Id: I452040883027b260c0480b9d7ac2e2ca476b72b3
this adds the new elastic-recheck-unclassified report, which
generates an html page of all the unclassified gate failures.
it also runs this command, and the elastic-recheck-graph command
not only on cron, but also on updates to the repository. In order
to do this safely, we create a new er_safe_run.sh in /usr/local
that provides a flock front end to ensure we won't race on our
html writes.
Change-Id: I624f2629e2ac19a1394866f73f1aa9ba18e968ce
Also, normalize some parameters to the status manifest to make it
more readable (since they follow the same pattern). And make the
ssh config for elastic-recheck bot match reviewday.
Change-Id: I2417f121e7b3685aab9540504cdd4c6db1754e67
* modules/elastic_recheck/manifests/bot.pp: The subscribe on the
install_elastic-recheck exec in ff1e7bc2 specified an explicit
scope, but exec resources are not scoped. Fix it.
Change-Id: I16ee575a24c9abe1db13f4c2164f8c49fa95980f
The installation of elastic-recheck is done on multiple machines but the
irc bot is only run on one machine. Therefore don't notify the bot
service from the install exec instead subscribe to the exec from the
service.
Change-Id: I11f3d32e40408a02d948d0533f5a4c45c983adae
this provides the changes needed to handle the new elastic-recheck
web console installed from the elastic-recheck package in the
share tree.
factor out bot code to elastic_recheck::bot
this change moves elastic_recheck bot starting code out into
a separate module, which lets us include all the elastic_recheck
base module into the static site config.
Change-Id: If53523754494a746c152c5d1384db5426b096fc1
this provides the changes needed to handle the new elastic-recheck
web console installed from the elastic-recheck package in the
share tree.
Change-Id: I8c8b76cb53b2f2eea026eb0f321e48903df71636
moving elastic-recheck to a queries directory instead of a file
means we need to make the graphs support that. Also refactor out
the er_state_dir to make changing that easier when we pull in the
web console.
2Bupdate elastic-recheck bot config to use queries directory
Change-Id: Ifcc71511c066e8aee5cb74e013555c628f270e29
in I86317fcf6f1886ab5b6c0ee154b29e71865c52b7 we want to move to
a directory of queries instead of a single file. In order to
support this we need to support linking in that directory as well.
This needs to land prior to that change.
Change-Id: I3af42cff1050754c6844db994004fd203063bb3d
The pip vs. easy_install interactions continually get the systems
into a weird state. pip install $path_to_repo, on the other hand,
works like a charm.
Left out two graphite installs because they are doing path manipulation
and I want to do further testing before touching that.
Change-Id: I373b29aca03f1ebd20e533ebaaf5de9ef2db017e
this ensures we notify the er daemon to restart after we've updated
the code, making it so we deploy instantly into the environment.
Change-Id: Ifa266c7c0248ce9d0431f44e76de87958503f401
* manifests/site.pp: Expect the SSH key in /home rather than in
/var/run.
* modules/elastic_recheck/manifests/init.pp: Change the recheck
shell account for elastic-recheck to use a homedir in /home.
Change-Id: I43f4a8cdcb803328388b7e92732e5cb9d6acb2a8
* modules/elastic_recheck/files/elastic-recheck.init: Minor
whitespace cleanup for space-before-tab indent on a comment line.
* modules/elastic_recheck/files/logging.config: New logging
configuration file for elastic-recheck IRC bot.
* modules/elastic_recheck/manifests/init.pp: Install the new logging
configuration file.
* modules/elastic_recheck/templates/elastic-recheck.conf.erb: Tell
the IRC bot to use the new logging configuration file.
Change-Id: I47cabb01df209c9f020846bcb272b9a2f796b0e5
* modules/elastic_recheck/files/elastic-recheck.init: Update the
daemon invocation now that it has an executable entrypoints wrapper.
* modules/elastic_recheck/manifests/init.pp: Correct the Puppet
dependency tree sufficiently to be able to apply the manifest
successfully.
Change-Id: Ib018564e66c29d73b30aaf50c9306dc6d67ba2b2
This elastic recheck thing that classifies failures based on logstash
queries is awesome. We should run it on the OpenStack developer
infrastructure. Deploy it on logstash.o.o to keep related services
together.
Change-Id: I16ae7a32c2c48474454df17931779d0f60c3ed85