276 Commits

Author SHA1 Message Date
guang-yee
fbc389ec43 Use auth_token from keystonemiddleware
auth_token middleware in python-keystoneclient is deprecated and has
been moved to the keystonemiddleware repo.

Change-Id: Ib43223c2cfb31198b9438bb02f0d68f2bab7eb7e
Closes-Bug: #1342274
2014-07-21 17:30:04 -07:00
Ionuț Arțăriși
54097e23b8 Make default extension path relative to pybasedir
Change the default extension path in the sample config files as these
take precedence over the files values in common/cfg.py

Change-Id: I3c473fb9cd7e81341a1963a35834ef4c2fd69717
Closes-Bug: 1316195
2014-07-21 13:48:36 +02:00
Michael Yu
61935d3512 Add Backup/Restore support for Couchbase
Backup strategy:
- Save bucket metadata config (as json file) to /tmp/backups
- This is needed on restore to create the destination buckets
- Handles root enabled case by also saving /etc/couchbase/secret_key to /tmp/backups
- Use cbbackup to do the backup
- /tmp/backups will be zipped up, encrypted, and sent to Swift

Restore strategy:
- If /tmp/backups/secret_key exists, set root password
- Iterate through bucket metadata config (json file) and use Couchbase
  REST API to create each bucket
- Wait until bucket finishes creating
- Use cbrestore (which requires destination buckets to be manually
  created first) to do the restore
- cbrestore can only restore one bucket at a time

DocImpact
backup_namespace and restore_namespace were in the default conf group, now they are per datastore

Change-Id: Iaf4370dcc978df68e7d01c6f63795eaf50aac89d
Implements: blueprint backups-single-instance-couchbase
2014-07-17 22:51:10 +00:00
Jenkins
cc6a281bd6 Merge "Enable usage of config-drive for file injection" 2014-07-15 14:08:25 +00:00
Vipul Sabhaya
5018dae112 Enable usage of config-drive for file injection
* Some nova deployments allow the use of config-drive to do file
  injection.  Trove, when booting an instance, needs to send this
  to Nova when config-drive is to be used.

Closes-Bug: #1329535

Change-Id: I950b93a32f6c7224f30cf930e5f95710c07cbc26
2014-07-10 16:34:36 -07:00
Greg Lucas
ba21c3a3ab Change default for update_status_on_fail
Default 'update_status_on_fail' to True so that instances do not get stuck
in the BUILD state.

When this option is set, instances that remain in BUILD for longer than
the datastore's configured 'usage_timeout' will be considered failed and
moved to the ERROR state.

Closes-Bug: 1339826
Change-Id: I5f4b734830a21a1fbb6f169edb5424bac6f883cd
2014-07-10 13:36:07 -04:00
Matt Riedemann
e932c4777d Use (# of CPUs) api/conductor workers by default
This changes the default number of trove API and conductor workers to be
equal to the number of CPUs available on the host, rather than
defaulting to 1 as it did before.

Commit 75c96a48fc7e5dfb59d8258142b01422f81b0253 did the same thing in
Nova in Icehouse. Similar changes are being made to Glance and Cinder
as well.

DocImpact: trove_api_workeres and trove_conductor_workers will now be
           equal to the number of CPUs available by default if not
           explicitly specified in the trove configuration files.

UpgradeImpact: Anyone upgrading to this change that does not have
           trove_api_workers or trove_conductor_workers specified in
           the trove configuration files will now be running multiple
           API and conductor workers by default when they restart the
           respective trove services.

Closes-Bug: #1335284

Change-Id: Id300bbe991436a0f826ea715630669ab5922a6a4
2014-07-07 10:43:06 -07:00
Jenkins
367f4e0bc8 Merge "Add sample admin_{user,tenant_name,password}" 2014-07-02 19:49:03 +00:00
boden
87f0229e87 Expose trove conductor manager class as conf property
The current implementation of trove conductor hard-codes the conductor
manager class name in the cmd entry point thus not allowing it to be set
by consumers. This change exposes that classname on the conductor conf
so that consumers could plug-in their own manager class. Additionally
unit tests are provided to ensure a user defined and conductor manager
can be used in addition to the default trove conductor manager.

Change-Id: I6bf88dd9cf119f0eb6a197f4bda48c309511297e
Implements: blueprint pluggable-conductor-manager
2014-06-27 04:15:46 -04:00
Shuichiro MAKIGAKI
fd9178aef4 Add sample admin_{user,tenant_name,password}
Other OpenStack components have the same configs in api-paste.ini.
admin_token was removed by Bug #1299332, and admin_{user,password,
tenant_name} should be added instead of admin_token.

Change-Id: Icabb056c315556db8064ded882fba13c25ec2bca
Closes-Bug: #1325482
2014-06-21 17:46:42 +00:00
daniel-a-nguyen
83c7aaf178 Remove unused xml config file
Change-Id: I2b0129fcf3fb9f2e206dc0d6ac422c5bcc82d6af
Closes-Bug: #1324317
2014-05-28 17:13:01 -07:00
Jamie Lennox
9db80e79ea Remove admin_token from configs
A previous review removed the admin_token from the sample config file.
It appears that devstack doesn't use that file but instead uses one of
the .test files.

Admin tokens are actively discouraged as a form of authenticating in
auth_token middleware and the behaviour when using admin_token in
combination with a username and password (as this default produces in
devstack) can lead to problems in authentication.

Closes-Bug: #1299332
Change-Id: I78d2c184f7937006b808d580e3663ab055f7c5fe
2014-05-26 17:20:24 +10:00
Mat Lowery
e360f83c02 Get service endpoints from catalog
* Add catalog to TroveContext
* Populate context with catalog in middleware
* No new keystone calls made
* If *_url from conf available, it overrides catalog
* Following clients now use service_catalog:
    * cinder
    * heat
    * nova
    * swift
* Add new conf options (necessary to maintain parity with flexibility
  of *_url in conf):
    * os_region_name
    * cinder_service_type
    * heat_service_type
    * nova_service_type
    * swift_service_type

Closes-Bug: #1222891
Change-Id: I669bfd9fe63e538f9412f53723c6606b04c2e2ae
2014-05-19 22:36:10 -05:00
daniel-a-nguyen
d07c8c0445 Added separate rate limit setting for mgmt POST
Created a simple unittest
Fixed typo granced --> granted
Fixed test to show that the 4th request will result in a delay

Change-Id: I95062a805aaae88432c081d5f1d0a2da81d7108c
Closes-Bug: #1294421
2014-04-17 16:00:46 -07:00
Jenkins
8fc4370680 Merge "fix default rabbitmq configuration values in sample cfgs" 2014-04-15 02:43:10 +00:00
Jenkins
611d3f1b0a Merge "Collapse mysql OptGroup Sections" 2014-04-10 04:03:20 +00:00
Jenkins
2baf5d8e21 Merge "Don't specify admin_token in paste config" 2014-04-08 22:10:54 +00:00
amcrn
8aea38ebac Collapse mysql OptGroup Sections
collapses two [mysql] sections into one.

Change-Id: I19317b392aa3c4d670eabbae079c6df406f0ab6f
Closes-Bug: #1302236
2014-04-03 16:11:08 -07:00
Denis Makogon
b5af5416ee Fix create call for security group rules
We need to fix the create security group call to
correctly use the per-datastore configs to set the
port, and protocol values for the security group rule.

The only parameter that the trove user should be able to
specify for the rule is the CIDR.

Changes:
- Use the datastore specific ports and protocol for creating
  the new security group rule.

Changes:
- using datastore-specific ports for building new rules, based upon income CIDR;

Change-Id: Ic1440f735b6cf2b8b4f29c5ab9f48bcb427ca9e6
Closes-Bug: #1298749
2014-04-01 19:11:54 -07:00
Jamie Lennox
ae696056fb Don't specify admin_token in paste config
This is not a value that is going to be useful on other systems and
shouldn't be included in the default pipeline.

Closes-Bug: #1299332
Change-Id: I714539b7536965fa08a3ddc64493b481df7397e3
2014-03-29 14:07:12 +10:00
Ionuț Arțăriși
c49caef4ae fix default rabbitmq configuration values in sample cfgs
The rabbit_* configuration options have default values in the library
code, but they are not visible in the sample configuration file. This
change adds the config values to the sample configuration files.

It also removes the other unused notifier_* config attributes.

Change-Id: I4ae76ef78b7a0a04c2ea5a24fd3000e9c1561f81
Closes-Bug: #1281975
2014-03-17 14:39:53 +01:00
Ramashri Umale
ce6e24bb86 Root_on_create per datastore
Reason: Not each datastore has root user entity;

Changes: root_on_create flag per datastore.
fixed test case root_on_create;

Change-Id: I5b2f665cfdb36e9f88d57d04b5e9470085b3362a
Closes-Bug: #1276858
2014-03-09 17:56:27 -07:00
Denis Makogon
ea51b7e625 Fix inconsistent usage of mount_point
Reasons:
- multiple datastores;
- inconsistent usage of mount_point at
taskmanager and guestagent;

Changes:
- oslo groups;
- mount point per datastore;
- guest API not changed.

Change-Id: I8044789c6ab2ed837ff0e1b16183dfeafbf955bb
Closes-Bug: #1248212
2014-03-03 17:11:07 -06:00
Jenkins
4f19ff62c7 Merge "Raise max header size to accommodate large tokens" 2014-03-03 18:20:25 +00:00
Khyati Sheth
3d44f7d467 Add security group rules for Couchbase
Closes-Bug: #1287037
Change-Id: I6529930e91cfa709bd87aa0ae97b286ec65d239c
2014-03-03 00:24:14 -08:00
Jenkins
ad69a73742 Merge "Make backup/restore pluggable" 2014-03-03 05:45:54 +00:00
Denis Makogon
bd04c531d8 Make backup/restore pluggable
Reason:
 - mutliple datastore support;
 - execute_backup expects that instance that
   is going to be backuped is a mysql server
   (expects that my.cnf exist).
Changes:
 - oslo groups added;
 - updating MySQLDump strategy;
 - cleaning runner parameters;
 - updating tests;

Change-Id: Id98ac37c048db79cd24adfa22f0db722648cf55b
Closes-Bug: #1281127
2014-02-27 12:08:53 +02:00
Denis Makogon
bfa17ed6fd Security groups workflow update
Reasons:
 - By telling to trove-taskmanager
   that it should create security group with
   datastore default port taskmanager should
   go into config and find default datastore
   specific ports and create custom rule
   for them. But now it creates rule
   which allows access to 3306 port - here comes bottleneck,
   this port should be custom for each datastore.

Changes:
 - Rules are not optional any more, because empty
   security group is meaningless. By allowing usage of
   security groups we telling that it should contain
   rule by default, which mean "not optional";
 - Adding option group for a specific datastore (mysql, redis);
 - Adding tcp and upd ports as possible options.

Implements: blueprint security-groups-workflow-update
Change-Id: Iaf3187ba89d0674f7921cbf92e5eb24de05abc6b
2014-02-27 11:49:44 +02:00
Jenkins
3ca0f3db6d Merge "Remove unused admin_context from conductor manager" 2014-02-27 09:13:31 +00:00
lvdongbing
4df858db82 Raise max header size to accommodate large tokens
The max header is exceeded in the following scenario
- Auth tokens built with a keystone v3 API catalog
- A catalog with approximately 8 or more endpoints defined

An equivalent fix will be required for all projects which use
eventlet.wsgi.

Closes-Bug: #1190149

Change-Id: Ia0882e43395f1025137173b51aab147433b54255
2014-02-26 10:46:59 +08:00
Tim Simpson
ae5c2fc8fb Remove unused admin_context from conductor manager
This variable was used nowhere, plus it referenced three config
variables which were also in the sample config file that Conductor
doesn't need.

Change-Id: I08ffe578158c73d387112b6c05b7ab2c6cae833b
2014-02-10 16:54:51 -06:00
Mat Lowery
9686867df8 Add clearer help kwarg for max_volumes_per_user
Add same wording as comment in trove.conf.sample.

Change-Id: I0ad8ca2dd2d6aaa4b7012c491c4a467b6004e4cb
Closes-Bug: #1274608
2014-02-04 18:08:03 -06:00
Tim Simpson
3cdb9ea817 Changing DNS to pass string to driver
This commit also moves the "get_ip_address" functions out of
instance.views and into the models code under the SimpleInstance class,
where it's renamed get_visible_ip_addresses().
Also, the content field (the ip address) is now passed to the driver
instead of set on the Entry object by the manager, which allows more
flexibility for dealing with various drivers and will hopefully prevent
issues in the future.
Finally, integration tests were added to keep this from breaking
in the future.

Closes-Bug: 1273446

Change-Id: I70bf37838cc5cecfe579fe6001df79d7f6f5d53e
2014-01-31 17:50:34 -06:00
Jenkins
339a811687 Merge "Instance details view shows hostname (if it has it) or IP" 2014-01-23 01:49:55 +00:00
Jenkins
62c3a203ba Merge "Make use of IP filtering when creating DNS records" 2014-01-22 02:01:47 +00:00
Steve Leon
ad6e39bac5 Make use of IP filtering when creating DNS records
Previously, the return list from get_ip_address in
trove.instance.views was not filtering by the CONF.ip_regex.
(See blueprint add-ip-addr-filter-to-instance-view for more
information). DNS uses this method to get the IP. Therefore
it was not creating record with the right IP. Moving the filtering
logic to the get_ip_address method will ensure that both DNS and
instance view will obtain the right IP.

Change-Id: I4ce7c65e24f6d105f555e6c36d788558daa25f12
Closes-Bug: #1260089
2014-01-06 09:03:42 -08:00
Steve Leon
33ebe3c21d Instance details view shows hostname (if it has it) or IP
Currently the hostname and IP is controlled by two config values:
trove_dns_support and add_addresses. I dont see a point of having
the hostname and IP indenpendent. It causes issues for old instances
when DNS is enabled.

I propose to simplify the view by showing hostname if it is set or
show the IP if it is not set.

Closes-bug: #1260567

Change-Id: Ia1fd47cbee89b10d53b0821e104a34eb31543846
2014-01-06 08:46:42 -08:00
Jenkins
4fa244071d Merge "Updates trove-guestagent.conf.sample" 2013-12-12 19:32:52 +00:00
Jenkins
21b7c2c444 Merge "Conductor Does Not Default to Verbose/Debug Logs" 2013-12-12 19:28:46 +00:00
Jenkins
453c58a800 Merge "Fixed misspellings of common words" 2013-12-12 18:29:47 +00:00
Jenkins
9b028c470e Merge "use_stderr should be False for tox tests" 2013-12-12 18:04:27 +00:00
Sushil Kumar
8b2b08f72d Updates trove-guestagent.conf.sample
Reason:
- Parameter service_registry_ext has been replaced
  by datastore_registry_ext.
- trove-guestagent.conf.sample contains
  service_registry_ext, should be
  datastore_registry_ext instead.

Changes:
- Updates service_registry_ext to
  datastore_registry_ext in
  trove-guestagent.conf.sample.

Change-Id: I27feb57790011da659f1f2c5e67def08623aa490
Closes-Bug: #1260294
2013-12-12 12:10:38 +00:00
Craig Vyvial
d8f2a75e7e use_stderr should be False for tox tests
Reverting the change to the trove.test.conf so that the tox tests are
not so verbose with all the logging when running through the tests. It
makes it very difficult to find out what failed and where without much
groking of the output.

This was a change i made so that when you do a './redstack start-fake'
it would show the logs in the screen session. But this turned out to be
a bad idea when running in tox.

Change-Id: I236b2de2e89e49fd937d917d2955fddb5fb35d6f
2013-12-11 10:27:40 -06:00
Cyril Roelandt
cb58f8ca0d Replace "tokenauth" by "authtoken" in configuration files
This is the standard word used by all other projects.

Change-Id: Ibd52056bd1214f5fd7b2f2fd315086542d34fd0c
Closes-Bug: 1234314
2013-12-10 15:34:06 +01:00
Jenkins
96b4870716 Merge "setting fake mode to print startup errors" 2013-12-06 19:13:02 +00:00
Nikhil Manchanda
f15038ec83 Added ability to pass custom command-line options to backup runner
Added a new DictOpt field "backup_runner_options" to the trove config
Custom command-line options for the backup runner can be passed in
through this

Closes-Bug: 1253849

Change-Id: I2ea45aa4899da4126776ef14a9cb2f0328a9e544
2013-12-04 16:55:41 -08:00
amcrn
4305cac40a Conductor Does Not Default to Verbose/Debug Logs
updates sample trove-conductor.conf to set verbose and debug to True

Change-Id: I92864e5d1db85e9e519719ec7c3fdcba195d69b9
Closes-Bug: #1257583
2013-12-03 19:37:05 -08:00
Nikhil Manchanda
be39d7be30 Fixed misspellings of common words
Fixed misspelling of common words found by the 'misspellings' tool.

Closes-Bug: #1257531
Change-Id: If317929be260c5efb0949921be1143d1ccd4e283
2013-12-03 15:52:47 -08:00
Craig Vyvial
851b309351 setting fake mode to print startup errors
Change-Id: I55a2e547ffaa02621093af8d9c7d0a48714fb6be
2013-11-26 15:59:22 -06:00
Ed Cranford
384576675f Conductor proxies host db access for guests
Previously, instances updated their status by
updating the database on the host directly.
Necessarily, each instance would need access to the
database to stay updated.

Trove's new conductor service eliminates that need by
working as a proxy for those instances. By sending a heartbeat
to conductor via RPC, conductor updates the database
on the host on behalf of the instance.

As backups also made use of the host database, the backup
code has been refactored to take richer inputs to remove
the need to query the host database, and now conductor is
also used to submit updates to backup states.

Implements: blueprint trove-conductor

Change-Id: I4cb34baedd0e3a50051f9e66de95c9028c66e4b5
2013-11-26 11:52:00 -06:00