diff --git a/integration/.gitignore b/integration/.gitignore new file mode 100644 index 0000000000..75bff19283 --- /dev/null +++ b/integration/.gitignore @@ -0,0 +1,25 @@ +*.sublime-project +*.sublime-workspace +scripts/.screenrc +scripts/test-def.conf +.tox +*.pyc +*.coverage +tests/integration/Reddwarf_Integration_Tests.egg-info/* +apidocs/target +scripts/.cache/ +scripts/.cinderclient/ +scripts/.mysql_history +scripts/.troveclient/ +conf.json +Vagrantfile +.my.cnf +.vagrant +.bash_history +.rnd +options.rc +.novaclient +*.log +.local.conf +scripts/local.conf.d/local.conf.d +*.config/ diff --git a/integration/.gitreview b/integration/.gitreview new file mode 100644 index 0000000000..725c7456d3 --- /dev/null +++ b/integration/.gitreview @@ -0,0 +1,4 @@ +[gerrit] +host=review.openstack.org +port=29418 +project=openstack/trove-integration.git diff --git a/integration/README.md b/integration/README.md new file mode 100644 index 0000000000..6b1fe76588 --- /dev/null +++ b/integration/README.md @@ -0,0 +1,210 @@ +## Integration dev scripts, tests and docs for Trove. + +*** + +### Steps to setup this environment: + +Install a fresh Ubuntu 14.04 (Trusty Tahr) image ( _We suggest creating a development virtual machine using the image_ ) + +#### Login to the machine as root + +#### Make sure we have git installed: + + # apt-get update + # apt-get install git-core -y + +#### Add a user named ubuntu if you do not already have one: + + # adduser ubuntu + # visudo + + add this line to the file below the root user + + ubuntu ALL=(ALL:ALL) ALL + + **OR use this if you dont want to type your password to sudo a command** + + ubuntu ALL=(ALL) NOPASSWD: ALL + + if /dev/pts/0 does not have read/write for your user + + # chmod 666 /dev/pts/0 + + *Note that this number can change and if you can not connect to the screen session then the /dev/pts/# needs modding like above.* + +#### Login with ubuntu: + + # su ubuntu + $ cd ~ + +#### Clone this repo: + + $ git clone https://github.com/openstack/trove-integration.git + +#### Go into the scripts directory: + + $ cd trove-integration/scripts/ + +#### Running redstack is the core script: +*Run this to get the command list with a short description of each* + + $ ./redstack + +#### Install all the dependencies and then install trove via redstack. +*This brings up trove (rd-api rd-tmgr) and initializes the trove database.* + + $ ./redstack install + +*** + +#### Connecting to the screen session + + $ screen -x stack + +*If that command fails with the error* + + Cannot open your terminal '/dev/pts/1' + +*If that command fails with the error chmod the corresponding /dev/pts/#* + + $ chmod 660 /dev/pts/1 + +#### Navigate the log screens +To produce the list of screens that you can scroll through and select + + ctrl+a then " + +Num Name + +..... (full list ommitted) + +20 c-vol +21 h-eng +22 h-api +23 h-api-cfn +24 h-api-cw +25 tr-api +26 tr-tmgr +27 tr-cond + +Alternatively, to go directly to a specific screen window + + ctrl+a then ' + +then enter a number (like 25) or name (like tr-api) + +#### Detach from the screen session +Allows the services to continue running in the background + + ctrl+a then d + +*** + +#### Kick start the build/test-init/build-image commands +*Add mysql as a parameter to set build and add the mysql guest image. This will also populate /etc/trove/test.conf with appropriate values for running the integration tests.* + + $ ./redstack kick-start mysql + +*Optional commands if you did not run kick-start* + +#### Initialize the test configuration and set up test users (overwrites /etc/trove/test.conf) + + $ ./redstack test-init + +#### Build the image and add it to glance + + $ ./redstack build-image mysql + +*** + +### Reset your environment + +#### Stop all the services running in the screens and refresh the environment: + + $ killall -9 screen + $ screen -wipe + $ RECLONE=yes ./redstack install + $ ./redstack kick-start mysql + + or + + $ RECLONE=yes ./redstack install + $ ./redstack test-init + $ ./redstack build-image mysql + +*** + +### Recover after reboot +If the VM was restarted, then the process for bringing up Openstack and Trove is quite simple + + $./redstack start-deps + $./redstack start + +Use screen to ensure all modules have started without error + + $screen -r stack + +*** + +### Running Integration Tests +Check the values in /etc/trove/test.conf in case it has been re-initialized prior to running the tests. For example, from the previous mysql steps: + + "dbaas_datastore": "%datastore_type%", + "dbaas_datastore_version": "%datastore_version%", + +should be: + + "dbaas_datastore": "mysql", + "dbaas_datastore_version": "5.5", + +Once Trove is running on DevStack, you can use the dev scripts to run the integration tests locally. + + $./redstack int-tests + +This will runs all of the blackbox tests by default. Use the --group option to run a different group: + + $./redstack int-tests --group=simple_blackbox + +You can also specify the TESTS_USE_INSTANCE_ID environment variable to have the integration tests use an existing instance for the tests rather than creating a new one. + + $./TESTS_DO_NOT_DELETE_INSTANCE=True TESTS_USE_INSTANCE_ID=INSTANCE_UUID ./redstack int-tests --group=simple_blackbox + +*** + +### VMware Fusion 5 speed improvement +Running Ubuntu with KVM or Qemu can be extremely slow without certain optimizations. The following are some VMware settings that can improve performance and may also apply to other virtualization platforms. + +1. Shutdown the Ubuntu VM. + +2. Go to VM Settings -> Processors & Memory -> Advanced Options. + Check the "Enable hypervisor applications in this virtual machine" + +3. Go to VM Settings -> Advanced. + Set the "Troubleshooting" option to "None" + +4. After setting these create a snapshot so that in cases where things break down you can revert to a clean snapshot. + +5. Boot up the VM and run the `./redstack install` + +6. To verify that KVM is setup properly after the devstack installation you can run these commands. +``` +ubuntu@ubuntu:~$ kvm-ok +INFO: /dev/kvm exists +KVM acceleration can be used +``` + +### VMware Workstation performance improvements + +In recent versions of VMWare, you can get much better performance if you enable the right virtualization options. For example, in VMWare Workstation (found in version 10.0.2), click on VM->Settings->Processor. + +You should see a box of "Virtualization Engine" options that can be changed only when the VM is shutdown. + +Make sure you check "Virtualize Intel VT-x/EPT or AMD-V/RVI" and "Virtualize CPU performance counters". Set the preferred mode to "Automatic". + +Then boot the VM and ensure that the proper virtualization is enabled. + +``` +ubuntu@ubuntu:~$ kvm-ok +INFO: /dev/kvm exists +KVM acceleration can be used +``` diff --git a/integration/apidocs/pom.xml b/integration/apidocs/pom.xml new file mode 100644 index 0000000000..9351619456 --- /dev/null +++ b/integration/apidocs/pom.xml @@ -0,0 +1,193 @@ + + + 4.0.0 + + com.rackspace.cloud.dbaas + dbaas-docs + 1.0.0 + + Database Public API Spec + jar + + + + Rackspace Research Repositories + + true + + + + rackspace-research + Rackspace Research Repository + http://maven.research.rackspacecloud.com/content/groups/public/ + + + + + rackspace-research + Rackspace Research Repository + http://maven.research.rackspacecloud.com/content/groups/public/ + + + + + + UTF-8 + 1.5.0 + + + + src + + + ../xsd + + + + + com.rackspace.cloud.api + clouddocs-maven-plugin + + + ${doctools.version} + + + g1 + + generate-pdf + generate-webhelp + + generate-sources + + true + src/resources + false + + cdb-mgmt-devguide.xml + ../../cdb-mgmt-devguide-internal.pdf + intranet + mike.asthalter@rackspace.com + + + + + + + + g2 + + generate-pdf + generate-webhelp + + generate-sources + + true + src/resources + false + 2 + + + ../../../cdb-devguide-latest.pdf + 1 + UA-23102455-4 + cdb-devguide.xml + http://docs.rackspace.com/cdb/api/v1.0/cdb-devguide/content + + + + + + + + + g3 + + generate-pdf + generate-webhelp + + generate-sources + + true + src/resources + false + + cdb-getting-started.xml + ../../../cdb-getting-started.pdf + 1 + UA-23102455-4 + http://docs.rackspace.com/cdb/api/v1.0/cdb-getting-started/content + + + + + + + + + + g4 + + generate-webhelp + generate-pdf + + generate-sources + + true + src/resources + false + ../../../cdb-releasenotes-latest.pdf + cdb-releasenotes.xml + 1 + mike.asthalter@rackspace.com + 1 + 0 + external + http://docs.rackspace.com/cdb/api/v1.0/cdb-releasenotes/content + + + + + + + + + + + g5 + + generate-pdf + generate-webhelp + + generate-sources + + true + src/resources + false + ../../../cdb-releasenotes-latest.pdf + cdb-releasenotes.xml + intranet + mike.asthalter@rackspace.com + 1 + 0 + internal + http://docs-internal.rackspace.com/cdb/api/v1.0/cdb-releasenotes/content + + + + + + + + + + + + 1 + mike.asthalter@rackspace.com + rackspace + + + + + diff --git a/integration/apidocs/src/resources/cdb-devguide.xml b/integration/apidocs/src/resources/cdb-devguide.xml new file mode 100644 index 0000000000..5c2ae69b31 --- /dev/null +++ b/integration/apidocs/src/resources/cdb-devguide.xml @@ -0,0 +1,1675 @@ + + + + + + + + GET'> + PUT'> + POST'> + DELETE'> + + + + + + + + + + + + '> + + + + + + '> +]> + + + + Rackspace Cloud Databases Developer Guide + Rackspace Cloud Databases Developer + Guide + + + + + + + + Rackspace Cloud + + + + 2010 + 2011 + 2012 + Rackspace US, Inc. + + API v1.0 + Rackspace Cloud Databases + 2012-09-04 + + + Copyright details are filled in by the template. + + + + This document is intended for software developers + interested in developing applications using the + Rackspace Cloud Databases Application Programming + Interface (API). + + + + + 2012-09-04 + + + + Added information for pricing and + service level (refer to ). + + + Updated maximum volume size for a + database instance (refer to ). + + + + + + 2012-08-21 + + + + Changed FAILED status for database + instance to ERROR instead (see ). + + + List reserved names that cannot be + used for creating databases (see ) and users (see ). + + + + + + 2012-08-01 + + + + Initial Unlimited Availability (UA) + release for Rackspace Cloud + Databases. + + + + + + + + this is a placeholder for the front cover + + + this is a placeholder for the back cover + + + cdb + 2 + + + + Overview + Rackspace Cloud Databases is an OpenStack-based MySQL + relational database service that allows Rackspace + customers to easily provision database instances of + varying virtual resource sizes without the need to + maintain and/or update MySQL. Interactions with Cloud + Databases occur programmatically via the Cloud Databases + API as described in this developer guide. + + Rackspace recommends that Cloud Databases users back + up their data using mysqldump until backups are supported + in Cloud Databases. + + The following figure shows an + overview of Cloud Databases Infrastructure: + + + + + + + + + + + Writer: need to get architecture diagram for DBaaS. + Emailed Daniel 5/1/12. + We welcome feedback, comments, and bug reports at http://feedback.rackspacecloud.com. + Writer: check whether following statement should be + added back in for public (not private) beta: Issues and + bug reports can be directed to your support team via + ticket, chat, email, or phone. + +
+ Intended Audience + This Guide is intended to assist software + developers who want to develop applications using the + Cloud Databases API. It assumes the reader has a + general understanding of databases and is familiar + with: + + + ReSTful web services + + + HTTP/1.1 conventions + + + JSON and/or XML data serialization + formats + + + ATOM Syndication Format + + +
+ +
+ Document Change History + This version of the Developer Guide replaces and + obsoletes all previous versions. The most recent + changes are described in the table below: + +
+
+ Additional Resources + Descriptive information about Cloud Databases is + also published in its Web Application Description + Language (WADL) and XML Schema Definition (XSD). You + are welcome to read this information here: + + + The WADL is . + + + The XSD is . + + + You can download the most current versions of other + API-related documents from http://docs.rackspace.com/. + For more details about Rackspace Cloud Databases, + refer to http://www.rackspace.com/cloud/cloud_hosting_products/databases/. + This site also offers links to Rackspace's official + support channels, including knowledge center articles, + forums, phone, chat, and email. + Using this API document, your Rackspace Cloud + account, and at least one Cloud Server, you can get + started whenever you'd like. See the + Getting Started with Rackspace Cloud + Databases and Servers at http://docs.rackspace.com/ for information + about getting started using the API. + Please visit our Product Feedback Forum and let us know + what you think about Cloud Databases! + You can also follow Rackspace updates and + announcements via twitter at: http://www.twitter.com/rackspace. + This API uses standard HTTP 1.1 response codes as + documented at: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html. +
+ +
+ Pricing and Service Level + Cloud Databases is part of the Rackspace Cloud and + your use through the API will be billed as per the + pricing schedule at http://www.rackspace.com/cloud/public/databases/pricing. + The Service Level Agreement (SLA) for Cloud + Databases is available at http://www.rackspace.com/cloud/legal/sla/#cloud_databases. +
+ + + + Concepts + + To use the Cloud Databases API effectively, you should + understand several key concepts: + Reviewer: Daniel Morris is asking Daniel Salinas to + do an initial write-up of this chapter. +
+ Database Instance + A database instance is an isolated MySQL instance in + a single tenant environment on a shared physical host + machine. + Writer: once we support + MSSQL, we need to describe here what is used for MSSQL + in place of database instance. +
+
+ Database + A MySQL database within a database instance. + Writer: once we support + MSSQL, we need to modify the wording here, such as: + The actual database, whether it is in MySQL or + MSSQL. +
+
+ Flavor + A flavor is an available hardware configuration for + a database instance. Each flavor has a unique + combination of memory capacity and priority for CPU + time. +
+
+ Volume + A volume is user-specified storage that contains the + MySQL data directory. Volumes are automatically + provisioned on shared Internet Small Computer System + Interface (iSCSI) storage area networks (SAN) that + provide for increased performance, scalability, + availability and manageability. Applications with high + I/O demands are performance optimized and data is + protected through both local and network RAID-10. + Additionally, network RAID provides synchronous + replication of volumes with automatic failover and + load balancing across available storage + clusters. +
+ + + General API Information + The Cloud Databases API is implemented using a ReSTful + web service interface. Like other products in the + Rackspace Cloud suite, the Database Service shares a + common token-based authentication system that allows + seamless access between products and services. + + All requests to authenticate against and operate the service are performed using + SSL over HTTP (HTTPS) on TCP port 443. + +
+ Authentication + Every ReST request against the Database Service + requires the inclusion of a specific authorization + token, supplied by the X-Auth-Token HTTP + header. Customers obtain this token by first using the + Rackspace Cloud Authentication Service and supplying a + valid username and API access key. +
+ Geographic Endpoints + The Rackspace Cloud Authentication Service + serves as the entry point to all Rackspace Cloud + APIs and is itself a ReSTful web service. + To access the Authentication Service, you must + know whether your account is US-based or UK-based: + + + US-based accounts authenticate through + &ENDPOINT-US-20;. + + + UK-based accounts authenticate through + &ENDPOINT-UK-20;. + + + Your account may be based in either the US or + the UK; this is not determined by your physical + location but by the location of the Rackspace + retail site which was used to create your account: + + + If your account was created via http://www.rackspacecloud.com, + it is a US-based account. + + + If your account was created via http://www.rackspace.co.uk, it + is a UK-based account. + + + If you are unsure how your account was created, + use the Rackspace contact information at either + site to ask for help. +
+
+ Retrieving the Authentication Token + + + + &POST; + v2.0/tokens + Authenticate to receive a + token and a service catalog. + + + + Normal Response Code(s): + 200, + 203 + + Error Response Code(s): unauthorized + (401), userDisabled + (403), badRequest + (400), authFault + (500), + serviceUnavailable (503) + The authenticate operation provides clients + with an authentication token and a list of + regional cloud endpoints. The sample requests and + responses in this section illustrate a general + case. In your authentication request, use your own + credentials rather than the sample values shown + here for username and + apiKey. When you authenticate + successfully, the response to your authentication + request will include a catalog of the services to + which you have subscribed rather than the sample + values shown here. + + Auth Request: XML + + + + + + + + + + + + Auth Request: JSON + + + + + + + + + + + + + The username supplied here is your + common Rackspace Cloud username. + + + The key is your API access key. The key + can be obtained from the Rackspace Cloud + Control Panel in the <Your + Account>/API + Access section (login + here: Control Panel Login). + + + + Auth Response: XML + + + + + + + + + + + + + + + + + + + + Auth Response: JSON + + + + + + + + + + + + + + + + + + + The information shown in the Auth Response + examples is for US-based accounts. If you + authenticate against the UK-endpoint for auth, + you will see the service catalog information + for UK-based accounts. + + + + + In XML responses only, + a list of namespaces identifies API extensions that add functionality to the core API. + + + + + This token can be presented to a service as evidence of authentication. + Tokens are valid for a finite duration; a token's default lifespan is twenty-four hours. + + The token's expires attribute denotes the time + after which the token will automatically become + invalid. A token may be manually revoked before + the time identified by the expires + attribute; expires predicts a token's + maximum possible lifespan but does not guarantee + that it will reach that lifespan. Clients are + encouraged to cache a token until it expires. + + + + Users can be assigned multiple roles, + with each role providing specific + privileges. In this example, + jsmith is the + administrative user for the account, + holding the fully-privileged + identity:admin role. + Other users might hold other roles with + different privileges. Roles need not be + associated with actual job functions such + as Administrator, Operator, Developer, + Tester, or Trainer. + + + + The service catalog lists the services + this user can access. In this example, the + user can access one database service, one + loadbalancing service, two compute + services (Cloud Servers OpenStack and + Cloud Servers), two object storage + services (Cloud Files Content Distribution + Network (CDN), and Cloud Files), and one + DNS service. The catalog listing for each + service provides at least one endpoint URL + for that service. Other information, such + as regions, versions, and tenants, is + provided if it's relevant to this user's + access to this service. + + + + + The service type attribute identifies services that perform similar functions, whatever those services might be named. + In this example, the services named cloudServers and cloudServersOpenstack are both identified as type="compute", + identifying them as compute services even though the word "compute" does not appear in their names. + + + Use service type as the primary value for + locating a service. If multiple endpoints of the + same service type exist in the same region, use + service name as the tiebreaker. + + + + + + The service name attribute identifies each unique service in the catalog. + Once a service is created, its name does not change. However, new services of the same service type may be added to the catalog with new names. + + + If you are programmatically parsing an authentication + response, use service type rather than service name as + the basis for determining whether a user has access to + a particular kind of service. Service type is stable + across all releases; new service types may be developed, + but existing service types are not renamed. + In this example, + type="compute" identifies all the + available compute services, one of which is named + cloudServers and one of which is named + cloudServersOpenStack. New compute service names may be added + in future releases; whatever the compute services are + named, you can always + recognize them by parsing for + type="compute" in the authentication + response's service catalog. + + + + + A service may expose endpoints in different regions. + Regional endpoints allow clients to provision + resources in a manner that provides high + availability. + Some services are not region-specific. These services supply a single + non-regional endpoint and do not provide access to internal URLs. + + + + Some services recognize specification of a tenant. If a + service does recognize tenants, the format of the + tenant specification is defined only by the + service; for details about whether and how to + specify a tenant, check the documentation for the + service you are using. + + + + + An endpoint can be assigned public and internal URLs. A + public URL is accessible from anywhere. Access to a public + URL usually incurs traffic charges. Internal URLs are only + accessible to services within the same region. Access to + an internal URL is free of charge. + + + + + Authentication tokens are typically valid for 24 + hours. Applications should be designed to + re-authenticate after receiving a 401 + (Unauthorized) response from a service endpoint. + + If you are programmatically parsing an + authentication response, please be aware that + service names are stable for the life of the + particular service and can be used as keys. + You should also be aware that a user's service + catalog can include multiple uniquely-named + services which perform similar functions. For + example, cloudServersOpenStack is the + OpenStack version of compute whereas + cloudServers is the legacy version of compute; + the same user can have access to both + services. In Auth 2.0, the service type + attribute can be used as a key by which to + recognize similar services; see the tip + below. + + + Beginning with Auth 2.0, the service catalog + includes a service type attribute to identify + services that perform similar functions but + have different names; for example, + type="compute" identifies + compute services such as cloudServers and + cloudServersOpenStack. Some developers have + found the service type attribute to be useful + in parsing the service catalog. For additional + information on Auth 2.0 (also known as the + Cloud Identity Service), refer to the + Cloud Identity Client Developer + Guide at http://docs.rackspace.com/. + + Databases service endpoints are published in the + service catalog in the Auth response with the + account number, which is a required element of the + service endpoints. The examples shown here are for + authentication for US customers. Customers with + UK-based accounts will see different values in the + service catalog. Refer to the next section for + more information about service endpoints. +
+
+
+ Service Access/Endpoints + The Database Service is a regionalized service. The + user of the service is therefore responsible for + appropriate replication, caching, and overall + maintenance of Cloud Databases data across regional + boundaries to other Cloud Databases servers. + You can find the available service access/endpoints + for Cloud Databases summarized in the table + below. + + + + + + + + + + + + + + + + + + + + + + + + +
Regionalized Service Endpoints
RegionEndpoint
Chicago (ORD)https://ord.databases.api.rackspacecloud.com/v1.0/1234/
Dallas/Ft. Worth (DFW)https://dfw.databases.api.rackspacecloud.com/v1.0/1234/
London (LON) + https://lon.databases.api.rackspacecloud.com/v1.0/1234/
+ + + Replace the sample account ID number, + 1234, with your actual + account number returned as part of the authentication + service response. + You will find the actual account number after the + final '/' in the publicURL field returned + by the authentication response. For example, in you can see + from the publicURL field for + cloudServers + ("https://servers.api.rackspacecloud.com/v1.0/1100111") that the account + number is 1100111. +
+
+ Cloud Databases Service Versions + The Cloud Databases version defines the contract + and build information for the API. +
+ Contract Version + The contract version denotes the data model and + behavior that the API supports. The requested + contract version is included in all request URLs. + Different contract versions of the API may be + available at any given time and are not guaranteed + to be compatible with one another. + + + Example Request URL (contract version + in <emphasis role="strong" + >bold</emphasis>) + https://ord.databases.api.rackspacecloud.com/v1.0/1234 + + + This document pertains to contract + version 1.0. + + +
+
+ API Version + The API List Versions call is available to show + the current API version as well as information + about all versions of the API. Refer to for details. +
+
+
+ Request/Response Types + The Cloud Databases API supports both the JSON and + XML data serialization formats. The request format is + specified using the Content-Type header + and is required for calls that + have a request body. The response format can be + specified in requests either by using the + Accept header or by adding an + .xml or .json extension + to the request URI. Note that it is possible for a + response to be serialized using a format different + from the request. If no response format is specified, + JSON is the default. If conflicting formats are + specified using both an Accept header and + a query extension, the query extension takes + precedence. + Some operations support an Atom representation that + can be used to efficiently determine when the state of + services has changed. + Reviewer: the previous sentence will be hidden + for the Private Beta, since it does not appear that + Atom will be supported yet. Correct? + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Response Formats
FormatAccept HeaderQuery ExtensionDefault
JSONapplication/json.jsonYes
XMLapplication/xml.xmlNo
ATOMapplication/atom+xml.atomNo
+ Reviewer: the ATOM row in the table above will be + hidden for the Private Beta, since it does not appear + that Atom will be supported yet. Correct? + In the request example below, notice that + Content-Type is set to + application/json, but + application/xml is + requested via the Accept + header: + + Request with Headers: JSON + + Missing code sample! + + Therefore an XML response format is returned: + + Response with Headers: XML + + Missing code sample! + +
+
+ Synchronous and Asynchronous Responses + Reviewer: please give me the updated info for this + section. Need to replace info about callback URL, + etc. + All successful &GET; requests are + synchronous calls, since they + are always retrieving (reading) existing information. + With these requests, the caller waits until the call + returns with the specified code and response body. For + an example, see XXXX. + &PUT;, &POST;, and &DELETE; calls are asynchronous, however, + since they may take some time to process. Therefore they return 202 ACCEPTED + responses containing information with a callback URL, which allows the progress, + status, and/or response information of the call to be retrieved at a later point in + time. The asynchronous response body will look similar to the following examples, + depending on the format requested: + + 202 ACCEPTED Response: XML + + Reviewer: need code example + + + 202 ACCEPTED Response: JSON + + Reviewer: need code example + + The following table shows the attributes for asynchronous responses: + + + + + + + + + + + + + + + + + + + +
Attributes for Asynchronous Responses
AttributeDescription
jobIdAn identifier for the specific request.
callbackUrlResource locator for querying the status of the request.
+ + The status for asynchronous calls is retained for up to 24 hours. + + + If a request body does not pass initial validation or an error condition + arises, you may receive an immediate error response from the request. + + When a request is made to the callback URL provided + and the job is still running, another + 202 ACCEPTED response + is returned with the same information as the previous + one. If the request is complete, the response will be + as if the original call returned as normal, without + waiting. For example, if a Create Database request was + issued and a 202 asynchronous response was returned, + the response from querying the callback URL for a + completed successful database creation would be a + 200 OK and contain the + information for the created database. See XXXX for a + specific example. + If an error occurs during the processing of the + create request, querying the callback URL will return + the details of the error, as if the original call + returned the error response. For example, if a + validation error occurs during the Create Database + request above, the response from querying the callback + URL would be a 400 BAD + REQUEST and contain details regarding the specific + validation error. + + If the response from querying a callback URL is a + 404 NOT FOUND, the details of the error in the + response body will contain information the caller may use to determine whether + the specified job itself was not found, or if the response from the original + request was a 404 NOT FOUND. + + The description of each &PUT;, &POST;, and &DELETE; + request identifies the response codes that can + indicate success or error for that request. For + example, see XXXX immediately below the table for a + list of the successful and error response codes for + the POST /xxxx call. +
+
+ Content Compression + Reviewer: I am hiding this entire section for the + Private Beta, since I'm not sure that it applies. Is + that correct? + Request and response body data may be encoded with gzip compression to accelerate + interactive performance of API calls and responses. This is controlled using the + Accept-Encoding header on the request from the client and indicated + by the Content-Encoding header in the server response. Unless the + header is explicitly set, encoding defaults to disabled. + + + + + + + + + + + + + + + + + + + + + + +
Encoding Headers
Header TypeNameValue
HTTP/1.1 RequestAccept-Encodinggzip
HTTP/1.1 ResponseContent-Encodinggzip
+
+
+ Persistent Connections + Reviewer: I am hiding this entire section for the + Private Beta, since I'm not sure that it applies. Is + that correct? + + By default, the API supports persistent connections + via HTTP/1.1 keepalives. All connections will be kept + alive unless the connection header is set to close. + + + To prevent abuse, HTTP sessions have a timeout of 20 + seconds before being closed. + + + + The server may close the connection at any time + and clients should not rely on this behavior. + + +
+ +
+ Limits + + All accounts, by default, have a preconfigured set of + thresholds (or limits) to manage capacity and prevent + abuse of the system. The system recognizes two kinds + of limits: rate limits and + absolute limits. Rate limits + are thresholds that are reset after a certain amount + of time passes. Absolute limits are fixed. + +
+ Rate Limits + Rate limits are specified in terms of both a + human-readable wild-card URI and a + machine-processable regular expression. The + regular expression boundary matcher '^' takes + effect after the root URI path. For example, the + regular expression ^/v1.0/instances would match + the bolded portion of the following URI: + https://ord.databases.api.rackspacecloud.com/v1.0/instances. + The following table specifies the default rate + limits for all API operations for all &GET;, + &POST;, &PUT;, and &DELETE; calls for databases + and database instances: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Default Rate Limits
VerbURIRegExDefault
&GET; changes-since*/instances/*^/vd+.d+/instances.*3/minute
&POST;*/instances/*^/vd+.d+/instances.*10/minute
&POST; instances*/instances/*^/vd+.d+/instances.*50/day
&PUT;*/instances/*^/vd+.d+/instances.*10/minute
&DELETE;*/instances/*^/vd+.d+/instances.*100/minute
+ Rate limits are applied in order relative to the + verb, going from least to most specific. For + example, although the threshold for &POST; to + /v1.0/* is 10 per minute, one cannot &POST; to + /v1.0/* more than 50 times within a single day. + If you exceed the thresholds established for + your account, a 413 (Rate + Control) HTTP response will be + returned with a Retry-After header to + notify the client when it can attempt to try + again. +
+
+ Absolute Limits + Reviewer: Need to update this entire section. + Please give me your updates. + Refer to the following table for the absolute + limits that are set. + + + + + + + + + + + + + + + + + + + + + +
Absolute Limits
NameDescriptionLimit
InstancesMaximum number of instances allowed + for your account5
Volume SizeMaximum volume size per + instance in gigabytes (GB) for your + account50
+
+
+
+ Date/Time Format + The Database Service uses an ISO-8601 compliant + date format for the display and consumption of + date/time values. + The system timezone is in UTC. MySQL converts TIMESTAMP values from + the current time zone to UTC for storage, and back + from UTC to the current time zone for retrieval. + This does not occur for other types, such as DATETIME. + + + DB Service Date/Time Format + yyyy-MM-dd'T'HH:mm:ss.SSSZ + See the table below for a description of the date/time format codes. + May 19th, 2011 at 8:07:08 AM, GMT-5 would have the following + format: + 2011-05-19T08:07:08-05:00 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Explanation of Date/Time Format Codes
CodeDescription
yyyyFour digit year
MMTwo digit month
ddTwo digit day of month
TSeparator for date/time
HHTwo digit hour of day (00-23)
mmTwo digit minutes of hour
ssTwo digit seconds of the minute
SSSThree digit milliseconds of the second
ZRFC-822 timezone
+ +
+
+ Pagination + To reduce load on the service, list operations + return a maximum of 20 items at a time. This is + referred to as pagination. Cloud + Databases has separate paging limits for instances, + databases, and users, which are currently all set to + 20. If a request supplies no limit or one that exceeds + the configured default limit, the default is used + instead. + Pagination provides the ability to limit the size + of the returned data as well as retrieve a specified + subset of a large data set. Pagination has two key + concepts: limit and marker. Limit + is the restriction on the maximum number of items for + that type that can be returned. + Marker is the ID of the last + item in the previous list returned. The ID is the UUID + in the case of instances, and the name in the case of + databases and users. For example, a query could + request the next 10 instances after the instance + "1234" as follows: + ?limit=10&marker=1234. Items are + displayed sorted by ID. + Pagination applies only to the calls listed in the + following table: + + + + Verb + URI + Description + + + + + &GET; + /instances/ + Lists the status and + information for all database + instances. + + + &GET; + + /instances/{instanceId}/databases + Lists databases for the + specified instance. + + + &GET; + /instances/{instanceId}/users + Lists the users in the + specified database instance. + + + + If the content returned by a call is paginated, the + response includes a structured link much like an + instance item's links, with the basic structure + {"href": "<url>", "rel": "next"}. + Any response that is truncated by pagination will have + a next link, which points to the + next item in the collection. If there are no more + items, no next link is + returned. + See the examples of paged List Instances calls that + follow. + + List Instances Paged Request: XML + + + + + + + List Instances Paged Request: JSON + + + + + + Notice that the paged request examples above set the + limit to 2 (?limit=2), so the responses + that follow each show 2 instances and return a + marker set to the UUID of the + last item in the returned list + (?marker=4137d6a4-03b7-4b66-b0ef-8c7c35c470d3). + Also a link is provided to retrieve the next 2 results + (limit=2) in the link element + identified by the attribute rel="next" + (XML) or "rel":"next" (JSON): + + List Instances Paged Response: XML + + + + + + + List Instances Paged Response: JSON + + + + + +
+
+ Efficient Polling with the + <parameter>Changes-Since</parameter> + Parameter + Reviewer: I have hidden this section, since it + does not appear that it will be supported for Private + Beta. Correct? + The ReST API allows you to poll for the status of + certain operations by performing a &GET; on various + URIs. Rather than re-downloading and re-parsing the + full status at each polling interval, your ReST client + may use the changes-since + parameter to check for changes since a previous + request. The changes-since time + is specified as Unix time (the number of seconds since + January 1, 1970, 00:00:00 UTC, not counting leap + seconds). If nothing has changed since the + changes-since time, a + 304 (Not Modified) + response will be returned. If data has changed, only + the items changed since the specified time will be + returned in the response. + Reviewer: does the following sentence apply, and + should it be included? + For example, performing a &GET; against + https://api.servers.rackspacecloud.com/v1.0/224532/servers?changes-since=1244012982 + would list all servers that have changed since Wed, 03 + Jun 2009 07:09:42 UTC. +
+
+ Faults + When an error occurs, the Database Service returns + a fault object containing an HTTP error response code + that denotes the type of error. In the body of the + response, the system will return additional + information about the fault. + The following table lists possible fault types with their associated error codes + and descriptions. + + + + Fault Type + Associated Error Code + Description + + + + + badRequest + 400 + There was one or more errors in the user request. + + + unauthorized + 401 + The supplied token is not authorized to access the resources, either it's expired or invalid. + + + forbidden + 403 + Access to the requested + resource was denied. + + + itemNotFound + 404 + The back-end services did not + find anything matching the + Request-URI. + + + badMethod + 405 + The request method is not allowed for this resource. + + + overLimit + 413 + Either the number of entities in the request is larger than + allowed limits, or the user has exceeded allowable request rate limits. + See the details element for more specifics. Contact support + if you think you need higher request rate limits. + + + badMediaType + 415 + The requested content type is not supported by this service. + + + unprocessableEntity + 422 + The requested resource could + not be processed on at the moment. + + + instanceFault + 500 + This is a generic server error and the message contains the reason for the error. This error could wrap several error messages and is a catch all. + + + notImplemented + 501 + The requested method or resource is not implemented. + + + serviceUnavailable + 503 + The Database Service is not + available. + + + + The following two instanceFault + examples show errors when the server has erred or + cannot perform the requested operation: + + + Example instanceFault Response: XML + + + + + + + Example Fault Response: JSON + + + + + + The error code (code) is returned in the body of the response for + convenience. The message element returns a human-readable message that + is appropriate for display to the end user. The details element is + optional and may contain information that is useful for tracking down an error, such + as a stack trace. The details element may or may not be appropriate for + display to an end user, depending on the role and experience of the end user. + The fault's root element (for example, + instanceFault) may change depending + on the type of error. + The following two badRequest examples + show errors when the volume size is invalid: + + Example badRequest Fault on Volume Size Errors: + XML + + + + + + + Example badRequest Fault on Volume Size Errors: + JSON + + + + + + The next two examples show + itemNotFound errors: + + Example itemNotFound Fault: XML + + + + + + + Example itemNotFound Fault: JSON + + + + + +
+
+ Database Instance Status + When making an API call to create, list, or delete + database instance(s), the following database instance + status values are possible: + + + BUILD – The database instance is being provisioned. + + + REBOOT – The database instance is + rebooting. + + + ACTIVE – The database instance is + online and available to take requests. + + + BLOCKED – The database instance is + unresponsive at the moment. + + + RESIZE – The database instance is being + resized at the moment. + + + SHUTDOWN – The database instance is + terminating services. Also, SHUTDOWN is + returned if for any reason the MySQL instance + is shut down but not the actual server. + + + If MySQL has crashed (causing the + SHUTDOWN status), please call support + for assistance. + + + + + ERROR – The last operation for the + database instance failed due to an + error. + + +
+
+ Database Instance Accessibility + Database instances are directly accessible only on + the internal ServiceNet network and using a Cloud + resource within the same regional datacenter. For + example, a database instance in DFW can only be + accessed by a Cloud Server in DFW. + Note that using a public Cloud Load Balancer allows + you to access your ServiceNet database instance from + the public network by performing the following steps: + + Using the Cloud Databases API, create a + database instance. For details see . + + + Using the Cloud Load Balancers API, + create a public load balancer and specify + your database instance hostname as a node, + or use the API to add your instance + hostname as a node to an existing public + load balancer. For details refer to the + Cloud Load Balancers + Developer Guide at http://docs.rackspace.com/api/. + + +
+ + + + API Operations + + Do not use trailing slashes (/) at the end of calls + to API operations, since this may cause the call to + fail. For example, do not use &GET; /instances/detail/ + (with the trailing slash at the end). Rather, use + &GET; /instances/detail instead. + + +
+ API Versions + This section describes the versions that are + supported for the Cloud Databases API. + + + + + + + + + +
+ +
+ Database Instances + This section describes the operations that are supported for database instances. + + + + + + + + + + + + + + + +
+ +
+ Database Instance Actions + This section describes the actions that are supported for database instances. + + + + + + + +
+ +
+ Databases + This section describes the operations that are + supported for databases. + + + + + + + + + +
+ +
+ Users + This section describes the operations that are + supported for managing database users. + + + + + + + + + +
+ +
+ Flavors + This section describes the operations that are + supported for flavors. + + + + + + + + + +
+ + + Glossary + + database + + A MySQL database within a database instance. + + + + database instance + + A database instance is an isolated MySQL instance in a single tenant environment on a + shared physical host machine. Also referred to as instance. + + + + flavor + + A flavor is an available hardware configuration for a database instance. Each flavor has a unique combination of memory capacity and priority for CPU time. + + + + volume + + A volume is user-specified storage that contains the MySQL data directory. + Volumes are automatically provisioned on shared Internet Small Computer System Interface (iSCSI) + storage area networks (SAN) that provide for increased performance, scalability, availability + and manageability. Applications with high I/O demands are performance optimized and data is + protected through both local and network RAID-10. Additionally, network RAID provides synchronous + replication of volumes with automatic failover and load balancing across available storage clusters. + + + + diff --git a/integration/apidocs/src/resources/cdb-getting-started.xml b/integration/apidocs/src/resources/cdb-getting-started.xml new file mode 100644 index 0000000000..ba3275b356 --- /dev/null +++ b/integration/apidocs/src/resources/cdb-getting-started.xml @@ -0,0 +1,1131 @@ + + + + + + + + + GET'> + PUT'> + POST'> + DELETE'> + + + + + + + + + MAY'> + SHOULD'> + MUST'> + MUST NOT'> +]> + + + + + Rackspace Cloud Databases Getting Started Guide + Rackspace Cloud Databases and Servers Getting Started Guide + + + + + 2011 + 2012 + Rackspace US, Inc. + + API v1.0 + Rackspace Cloud Databases + 2012-09-04 + + + Copyright details are filled in by the template. + + + + + 2012-09-04 + + + + Added information for pricing and + service level (refer to ). + + + + + + 2012-08-02 + + + + Corrected request examples for + authentication (refer to ). + + + + + + 2012-08-01 + + + + Initial Unlimited Availability (UA) + release for Rackspace Cloud + Databases. + + + + + + + cdb + 1 + + + + Document Change History + This version of the Getting Started replaces and + obsoletes all previous versions. The most recent changes + are described in the table below: + + + + Overview + Follow the steps described in this guide to use the + Rackspace Cloud Databases API and the Cloud + Servers section of the Rackspace Cloud Control + Panel to create and access your database instances via + Rackspace Cloud Servers. + For details about using the Cloud Databases API, refer + to the Cloud Databases Developer + Guide at http://docs.rackspace.com/. + For more details about Rackspace Cloud Databases, refer + to http://www.rackspace.com/cloud/cloud_hosting_products/databases/. + This site also offers links to Rackspace's official + support channels, including knowledge center articles, + forums, phone, chat, and email. + Please visit our Product Feedback Forum and let us know what + you think about Cloud Databases! +
+ Prerequisites for Running Examples + In order to run the examples in this guide, you must + have the following prerequisites: + + Rackspace Cloud account + + + Rackspace Cloud username and password, + as specified during registration + + +
+
+ Pricing and Service Level + Cloud Databases is part of the Rackspace Cloud and + your use through the API will be billed as per the + pricing schedule at http://www.rackspace.com/cloud/public/databases/pricing. + Cloud Servers is also part of the Rackspace Cloud and + your use through the Control Panel will be billed as + per the pricing schedule at http://www.rackspace.com/cloud/public/servers/pricing. + The Service Level Agreements (SLAs) for Cloud + Databases and Cloud Servers are available at http://www.rackspace.com/cloud/legal/sla/#cloud_databases + and http://www.rackspace.com/cloud/legal/sla + respectively. +
+ + + Send Requests to the API + You have several options for sending requests through an + API: + + + Developers and testers may prefer to use cURL, + the command-line tool from http://curl.haxx.se/. + With cURL you can send HTTP requests and receive + responses back from the command line. + + + If you like to use a more graphical interface, + the ReST client for Firefox also works well for + testing and trying out commands, see https://addons.mozilla.org/en-US/firefox/addon/restclient/. + + + You can also download and install rest-client, a + Java application to test ReSTful web services, + from http://code.google.com/p/rest-client/. + + +
+ Send API Requests Using cURL + cURL is a command-line tool that is available on + most UNIX®-like environments and Mac OS X® and can be + downloaded for Windows® in order to interact with the + ReST interfaces. For more information on cURL, visit + http://curl.haxx.se/. + cURL allows you to transmit and receive HTTP requests and responses from the + command-line or from within a shell script. This makes it possible to work with the ReST API + directly without using one of the client APIs. + The following cURL command-line options will be used + in this guide to run the examples: + + cURL Command-Line Options + + + + Sends the specified data in a post + request to the HTTP server. + + + + + + Includes the HTTP header in the + output. + + + + + + Specify an HTTP header in the request. + + + + + If you have the tools, you can run the cURL JSON + request examples with the following options to + format the output from cURL: <curl + JSON request example> | python + -mjson.tool. + +
+
+ Copying and Pasting cURL Request Examples into a + Terminal Window + To run the cURL request examples shown in this guide + on Linux or Mac systems, copy and paste each example + from the HTML version of this guide into an ASCII + editor (for example vi or TextEdit). Then modify each example + with your required account information and so forth, + as detailed in this guide. + + The carriage returns in the cURL request + examples that are part of the cURL syntax are + escaped with a backslash ('\') in order to avoid + prematurely terminating the command. However you + should not escape carriage returns inside the xml + or json message within the command. + + Consider the following cURL Authenticate Request: + XML example that is described in detail in : + + cURL Authenticate Request: XML + + curl \ +'<?xml version="1.0" encoding="UTF-8"?> + <auth>    + <apiKeyCredentials      + xmlns="http://docs.rackspace.com/identity/api/ext/RAX-KSKEY/v1.0"      + username="your_username"      + apiKey="your_api_key"/>    + </auth> \ +'https://identity.api.rackspacecloud.com/v2.0/tokens' + + You can see that the lines that are part of the cURL + command syntax have all been escaped with a backslash + ('\') to indicate that the command continues on the + next line: + curl \ + +    +(... lines within the xml portion of the message are not shown in this example) +(... the example only shows lines that are part of cURL syntax)      +      +     +    + </auth> \ +'https://identity.api.rackspacecloud.com/v2.0/tokens' + However the lines within the + xml portion of the message are + not escaped with a backslash + ('\') in order to avoid issues with the xml + processing: + '<?xml version="1.0" encoding="UTF-8"?> + <auth>    + <apiKeyCredentials      + xmlns="http://docs.rackspace.com/identity/api/ext/RAX-KSKEY/v1.0"      + username="your_username"      + apiKey="your_api_key"/>    + </auth> \ + + + The final line of the xml message is escaped + since the backslash lies + outside the xml message + and continues the cURL command to the next + line. + + After you are finished modifying the text for the + cURL request example with your information (for + example your_username and + your_api_key), paste it + into your terminal window. Then execute the cURL + command by pressing Enter. + If you have trouble copying and pasting the examples + as described, try typing the entire example on one + long line, removing all the backslash line + continuation characters. +
+ + + Generate an Authentication Token + You need to generate a token whether you use cURL or a + ReST client. + In order to use the ReST API, you will first need to + obtain an authentication token, which will need to be + passed in for each request using the + X-Auth-Token header. + The following example demonstrates how to use cURL to + obtain the authentication token and the account number. + You will need to supply the authentication token and + account number when making subsequent Cloud Databases API + calls. + Remember to replace the names in the Authenticate + Request examples below with their respective + values: + + your_username — The + username is your common Rackspace Cloud + username, as supplied during + registration. + + + your_api_key — The key + is your API access key. The key can be + obtained from the Rackspace Cloud Control + Panel in the <Your + Account> / API + Keys section (login here: + Control Panel Login). + + + To access the Authentication Service, you must know + whether your account is US-based or UK-based: + + + US-based accounts authenticate through &ENDPOINT-US-20;. + + + UK-based accounts authenticate through &ENDPOINT-UK-20;. + + + Your account may be based in either the US or the UK; + this is not determined by your physical location but by + the location of the Rackspace retail site which was used + to create your account: + + + If your account was created via http://www.rackspacecloud.com, it is a + US-based account. + + + If your account was created via http:/www.rackspace.co.uk, it is a + UK-based account. + + + If you are unsure how your account was created, use the + Rackspace contact information at either site to ask for + help. + Notice that you authenticate using a special URL for + Cloud authentication services. For example, for US-based + accounts, you use + https://identity.api.rackspacecloud.com/v2.0/tokens, + as shown in the following Authenticate Request examples. + Note that the v2.0 component in the URL + indicates that you are using version 2.0 of the Cloud Auth + API. + + cURL Authenticate Request: XML + + curl \ +'<?xml version="1.0" encoding="UTF-8"?> +<auth> + <apiKeyCredentials + xmlns="http://docs.rackspace.com/identity/api/ext/RAX-KSKEY/v1.0" + username="your_username" + apiKey="your_api_key"/> +</auth>' \ + \ + \ +'https://identity.api.rackspacecloud.com/v2.0/tokens' + + + cURL Authenticate Request: JSON + + curl \ +'{ + "auth": + { + "RAX-KSKEY:apiKeyCredentials": + { + "username": "your_username", + "apiKey": "your_api_key"} + } +}' \ + \ +'https://identity.api.rackspacecloud.com/v2.0/tokens' + + + For UK-based accounts, you would need to modify the + URL shown in the last line of each of the Authenticate + Request examples above to be + 'https://lon.identity.api.rackspacecloud.com/v2.0/tokens' + instead. + + + Authenticate Response: XML + Missing code sample! + + + Authenticate Response: JSON + Missing code sample! + + The authentication token id is returned + along with an expires attribute that + specifies when the token expires. + + Notes + + + For all response examples in this guide, the + field values you receive in your responses + will vary from those shown here since they + will be specific to your account. + + + The information shown in the Authenticate + Response examples above is for US-based + accounts. If you authenticate against the + UK-endpoint for auth, you will see the service + catalog information for UK-based + accounts. + + + The id attribute in the + Authenticate Response specifies the + authentication token. Tokens are valid for a + finite duration. + Remember to supply your authentication token + wherever you see the field your_auth_token in + the examples in this guide. + + + The expires attribute denotes + the time after which the token will + automatically become invalid. A token may be + manually revoked before the time identified by + the expires attribute; expires + predicts a token's maximum possible lifespan + but does not guarantee that it will reach that + lifespan. Clients are encouraged to cache a + token until it expires. + + + Applications should be designed to + re-authenticate after receiving a 401 + (Unauthorized) response from a service + endpoint. + + + + The publicURL endpoints for + cloudDatabases (for example + https://ord.databases.api.rackspacecloud.com/v1.0/1100111) + are also returned in the response. + You will find the actual account number after the final + '/' in the publicURL field. In this example, + you can see that the account number is 1100111. You need + to specify your account number on most of the Cloud + Databases API calls, wherever you see the field your_acct_id specified in the + examples in this guide. + After authentication, you can use cURL to perform &GET;, + &DELETE;, and &POST; requests for the Cloud Databases + API. + + + Service Access/Endpoints + The endpoints to use for your Cloud Databases API calls + are summarized in the table below. + + + + + + + + + + + + + + + + + + + + + + + + +
Regionalized Service Endpoints
RegionEndpoint
Chicago (ORD)https://ord.databases.api.rackspacecloud.com/v1.0/1234/ +
Dallas/Ft. Worth (DFW)https://dfw.databases.api.rackspacecloud.com/v1.0/1234/ +
London (LON) + https://lon.databases.api.rackspacecloud.com/v1.0/1234/ +
+ + + Notes: + + + Choose the endpoint from the table for the + datacenter where your Cloud resources are + located. + + + The Cloud Server that you use in must be + located in the same datacenter where your + database resides. + + + All examples in this guide assume that you + are operating against the ORD datacenter, + however if you are using a different + datacenter, be sure to use the associated + endpoint from the table above instead. + + + + Replace the sample account ID number, + 1234, with your actual account + number returned as part of the authentication response. + Use your actual account number wherever you see the field + your_acct_id + specified in this guide. Refer to . + When making a Cloud Databases API call, place the + endpoint at the beginning of the request URL, for example: + (https://ord.databases.api.rackspacecloud.com/v1.0/your_acct_id/), + as you can see in the cURL List Flavors Details Request + examples in . + + + List Flavors + A flavor is an available hardware configuration for a + database instance. Each flavor has a unique combination of + memory capacity and priority for CPU time. The larger the + flavor size you use, the larger the amount of RAM and + priority for CPU time your database instance will + receive. + You need to use the List Flavors API call + (/flavors) to find the available + configurations for your database instance, and then decide + which size you need. + This operation does not require a request body. + The following examples show the cURL requests for List + Flavors: + + cURL List Flavors Request: XML + curl -i \ +-H 'X-Auth-Token: your_auth_token' \ + \ +'https://ord.databases.api.rackspacecloud.com/v1.0/your_acct_id/flavors' + + + cURL List Flavors Request: JSON + curl -i \ +-H 'X-Auth-Token: your_auth_token' \ + \ +'https://ord.databases.api.rackspacecloud.com/v1.0/your_acct_id/flavors' + + Remember to replace the names in the examples above with + their actual respective values for all the cURL examples + that follow: + + your_auth_token — as + returned in your authentication response (see + the response examples in ) + + + your_acct_id — as + returned in your authentication response (must + be replaced in the request URL) + + + The following examples show the responses for List + Flavors: + + List Flavors Response: XML + Missing code sample! + + + List Flavors Response: JSON + Missing code sample! + + In the previous examples, you can see from the flavor + name that there are multiple flavors + available, including medium (with 1 virtual + CPU and 2 gigabytes of memory) and tiny (with + 1 virtual CPU and 0.5 gigabytes of memory). + In this example, assume that you decide to use the tiny + flavor (id 1) to provide the needed capacity for your + database instance. + Notice that there are two kinds of link relations + associated with flavor resources. A self link + contains a versioned link to the + flavor resource. These links should be used in cases where + the link will be followed immediately (as you will see in + the next section). A bookmark link provides a + permanent link to a flavor resource that is appropriate + for long term storage and works across API + versions. + + + + Create a Database Instance with a Database and a + User + A database instance is an isolated MySQL instance in + a single tenant environment on a shared physical host + machine. In the example below, you create a database + instance with a database and a user. The example instance + uses the tiny flavor and a volume size of 2 gigabytes + (GB). + Refer to Create Database in the Cloud + Databases Developer Guide for the + restrictions for choosing the database name. + The example creates a database instance + myrackinstance, with the following: + + the tiny flavor + + + volume size of 2 gigabytes (GB) + + + a database named sampledb with: + + utf8 character + set + + + utf8_general_ci + collation + + + + + a user simplestUser with + password password + + + + Notice that the request specifies the flavor reference + (flavorRef) to the tiny flavor (id 1) + that was returned by the List Flavors Details call: + "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/1". + This is the self link that is the versioned + link to the flavor resource. Refer to the response + examples in . + The following examples show the cURL requests for Create + Instance: + + cURL Create Instance Request: XML + curl \ +'<?xml version="1.0" ?> +<instance xmlns="http://docs.openstack.org/database/api/v1.0" + name="myrackinstance" + flavorRef= + "https://ord.databases.api.rackspacecloud.com/v1.0/your_acct_id/flavors/1"> + <databases> + <database name="sampledb" character_set="utf8" + collate="utf8_general_ci" /> + </databases> + <users> + <user name="simplestUser" password="password"> + <databases> + <database name="sampledb"/> + </databases> + </user> + </users> + <volume size="2" /> +</instance>' \ +-H 'X-Auth-Token: your_auth_token' \ + \ + \ +'https://ord.databases.api.rackspacecloud.com/v1.0/your_acct_id/instances' + + + cURL Create Instance Request: JSON + curl \ +'{ + "instance": { + "databases": [ + { + "character_set": "utf8", + "collate": "utf8_general_ci", + "name": "sampledb" + } + ], + "flavorRef": "https://ord.databases.api.rackspacecloud.com/v1.0/your_acct_id/flavors/1", + "name": "myrackinstance", + "users": [ + { + "databases": [ + { + "name": "sampledb" + } + ], + "name": "simplestUser", + "password": "password" + } + ], + "volume": + { + "size": 2 + } + } +}' \ +-H 'X-Auth-Token: your_auth_token' \ + \ +'https://ord.databases.api.rackspacecloud.com/v1.0/your_acct_id/instances' + + Remember to replace the names in the examples above with + their actual respective values for all the cURL examples + that follow: + + your_auth_token — as + returned in your authentication response (see + the response examples in ) + + + your_acct_id — as + returned in your authentication response (must + be replaced in the request URL) + + + Remember to replace the account id shown in the + flavorRef property for the examples with + your actual account id: + "https://ord.databases.api.rackspacecloud.com/v1.0/your_acct_id/flavors/1". + Rather than the flavor URI shown in the previous sentence, + you can also pass the flavor id (integer) as the value for + flavorRef. For example, the flavor id for the flavor URI + shown above is "1". + The following examples show the Create Instance + responses: + + + Create Instance Response: XML + Missing code sample! + + + + Create Instance Response: JSON + Missing code sample! + + You will need to specify the instance id returned (in + the response examples above: + id="d379ba5c-9a1f-4aa9-9a17-afe237d04c65") + on subsequent API calls that require it, for example List + Databases for Instance. + Note that the database and user are not listed in the + Create Instance responses. Next you will verify that both + were successfully created. + The operation of creating the database instance + may take up to several minutes. You will not be + able to perform the operations to List Databases + for Instance and List Users for Instance in the + sections that follow until the instance has ACTIVE + status. + + + + List Databases for Instance + In this section you will list the databases in the + specified database instance. + This operation does not require a request body. + The following + examples show the cURL requests for List Databases for + Instance: + + cURL List Databases for Instance Request: + XML + curl \ +-H 'X-Auth-Token: your_auth_token' \ + \ + \ +'https://ord.databases.api.rackspacecloud.com/v1.0/your_acct_id/instances/instance_id/databases' + + + + cURL List Databases for Instance Request: + JSON + curl \ +-H 'X-Auth-Token: your_auth_token' \ + \ +'https://ord.databases.api.rackspacecloud.com/v1.0/your_acct_id/instances/instance_id/databases' + + Remember to replace the names in the examples above with + their actual respective values: + + your_auth_token — as + returned in your authentication response (see + the examples in ) + + + your_acct_id — as + returned in your authentication response (see + the examples in ) + + + instance_id + — as returned in your create instance + response (see the examples in ) + + + The following examples show the responses for Create + User: + + List Databases for Instance Response: XML + HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 109 +Date: Thu, 05 Apr 2012 18:20:18 GMT + +<databases xmlns="http://docs.openstack.org/database/api/v1.0"> + <database name="sampledb"/> +</databases> + + + + List Databases for Instance Response: JSON + HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 37 +Date: Thu, 05 Apr 2012 18:13:53 GMT + +{ + "databases": [ + { + "name": "sampledb" + } + ] +} + + You can see that the database sampledb was + successfully created. Next you will list the users. + + + List Users in Database Instance + In this section you will list the users in the specified + database instance. + This operation does not require a request body. + The following examples show the cURL requests for List + Users in Database Instance: + + cURL List Users in Database Instance Request: + XML + curl \ +-H 'X-Auth-Token: your_auth_token' \ + \ + \ +'https://ord.databases.api.rackspacecloud.com/v1.0/your_acct_id/instances/instance_id/users' + + + + cURL List Users in Database Instance Request: + JSON + curl \ +-H 'X-Auth-Token: your_auth_token' \ + \ +'https://ord.databases.api.rackspacecloud.com/v1.0/your_acct_id/instances/instance_id/users' + + Remember to replace the names in the examples above with + their actual respective values: + + your_auth_token — as + returned in your authentication response (see + the examples in ) + + + your_acct_id — as + returned in your authentication response (see + the examples in ) + + + instance_id + — as returned in your create instance + response (see the examples in ) + + + The following examples show the responses for List + Users in Database Instance: + + List Users in Database Instance Response: + XML + HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 109 +Date: Thu, 05 Apr 2012 18:20:18 GMT + +<?xml version="1.0" ?> +<users xmlns="http://docs.openstack.org/database/api/v1.0"> + <user name="simplestUser"> + <databases> + <database> + <name> + sampledb + </name> + </database> + </databases> + </user> +</users> + + + + List Users in Database Instance Response: + JSON + HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 113 +Date: Thu, 05 Apr 2012 18:13:53 GMT + +{ + "users": [ + { + "databases": [ + { + "name": "sampledb" + } + ], + "name": "simplestUser" + } + ] +} + + You can see that the user simplestUser was + successfully created. + + + Create a New Cloud Server + Create a Cloud Server (or use an existing Cloud Server) + to access your database. + + If you are using an existing Cloud Server on your + account, skip this step and go directly to . + + Create a Cloud Server using the Cloud Servers section of + the Cloud Control Panel (login here: Control Panel Login). + + To create a Cloud Server using the Cloud Control + Panel: + + Click Servers to view + the Cloud Servers page. + + + Using the Region drop-down + menu, select the appropriate region, depending on + whether you want to create a first generation + Cloud Server or a next generation Cloud Server. + (The type of Cloud Server does not matter for this + exercise.) + + + Select an image from a list of different + operating systems, including Linux Distributions + and Windows Images: + + + + + + + + Specify the Server Name + and select a Size for your + Cloud Server, then click Create + Server. + + + + You can also create a Cloud Server using the Cloud + Servers API. Refer to the Next Generation Cloud + Servers Developer Guide for + details. + + + + Configuring an Application to Use Your Cloud + Database + If you are configuring an application to use your + database on a Cloud Server, you need to configure the + application with the hostname for the database instance + and user name / password for the database. Refer to the + response examples in + for the hostname returned and the request + examples for the user name and password you + specified. + + + MySQL GUI Administration + If you want to access your database using the command + line MySQL client, then you have now completed the + Getting Started. + Otherwise, you can use a GUI tool such as phpMyAdmin to + interact with your database instance. Common operations + include managing databases, tables, fields, relations, + indexes, users, and permissions. Included below is a + procedure to set up phpMyAdmin on an Ubuntu 11.04 Cloud + Server. + For more detailed installation configuration + instructions see the phpMyAdmin documentation at: http://www.phpmyadmin.net/documentation/. + Rackspace does not provide phpMyAdmin support, + and the user is responsible for any security + related configuration. + + + To install and configure phpMyAdmin on an Ubuntu + 11.04 Cloud Server: + + Install phpMyAdmin: + + sudo apt-get install + phpmyadmin + + + Set up a symbolic link to the phpmyadmin config + file: + sudo ln -s /etc/phpmyadmin/apache.conf + /etc/apache2/conf.d/phpmyadmin.conf + + + + Edit the + /etc/phpmyadmin/config-db.php + config file to point to your database + instance: + $dbserver='<cloud database + hostname>'; + + + + Restart apache: + sudo apachectl restart + + + Access phpMyAdmin at + http://<your_ipaddress>/phpMyAdmin: + + + + + + + + This concludes the Getting + Started. Thank you for using Rackspace Cloud + products. + + diff --git a/integration/apidocs/src/resources/cdb-mgmt-devguide.xml b/integration/apidocs/src/resources/cdb-mgmt-devguide.xml new file mode 100644 index 0000000000..ea7d784cbb --- /dev/null +++ b/integration/apidocs/src/resources/cdb-mgmt-devguide.xml @@ -0,0 +1,1267 @@ + + + + + + + + GET'> + PUT'> + POST'> + DELETE'> + + + + + + '> + + + + + + '> +]> + + + + Rackspace Cloud Databases Developer Guide for Service + Management + + Rackspace Cloud Databases Management Developer + Guide + + + + + + + + Rackspace Cloud + + + + 2011 + 2012 + Rackspace US, Inc. + + API v1.0 + Rackspace Cloud Databases + 2012-10-04 + + + Copyright details are filled in by the template. + + + + This document is intended for software developers + interested in developing service management + applications using the Rackspace Cloud Databases + Application Programming Interface + (API). + + + + + 2012-10-04 + + + + Added new API call List All Active + Accounts (see ). + + + + + + 2012-10-02 + + + + Added new API calls Get Hardware + Info (see ) and Update All Instances on Host + (see ). + + + Added tenant_id field to sample + responses for List All Instances for a + Host API call (see ). + + + + + + 2012-08-21 + + + + Changed FAILED status for database + instance to ERROR instead (see ). + + + + + + 2012-08-01 + + + + Initial Unlimited Availability (UA) + release for Rackspace Cloud + Databases. + + + + + + + + this is a placeholder for the front cover + + + this is a placeholder for the back cover + + + + Overview + Rackspace Cloud Databases is a database available to + Rackspace Open Cloud customers. Interactions with Cloud + Databases occur programmatically via the Cloud Databases + API as described in this Cloud Databases + Developer Guide for Service + Management. + Writer: need to synch up Overview section with + marketing blurb for web. These should be + consistent. + The API operations described in this manual relate to + support of Cloud Databases service activities on behalf of + a specific customer. The scope of these operations is + narrow, relating to the needs of a single customer. A + customer might directly request these operations, however + only Rackers are authorized to perform them. + The following figure shows an + overview of Cloud Databases Infrastructure: + + + + + + + + + + + Writer: need to get architecture diagram for DBaaS. + Daniel emailed 11/17 that he would work on this during the + Private Beta and have it done for the Public + Beta. + We welcome feedback, comments, and bug reports at http://feedback.rackspacecloud.com. + Writer: check whether following statement should be + added back in for public (not private) beta: Issues and + bug reports can be directed to your support team via + ticket, chat, email, or phone. +
+ Intended Audience + Two APIs provide access to Rackspace Cloud + Databases: + + + The management API, + available only to developers, Support + personnel, and Operations administrators + within Rackspace, depending on the granted + LDAP roles/permissions, is the subject of this + document. + + + The public API, + available to developers within Rackspace and + Rackspace customers, is the subject of a + companion document, the Cloud + Databases Developer + Guide. + + + To use the information provided here, you should + first have a general understanding of the Database + service. You should also be familiar with: + + + Database terminology + + + ReSTful web services + + + HTTP/1.1 conventions + + + JSON and/or XML data serialization + formats + + + ATOM Syndication Format + + +
+ +
+ Document Change History + This version of the Developer Guide replaces and + obsoletes all previous versions. The most recent + changes are described in the table below: + +
+
+ Additional Resources + Descriptive information about Cloud Databases is + also published in its Web Application Description + Language (WADL) and XML Schema Definition (XSD). You + are welcome to read this information here: + + + The WADL is . + + + The XSD is . + + + You can download the most current versions of other + API-related documents from http://docs.rackspace.com/. + For information about getting started using Cloud + Databases and Cloud Servers, refer to + Getting Started with Rackspace Cloud + Databases and Servers. + For more details about Rackspace Cloud Databases, + refer to http://www.rackspace.com/cloud/cloud_hosting_products/databases/. + This site also offers links to Rackspace's official + support channels, including knowledge center articles, + forums, phone, chat, and email. + Please visit our Product Feedback Forum to find out what + customers think about Cloud Databases. + You can also follow Rackspace updates and + announcements via twitter at http://www.twitter.com/rackspace. + This API uses standard HTTP 1.1 response codes as + documented at http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html. + +
+ + + Concepts + + To use the Cloud Databases API effectively, you should + understand several key concepts: + Reviewer: Daniel Morris is asking Daniel Salinas to + do an initial write-up of this chapter. +
+ Database Instance + A database instance is an isolated MySQL instance in + a single tenant environment on a shared physical host + machine. + Writer: once we support + MSSQL, we need to describe here what is used for MSSQL + in place of database instance. +
+
+ Database + A MySQL database within a database instance. + Writer: once we support + MSSQL, we need to modify the wording here, such as: + The actual database, whether it is in MySQL or + MSSQL. +
+
+ Flavor + A flavor is an available hardware configuration for + a database instance. Each flavor has a unique + combination of memory capacity and priority for CPU + time. +
+
+ Volume + A volume is user-specified storage that contains the + MySQL data directory. Volumes are automatically + provisioned on dedicated Internet Small Computer + System Interface (iSCSI) storage area networks (SAN) + that provide for increased performance, scalability, + availability and manageability. Applications with high + I/O demands are performance optimized and data is + protected through both local and network RAID-10. + Additionally, network RAID provides synchronous + replication of volumes with automatic failover and + load balancing across available storage + clusters. +
+ + + General API Information + The Cloud Databases Management API is implemented using + a ReSTful web service interface. Most functions of the + customer-facing Cloud Databases API may be accessed on + behalf of a customer. The Cloud Databases Management API + also extends this functionality to include operations for + managing features of the public APIas well as + additional data fields not available to + customers. +
+ Authentication + Every ReST request against the Cloud Databases + Management API requires that the caller be + authenticated with HTTP Basic Auth. + + If you cannot access the Cloud Databases + Management API, please email the Cloud DB team at + clouddb_all@rackspace.com to + request access. Access will be granted on an + as-needed basis until access to the Management API + is integrated with Global Auth. + +
+
+ Service Access + The Database Service is a regionalized service. The + user of the service is therefore responsible for + appropriate replication, caching, and overall + maintenance of Cloud Databases data across regional + boundaries to other Cloud Databases servers. + + Replace the sample account ID number, + 1234, as shown in the URLs + in this guide, with your actual account number + returned as part of the authentication service + response. + You will find the actual account number after the + final '/' in the publicURL field returned + by the authentication response. In the following + example, you can see from the publicURL + field for cloudServers + (publicURL="https://servers.api.rackspacecloud.com/v1.0/322781") + that the account number is 322781. +
+
+ Request/Response Types + The Cloud Databases API supports both the JSON and + XML data serialization formats. The request format is + specified using the Content-Type header + and is required for calls that + have a request body. The response format can be + specified in requests either by using the + Accept header or by adding an + .xml or .json extension + to the request URI. Note that it is possible for a + response to be serialized using a format different + from the request. If no response format is specified, + JSON is the default. If conflicting formats are + specified using both an Accept header and + a query extension, the query extension takes + precedence. + Some operations support an Atom representation that + can be used to efficiently determine when the state of + services has changed. + Reviewer: the previous sentence will be hidden + for the Private Beta, since it does not appear that + Atom will be supported yet. Correct? + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Response Formats
FormatAccept HeaderQuery ExtensionDefault
JSONapplication/json.jsonYes
XMLapplication/xml.xmlNo
ATOMapplication/atom+xml.atomNo
+ Reviewer: the ATOM row in the table above will be + hidden for the Private Beta, since it does not appear + that Atom will be supported yet. Correct? + In the request example below, notice that + Content-Type is set to + application/json, but + application/xml is + requested via the Accept + header: + + Request with Headers: JSON + + Missing code sample! + + Therefore an XML response format is returned: + + Response with Headers: XML + + Missing code sample! + +
+
+ Synchronous and Asynchronous Responses + Reviewer: please give me the updated info for this + section. Need to replace info about callback URL, + etc. + All successful &GET; requests are + synchronous calls, since they + are always retrieving (reading) existing information. + With these requests, the caller waits until the call + returns with the specified code and response body. For + an example, see XXXX. + &PUT;, &POST;, and &DELETE; calls are asynchronous, however, + since they may take some time to process. Therefore they return 202 ACCEPTED + responses containing information with a callback URL, which allows the progress, + status, and/or response information of the call to be retrieved at a later point in + time. The asynchronous response body will look similar to the following examples, + depending on the format requested: + + 202 ACCEPTED Response: XML + + Reviewer: need code example + + + 202 ACCEPTED Response: JSON + + Reviewer: need code example + + The following table shows the attributes for asynchronous responses: + + + + + + + + + + + + + + + + + + + +
Attributes for Asynchronous Responses
AttributeDescription
jobIdAn identifier for the specific request.
callbackUrlResource locator for querying the status of the request.
+ + The status for asynchronous calls is retained for up to 24 hours. + + + If a request body does not pass initial validation or an error condition + arises, you may receive an immediate error response from the request. + + When a request is made to the callback URL provided + and the job is still running, another + 202 ACCEPTED response + is returned with the same information as the previous + one. If the request is complete, the response will be + as if the original call returned as normal, without + waiting. For example, if a Create Database request was + issued and a 202 asynchronous response was returned, + the response from querying the callback URL for a + completed successful database creation would be a + 200 OK and contain the + information for the created database. See XXXX for a + specific example. + If an error occurs during the processing of the + create request, querying the callback URL will return + the details of the error, as if the original call + returned the error response. For example, if a + validation error occurs during the Create Database + request above, the response from querying the callback + URL would be a 400 BAD + REQUEST and contain details regarding the specific + validation error. + + If the response from querying a callback URL is a + 404 NOT FOUND, the details of the error in the + response body will contain information the caller may use to determine whether + the specified job itself was not found, or if the response from the original + request was a 404 NOT FOUND. + + The description of each &PUT;, &POST;, and &DELETE; + request identifies the response codes that can + indicate success or error for that request. For + example, see XXXX immediately below the table for a + list of the successful and error response codes for + the POST /xxxx call. +
+
+ Content Compression + Request and response body data may be encoded with gzip compression to accelerate + interactive performance of API calls and responses. This is controlled using the + Accept-Encoding header on the request from the client and indicated + by the Content-Encoding header in the server response. Unless the + header is explicitly set, encoding defaults to disabled. + + + + + + + + + + + + + + + + + + + + + + +
Encoding Headers
Header TypeNameValue
HTTP/1.1 RequestAccept-Encodinggzip
HTTP/1.1 ResponseContent-Encodinggzip
+
+
+ Persistent Connections + + By default, the API supports persistent connections + via HTTP/1.1 keepalives. All connections will be kept + alive unless the connection header is set to close. + + + To prevent abuse, HTTP sessions have a timeout of 20 + seconds before being closed. + + + + The server may close the connection at any time + and clients should not rely on this behavior. + + +
+ +
+ Limits + + All accounts, by default, have a preconfigured set of + thresholds (or limits) to manage capacity and prevent + abuse of the system. The system recognizes two kinds + of limits: rate limits and + absolute limits. Rate limits + are thresholds that are reset after a certain amount + of time passes. Absolute limits are fixed. + +
+ Rate Limits + Rate limits are specified in terms of both a + human-readable wild-card URI and a + machine-processable regular expression. The + regular expression boundary matcher '^' takes + effect after the root URI path. For example, the + regular expression ^/v1.0/instances would match + the bolded portion of the following URI: + https://ord.databases.api.rackspacecloud.com/v1.0/instances. + The following table specifies the default rate + limits for all API operations for all &GET;, + &POST;, &PUT;, and &DELETE; calls for databases + and database instances: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Default Rate Limits
VerbURIRegExDefault
&GET; changes-since*/instances/*^/vd+.d+/instances.*3/minute
&POST;*/instances/*^/vd+.d+/instances.*10/minute
&POST; instances*/instances/*^/vd+.d+/instances.*50/day
&PUT;*/instances/*^/vd+.d+/instances.*10/minute
&DELETE;*/instances/*^/vd+.d+/instances.*100/minute
+ Rate limits are applied in order relative to the + verb, going from least to most specific. For + example, although the threshold for &POST; to + /v1.0/* is 10 per minute, one cannot &POST; to + /v1.0/* more than 50 times within a single day. + If you exceed the thresholds established for + your account, a 413 (Rate + Control) HTTP response will be + returned with a Retry-After header to + notify the client when it can attempt to try + again. +
+
+ Absolute Limits + Reviewer: Need to update this entire section. + Please give me your updates. + Refer to the following table for the absolute + limits that are set. + + + + + + + + + + + + + + + + + + + + + +
Absolute Limits
NameDescriptionLimit
InstancesMaximum number of instances allowed + for your account5
Volume SizeMaximum volume size per + instance in gigabytes (GB) for your + account25
+
+
+
+ Date/Time Format + The Database Service uses an ISO-8601 compliant + date format for the display and consumption of + date/time values. + The system timezone is in UTC. MySQL converts + TIMESTAMP values from the current time zone to UTC for + storage, and back from UTC to the current time zone + for retrieval. This does not occur for other types, + such as DATETIME. + + DB Service Date/Time Format + yyyy-MM-dd'T'HH:mm:ss.SSSZ + See the table below for a description of the date/time format codes. + May 19th, 2011 at 8:07:08 AM, GMT-5 would have the following + format: + 2011-05-19T08:07:08-05:00 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Explanation of Date/Time Format Codes
CodeDescription
yyyyFour digit year
MMTwo digit month
ddTwo digit day of month
TSeparator for date/time
HHTwo digit hour of day (00-23)
mmTwo digit minutes of hour
ssTwo digit seconds of the minute
SSSThree digit milliseconds of the second
ZRFC-822 timezone
+ +
+
+ Pagination + Reviewer: please review and give me your updates + for this section. Dev needs to determine which API + calls support pagination and then supply examples. I + have hidden this entire section for now, since we do + not believe that it will be supported for Private + Beta. Correct? + To reduce load on the service, list operations will + return a maximum of 100 items at a time. This is + referred to as pagination. + Pagination is the ability to limit the size of the returned data as well as + retrieve a specified subset of a large data set. Pagination has two key concepts: + limit and offset. Limit is the restriction on the maximum + number of items for that type that can be returned. Offset is + the starting point for the return data. For example, an offset of 50 specifies that + the items that are returned should start with item number 51 (since the numbering is + one-based) in the collection. + It is important to note that offset must be a multiple of the + limit (or zero), otherwise a Bad Request Exception will be thrown. Both limit and + offset are specified via request parameters on the URI. The parameters are named + limit and offset respectively, and both apply only to + &GET; calls. If unspecified, they default to limit=100 and + offset=0. See the examples that follow. + + Examples of Limits and Offsets for Paging Calls + + Reviewer: Need code example. + + Pagination applies only to the calls listed in the following table: + Reviewer: need to update the table of paginated API + calls below: + + + + Verb + URI + Description + + + + + &GET; + /URI/ + List all databases manageable + by the account specified. + + + + &GET; + /URI/?name=Name + Filter databases. + + + + &GET; + /URI/ID + List details of the specified + database. + + + + &GET; + /URI/ID/xxx + List details xxx. + + + + &GET; + /URI/ID/yyy + List details yyy. + + + + + + + See the following section for examples of paged List + Databases calls. +
+ Pagination Elements and Attributes + Reviewer: please review and give me your + updates for this section. + For any collection in a result, there is a totalEntries attribute + representing the total number of entries there are for this item type. If the + number of items requested in the &GET; call is less then the total number of + items for this type, then there will be pagination links previous + and/or next, specifying how to get to the previous and/or next set + of records. + + The previous and/or next link elements are + displayed only if there are items available in the corresponding link. See + the following examples for details. + + + List Databases Request with limit: + XML + + Reviewer: Need code example. + + + List Databases Request with limit: + JSON + + Reviewer: Need code example. + + + List Databases Response with totalEntries: + XML + + Reviewer: Need code example. + + + List Databases Response with totalEntries: + JSON + + Reviewer: Need code example. + + In the previous two response examples, note that + totalEntries=112 and that a link has been provided to retrieve + the next 3 results (limit=3) in the link element identified by the + attribute rel="next" (XML) or "rel":"next" (JSON). + The following example shows links to both previous and next results in the + responses, since the request specified to start with the fourth item in the + collection (offset=3): + + List Databases Request with limit and + offset: XML + + Reviewer: Need code example. + + + List Databases Request with limit and + offset: JSON + + Reviewer: Need code example. + + + List Databases Response with Links to + previous and next Results: XML + + Reviewer: Need code example. + + + List Databases Response with Links to + previous and next Results: JSON + + Reviewer: Need code example. + + + In the previous two response examples, note that + totalEntries=112 and two links have been provided to: + + Retrieve the next 3 results (limit=3) via the link + element identified by the attribute rel="next" (XML) or + "rel":"next" (JSON) + + + Retrieve the previous 3 results via the link element identified by + the attribute rel="previous" (XML) or + "rel":"previous" (JSON) + + +
+
+
+ Efficient Polling with the + <parameter>Changes-Since</parameter> + Parameter + Reviewer: I have hidden this section, since it + does not appear that it will be supported for Private + Beta. Correct? + The ReST API allows you to poll for the status of + certain operations by performing a &GET; on various + URIs. Rather than re-downloading and re-parsing the + full status at each polling interval, your ReST client + may use the changes-since + parameter to check for changes since a previous + request. The changes-since time + is specified as Unix time (the number of seconds since + January 1, 1970, 00:00:00 UTC, not counting leap + seconds). If nothing has changed since the + changes-since time, a + 304 (Not Modified) + response will be returned. If data has changed, only + the items changed since the specified time will be + returned in the response. + Reviewer: does the following sentence apply, and + should it be included? + For example, performing a &GET; against + https://api.servers.rackspacecloud.com/v1.0/224532/servers?changes-since=1244012982 + would list all servers that have changed since Wed, 03 + Jun 2009 07:09:42 UTC. +
+
+ Faults + Reviewer: need to update this section as needed + for Cloud Databases. + When an error occurs, the Database Service returns + a fault object containing an HTTP error response code + that denotes the type of error. In the body of the + response, the system will return additional + information about the fault. + The following table lists possible fault types with their associated error codes + and descriptions. + + + + Fault Type + Associated Error Code + Description + + + + + badRequest + 400 + There was one or more errors in the user request. + + + unauthorized + 401 + The supplied token is not authorized to access the resources, either it's expired or invalid. + + + forbidden + 403 + Access to the requested + resource was denied. + + + itemNotFound + 404 + The back-end services did not + find anything matching the + Request-URI. + + + badMethod + 405 + The request method is not allowed for this resource. + + + overLimit + 413 + Either the number of entities in the request is larger than + allowed limits, or the user has exceeded allowable request rate limits. + See the details element for more specifics. Contact support + if you think you need higher request rate limits. + + + badMediaType + 415 + The requested content type is not supported by this service. + + + unprocessableEntity + 422 + The requested resource could + not be processed on at the moment. + + + instanceFault + 500 + This is a generic server error and the message contains the reason for the error. This error could wrap several error messages and is a catch all. + + + notImplemented + 501 + The requested method or resource is not implemented. + + + serviceUnavailable + 503 + The Database Service is not + available. + + + + The following two instanceFault + examples show errors when the server has erred or + cannot perform the requested operation: + + + Example instanceFault Response: XML + + + + + + + Example instanceFault Response: JSON + + + + + + The error code (code) is returned in the body of the response for + convenience. The message element returns a human-readable message that + is appropriate for display to the end user. The details element is + optional and may contain information that is useful for tracking down an error, such + as a stack trace. The details element may or may not be appropriate for + display to an end user, depending on the role and experience of the end user. + The fault's root element (for example, + instanceFault) may change depending + on the type of error. + The following two badRequest examples + show errors when the volume size is invalid: + + Example badRequest Fault on Volume Size Errors: + XML + + + + + + + Example badRequest Fault on Volume Size Errors: + JSON + + + + + + The next two examples show + itemNotFound errors: + + Example itemNotFound Fault: XML + + + + + + + Example itemNotFound Fault: JSON + + + + + +
+
+ Database Instance Status + When making an API call to create, list, or delete + database instance(s), the following database instance + status values are possible: + + + BUILD – The database instance is being provisioned. + + + REBOOT – The database instance is + rebooting. + + + ACTIVE – The database instance is + online and available to take requests. + + + BLOCKED – The database instance is + unresponsive at the moment. + + + RESIZE – The database instance is being + resized at the moment. + + + SHUTDOWN – The database instance is + terminating services. Also, SHUTDOWN is + returned if for any reason the MySQL instance + is shut down but not the actual server. + + + If MySQL has crashed (causing the + SHUTDOWN status), you can try + rebooting the database instance to see + if that corrects the problem. If the + instance still does not recover, you + can escalate the issue to + NebOps. + + + + + ERROR – The last operation for the + database instance failed due to an + error. + + +
+ + + + API Operations + +
+ Database Instance Management + This section describes the operations that are supported for managing database instances. + + + + + + + + + + + + + + + + + +
+ +
+ Host Information + This section describes the operations that are supported for getting host information. + + + + + + + + + + + +
+ +
+ Management Instance Actions + This section describes the actions that are supported for database instances. + + + + + + +
+ +
+ Storage Management + This section describes the operations that are supported for getting information about storage devices. + + + + + +
+ +
+ Account Information + This section describes the operations that are + supported for accounts. + + + + + + + + +
+ + + diff --git a/integration/apidocs/src/resources/cdb-releasenotes.xml b/integration/apidocs/src/resources/cdb-releasenotes.xml new file mode 100644 index 0000000000..205ff6530e --- /dev/null +++ b/integration/apidocs/src/resources/cdb-releasenotes.xml @@ -0,0 +1,711 @@ + + + + + Rackspace Cloud Databases Release Notes + Rackspace Cloud DBs Rel Notes + + + + + + + + Rackspace Cloud + + + + 2010 + 2011 + 2012 + Rackspace US, Inc. + + API v1.0 + Rackspace Cloud Databases + 2012-10-02 + + + Copyright details are filled in by the template. + + + + This document is intended for software developers + interested in developing applications using the + Rackspace Cloud Databases Application Program + Interface (API). + + + + 2012-10-02 + + Updated for v1.0.13. + + + + 2012-09-17 + + Updated for v1.0.12. + + + + 2012-09-13 + + Updated for v1.0.11. + + + + 2012-08-21 + + Updated for v1.0.8. + + + + 2012-08-02 + + Updated for v1.0.7. + + + + 2012-08-01 + + Initial Unlimited Availability (UA) release + for Rackspace Cloud Databases. + + + + + cdb + 10 + + + + Document Change History + This version of the Release Notes replaces and obsoletes all + previous versions. The most recent changes are described + in the table below: + + + + v1.0.13, October 2, 2012 + + Bug Fixes + + D-08245: Formatted code to comply with + pep8 code style. + + + D-08357: Initialized and updated usage + data for instances that existed prior to billing + enablement. + + + D-08648: Fixed guest agent to properly + address lost connections with message bus. + + + + Resources + + Get started using the Cloud Databases API to + create databases in the Getting Started Guide at: http://docs.rackspace.com/api/. + + + Get reference information and examples in + the Cloud Databases + Developer Guide at: http://docs.rackspace.com/api/. + + + Support for Cloud Databases is available for US + and UK customers 24x7x365 via phone, chat, or you + may also File + a Ticket. + + + Please visit our Product Feedback Forum and let us know + what you think about Cloud Databases! + + + + + v1.0.12 (Hotfix), September 17, 2012 + + Bug Fixes + + D-08422: Fixed billing code to + properly send events to Yagi and fixed billing + timestamp format. + + + + Known Issues / Limitations + + List Versions API call does not work and returns + a "403 Forbidden" response. + + + Users' database privileges are not modifiable. + Users must be deleted and recreated to add + additional database privileges. + + + + Resources + + Get started using the Cloud Databases API to + create databases in the Getting Started Guide at: http://docs.rackspace.com/api/. + + + Get reference information and examples in + the Cloud Databases + Developer Guide at: http://docs.rackspace.com/api/. + + + Support for Cloud Databases is available for US + and UK customers 24x7x365 via phone, chat, or you + may also File + a Ticket. + + + Please visit our Product Feedback Forum and let us know + what you think about Cloud Databases! + + + + + v1.0.11, September 13, 2012 + + Bug Fixes + + D-08051: Fixed issue preventing users + from increasing/decreasing their instance flavor + size. + + + D-06654: Resizing instance using same + volume size no longer generates misleading error + message. + + + D-07897: Fixed issue where instance in + ERROR fails to delete. + + + D-07819: Fixed issue causing usage + information to be applied when instance goes into + error status after failed provisioning. + + + D-07213: Fixed guest update Mgmt API + call. + + + + Maintenance + + D-08217: Corrected yagi logrotate + configuration to prevent log file handle from + disappearing. + + + B-27788: Updated usage modify events + (resize volume/flavor). + + + B-29438: Added Environment Variable to + usage event. + + + + Known Issues / Limitations + + List Versions API call does not work and returns + a "403 Forbidden" response. + + + Users' database privileges are not modifiable. Users must be + deleted and recreated to add additional database + privileges. + + + + Resources + + Get started using the Cloud Databases API to + create databases in the Getting Started Guide at: http://docs.rackspace.com/api/. + + + Get reference information and examples in + the Cloud Databases + Developer Guide at: http://docs.rackspace.com/api/. + + + Support for Cloud Databases is available for US + and UK customers 24x7x365 via phone, chat, or you + may also File + a Ticket. + + + Please visit our Product Feedback Forum and let us know + what you think about Cloud Databases! + + + + + v1.0.8, August 21, 2012 + + What's New + + Added reserved database names that cannot be used for + creating databases (see http://docs.rackspace.com/cdb/api/v1.0/cdb-devguide/content/POST_createDatabase__version___accountId__instances__instanceId__databases_.html + for details). + + + Added reserved user names that cannot be used for creating + users (see http://docs.rackspace.com/cdb/api/v1.0/cdb-devguide/content/POST_createUser__version___accountId__instances__instanceId__users_.html + for details). + + + Added support for the InnoDB plugin. + + + + Bug Fixes + + D-07436: Added updated + attribute to the response from the List Versions + API call. + + + D-07488: Increased + max_allowed_packet default to 16 + MB. + + + D-07278: Fixed + open_files_limit based on flavor + size. + + + D-06636: Fixed issue where rabbit + restart causes delete_queue to fail. + + + D-07273: Added check for account id in + Mgmt API calls. + + + D-07644: Fixed the issue where the + Usage events for "Exists" do not have a "T" in the + date/time format. + + + D-06783: Split usage events of over 24 + hours into "exists" events. + + + D-07366: Simplified ioprio + calculation. + + + Story:317533: Added cpulimit + overcommit multiplier. + + + Fixed the trove-api issues with rabbit + connections. Using openstack-common rpc. + + + Updated the usage event with + the environment set to STAGE instead of + STAGING. + + + Changed set_cpus to not use a multiplier so that values in + the flavors database have the correct vcpus that + are then applied to the containers. + + + + Known Issues / Limitations + + List Versions API call does not work and returns + a "403 Forbidden" response. + + + Users' database privileges are not modifiable. +  Users must be deleted and recreated to add + additional database privileges. + + + + Resources + + Get started using the Cloud Databases API to + create databases in the Getting Started Guide at: http://docs.rackspace.com/api/. + + + Get reference information and examples in + the Cloud Databases + Developer Guide at: http://docs.rackspace.com/api/. + + + Support for Cloud Databases is available for US + and UK customers 24x7x365 via phone, chat, or you + may also File + a Ticket. + + + Please visit our Product Feedback Forum and let us know + what you think about Cloud Databases! + + + + + v1.0.7, August 2, 2012 + + What's New + + Multi-cab scheduler. + + + + Bug Fixes + + Fixed usage bug with httplib2. + + + + Resources + + Get started using the Cloud Databases API to + create databases in the Getting Started Guide at: http://docs.rackspace.com/api/. + + + Get reference information and examples in + the Cloud Databases + Developer Guide at: http://docs.rackspace.com/api/. + + + Support for Cloud Databases is available for US + and UK customers 24x7x365 via phone, chat, or you + may also File + a Ticket. + + + Please visit our Product Feedback Forum and let us know + what you think about Cloud Databases! + + + + + v1.0.6, August 1, 2012 + We’re pleased to announce the availability of Rackspace Cloud + Databases to all US and UK Open Cloud customers. These + release notes cover the Unlimited Availability (UA) + release of Rackspace Cloud Databases. + + Product Features + + + Performance Optimized MySQL + Instances + + Provision MySQL database instances of varying + flavor (RAM) sizes without the need to maintain + and/or update MySQL. Optionally create users and + databases during instance creation for quick + deployment. + + + + + High-performance block level + storage + + Critical loads with high I/O demands are + performance-optimized and protected with local + RAID 10 and network RAID 10. Additionally, network + RAID provides synchronous replication of volumes + with automatic failover and load balancing across + available storage clusters. + + + + + Storage + Management + + Increase storage allocation seamlessly without + downtime as your dataset size increases. + + + + + Database Instance + Management + + Increase or decrease flavor (RAM) size as + database input/output (I/O) demands + increase. + + + + + Database + Management + + Create, list, and delete databases on your MySQL + database instance. + + + + + User Management + + Control database access by adding and removing + MySQL users. Optionally enable root user access + for additional control. + + + + What's New + + Performance improvements. + + + Documentation updates to the CDB Developer Guide. + + + + Bug Fixes + + Removed root user from the users list. + + + List users no longer shows root (or any other + specified users we define). + + + XML Pagination works for list instances. + + + Kernel openvz updates. + + + Added Environment field to usage payload + xml. + + + Separate the modify event into modify + volume/flavor types. + + + Added categories searching for atom hopper + billing. + + + + Known Issues / Limitations + + Users' database privileges are not modifiable. +  Users must be deleted and recreated to add + additional database privileges. + + + + Resources + + Get started using the Cloud Databases API to create + databases in the Getting + Started Guide at: http://docs.rackspace.com/api/. + + + Get reference information and examples in the Cloud Databases Developer + Guide at: http://docs.rackspace.com/api/. + + + Support for Cloud Databases is available for US and UK + customers 24x7x365 via phone, chat, or you may + also File a Ticket. + + + Please visit our Product Feedback Forum and let us know + what you think about Cloud Databases! + + + + + diff --git a/integration/apidocs/src/resources/images/Choose_CS_Image.png b/integration/apidocs/src/resources/images/Choose_CS_Image.png new file mode 100644 index 0000000000..b60a9df8a9 Binary files /dev/null and b/integration/apidocs/src/resources/images/Choose_CS_Image.png differ diff --git a/integration/apidocs/src/resources/images/Choose_CS_Image_CCP.png b/integration/apidocs/src/resources/images/Choose_CS_Image_CCP.png new file mode 100644 index 0000000000..2e6f23ebea Binary files /dev/null and b/integration/apidocs/src/resources/images/Choose_CS_Image_CCP.png differ diff --git a/integration/apidocs/src/resources/images/Choose_Image_CCP.png b/integration/apidocs/src/resources/images/Choose_Image_CCP.png new file mode 100644 index 0000000000..14a555722b Binary files /dev/null and b/integration/apidocs/src/resources/images/Choose_Image_CCP.png differ diff --git a/integration/apidocs/src/resources/images/Choose_Image_CCP.tiff b/integration/apidocs/src/resources/images/Choose_Image_CCP.tiff new file mode 100644 index 0000000000..79d5a45faf Binary files /dev/null and b/integration/apidocs/src/resources/images/Choose_Image_CCP.tiff differ diff --git a/integration/apidocs/src/resources/images/Cloud_DB_Infographic-1.png b/integration/apidocs/src/resources/images/Cloud_DB_Infographic-1.png new file mode 100644 index 0000000000..7a00ae001b Binary files /dev/null and b/integration/apidocs/src/resources/images/Cloud_DB_Infographic-1.png differ diff --git a/integration/apidocs/src/resources/images/Cloud_DB_Infographic-1.svg b/integration/apidocs/src/resources/images/Cloud_DB_Infographic-1.svg new file mode 100644 index 0000000000..4a3f125abb --- /dev/null +++ b/integration/apidocs/src/resources/images/Cloud_DB_Infographic-1.svg @@ -0,0 +1,4533 @@ + + + + +]> + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/images/Create_CS.png b/integration/apidocs/src/resources/images/Create_CS.png new file mode 100644 index 0000000000..32c291b347 Binary files /dev/null and b/integration/apidocs/src/resources/images/Create_CS.png differ diff --git a/integration/apidocs/src/resources/images/phpMyAdmin.png b/integration/apidocs/src/resources/images/phpMyAdmin.png new file mode 100644 index 0000000000..80f909f99f Binary files /dev/null and b/integration/apidocs/src/resources/images/phpMyAdmin.png differ diff --git a/integration/apidocs/src/resources/samples/db-auth-20.json b/integration/apidocs/src/resources/samples/db-auth-20.json new file mode 100644 index 0000000000..43a99e1f46 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-auth-20.json @@ -0,0 +1,128 @@ +HTTP/1.1 200 OK +Content-Type: application/json; charset=UTF-8 +Content-Length: 477 +Date: Thu, 12 Apr 2012 18:45:13 GMT + +{ + "access": { + + "token": { + "expires": "2011-12-08T22:51:02.000-06:00", + "id": "vvvvvvvv-wwww-xxxx-yyyy-zzzzzzzzzzzz" + }, + "user": { + "id": "123456", + "name": "jsmith", + "roles": [ + { + "description": "Admin Role.", + "id": "identity:admin", + "name": "identity:admin" + }, + { + "description": "Default Role.", + "id": "identity:default", + "name": "identity:default" + } + ] + }, + "serviceCatalog": [ + { + "endpoints": [ + { + "publicURL": "https://ord.databases.api.rackspacecloud.com/v1.0/1100111", + "region": "ORD", + "tenantId": "1100111" + }, + { + "publicURL": "https://dfw.databases.api.rackspacecloud.com/v1.0/1100111", + "region": "DFW", + "tenantId": "1100111" + } + ], + "name": "cloudDatabases", + "type": "rax:database" + }, + { + "endpoints": [ + { + "publicURL": "https://dfw.loadbalancers.api.rackspacecloud.com/v1.0/1100111", + "region": "DFW", + "tenantId": "1100111" + }, + { + "publicURL": "https://ord.loadbalancers.api.rackspacecloud.com/v1.0/1100111", + "region": "ORD", + "tenantId": "1100111" + } + ], + "name": "cloudLoadBalancers", + "type": "rax:load-balancer" + }, + { + "endpoints": [ + { + "tenantId": "1100111", + "publicURL": "https://servers.api.rackspacecloud.com/v1.0/1100111", + "versionId": "1.0", + "versionInfo": "https://servers.api.rackspacecloud.com/v1.0/", + "versionList": "https://servers.api.rackspacecloud.com/" + } + ], + "name": "cloudServersOpenStack", + "type": "compute" + }, + { + "endpoints": [ + { + "tenantId": "1100111", + "publicURL": "https://servers.api.rackspacecloud.com/v1.0/1100111", + "versionId": "1.0", + "versionInfo": "https://servers.api.rackspacecloud.com/v1.0/", + "versionList": "https://servers.api.rackspacecloud.com/" + } + ], + "name": "cloudServers", + "type": "compute" + }, + { + "endpoints": [ + { + "tenantId": "MossoCloudFS_aaaaaaaa-bbbb-cccc-dddd-eeeeeeee", + "publicURL": "https://storage101.ord1.clouddrive.com/v1/MossoCloudFS_aaaaaaaa-bbbb-cccc-dddd-eeeeeeee", + "internalURL": "https://snet-storage101.ord1.clouddrive.com/v1/MossoCloudFS_aaaaaaaa-bbbb-cccc-dddd-eeeeeeee", + "region": "ORD", + "versionId": "1", + "versionInfo": "https://cdn2.clouddrive.com/v1/", + "versionList": "https://cdn2.clouddrive.com/" + + } + ], + "name": "cloudFiles", + "type": "object-store" + }, + { + "endpoints": [ + { + "tenantId": "MossoCloudFS_aaaaaaaa-bbbb-cccc-dddd-eeeeeeee", + "publicURL": "https://cdn2.clouddrive.com/v1/MossoCloudFS_aaaaaaaa-bbbb-cccc-dddd-eeeeeeee", + "region": "ORD" + } + ], + "name": "cloudFilesCDN", + "type": "rax:object-cdn" + }, + { + + "endpoints": [ + { + "tenantId": "1100111", + "publicURL": "https://dns.api.rackspacecloud.com/v1.0/1100111" + } + ], + "name": "cloudDNS", + "type": "rax:dns" + } + ] + } +} diff --git a/integration/apidocs/src/resources/samples/db-auth-20.xml b/integration/apidocs/src/resources/samples/db-auth-20.xml new file mode 100644 index 0000000000..e0bc3b8c3e --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-auth-20.xml @@ -0,0 +1,63 @@ +HTTP/1.1 200 OK +Content-Type: application/xml; charset=UTF-8 +Content-Length: 477 +Date: Thu, 12 Apr 2012 18:50:20 GMT + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-auth.json b/integration/apidocs/src/resources/samples/db-auth.json new file mode 100644 index 0000000000..583b584c0b --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-auth.json @@ -0,0 +1,72 @@ +HTTP/1.1 200 OK +Content-Type: application/json; charset=UTF-8 +Content-Length: 477 +Date: Thu, 12 Apr 2012 18:45:13 GMT + +{ + "auth": { + "serviceCatalog": { + "cloudDNS": [ + { + "publicURL": "https://dns.api.rackspacecloud.com/v1.0/1234" + } + ], + "cloudDatabases": [ + { + "publicURL": "https://dfw.databases.api.rackspacecloud.com/v1.0/1234", + "region": "DFW" + }, + { + "publicURL": "https://ord.databases.api.rackspacecloud.com/v1.0/1234", + "region": "ORD" + } + ], + "cloudFiles": [ + { + "internalURL": "https://snet-storage101.dfw1.clouddrive.com/v1/ MossoCloudFS_aaaa-bbbbb-cccc-ddddd ", + "publicURL": "https://storage101.dfw1.clouddrive.com/v1/ MossoCloudFS_aaaa-bbbbb-cccc-ddddd ", + "region": "DFW", + "v1Default": true + } + ], + "cloudFilesCDN": [ + { + "publicURL": "https://cdn1.clouddrive.com/v1/MossoCloudFS_aaaa-bbbbb-cccc-ddddd", + "region": "DFW", + "v1Default": true + } + ], + "cloudLoadBalancers": [ + { + "publicURL": "https://ord.loadbalancers.api.rackspacecloud.com/v1.0/1234", + "region": "ORD" + }, + { + "publicURL": "https://dfw.loadbalancers.api.rackspacecloud.com/v1.0/1234", + "region": "DFW" + } + ], + "cloudMonitoring": [ + { + "publicURL": "https://monitoring.api.rackspacecloud.com/v1.0/1234" + } + ], + "cloudServers": [ + { + "publicURL": "https://servers.api.rackspacecloud.com/v1.0/1234", + "v1Default": true + } + ], + "cloudServersOpenStack": [ + { + "publicURL": "https://dfw.servers.api.rackspacecloud.com/v2/1234", + "region": "DFW" + } + ] + }, + "token": { + "expires": "2012-04-12T13:15:52.000-05:00", + "id": "aaaaa-bbbbbb-cccccc-ddddd" + } + } +} diff --git a/integration/apidocs/src/resources/samples/db-auth.xml b/integration/apidocs/src/resources/samples/db-auth.xml new file mode 100644 index 0000000000..de97e59a0b --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-auth.xml @@ -0,0 +1,50 @@ +HTTP/1.1 200 OK +Content-Type: application/xml; charset=UTF-8 +Content-Length: 477 +Date: Thu, 12 Apr 2012 18:50:20 GMT + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-check-root-user-request.json b/integration/apidocs/src/resources/samples/db-check-root-user-request.json new file mode 100644 index 0000000000..79a6d18e1e --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-check-root-user-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f/root HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-check-root-user-request.xml b/integration/apidocs/src/resources/samples/db-check-root-user-request.xml new file mode 100644 index 0000000000..46f2cf2918 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-check-root-user-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f/root HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-check-root-user-response.json b/integration/apidocs/src/resources/samples/db-check-root-user-response.json new file mode 100644 index 0000000000..3e121e4e76 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-check-root-user-response.json @@ -0,0 +1,8 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 21 +Date: Wed, 25 Jan 2012 21:58:13 GMT + +{ + "rootEnabled": true +} diff --git a/integration/apidocs/src/resources/samples/db-check-root-user-response.xml b/integration/apidocs/src/resources/samples/db-check-root-user-response.xml new file mode 100644 index 0000000000..fd75a745f2 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-check-root-user-response.xml @@ -0,0 +1,6 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 90 +Date: Wed, 25 Jan 2012 21:58:14 GMT + +True diff --git a/integration/apidocs/src/resources/samples/db-create-database-instance-response.json b/integration/apidocs/src/resources/samples/db-create-database-instance-response.json new file mode 100644 index 0000000000..78b1743243 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-create-database-instance-response.json @@ -0,0 +1,41 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 632 +Date: Mon, 28 Nov 2011 21:35:41 GMT + +{ + "instance": { + "created": "2011-11-03T15:55:26Z", + "flavor": { + "id": "1", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/1", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/flavors/1", + "rel": "bookmark" + } + ] + }, + "hostname": "d735f61b985bb003a61dc72948dbf4e7174da12c.rackspaceclouddb.com", + "id": "00ff11ee-22dd-33cc-44bb-55aa66007799", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances/00ff11ee-22dd-33cc-44bb-55aa66007799", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/instances/00ff11ee-22dd-33cc-44bb-55aa66007799", + "rel": "bookmark" + } + ], + "name": "myrackinstance", + "status": "BUILD", + "updated": "2011-11-03T15:55:27Z", + "volume": { + "size": "2" + } + } +} diff --git a/integration/apidocs/src/resources/samples/db-create-database-instance-response.xml b/integration/apidocs/src/resources/samples/db-create-database-instance-response.xml new file mode 100644 index 0000000000..9f0cc3550f --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-create-database-instance-response.xml @@ -0,0 +1,24 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 639 +Date: Mon, 28 Nov 2011 16:19:34 GMT + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-create-databases-request.json b/integration/apidocs/src/resources/samples/db-create-databases-request.json new file mode 100644 index 0000000000..1ef2bab6b5 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-create-databases-request.json @@ -0,0 +1,19 @@ +POST /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f/databases HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + +{ + "databases": [ + { + "character_set": "utf8", + "collate": "utf8_general_ci", + "name": "testingdb" + }, + { + "name": "sampledb" + } + ] +} diff --git a/integration/apidocs/src/resources/samples/db-create-databases-request.xml b/integration/apidocs/src/resources/samples/db-create-databases-request.xml new file mode 100644 index 0000000000..a8e584d0d1 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-create-databases-request.xml @@ -0,0 +1,13 @@ +POST /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f/databases HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-create-databases-response.json b/integration/apidocs/src/resources/samples/db-create-databases-response.json new file mode 100644 index 0000000000..6a83f7fcb1 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-create-databases-response.json @@ -0,0 +1,4 @@ +HTTP/1.1 202 Accepted +Content-Type: application/json +Content-Length: 0 +Date: Wed, 27 Jun 2012 23:11:18 GMT diff --git a/integration/apidocs/src/resources/samples/db-create-databases-response.xml b/integration/apidocs/src/resources/samples/db-create-databases-response.xml new file mode 100644 index 0000000000..3a2ce5f652 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-create-databases-response.xml @@ -0,0 +1,4 @@ +HTTP/1.1 202 Accepted +Content-Type: application/xml +Content-Length: 0 +Date: Wed, 27 Jun 2012 23:11:18 GMT diff --git a/integration/apidocs/src/resources/samples/db-create-instance-request.json b/integration/apidocs/src/resources/samples/db-create-instance-request.json new file mode 100644 index 0000000000..cbcfc90dc9 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-create-instance-request.json @@ -0,0 +1,37 @@ +POST /v1.0/1234/instances HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: d6cafa5b-e0c7-4ab8-948e-7c95f2acd031 +Accept: application/json +Content-Type: application/json + +{ + "instance": { + "databases": [ + { + "character_set": "utf8", + "collate": "utf8_general_ci", + "name": "sampledb" + }, + { + "name": "nextround" + } + ], + "flavorRef": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/1", + "name": "json_rack_instance", + "users": [ + { + "databases": [ + { + "name": "sampledb" + } + ], + "name": "demouser", + "password": "demopassword" + } + ], + "volume": { + "size": 2 + } + } +} diff --git a/integration/apidocs/src/resources/samples/db-create-instance-request.xml b/integration/apidocs/src/resources/samples/db-create-instance-request.xml new file mode 100644 index 0000000000..7f19a4e748 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-create-instance-request.xml @@ -0,0 +1,23 @@ +POST /v1.0/1234/instances HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: d6cafa5b-e0c7-4ab8-948e-7c95f2acd031 +Accept: application/xml +Content-Type: application/xml + + + + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-create-instance-response.json b/integration/apidocs/src/resources/samples/db-create-instance-response.json new file mode 100644 index 0000000000..f5383ce2fd --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-create-instance-response.json @@ -0,0 +1,41 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 636 +Date: Wed, 25 Jan 2012 21:53:10 GMT + +{ + "instance": { + "created": "2012-01-25T21:53:09Z", + "flavor": { + "id": "1", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/1", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/flavors/1", + "rel": "bookmark" + } + ] + }, + "hostname": "e09ad9a3f73309469cf1f43d11e79549caf9acf2.rackspaceclouddb.com", + "id": "dea5a2f7-3ec7-4496-adab-0abb5a42d635", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances/dea5a2f7-3ec7-4496-adab-0abb5a42d635", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/instances/dea5a2f7-3ec7-4496-adab-0abb5a42d635", + "rel": "bookmark" + } + ], + "name": "json_rack_instance", + "status": "BUILD", + "updated": "2012-01-25T21:53:10Z", + "volume": { + "size": 2 + } + } +} diff --git a/integration/apidocs/src/resources/samples/db-create-instance-response.xml b/integration/apidocs/src/resources/samples/db-create-instance-response.xml new file mode 100644 index 0000000000..8ed9e306ec --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-create-instance-response.xml @@ -0,0 +1,19 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 748 +Date: Wed, 25 Jan 2012 21:53:23 GMT + + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-create-users-request.json b/integration/apidocs/src/resources/samples/db-create-users-request.json new file mode 100644 index 0000000000..022a12b2d1 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-create-users-request.json @@ -0,0 +1,32 @@ +POST /v1.0/1234/instances/1c59bdb8-03b6-4079-a7db-ba92d23a98b3/users HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: bb64d788-2dec-4a6b-a670-7151d108cacf +Accept: application/json +Content-Type: application/json + +{ + "users": [ + { + "databases": [ + { + "name": "databaseA" + } + ], + "name": "dbuser3", + "password": "password" + }, + { + "databases": [ + { + "name": "databaseB" + }, + { + "name": "databaseC" + } + ], + "name": "dbuser4", + "password": "password" + } + ] +} diff --git a/integration/apidocs/src/resources/samples/db-create-users-request.xml b/integration/apidocs/src/resources/samples/db-create-users-request.xml new file mode 100644 index 0000000000..2f24166560 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-create-users-request.xml @@ -0,0 +1,21 @@ +POST /v1.0/1234/instances/1f19ee9e-6124-46b3-a6c6-fc46a5756814/users HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: bb64d788-2dec-4a6b-a670-7151d108cacf +Accept: application/xml +Content-Type: application/xml + + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-create-users-response.json b/integration/apidocs/src/resources/samples/db-create-users-response.json new file mode 100644 index 0000000000..6a83f7fcb1 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-create-users-response.json @@ -0,0 +1,4 @@ +HTTP/1.1 202 Accepted +Content-Type: application/json +Content-Length: 0 +Date: Wed, 27 Jun 2012 23:11:18 GMT diff --git a/integration/apidocs/src/resources/samples/db-create-users-response.xml b/integration/apidocs/src/resources/samples/db-create-users-response.xml new file mode 100644 index 0000000000..9b6fa1dc90 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-create-users-response.xml @@ -0,0 +1,4 @@ +HTTP/1.1 202 Accepted +Content-Type: application/xml +Content-Length: 0 +Date: Wed, 27 Jun 2012 23:11:19 GMT diff --git a/integration/apidocs/src/resources/samples/db-credentials-20.json b/integration/apidocs/src/resources/samples/db-credentials-20.json new file mode 100644 index 0000000000..1ae90cd4a3 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-credentials-20.json @@ -0,0 +1,18 @@ +POST /v2.0/tokens HTTP/1.1 +User-Agent: curl/7.21.0 (x86_64-pc-linux-gnu) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6 +Host: identity.api.rackspacecloud.com +Accept: application/json +Content-Type: application/json +Content-Length: 54 + +{ + "auth": + { + "RAX-KSKEY:apiKeyCredentials": + { + "username": "jsmith", + "apiKey": "aaaaa-bbbbb-ccccc-12345678" + } + } +} + diff --git a/integration/apidocs/src/resources/samples/db-credentials-20.xml b/integration/apidocs/src/resources/samples/db-credentials-20.xml new file mode 100644 index 0000000000..cd3298b4d0 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-credentials-20.xml @@ -0,0 +1,14 @@ +POST /v2.0/tokens HTTP/1.1 +User-Agent: curl/7.21.0 (x86_64-pc-linux-gnu) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6 +Host: identity.api.rackspacecloud.com +Accept: application/xml +Content-Type: application/xml +Content-Length: 88 + + + + + diff --git a/integration/apidocs/src/resources/samples/db-credentials.json b/integration/apidocs/src/resources/samples/db-credentials.json new file mode 100644 index 0000000000..bf15936f9c --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-credentials.json @@ -0,0 +1,13 @@ +POST /v1.1/auth HTTP/1.1 +User-Agent: curl/7.21.0 (x86_64-pc-linux-gnu) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6 +Host: auth.api.rackspacecloud.com +Accept: application/json +Content-Type: application/json +Content-Length: 54 + +{ + "credentials" : { + "username" : "hub_cap", + "key" : "a86850deb2742ec3cb41518e26aa2d89" + } +} diff --git a/integration/apidocs/src/resources/samples/db-credentials.xml b/integration/apidocs/src/resources/samples/db-credentials.xml new file mode 100644 index 0000000000..ae7f8971e1 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-credentials.xml @@ -0,0 +1,11 @@ +POST /v1.1/auth HTTP/1.1 +User-Agent: curl/7.21.0 (x86_64-pc-linux-gnu) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6 +Host: auth.api.rackspacecloud.com +Accept: application/xml +Content-Type: application/xml +Content-Length: 88 + + + diff --git a/integration/apidocs/src/resources/samples/db-delete-databases-request.json b/integration/apidocs/src/resources/samples/db-delete-databases-request.json new file mode 100644 index 0000000000..eb958e9549 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-delete-databases-request.json @@ -0,0 +1,8 @@ +DELETE /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f/databases/exampledb HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-delete-databases-request.xml b/integration/apidocs/src/resources/samples/db-delete-databases-request.xml new file mode 100644 index 0000000000..94cfc0bd96 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-delete-databases-request.xml @@ -0,0 +1,8 @@ +DELETE /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f/databases/exampledb HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-delete-databases-response.json b/integration/apidocs/src/resources/samples/db-delete-databases-response.json new file mode 100644 index 0000000000..6a83f7fcb1 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-delete-databases-response.json @@ -0,0 +1,4 @@ +HTTP/1.1 202 Accepted +Content-Type: application/json +Content-Length: 0 +Date: Wed, 27 Jun 2012 23:11:18 GMT diff --git a/integration/apidocs/src/resources/samples/db-delete-databases-response.xml b/integration/apidocs/src/resources/samples/db-delete-databases-response.xml new file mode 100644 index 0000000000..3a2ce5f652 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-delete-databases-response.xml @@ -0,0 +1,4 @@ +HTTP/1.1 202 Accepted +Content-Type: application/xml +Content-Length: 0 +Date: Wed, 27 Jun 2012 23:11:18 GMT diff --git a/integration/apidocs/src/resources/samples/db-delete-instance-request.json b/integration/apidocs/src/resources/samples/db-delete-instance-request.json new file mode 100644 index 0000000000..fdf11e6494 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-delete-instance-request.json @@ -0,0 +1,8 @@ +DELETE /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-delete-instance-request.xml b/integration/apidocs/src/resources/samples/db-delete-instance-request.xml new file mode 100644 index 0000000000..b960fbf927 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-delete-instance-request.xml @@ -0,0 +1,8 @@ +DELETE /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-delete-instance-response.json b/integration/apidocs/src/resources/samples/db-delete-instance-response.json new file mode 100644 index 0000000000..a1ecb28dac --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-delete-instance-response.json @@ -0,0 +1,4 @@ +HTTP/1.1 202 Accepted +Content-Type: application/json +Content-Length: 0 +Date: Wed, 27 Jun 2012 23:15:51 GMT diff --git a/integration/apidocs/src/resources/samples/db-delete-instance-response.xml b/integration/apidocs/src/resources/samples/db-delete-instance-response.xml new file mode 100644 index 0000000000..3dcda464bf --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-delete-instance-response.xml @@ -0,0 +1,4 @@ +HTTP/1.1 202 Accepted +Content-Type: application/xml +Content-Length: 0 +Date: Wed, 27 Jun 2012 23:15:51 GMT diff --git a/integration/apidocs/src/resources/samples/db-delete-users-request.json b/integration/apidocs/src/resources/samples/db-delete-users-request.json new file mode 100644 index 0000000000..76ba9a600d --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-delete-users-request.json @@ -0,0 +1,8 @@ +DELETE /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f/users/testuser HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-delete-users-request.xml b/integration/apidocs/src/resources/samples/db-delete-users-request.xml new file mode 100644 index 0000000000..3be87be7ea --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-delete-users-request.xml @@ -0,0 +1,8 @@ +DELETE /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f/users/testuser HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-delete-users-response.json b/integration/apidocs/src/resources/samples/db-delete-users-response.json new file mode 100644 index 0000000000..e5f311463d --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-delete-users-response.json @@ -0,0 +1,4 @@ +HTTP/1.1 202 Accepted +Content-Type: application/json +Content-Length: 0 +Date: Wed, 27 Jun 2012 23:11:19 GMT diff --git a/integration/apidocs/src/resources/samples/db-delete-users-response.xml b/integration/apidocs/src/resources/samples/db-delete-users-response.xml new file mode 100644 index 0000000000..9b6fa1dc90 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-delete-users-response.xml @@ -0,0 +1,4 @@ +HTTP/1.1 202 Accepted +Content-Type: application/xml +Content-Length: 0 +Date: Wed, 27 Jun 2012 23:11:19 GMT diff --git a/integration/apidocs/src/resources/samples/db-enable-root-user-request.json b/integration/apidocs/src/resources/samples/db-enable-root-user-request.json new file mode 100644 index 0000000000..85ba63792a --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-enable-root-user-request.json @@ -0,0 +1,8 @@ +POST /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f/root HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-enable-root-user-request.xml b/integration/apidocs/src/resources/samples/db-enable-root-user-request.xml new file mode 100644 index 0000000000..66485f815f --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-enable-root-user-request.xml @@ -0,0 +1,8 @@ +POST /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f/root HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-enable-root-user-response.json b/integration/apidocs/src/resources/samples/db-enable-root-user-response.json new file mode 100644 index 0000000000..a7868ee712 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-enable-root-user-response.json @@ -0,0 +1,11 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 78 +Date: Wed, 25 Jan 2012 21:58:11 GMT + +{ + "user": { + "name": "root", + "password": "d4311cb1-d912-45be-8517-e8a46f54df66" + } +} diff --git a/integration/apidocs/src/resources/samples/db-enable-root-user-response.xml b/integration/apidocs/src/resources/samples/db-enable-root-user-response.xml new file mode 100644 index 0000000000..c2f99a9784 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-enable-root-user-response.xml @@ -0,0 +1,7 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 120 +Date: Wed, 25 Jan 2012 21:58:12 GMT + + + diff --git a/integration/apidocs/src/resources/samples/db-faults-badRequest.json b/integration/apidocs/src/resources/samples/db-faults-badRequest.json new file mode 100644 index 0000000000..fcb4571afc --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-faults-badRequest.json @@ -0,0 +1,11 @@ +HTTP/1.1 400 None +Content-Length: 120 +Content-Type: application/json; charset=UTF-8 +Date: Tue, 29 Nov 2011 00:33:48 GMT + +{ + "badRequest": { + "code": 400, + "message": "Volume 'size' needs to be a positive integer value, -1.0 cannot be accepted." + } +} diff --git a/integration/apidocs/src/resources/samples/db-faults-badRequest.xml b/integration/apidocs/src/resources/samples/db-faults-badRequest.xml new file mode 100644 index 0000000000..6067a4d9b9 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-faults-badRequest.xml @@ -0,0 +1,10 @@ +HTTP/1.1 400 None +Content-Type: application/xml +Content-Length: 121 +Date: Mon, 28 Nov 2011 18:19:37 GMT + + + + Volume 'size' needs to be a positive integer value, -1.0 cannot be accepted. + + diff --git a/integration/apidocs/src/resources/samples/db-faults-instanceFault.json b/integration/apidocs/src/resources/samples/db-faults-instanceFault.json new file mode 100644 index 0000000000..97299fc4fd --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-faults-instanceFault.json @@ -0,0 +1,11 @@ +HTTP/1.1 500 Internal Server Error +Content-Length: 120 +Content-Type: application/json; charset=UTF-8 +Date: Tue, 29 Nov 2011 00:33:48 GMT + +{ + "instanceFault": { + "code": 500, + "message": "The server has either erred or is incapable of performing the requested operation." + } +} diff --git a/integration/apidocs/src/resources/samples/db-faults-instanceFault.xml b/integration/apidocs/src/resources/samples/db-faults-instanceFault.xml new file mode 100644 index 0000000000..4864b0b483 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-faults-instanceFault.xml @@ -0,0 +1,10 @@ +HTTP/1.1 500 Internal Server Error +Content-Type: application/xml +Content-Length: 121 +Date: Mon, 28 Nov 2011 18:19:37 GMT + + + + The server has either erred or is incapable of performing the requested operation. + + diff --git a/integration/apidocs/src/resources/samples/db-faults-itemNotFound.json b/integration/apidocs/src/resources/samples/db-faults-itemNotFound.json new file mode 100644 index 0000000000..866329b5e5 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-faults-itemNotFound.json @@ -0,0 +1,11 @@ +HTTP/1.1 404 Not Found +Content-Length: 78 +Content-Type: application/json; charset=UTF-8 +Date: Tue, 29 Nov 2011 00:35:24 GMT + +{ + "itemNotFound": { + "code": 404, + "message": "The resource could not be found." + } +} diff --git a/integration/apidocs/src/resources/samples/db-faults-itemNotFound.xml b/integration/apidocs/src/resources/samples/db-faults-itemNotFound.xml new file mode 100644 index 0000000000..806bc2c7cb --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-faults-itemNotFound.xml @@ -0,0 +1,10 @@ +HTTP/1.1 404 Not Found +Content-Length: 147 +Content-Type: application/xml; charset=UTF-8 +Date: Mon, 28 Nov 2011 19:50:15 GMT + + + + The resource could not be found. + + diff --git a/integration/apidocs/src/resources/samples/db-flavors-by-id-request.json b/integration/apidocs/src/resources/samples/db-flavors-by-id-request.json new file mode 100644 index 0000000000..1944f3ae66 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-flavors-by-id-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/flavors/1 HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-flavors-by-id-request.xml b/integration/apidocs/src/resources/samples/db-flavors-by-id-request.xml new file mode 100644 index 0000000000..d2beb31de8 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-flavors-by-id-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/flavors/1 HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-flavors-by-id-response.json b/integration/apidocs/src/resources/samples/db-flavors-by-id-response.json new file mode 100644 index 0000000000..685d02ca41 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-flavors-by-id-response.json @@ -0,0 +1,22 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 209 +Date: Wed, 25 Jan 2012 21:53:05 GMT + +{ + "flavor": { + "id": 1, + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/1", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/flavors/1", + "rel": "bookmark" + } + ], + "name": "m1.tiny", + "ram": 512 + } +} diff --git a/integration/apidocs/src/resources/samples/db-flavors-by-id-response.xml b/integration/apidocs/src/resources/samples/db-flavors-by-id-response.xml new file mode 100644 index 0000000000..d1d5fec0a8 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-flavors-by-id-response.xml @@ -0,0 +1,11 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 284 +Date: Wed, 25 Jan 2012 21:53:05 GMT + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-flavors-request.json b/integration/apidocs/src/resources/samples/db-flavors-request.json new file mode 100644 index 0000000000..0d6817a504 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-flavors-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/flavors HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-flavors-request.xml b/integration/apidocs/src/resources/samples/db-flavors-request.xml new file mode 100644 index 0000000000..7981dd7577 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-flavors-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/flavors HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-flavors-response.json b/integration/apidocs/src/resources/samples/db-flavors-response.json new file mode 100644 index 0000000000..901d65ab6c --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-flavors-response.json @@ -0,0 +1,69 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 1768 +Date: Tue, 19 Jun 2012 19:52:45 GMT + +{ + "flavors": [ + { + "id": 1, + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/1", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/flavors/1", + "rel": "bookmark" + } + ], + "name": "m1.tiny", + "ram": 512 + }, + { + "id": 2, + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/2", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/flavors/2", + "rel": "bookmark" + } + ], + "name": "m1.small", + "ram": 1024 + }, + { + "id": 3, + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/3", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/flavors/3", + "rel": "bookmark" + } + ], + "name": "m1.medium", + "ram": 2048 + }, + { + "id": 4, + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/4", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/flavors/4", + "rel": "bookmark" + } + ], + "name": "m1.large", + "ram": 4096 + }, + ] +} diff --git a/integration/apidocs/src/resources/samples/db-flavors-response.xml b/integration/apidocs/src/resources/samples/db-flavors-response.xml new file mode 100644 index 0000000000..516da13703 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-flavors-response.xml @@ -0,0 +1,32 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 2300 +Date: Tue, 19 Jun 2012 19:52:45 GMT + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-gs-create-database-instance-response.json b/integration/apidocs/src/resources/samples/db-gs-create-database-instance-response.json new file mode 100644 index 0000000000..69f69b4107 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-gs-create-database-instance-response.json @@ -0,0 +1,41 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 756 +Date: Thu, 05 Apr 2012 16:48:44 GMT + +{ + "instance": { + "status": "BUILD", + "updated": "2012-04-05T16:48:44Z", + "name": "myrackinstance", + "links": [ + { + "href": "http://ord.databases.api.rackspacecloud.com/v1.0/1234/instances/d379ba5c-9a1f-4aa9-9a17-afe237d04c65", + "rel": "self" + }, + { + "href": "http://ord.databases.api.rackspacecloud.com/instances/d379ba5c-9a1f-4aa9-9a17-afe237d04c65", + "rel": "bookmark" + } + ], + "created": "2012-04-05T16:48:44Z", + "hostname": "ca9fa2985e47b351915c75f1a8e95d0729068892.rackspaceclouddb.com", + "volume": { + "size": 2 + }, + "flavor": { + "id": "1", + "links": [ + { + "href": "http://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/1", + "rel": "self" + }, + { "href": "http://ord.databases.api.rackspacecloud.com/flavors/1", + "rel": "bookmark" + } + ] + }, + "id": "d379ba5c-9a1f-4aa9-9a17-afe237d04c65" + } +} + diff --git a/integration/apidocs/src/resources/samples/db-gs-create-database-instance-response.xml b/integration/apidocs/src/resources/samples/db-gs-create-database-instance-response.xml new file mode 100644 index 0000000000..967a5dd8e6 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-gs-create-database-instance-response.xml @@ -0,0 +1,22 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 870 +Date: Thu, 05 Apr 2012 16:17:29 GMT + + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-guest-update-request.json b/integration/apidocs/src/resources/samples/db-guest-update-request.json new file mode 100644 index 0000000000..9d03d2bad6 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-guest-update-request.json @@ -0,0 +1,10 @@ +POST /v1.0/1234/mgmt/instances/3b16f448-4387-44f5-b464-45111dd7a934/action HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 570ef429-78eb-43ec-ad69-de9456cd88c9 +Accept: application/json +Content-Type: application/json + +{ + "update": {} +} diff --git a/integration/apidocs/src/resources/samples/db-guest-update-request.xml b/integration/apidocs/src/resources/samples/db-guest-update-request.xml new file mode 100644 index 0000000000..7d64bdda04 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-guest-update-request.xml @@ -0,0 +1,10 @@ +POST /v1.0/1234/mgmt/instances/f3a9e303-1542-4663-9dca-bfdcd7abe095/action HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 570ef429-78eb-43ec-ad69-de9456cd88c9 +Accept: application/xml +Content-Type: application/xml + + + + diff --git a/integration/apidocs/src/resources/samples/db-guest-update-response.json b/integration/apidocs/src/resources/samples/db-guest-update-response.json new file mode 100644 index 0000000000..1b029992e4 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-guest-update-response.json @@ -0,0 +1,6 @@ +HTTP/1.1 202 Accepted +Content-Type: text/plain; charset=UTF-8 +Content-Length: 58 +Date: Mon, 05 Mar 2012 23:48:46 GMT + + diff --git a/integration/apidocs/src/resources/samples/db-guest-update-response.xml b/integration/apidocs/src/resources/samples/db-guest-update-response.xml new file mode 100644 index 0000000000..99f5908ace --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-guest-update-response.xml @@ -0,0 +1,6 @@ +HTTP/1.1 202 Accepted +Content-Type: text/plain; charset=UTF-8 +Content-Length: 58 +Date: Mon, 05 Mar 2012 23:48:47 GMT + + diff --git a/integration/apidocs/src/resources/samples/db-instance-reboot-request.json b/integration/apidocs/src/resources/samples/db-instance-reboot-request.json new file mode 100644 index 0000000000..892243ae58 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-reboot-request.json @@ -0,0 +1,10 @@ +POST /v1.0/1234/mgmt/instances/0617673d-a280-40c3-8151-29800a8f9d70/action HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 73d60021-96e1-4d06-843d-72d20fa5377b +Accept: application/json +Content-Type: application/json + +{ + "reboot": {} +} diff --git a/integration/apidocs/src/resources/samples/db-instance-reboot-request.xml b/integration/apidocs/src/resources/samples/db-instance-reboot-request.xml new file mode 100644 index 0000000000..b93a9f9eca --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-reboot-request.xml @@ -0,0 +1,10 @@ +POST /v1.0/1234/mgmt/instances/475baf88-53b8-4ebb-932a-a2518227a6c6/action HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 73d60021-96e1-4d06-843d-72d20fa5377b +Accept: application/xml +Content-Type: application/xml + + + + diff --git a/integration/apidocs/src/resources/samples/db-instance-reboot-response.json b/integration/apidocs/src/resources/samples/db-instance-reboot-response.json new file mode 100644 index 0000000000..2308734cac --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-reboot-response.json @@ -0,0 +1,6 @@ +HTTP/1.1 202 Accepted +Content-Type: text/plain; charset=UTF-8 +Content-Length: 58 +Date: Tue, 07 Feb 2012 23:56:52 GMT + + diff --git a/integration/apidocs/src/resources/samples/db-instance-reboot-response.xml b/integration/apidocs/src/resources/samples/db-instance-reboot-response.xml new file mode 100644 index 0000000000..b8ad7430da --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-reboot-response.xml @@ -0,0 +1,6 @@ +HTTP/1.1 202 Accepted +Content-Type: text/plain; charset=UTF-8 +Content-Length: 58 +Date: Tue, 07 Feb 2012 23:56:55 GMT + + diff --git a/integration/apidocs/src/resources/samples/db-instance-resize-flavor-request.json b/integration/apidocs/src/resources/samples/db-instance-resize-flavor-request.json new file mode 100644 index 0000000000..c80b418c34 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-resize-flavor-request.json @@ -0,0 +1,12 @@ +POST /v1.0/1234/instances/aeb0b280-7c21-42cc-938f-275c0cc83c08/action HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: d45346d7-b313-41fb-a29b-e8f5f7c59de2 +Accept: application/json +Content-Type: application/json + +{ + "resize": { + "flavorRef": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/3" + } +} diff --git a/integration/apidocs/src/resources/samples/db-instance-resize-flavor-request.xml b/integration/apidocs/src/resources/samples/db-instance-resize-flavor-request.xml new file mode 100644 index 0000000000..274ef34fdf --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-resize-flavor-request.xml @@ -0,0 +1,10 @@ +POST /v1.0/1234/instances/51a5576d-c2cb-449b-b2be-176d65566ac3/action HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: d45346d7-b313-41fb-a29b-e8f5f7c59de2 +Accept: application/xml +Content-Type: application/xml + + + + diff --git a/integration/apidocs/src/resources/samples/db-instance-resize-flavor-response.json b/integration/apidocs/src/resources/samples/db-instance-resize-flavor-response.json new file mode 100644 index 0000000000..fe14f4dd49 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-resize-flavor-response.json @@ -0,0 +1,4 @@ +HTTP/1.1 202 Accepted +Content-Type: application/json +Content-Length: 0 +Date: Wed, 27 Jun 2012 23:14:20 GMT diff --git a/integration/apidocs/src/resources/samples/db-instance-resize-flavor-response.xml b/integration/apidocs/src/resources/samples/db-instance-resize-flavor-response.xml new file mode 100644 index 0000000000..6271164b1f --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-resize-flavor-response.xml @@ -0,0 +1,4 @@ +HTTP/1.1 202 Accepted +Content-Type: application/xml +Content-Length: 0 +Date: Wed, 27 Jun 2012 23:14:20 GMT diff --git a/integration/apidocs/src/resources/samples/db-instance-resize-instance-request.json b/integration/apidocs/src/resources/samples/db-instance-resize-instance-request.json new file mode 100644 index 0000000000..ec1ef6d5e1 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-resize-instance-request.json @@ -0,0 +1,12 @@ +POST /v1.0/1234/instances/23a3d4fb-3731-497b-afd4-bf25bde2b5fc/action HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 2eeb3252-0164-40f5-8fb7-85df5faa2698 +Accept: application/json +Content-Type: application/json + +{ + "resize": { + "flavorRef": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/2" + } +} diff --git a/integration/apidocs/src/resources/samples/db-instance-resize-instance-request.xml b/integration/apidocs/src/resources/samples/db-instance-resize-instance-request.xml new file mode 100644 index 0000000000..fd51dc85cd --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-resize-instance-request.xml @@ -0,0 +1,12 @@ +POST /v1.0/1234/instances/5d891bb6-6c61-4b0a-8b85-26f4ee461c9d/action HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 2eeb3252-0164-40f5-8fb7-85df5faa2698 +Accept: application/xml +Content-Type: application/xml + + + + https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/2 + + diff --git a/integration/apidocs/src/resources/samples/db-instance-resize-instance-response.json b/integration/apidocs/src/resources/samples/db-instance-resize-instance-response.json new file mode 100644 index 0000000000..48a2e91a0a --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-resize-instance-response.json @@ -0,0 +1,6 @@ +HTTP/1.1 202 Accepted +Content-Type: text/plain; charset=UTF-8 +Content-Length: 58 +Date: Mon, 06 Feb 2012 21:28:10 GMT + + diff --git a/integration/apidocs/src/resources/samples/db-instance-resize-instance-response.xml b/integration/apidocs/src/resources/samples/db-instance-resize-instance-response.xml new file mode 100644 index 0000000000..549a5f80ed --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-resize-instance-response.xml @@ -0,0 +1,6 @@ +HTTP/1.1 202 Accepted +Content-Type: text/plain; charset=UTF-8 +Content-Length: 58 +Date: Mon, 06 Feb 2012 21:28:11 GMT + + diff --git a/integration/apidocs/src/resources/samples/db-instance-resize-volume-request.json b/integration/apidocs/src/resources/samples/db-instance-resize-volume-request.json new file mode 100644 index 0000000000..7cec8fd88c --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-resize-volume-request.json @@ -0,0 +1,14 @@ +POST /v1.0/1234/instances/23a3d4fb-3731-497b-afd4-bf25bde2b5fc/action HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 2eeb3252-0164-40f5-8fb7-85df5faa2698 +Accept: application/json +Content-Type: application/json + +{ + "resize": { + "volume": { + "size": 4 + } + } +} diff --git a/integration/apidocs/src/resources/samples/db-instance-resize-volume-request.xml b/integration/apidocs/src/resources/samples/db-instance-resize-volume-request.xml new file mode 100644 index 0000000000..fe567b353e --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-resize-volume-request.xml @@ -0,0 +1,12 @@ +POST /v1.0/1234/instances/5d891bb6-6c61-4b0a-8b85-26f4ee461c9d/action HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 2eeb3252-0164-40f5-8fb7-85df5faa2698 +Accept: application/xml +Content-Type: application/xml + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-instance-resize-volume-response.json b/integration/apidocs/src/resources/samples/db-instance-resize-volume-response.json new file mode 100644 index 0000000000..319bbadbbd --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-resize-volume-response.json @@ -0,0 +1,4 @@ +HTTP/1.1 202 Accepted +Content-Type: application/json +Content-Length: 0 +Date: Wed, 27 Jun 2012 23:12:20 GMT diff --git a/integration/apidocs/src/resources/samples/db-instance-resize-volume-response.xml b/integration/apidocs/src/resources/samples/db-instance-resize-volume-response.xml new file mode 100644 index 0000000000..e7c611031d --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-resize-volume-response.xml @@ -0,0 +1,4 @@ +HTTP/1.1 202 Accepted +Content-Type: application/xml +Content-Length: 0 +Date: Wed, 27 Jun 2012 23:12:20 GMT diff --git a/integration/apidocs/src/resources/samples/db-instance-restart-request.json b/integration/apidocs/src/resources/samples/db-instance-restart-request.json new file mode 100644 index 0000000000..ee27681a34 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-restart-request.json @@ -0,0 +1,10 @@ +POST /v1.0/1234/instances/13d940c4-70bb-4ff4-8866-6ee9ab5e5cae/action HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + +{ + "restart": {} +} diff --git a/integration/apidocs/src/resources/samples/db-instance-restart-request.xml b/integration/apidocs/src/resources/samples/db-instance-restart-request.xml new file mode 100644 index 0000000000..1f2165afc0 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-restart-request.xml @@ -0,0 +1,9 @@ +POST /v1.0/1234/instances/ab585bea-2b42-4f95-8655-4afdd5037966/action HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + + diff --git a/integration/apidocs/src/resources/samples/db-instance-restart-response.json b/integration/apidocs/src/resources/samples/db-instance-restart-response.json new file mode 100644 index 0000000000..e5f311463d --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-restart-response.json @@ -0,0 +1,4 @@ +HTTP/1.1 202 Accepted +Content-Type: application/json +Content-Length: 0 +Date: Wed, 27 Jun 2012 23:11:19 GMT diff --git a/integration/apidocs/src/resources/samples/db-instance-restart-response.xml b/integration/apidocs/src/resources/samples/db-instance-restart-response.xml new file mode 100644 index 0000000000..9b6fa1dc90 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-restart-response.xml @@ -0,0 +1,4 @@ +HTTP/1.1 202 Accepted +Content-Type: application/xml +Content-Length: 0 +Date: Wed, 27 Jun 2012 23:11:19 GMT diff --git a/integration/apidocs/src/resources/samples/db-instance-status-detail-request.json b/integration/apidocs/src/resources/samples/db-instance-status-detail-request.json new file mode 100644 index 0000000000..b76d404463 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-status-detail-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-instance-status-detail-request.xml b/integration/apidocs/src/resources/samples/db-instance-status-detail-request.xml new file mode 100644 index 0000000000..08090d3348 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-status-detail-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-instance-status-detail-response.json b/integration/apidocs/src/resources/samples/db-instance-status-detail-response.json new file mode 100644 index 0000000000..e4e2de53fa --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-status-detail-response.json @@ -0,0 +1,42 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 685 +Date: Wed, 28 Mar 2012 21:37:29 GMT + +{ + "instance": { + "created": "2012-03-28T21:31:02Z", + "flavor": { + "id": "1", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/1", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/flavors/1", + "rel": "bookmark" + } + ] + }, + "hostname": "e09ad9a3f73309469cf1f43d11e79549caf9acf2.rackspaceclouddb.com", + "id": "2450c73f-7805-4afe-a42c-4094ab42666b", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances/2450c73f-7805-4afe-a42c-4094ab42666b", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/instances/2450c73f-7805-4afe-a42c-4094ab42666b", + "rel": "bookmark" + } + ], + "name": "xml_rack_instance", + "status": "ACTIVE", + "updated": "2012-03-28T21:34:25Z", + "volume": { + "size": 2, + "used": 0.124542236328125 + } + } +} diff --git a/integration/apidocs/src/resources/samples/db-instance-status-detail-response.xml b/integration/apidocs/src/resources/samples/db-instance-status-detail-response.xml new file mode 100644 index 0000000000..2b839e2e2b --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-status-detail-response.xml @@ -0,0 +1,20 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 796 +Date: Wed, 28 Mar 2012 21:37:31 GMT + + + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-instance-update-guest-request.json b/integration/apidocs/src/resources/samples/db-instance-update-guest-request.json new file mode 100644 index 0000000000..c73ff9f200 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-update-guest-request.json @@ -0,0 +1,11 @@ +GET /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f/action HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + +{ + "update": { + } +} diff --git a/integration/apidocs/src/resources/samples/db-instance-update-guest-request.xml b/integration/apidocs/src/resources/samples/db-instance-update-guest-request.xml new file mode 100644 index 0000000000..a146f7887c --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-update-guest-request.xml @@ -0,0 +1,10 @@ +GET /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f/action HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + + + diff --git a/integration/apidocs/src/resources/samples/db-instance-update-guest-response.json b/integration/apidocs/src/resources/samples/db-instance-update-guest-response.json new file mode 100644 index 0000000000..23bd9c1201 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-update-guest-response.json @@ -0,0 +1,8 @@ +HTTP/1.1 202 Accepted +Content-Type: text/plain; charset=UTF-8 +Content-Length: 58 +Date: Wed, 29 Feb 2012 23:53:42 GMT + +202 Accepted + +The request is accepted for processing. diff --git a/integration/apidocs/src/resources/samples/db-instance-update-guest-response.xml b/integration/apidocs/src/resources/samples/db-instance-update-guest-response.xml new file mode 100644 index 0000000000..23bd9c1201 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instance-update-guest-response.xml @@ -0,0 +1,8 @@ +HTTP/1.1 202 Accepted +Content-Type: text/plain; charset=UTF-8 +Content-Length: 58 +Date: Wed, 29 Feb 2012 23:53:42 GMT + +202 Accepted + +The request is accepted for processing. diff --git a/integration/apidocs/src/resources/samples/db-instances-index-pagination-request.json b/integration/apidocs/src/resources/samples/db-instances-index-pagination-request.json new file mode 100644 index 0000000000..4a55cfb7e0 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instances-index-pagination-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/instances?limit=2 HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-instances-index-pagination-request.xml b/integration/apidocs/src/resources/samples/db-instances-index-pagination-request.xml new file mode 100644 index 0000000000..bd9a930c1e --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instances-index-pagination-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/instances?limit=2 HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-instances-index-pagination-response.json b/integration/apidocs/src/resources/samples/db-instances-index-pagination-response.json new file mode 100644 index 0000000000..bdaf6b4f91 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instances-index-pagination-response.json @@ -0,0 +1,45 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 868 +Date: Tue, 19 Jun 2012 15:44:54 GMT + +{ + "instances": [ + { + "id": "13d940c4-70bb-4ff4-8866-6ee9ab5e5cae", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances/13d940c4-70bb-4ff4-8866-6ee9ab5e5cae", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/instances/13d940c4-70bb-4ff4-8866-6ee9ab5e5cae", + "rel": "bookmark" + } + ], + "name": "xml_rack_instance", + "status": "ACTIVE" + }, + { + "id": "4137d6a4-03b7-4b66-b0ef-8c7c35c470d3", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances/4137d6a4-03b7-4b66-b0ef-8c7c35c470d3", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/instances/4137d6a4-03b7-4b66-b0ef-8c7c35c470d3", + "rel": "bookmark" + } + ], + "name": "xml_rack_instance", + "status": "ACTIVE" + } + ], + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances?marker=4137d6a4-03b7-4b66-b0ef-8c7c35c470d3&limit=2", + "rel": "next" + } + ] +} diff --git a/integration/apidocs/src/resources/samples/db-instances-index-pagination-response.xml b/integration/apidocs/src/resources/samples/db-instances-index-pagination-response.xml new file mode 100644 index 0000000000..4aab91cc97 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instances-index-pagination-response.xml @@ -0,0 +1,23 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 887 +Date: Tue, 19 Jun 2012 15:44:54 GMT + + + + + + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-instances-index-request.json b/integration/apidocs/src/resources/samples/db-instances-index-request.json new file mode 100644 index 0000000000..ef941e56bf --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instances-index-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/instances HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-instances-index-request.xml b/integration/apidocs/src/resources/samples/db-instances-index-request.xml new file mode 100644 index 0000000000..f03169746c --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instances-index-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/instances HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-instances-index-response.json b/integration/apidocs/src/resources/samples/db-instances-index-response.json new file mode 100644 index 0000000000..5a885c741a --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instances-index-response.json @@ -0,0 +1,71 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 1150 +Date: Tue, 19 Jun 2012 19:53:04 GMT + +{ + "instances": [ + { + "flavor": { + "id": "1", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/1", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/flavors/1", + "rel": "bookmark" + } + ] + }, + "id": "28d1b8f3-172a-4f6d-983d-36021508444a", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances/28d1b8f3-172a-4f6d-983d-36021508444a", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/instances/28d1b8f3-172a-4f6d-983d-36021508444a", + "rel": "bookmark" + } + ], + "name": "json_rack_instance", + "status": "ACTIVE", + "volume": { + "size": 2 + } + }, + { + "flavor": { + "id": "1", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/1", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/flavors/1", + "rel": "bookmark" + } + ] + }, + "id": "8fb081af-f237-44f5-80cc-b46be1840ca9", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances/8fb081af-f237-44f5-80cc-b46be1840ca9", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/instances/8fb081af-f237-44f5-80cc-b46be1840ca9", + "rel": "bookmark" + } + ], + "name": "xml_rack_instance", + "status": "ACTIVE", + "volume": { + "size": 2 + } + } + ] +} diff --git a/integration/apidocs/src/resources/samples/db-instances-index-response.xml b/integration/apidocs/src/resources/samples/db-instances-index-response.xml new file mode 100644 index 0000000000..c5e3ea129d --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instances-index-response.xml @@ -0,0 +1,34 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 1492 +Date: Tue, 19 Jun 2012 19:53:04 GMT + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-instances-paged-index-request.json b/integration/apidocs/src/resources/samples/db-instances-paged-index-request.json new file mode 100644 index 0000000000..d74ea74cea --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instances-paged-index-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/instances?marker=369018b0-30de-2775-ab39-a19bd20006e9&limit=5 HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-instances-paged-index-request.xml b/integration/apidocs/src/resources/samples/db-instances-paged-index-request.xml new file mode 100644 index 0000000000..d7fd876308 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instances-paged-index-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/instances?marker=369018b0-30de-2775-ab39-a19bd20006e9&limit=5 HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-instances-paged-index-response.json b/integration/apidocs/src/resources/samples/db-instances-paged-index-response.json new file mode 100644 index 0000000000..d8de6465e8 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instances-paged-index-response.json @@ -0,0 +1,90 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 648 +Date: Wed, 25 Jan 2012 21:58:14 GMT + +{ + "instances": [ + { + "id": "28290db6-2f47-80d4-1b62-92bde6ff1a6e", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances/28290db6-2f47-80d4-1b62-92bde6ff1a6e", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/instances/28290db6-2f47-80d4-1b62-92bde6ff1a6e", + "rel": "bookmark" + } + ], + "name": "example_rack_instance", + "status": "ACTIVE" + }, + { + "id": "ae939ae2-3827-d005-4de6-0028a5f438d0", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances/ae939ae2-3827-d005-4de6-0028a5f438d0", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/instances/ae939ae2-3827-d005-4de6-0028a5f438d0", + "rel": "bookmark" + } + ], + "name": "example_rack_instance", + "status": "ACTIVE" + }, + { + "id": "006e4db2-5729-3f8e-77b0-075a1723ad37", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances/006e4db2-5729-3f8e-77b0-075a1723ad37", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/instances/006e4db2-5729-3f8e-77b0-075a1723ad37", + "rel": "bookmark" + } + ], + "name": "example_rack_instance", + "status": "ACTIVE" + }, + { + "id": "2806d85e-d363-2a27-f7fd-4e9911ebd8a9", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances/2806d85e-d363-2a27-f7fd-4e9911ebd8a9", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/instances/2806d85e-d363-2a27-f7fd-4e9911ebd8a9", + "rel": "bookmark" + } + ], + "name": "example_rack_instance", + "status": "ACTIVE" + }, + { + "id": "e0846e3b-3df4-398f-1e9a-d53650573dd1", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances/e0846e3b-3df4-398f-1e9a-d53650573dd1", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/instances/e0846e3b-3df4-398f-1e9a-d53650573dd1", + "rel": "bookmark" + } + ], + "name": "example_rack_instance", + "status": "ACTIVE" + } + ], + "links": [ + { + "rel": "next", + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances?marker=e0846e3b-3df4-398f-1e9a-d53650573dd1&limit=5" + } + ] +} diff --git a/integration/apidocs/src/resources/samples/db-instances-paged-index-response.xml b/integration/apidocs/src/resources/samples/db-instances-paged-index-response.xml new file mode 100644 index 0000000000..05dc5261fb --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-instances-paged-index-response.xml @@ -0,0 +1,41 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 828 +Date: Wed, 25 Jan 2012 21:58:14 GMT + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-list-databases-request.json b/integration/apidocs/src/resources/samples/db-list-databases-request.json new file mode 100644 index 0000000000..b7a3db2fa5 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-list-databases-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f/databases HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-list-databases-request.xml b/integration/apidocs/src/resources/samples/db-list-databases-request.xml new file mode 100644 index 0000000000..d0e2bb86ce --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-list-databases-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f/databases HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-list-databases-response.json b/integration/apidocs/src/resources/samples/db-list-databases-response.json new file mode 100644 index 0000000000..879da3d0ab --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-list-databases-response.json @@ -0,0 +1,24 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 136 +Date: Wed, 25 Jan 2012 21:58:01 GMT + +{ + "databases": [ + { + "name": "anotherexampledb" + }, + { + "name": "exampledb" + }, + { + "name": "nextround" + }, + { + "name": "sampledb" + }, + { + "name": "testingdb" + } + ] +} diff --git a/integration/apidocs/src/resources/samples/db-list-databases-response.xml b/integration/apidocs/src/resources/samples/db-list-databases-response.xml new file mode 100644 index 0000000000..d6643d6be4 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-list-databases-response.xml @@ -0,0 +1,12 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 248 +Date: Wed, 25 Jan 2012 21:58:02 GMT + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-list-users-request.json b/integration/apidocs/src/resources/samples/db-list-users-request.json new file mode 100644 index 0000000000..43c813d086 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-list-users-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f/users HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-list-users-request.xml b/integration/apidocs/src/resources/samples/db-list-users-request.xml new file mode 100644 index 0000000000..0b9a2bb583 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-list-users-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/instances/692d8418-7a8f-47f1-8060-59846c6e024f/users HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-list-users-response.json b/integration/apidocs/src/resources/samples/db-list-users-response.json new file mode 100644 index 0000000000..e2a94c60f9 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-list-users-response.json @@ -0,0 +1,28 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 152 +Date: Wed, 21 Mar 2012 17:46:46 GMT + +{ + "users": [ + { + "databases": [ + { + "name": "databaseA" + } + ], + "name": "dbuser3" + }, + { + "databases": [ + { + "name": "databaseB" + }, + { + "name": "databaseC" + } + ], + "name": "dbuser4" + } + ] +} diff --git a/integration/apidocs/src/resources/samples/db-list-users-response.xml b/integration/apidocs/src/resources/samples/db-list-users-response.xml new file mode 100644 index 0000000000..69a60eded7 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-list-users-response.xml @@ -0,0 +1,24 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 501 +Date: Wed, 27 Jun 2012 21:56:06 GMT + + + + + + + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-account-details-request.json b/integration/apidocs/src/resources/samples/db-mgmt-get-account-details-request.json new file mode 100644 index 0000000000..fc35c39785 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-account-details-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/accounts/1234 HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-account-details-request.xml b/integration/apidocs/src/resources/samples/db-mgmt-get-account-details-request.xml new file mode 100644 index 0000000000..467501c28c --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-account-details-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/accounts/1234 HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-account-details-response.json b/integration/apidocs/src/resources/samples/db-mgmt-get-account-details-response.json new file mode 100644 index 0000000000..8080f2e8de --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-account-details-response.json @@ -0,0 +1,24 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 280 +Date: Tue, 24 Jul 2012 19:48:33 GMT + +{ + "account": { + "id": "1234", + "instances": [ + { + "host": "hostname", + "id": "6e038c18-902f-4a41-b2a2-5f46fa62fc1b", + "name": "json_rack_instance", + "status": "ACTIVE" + }, + { + "host": "hostname", + "id": "72eed896-134e-4212-9f25-f81b4e0795e2", + "name": "xml_rack_instance", + "status": "ACTIVE" + } + ] + } +} diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-account-details-response.xml b/integration/apidocs/src/resources/samples/db-mgmt-get-account-details-response.xml new file mode 100644 index 0000000000..ae735efee6 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-account-details-response.xml @@ -0,0 +1,12 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 358 +Date: Tue, 24 Jul 2012 19:48:33 GMT + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-host-detail-request.json b/integration/apidocs/src/resources/samples/db-mgmt-get-host-detail-request.json new file mode 100644 index 0000000000..b30ee85b15 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-host-detail-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/hosts/hostname HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-host-detail-request.xml b/integration/apidocs/src/resources/samples/db-mgmt-get-host-detail-request.xml new file mode 100644 index 0000000000..d2b20fd825 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-host-detail-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/hosts/hostname HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-host-detail-response.json b/integration/apidocs/src/resources/samples/db-mgmt-get-host-detail-response.json new file mode 100644 index 0000000000..6466a5efef --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-host-detail-response.json @@ -0,0 +1,29 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 411 +Date: Tue, 24 Jul 2012 19:48:33 GMT + +{ + "host": { + "instances": [ + { + "id": "6e038c18-902f-4a41-b2a2-5f46fa62fc1b", + "name": "json_rack_instance", + "server_id": "64438fb6-4b98-4bd8-975e-222bc11daf74", + "status": "ACTIVE", + "tenant_id": "1234" + }, + { + "id": "72eed896-134e-4212-9f25-f81b4e0795e2", + "name": "xml_rack_instance", + "server_id": "6d93bc48-f736-41db-9ad7-c8594aac25fb", + "status": "ACTIVE", + "tenant_id": "1234" + } + ], + "name": "hostname", + "percentUsed": 51, + "totalRAM": 2004, + "usedRAM": 1024 + } +} diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-host-detail-response.xml b/integration/apidocs/src/resources/samples/db-mgmt-get-host-detail-response.xml new file mode 100644 index 0000000000..c12da22a1d --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-host-detail-response.xml @@ -0,0 +1,12 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 480 +Date: Tue, 24 Jul 2012 19:48:33 GMT + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-instance-details-request.json b/integration/apidocs/src/resources/samples/db-mgmt-get-instance-details-request.json new file mode 100644 index 0000000000..b0de95071d --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-instance-details-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/instances/6e038c18-902f-4a41-b2a2-5f46fa62fc1b HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-instance-details-request.xml b/integration/apidocs/src/resources/samples/db-mgmt-get-instance-details-request.xml new file mode 100644 index 0000000000..dd2ff53caf --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-instance-details-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/instances/72eed896-134e-4212-9f25-f81b4e0795e2 HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-instance-details-response.json b/integration/apidocs/src/resources/samples/db-mgmt-get-instance-details-response.json new file mode 100644 index 0000000000..3843ad8323 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-instance-details-response.json @@ -0,0 +1,51 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 903 +Date: Tue, 24 Jul 2012 19:48:33 GMT + +{ + "instance": { + "created": "2012-07-24T19:48:27", + "deleted": false, + "deleted_at": null, + "flavor": { + "id": "1", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/1", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/flavors/1", + "rel": "bookmark" + } + ] + }, + "guest_status": { + "state_description": "running" + }, + "host": "hostname", + "id": "6e038c18-902f-4a41-b2a2-5f46fa62fc1b", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances/6e038c18-902f-4a41-b2a2-5f46fa62fc1b", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/instances/6e038c18-902f-4a41-b2a2-5f46fa62fc1b", + "rel": "bookmark" + } + ], + "local_id": 0, + "name": "json_rack_instance", + "server_id": "64438fb6-4b98-4bd8-975e-222bc11daf74", + "status": "ACTIVE", + "task_description": "No tasks for the instance.", + "tenant_id": "1234", + "updated": "2012-07-24T19:48:29", + "volume": { + "id": "VOL_0fb3073c-15de-483b-bb85-376a8997dc8a", + "size": 2 + } + } +} diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-instance-details-response.xml b/integration/apidocs/src/resources/samples/db-mgmt-get-instance-details-response.xml new file mode 100644 index 0000000000..eaa95339cb --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-instance-details-response.xml @@ -0,0 +1,20 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 1008 +Date: Tue, 24 Jul 2012 19:48:33 GMT + + + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-root-details-request.json b/integration/apidocs/src/resources/samples/db-mgmt-get-root-details-request.json new file mode 100644 index 0000000000..551d6f0808 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-root-details-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/instances/6e038c18-902f-4a41-b2a2-5f46fa62fc1b/root HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-root-details-request.xml b/integration/apidocs/src/resources/samples/db-mgmt-get-root-details-request.xml new file mode 100644 index 0000000000..72f4c15661 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-root-details-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/instances/72eed896-134e-4212-9f25-f81b4e0795e2/root HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-root-details-response.json b/integration/apidocs/src/resources/samples/db-mgmt-get-root-details-response.json new file mode 100644 index 0000000000..3c5d815037 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-root-details-response.json @@ -0,0 +1,12 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 102 +Date: Tue, 24 Jul 2012 19:48:33 GMT + +{ + "root_history": { + "enabled": "Never", + "id": "6e038c18-902f-4a41-b2a2-5f46fa62fc1b", + "user": "Nobody" + } +} diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-root-details-response.xml b/integration/apidocs/src/resources/samples/db-mgmt-get-root-details-response.xml new file mode 100644 index 0000000000..87458a104c --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-root-details-response.xml @@ -0,0 +1,7 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 140 +Date: Tue, 24 Jul 2012 19:48:33 GMT + + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-storage-request.json b/integration/apidocs/src/resources/samples/db-mgmt-get-storage-request.json new file mode 100644 index 0000000000..28a7a894e5 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-storage-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/storage HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-storage-request.xml b/integration/apidocs/src/resources/samples/db-mgmt-get-storage-request.xml new file mode 100644 index 0000000000..6b8ba36065 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-storage-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/storage HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-storage-response.json b/integration/apidocs/src/resources/samples/db-mgmt-get-storage-response.json new file mode 100644 index 0000000000..3f5e7e7d16 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-storage-response.json @@ -0,0 +1,23 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 177 +Date: Tue, 24 Jul 2012 19:48:35 GMT + +{ + "devices": [ + { + "capacity": { + "available": 90, + "total": 100 + }, + "name": "fake_storage", + "provision": { + "available": 40, + "percent": 10, + "total": 50 + }, + "type": "test_type", + "used": 10 + } + ] +} diff --git a/integration/apidocs/src/resources/samples/db-mgmt-get-storage-response.xml b/integration/apidocs/src/resources/samples/db-mgmt-get-storage-response.xml new file mode 100644 index 0000000000..e154a26940 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-get-storage-response.xml @@ -0,0 +1,12 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 254 +Date: Tue, 24 Jul 2012 19:48:35 GMT + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-host-update-request.json b/integration/apidocs/src/resources/samples/db-mgmt-host-update-request.json new file mode 100644 index 0000000000..56f6ce432d --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-host-update-request.json @@ -0,0 +1,12 @@ +POST /v1.0/1234/mgmt/hosts/hostname/instances/action HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 73d60021-96e1-4d06-843d-72d20fa5377b +Accept: application/json +Content-Type: application/json + +{ + "update": {} +} + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-host-update-request.xml b/integration/apidocs/src/resources/samples/db-mgmt-host-update-request.xml new file mode 100644 index 0000000000..2662bd28a7 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-host-update-request.xml @@ -0,0 +1,10 @@ +POST /v1.0/1234/mgmt/hosts/hostname/instances/action HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 73d60021-96e1-4d06-843d-72d20fa5377b +Accept: application/xml +Content-Type: application/xml + + + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-host-update-response.json b/integration/apidocs/src/resources/samples/db-mgmt-host-update-response.json new file mode 100644 index 0000000000..0574b12a5b --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-host-update-response.json @@ -0,0 +1,6 @@ +HTTP/1.1 202 Accepted +Content-Type: text/plain; charset=UTF-8 +Content-Length: 58 +Date: Tue, 07 Feb 2012 23:56:52 GMT + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-host-update-response.xml b/integration/apidocs/src/resources/samples/db-mgmt-host-update-response.xml new file mode 100644 index 0000000000..bcea5bff07 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-host-update-response.xml @@ -0,0 +1,6 @@ +HTTP/1.1 202 Accepted +Content-Type: text/plain; charset=UTF-8 +Content-Length: 58 +Date: Tue, 07 Feb 2012 23:56:55 GMT + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-instance-diagnostics-request.json b/integration/apidocs/src/resources/samples/db-mgmt-instance-diagnostics-request.json new file mode 100644 index 0000000000..89ba631133 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-instance-diagnostics-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/instances/6e038c18-902f-4a41-b2a2-5f46fa62fc1b/diagnostics HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-instance-diagnostics-request.xml b/integration/apidocs/src/resources/samples/db-mgmt-instance-diagnostics-request.xml new file mode 100644 index 0000000000..cfaa3ea029 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-instance-diagnostics-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/instances/72eed896-134e-4212-9f25-f81b4e0795e2/diagnostics HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-instance-diagnostics-response.json b/integration/apidocs/src/resources/samples/db-mgmt-instance-diagnostics-response.json new file mode 100644 index 0000000000..2755708db5 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-instance-diagnostics-response.json @@ -0,0 +1,16 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 125 +Date: Tue, 24 Jul 2012 19:48:33 GMT + +{ + "diagnostics": { + "fdSize": 64, + "threads": 2, + "version": "1", + "vmHwm": 2872, + "vmPeak": 29160, + "vmRss": 2872, + "vmSize": 29096 + } +} diff --git a/integration/apidocs/src/resources/samples/db-mgmt-instance-diagnostics-response.xml b/integration/apidocs/src/resources/samples/db-mgmt-instance-diagnostics-response.xml new file mode 100644 index 0000000000..edd8f0e1e9 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-instance-diagnostics-response.xml @@ -0,0 +1,7 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 159 +Date: Tue, 24 Jul 2012 19:48:33 GMT + + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-instance-hwinfo-request.json b/integration/apidocs/src/resources/samples/db-mgmt-instance-hwinfo-request.json new file mode 100644 index 0000000000..c879a96d00 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-instance-hwinfo-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/instances/6e038c18-902f-4a41-b2a2-5f46fa62fc1b/hwinfo HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-instance-hwinfo-request.xml b/integration/apidocs/src/resources/samples/db-mgmt-instance-hwinfo-request.xml new file mode 100644 index 0000000000..18243538ff --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-instance-hwinfo-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/instances/72eed896-134e-4212-9f25-f81b4e0795e2/hwinfo HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-instance-hwinfo-response.json b/integration/apidocs/src/resources/samples/db-mgmt-instance-hwinfo-response.json new file mode 100644 index 0000000000..613f82c83f --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-instance-hwinfo-response.json @@ -0,0 +1,13 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 48 +Date: Tue, 24 Jul 2012 19:48:33 GMT + +{ + "hwinfo": { + "mem_total": 524288, + "num_cpus": 1 + } +} + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-instance-hwinfo-response.xml b/integration/apidocs/src/resources/samples/db-mgmt-instance-hwinfo-response.xml new file mode 100644 index 0000000000..129d41f0ff --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-instance-hwinfo-response.xml @@ -0,0 +1,15 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 160 +Date: Tue, 24 Jul 2012 19:48:33 GMT + + + + 1 + + + 524288 + + + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-instance-index-request.json b/integration/apidocs/src/resources/samples/db-mgmt-instance-index-request.json new file mode 100644 index 0000000000..0b8e59ebb6 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-instance-index-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/instances?deleted=false HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-instance-index-request.xml b/integration/apidocs/src/resources/samples/db-mgmt-instance-index-request.xml new file mode 100644 index 0000000000..0de1eb4cf7 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-instance-index-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/instances?deleted=false HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-instance-index-response.json b/integration/apidocs/src/resources/samples/db-mgmt-instance-index-response.json new file mode 100644 index 0000000000..9486f6bbd8 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-instance-index-response.json @@ -0,0 +1,89 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 1586 +Date: Tue, 24 Jul 2012 19:48:33 GMT + +{ + "instances": [ + { + "created": "2012-07-24T19:48:27", + "deleted": false, + "deleted_at": null, + "flavor": { + "id": "1", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/1", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/flavors/1", + "rel": "bookmark" + } + ] + }, + "host": "hostname", + "id": "6e038c18-902f-4a41-b2a2-5f46fa62fc1b", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances/6e038c18-902f-4a41-b2a2-5f46fa62fc1b", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/instances/6e038c18-902f-4a41-b2a2-5f46fa62fc1b", + "rel": "bookmark" + } + ], + "local_id": 0, + "name": "json_rack_instance", + "server_id": "64438fb6-4b98-4bd8-975e-222bc11daf74", + "status": "ACTIVE", + "task_description": "No tasks for the instance.", + "tenant_id": "1234", + "updated": "2012-07-24T19:48:29", + "volume": { + "size": 2 + } + }, + { + "created": "2012-07-24T19:48:27", + "deleted": false, + "deleted_at": null, + "flavor": { + "id": "1", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/1", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/flavors/1", + "rel": "bookmark" + } + ] + }, + "host": "hostname", + "id": "72eed896-134e-4212-9f25-f81b4e0795e2", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/instances/72eed896-134e-4212-9f25-f81b4e0795e2", + "rel": "self" + }, + { + "href": "https://ord.databases.api.rackspacecloud.com/instances/72eed896-134e-4212-9f25-f81b4e0795e2", + "rel": "bookmark" + } + ], + "local_id": 0, + "name": "xml_rack_instance", + "server_id": "6d93bc48-f736-41db-9ad7-c8594aac25fb", + "status": "ACTIVE", + "task_description": "No tasks for the instance.", + "tenant_id": "1234", + "updated": "2012-07-24T19:48:30", + "volume": { + "size": 2 + } + } + ] +} diff --git a/integration/apidocs/src/resources/samples/db-mgmt-instance-index-response.xml b/integration/apidocs/src/resources/samples/db-mgmt-instance-index-response.xml new file mode 100644 index 0000000000..c607012d5c --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-instance-index-response.xml @@ -0,0 +1,34 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 1896 +Date: Tue, 24 Jul 2012 19:48:33 GMT + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-list-accounts-request.json b/integration/apidocs/src/resources/samples/db-mgmt-list-accounts-request.json new file mode 100644 index 0000000000..6d253741cf --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-list-accounts-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/accounts HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-list-accounts-request.xml b/integration/apidocs/src/resources/samples/db-mgmt-list-accounts-request.xml new file mode 100644 index 0000000000..08641a9eb1 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-list-accounts-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/accounts HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-list-accounts-response.json b/integration/apidocs/src/resources/samples/db-mgmt-list-accounts-response.json new file mode 100644 index 0000000000..d65c4e2056 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-list-accounts-response.json @@ -0,0 +1,13 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 51 +Date: Tue, 24 Jul 2012 19:48:33 GMT + +{ + "accounts": [ + { + "id": "1234", + "num_instances": 2 + } + ] +} diff --git a/integration/apidocs/src/resources/samples/db-mgmt-list-accounts-response.xml b/integration/apidocs/src/resources/samples/db-mgmt-list-accounts-response.xml new file mode 100644 index 0000000000..99dded2c5d --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-list-accounts-response.xml @@ -0,0 +1,9 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 119 +Date: Tue, 24 Jul 2012 19:48:33 GMT + + + + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-list-hosts-request.json b/integration/apidocs/src/resources/samples/db-mgmt-list-hosts-request.json new file mode 100644 index 0000000000..a8f7b2c200 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-list-hosts-request.json @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/hosts HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-list-hosts-request.xml b/integration/apidocs/src/resources/samples/db-mgmt-list-hosts-request.xml new file mode 100644 index 0000000000..a7c26dd586 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-list-hosts-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/1234/mgmt/hosts HTTP/1.1 +User-Agent: python-troveclient +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-mgmt-list-hosts-response.json b/integration/apidocs/src/resources/samples/db-mgmt-list-hosts-response.json new file mode 100644 index 0000000000..db62c38114 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-list-hosts-response.json @@ -0,0 +1,13 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 54 +Date: Tue, 24 Jul 2012 19:48:33 GMT + +{ + "hosts": [ + { + "instanceCount": 2, + "name": "hostname" + } + ] +} diff --git a/integration/apidocs/src/resources/samples/db-mgmt-list-hosts-response.xml b/integration/apidocs/src/resources/samples/db-mgmt-list-hosts-response.xml new file mode 100644 index 0000000000..42af1c3f2d --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-mgmt-list-hosts-response.xml @@ -0,0 +1,9 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 116 +Date: Tue, 24 Jul 2012 19:48:33 GMT + + + + + diff --git a/integration/apidocs/src/resources/samples/db-request-types.json b/integration/apidocs/src/resources/samples/db-request-types.json new file mode 100644 index 0000000000..44bd6d67ac --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-request-types.json @@ -0,0 +1,15 @@ +POST /v1.0/1234/instances HTTP/1.1 +Host: ord.databases.api.rackspacecloud.com +Content-Type: application/json +Accept: application/xml +X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb + +{ + "instance": { + "name": "'my_instance_name'", + "flavorRef": "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/1", + "volume": { + "size": "2" + } + } +} diff --git a/integration/apidocs/src/resources/samples/db-response-types.xml b/integration/apidocs/src/resources/samples/db-response-types.xml new file mode 100644 index 0000000000..957b65c197 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-response-types.xml @@ -0,0 +1,24 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 380 +Date: Thu, 17 Nov 2011 02:37:37 GMT + + + + + + + + + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-version-request.json b/integration/apidocs/src/resources/samples/db-version-request.json new file mode 100644 index 0000000000..45e1b1b6ff --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-version-request.json @@ -0,0 +1,8 @@ +GET /v1.0/ HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-version-request.xml b/integration/apidocs/src/resources/samples/db-version-request.xml new file mode 100644 index 0000000000..eb4ce6c716 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-version-request.xml @@ -0,0 +1,8 @@ +GET /v1.0/ HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-version-response.json b/integration/apidocs/src/resources/samples/db-version-response.json new file mode 100644 index 0000000000..d5618703af --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-version-response.json @@ -0,0 +1,18 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 150 +Date: Wed, 25 Jan 2012 21:53:04 GMT + +{ + "version": { + "id": "v1.0", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/", + "rel": "self" + } + ], + "status": "CURRENT", + "updated": "2012-01-01T00:00:00Z" + } +} diff --git a/integration/apidocs/src/resources/samples/db-version-response.xml b/integration/apidocs/src/resources/samples/db-version-response.xml new file mode 100644 index 0000000000..d978f23295 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-version-response.xml @@ -0,0 +1,10 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 218 +Date: Wed, 25 Jan 2012 21:53:04 GMT + + + + + + diff --git a/integration/apidocs/src/resources/samples/db-versions-request.json b/integration/apidocs/src/resources/samples/db-versions-request.json new file mode 100644 index 0000000000..719a55667b --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-versions-request.json @@ -0,0 +1,8 @@ +GET / HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/json +Content-Type: application/json + + diff --git a/integration/apidocs/src/resources/samples/db-versions-request.xml b/integration/apidocs/src/resources/samples/db-versions-request.xml new file mode 100644 index 0000000000..0b9d413a0b --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-versions-request.xml @@ -0,0 +1,8 @@ +GET / HTTP/1.1 +User-Agent: python-example-client +Host: ord.databases.api.rackspacecloud.com +X-Auth-Token: 87c6033c-9ff6-405f-943e-2deb73f278b7 +Accept: application/xml +Content-Type: application/xml + + diff --git a/integration/apidocs/src/resources/samples/db-versions-response.json b/integration/apidocs/src/resources/samples/db-versions-response.json new file mode 100644 index 0000000000..0028b4e4b1 --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-versions-response.json @@ -0,0 +1,20 @@ +HTTP/1.1 200 OK +Content-Type: application/json +Content-Length: 153 +Date: Wed, 25 Jan 2012 21:53:04 GMT + +{ + "versions": [ + { + "id": "v1.0", + "links": [ + { + "href": "https://ord.databases.api.rackspacecloud.com/v1.0/", + "rel": "self" + } + ], + "status": "CURRENT", + "updated": "2012-01-01T00:00:00Z" + } + ] +} diff --git a/integration/apidocs/src/resources/samples/db-versions-response.xml b/integration/apidocs/src/resources/samples/db-versions-response.xml new file mode 100644 index 0000000000..6ec42452ae --- /dev/null +++ b/integration/apidocs/src/resources/samples/db-versions-response.xml @@ -0,0 +1,12 @@ +HTTP/1.1 200 OK +Content-Type: application/xml +Content-Length: 261 +Date: Wed, 25 Jan 2012 21:53:04 GMT + + + + + + + + diff --git a/integration/scripts/conf.json.example b/integration/scripts/conf.json.example new file mode 100644 index 0000000000..ee6bc7bf0a --- /dev/null +++ b/integration/scripts/conf.json.example @@ -0,0 +1,12 @@ +{ + "devstack":null, + "glance":null, + "horizon":null, + "keystone":null, + "nova":null, + "python_openstackclient":null, + "python_novaclient":null, + "trove":null, + "python_troveclient":null, + "tempest":null +} diff --git a/integration/scripts/conf/cassandra.conf b/integration/scripts/conf/cassandra.conf new file mode 100644 index 0000000000..5a2f0accce --- /dev/null +++ b/integration/scripts/conf/cassandra.conf @@ -0,0 +1,6 @@ + "instance_flavor_name": "test.large-5", + "instance_bigger_flavor_name": "test.large-5.resize", + "instance_eph_flavor_name": "test.eph.large-5", + "instance_bigger_eph_flavor_name": "test.eph.large-5.resize", + "trove_volume_support": true, + "trove_volume_size": 1, diff --git a/integration/scripts/conf/couchbase.conf b/integration/scripts/conf/couchbase.conf new file mode 100644 index 0000000000..5a2f0accce --- /dev/null +++ b/integration/scripts/conf/couchbase.conf @@ -0,0 +1,6 @@ + "instance_flavor_name": "test.large-5", + "instance_bigger_flavor_name": "test.large-5.resize", + "instance_eph_flavor_name": "test.eph.large-5", + "instance_bigger_eph_flavor_name": "test.eph.large-5.resize", + "trove_volume_support": true, + "trove_volume_size": 1, diff --git a/integration/scripts/conf/couchdb.conf b/integration/scripts/conf/couchdb.conf new file mode 100644 index 0000000000..65bd382c90 --- /dev/null +++ b/integration/scripts/conf/couchdb.conf @@ -0,0 +1,6 @@ + "instance_flavor_name": "test.tiny-3", + "instance_bigger_flavor_name": "test.tiny-3.resize", + "instance_eph_flavor_name": "test.eph.tiny-3", + "instance_bigger_eph_flavor_name": "test.eph.tiny-3.resize", + "trove_volume_support": true, + "trove_volume_size": 1, diff --git a/integration/scripts/conf/db2.conf b/integration/scripts/conf/db2.conf new file mode 100644 index 0000000000..a842ea02f8 --- /dev/null +++ b/integration/scripts/conf/db2.conf @@ -0,0 +1,6 @@ + "instance_flavor_name": "test.small-5", + "instance_bigger_flavor_name": "test.small-5.resize", + "instance_eph_flavor_name": "test.eph.small-5", + "instance_bigger_eph_flavor_name": "test.eph.small-5.resize", + "trove_volume_support": true, + "trove_volume_size": 1, diff --git a/integration/scripts/conf/mariadb.conf b/integration/scripts/conf/mariadb.conf new file mode 100644 index 0000000000..0e29d1140a --- /dev/null +++ b/integration/scripts/conf/mariadb.conf @@ -0,0 +1,6 @@ + "instance_flavor_name": "test.small-4", + "instance_bigger_flavor_name": "test.small-4.resize", + "instance_eph_flavor_name": "test.eph.small-4", + "instance_bigger_eph_flavor_name": "test.eph.small-4.resize", + "trove_volume_support": true, + "trove_volume_size": 1, diff --git a/integration/scripts/conf/mongodb.conf b/integration/scripts/conf/mongodb.conf new file mode 100644 index 0000000000..0e0aae02a2 --- /dev/null +++ b/integration/scripts/conf/mongodb.conf @@ -0,0 +1,6 @@ + "instance_flavor_name": "test.large-5", + "instance_bigger_flavor_name": "test.large-5.resize", + "instance_eph_flavor_name": "test.eph.large-5", + "instance_bigger_eph_flavor_name": "test.eph.large-5.resize", + "trove_volume_support": true, + "trove_volume_size": 5, diff --git a/integration/scripts/conf/mysql.conf b/integration/scripts/conf/mysql.conf new file mode 100644 index 0000000000..eac3c5790e --- /dev/null +++ b/integration/scripts/conf/mysql.conf @@ -0,0 +1,6 @@ + "instance_flavor_name": "test.small-3", + "instance_bigger_flavor_name": "test.small-3.resize", + "instance_eph_flavor_name": "test.eph.small-3", + "instance_bigger_eph_flavor_name": "test.eph.small-3.resize", + "trove_volume_support": true, + "trove_volume_size": 1, diff --git a/integration/scripts/conf/percona.conf b/integration/scripts/conf/percona.conf new file mode 100644 index 0000000000..eac3c5790e --- /dev/null +++ b/integration/scripts/conf/percona.conf @@ -0,0 +1,6 @@ + "instance_flavor_name": "test.small-3", + "instance_bigger_flavor_name": "test.small-3.resize", + "instance_eph_flavor_name": "test.eph.small-3", + "instance_bigger_eph_flavor_name": "test.eph.small-3.resize", + "trove_volume_support": true, + "trove_volume_size": 1, diff --git a/integration/scripts/conf/postgresql.conf b/integration/scripts/conf/postgresql.conf new file mode 100644 index 0000000000..8033f5826b --- /dev/null +++ b/integration/scripts/conf/postgresql.conf @@ -0,0 +1,6 @@ + "instance_flavor_name": "test.medium-4", + "instance_bigger_flavor_name": "test.medium-4.resize", + "instance_eph_flavor_name": "test.eph.medium-4", + "instance_bigger_eph_flavor_name": "test.eph.medium-4.resize", + "trove_volume_support": true, + "trove_volume_size": 1, diff --git a/integration/scripts/conf/pxc.conf b/integration/scripts/conf/pxc.conf new file mode 100644 index 0000000000..eac3c5790e --- /dev/null +++ b/integration/scripts/conf/pxc.conf @@ -0,0 +1,6 @@ + "instance_flavor_name": "test.small-3", + "instance_bigger_flavor_name": "test.small-3.resize", + "instance_eph_flavor_name": "test.eph.small-3", + "instance_bigger_eph_flavor_name": "test.eph.small-3.resize", + "trove_volume_support": true, + "trove_volume_size": 1, diff --git a/integration/scripts/conf/redis.conf b/integration/scripts/conf/redis.conf new file mode 100644 index 0000000000..65bd382c90 --- /dev/null +++ b/integration/scripts/conf/redis.conf @@ -0,0 +1,6 @@ + "instance_flavor_name": "test.tiny-3", + "instance_bigger_flavor_name": "test.tiny-3.resize", + "instance_eph_flavor_name": "test.eph.tiny-3", + "instance_bigger_eph_flavor_name": "test.eph.tiny-3.resize", + "trove_volume_support": true, + "trove_volume_size": 1, diff --git a/integration/scripts/conf/test_begin.conf b/integration/scripts/conf/test_begin.conf new file mode 100644 index 0000000000..eff96dea6f --- /dev/null +++ b/integration/scripts/conf/test_begin.conf @@ -0,0 +1,101 @@ +{ + "dbaas_url":"http://%service_host%:8779/v1.0", + "version_url":"http://%service_host%:8779", + "trove_auth_url":"http://%service_host%:35357/v2.0/tokens", + "trove_client_insecure":false, + "auth_strategy":null, + "trove_client_region_name": "%region_name%", + + "nova_client": { + "url":"http://%service_host%:8774/v1.1", + "auth_url":"http://%service_host%:35357/v2.0", + "nova_service_type":"compute", + "volume_service_type":"volume" + }, + + "flavors": null, + + "white_box":false, + "start_services": true, + "test_mgmt":false, + "use_local_ovz":false, + "use_venv":false, + "glance_code_root":"%glance_path%", + "glance_api_conf":"/vagrant/conf/glance-api.conf", + "glance_reg_conf":"/vagrant/conf/glance-reg.conf", + "glance_images_directory": "/glance_images", + "glance_image": "debian-squeeze-x86_64-openvz.tar.gz", + "report_directory":"%report_directory%", + "usr_bin_dir":"%bin_path%", + "nova_code_root":"%nova_path%", + "nova_conf":"/home/vagrant/nova.conf", + "keystone_code_root":"%keystone_path%", + "keystone_conf":"/etc/keystone/keystone.conf", + "keystone_use_combined":true, + "trove_code_root":"%trove_path%", + "trove_conf":"/tmp/trove.conf", + "trove_version":"v1.0", + "trove_api_updated":"2012-08-01T00:00:00Z", + "trove_max_accepted_volume_size": 1000, + "trove_max_instances_per_user": 55, + "trove_max_volumes_per_user": 100, + "use_reaper":false, + "users": [ + { "auth_user":"admin", + "auth_key":"%admin_password%", + "tenant":"admin", + "tenant_id":"%admin_tenant_id%", + "requirements": { + "is_admin":true, + "services": ["trove"] + } + }, + { "auth_user":"alt_demo", + "auth_key":"%admin_password%", + "tenant":"alt_demo", + "tenant_id":"%alt_demo_tenant_id%", + "requirements": { + "is_admin":false, + "services": ["trove"] + } + }, + { "auth_user":"admin_alt_demo", + "auth_key":"%admin_password%", + "tenant":"alt_demo", + "tenant_id":"%alt_demo_tenant_id%", + "requirements": { + "is_admin":true, + "services": ["swift"] + } + }, + { "auth_user":"demo", + "auth_key":"%admin_password%", + "tenant":"demo", + "tenant_id":"%demo_tenant_id%", + "requirements": { + "is_admin":false, + "services": ["nova", "trove"] + } + } + ], + "root_removed_from_instance_api": true, + "root_timestamp_disabled": false, + "openvz_disabled": true, + "management_api_disabled": true, + "dbaas_image": 1, + "dns_driver":"trove.dns.rsdns.driver.RsDnsDriver", + "dns_instance_entry_factory":"trove.dns.rsdns.driver.RsDnsInstanceEntryFactory", + "trove_dns_support": false, + "databases_page_size": 20, + "instances_page_size": 20, + "users_page_size": 20, + "rabbit_runs_locally": true, + "dbaas_datastore": "%datastore_type%", + "dbaas_datastore_version": "%datastore_version%", + "neutron_enabled": %neutron_enabled%, + "shared_network": "%shared_network%", + "shared_network_subnet": "%shared_network_subnet%", + "instance_fault_1_flavor_name": "test.fault_1-1", + "instance_fault_1_eph_flavor_name": "test.eph.fault_1-1", + "instance_fault_2_flavor_name": "test.fault_2-5", + "instance_fault_2_eph_flavor_name": "test.eph.fault_2-5", diff --git a/integration/scripts/conf/test_end.conf b/integration/scripts/conf/test_end.conf new file mode 100644 index 0000000000..10fc2de14a --- /dev/null +++ b/integration/scripts/conf/test_end.conf @@ -0,0 +1,2 @@ + "sentinel": null +} diff --git a/integration/scripts/conf/vertica.conf b/integration/scripts/conf/vertica.conf new file mode 100644 index 0000000000..242d7fa140 --- /dev/null +++ b/integration/scripts/conf/vertica.conf @@ -0,0 +1,6 @@ + "instance_flavor_name": "test.large-10", + "instance_bigger_flavor_name": "test.large-10.resize", + "instance_eph_flavor_name": "test.eph.large-10", + "instance_bigger_eph_flavor_name": "test.eph.large-10.resize", + "trove_volume_support": true, + "trove_volume_size": 5, diff --git a/integration/scripts/create_vm b/integration/scripts/create_vm new file mode 100755 index 0000000000..7b1b376779 --- /dev/null +++ b/integration/scripts/create_vm @@ -0,0 +1,84 @@ +#!/usr/bin/env python +""" +Sets up a VM with hardcoded paths to multiple source trees. +Creates a script for use with VMWare or a Vagrantfile. + +Uses a configuration file (in JSON format) that stores the paths to checked-out +copies of OpenStack projects on the host machine. If the path is None then +it lets devstack download them. +""" + +import json + + +class Config(object): + """ + Very simple configuration file thats just some JSON. + """ + + vm_paths = { + 'devstack':"/devstack", + 'glance': '/opt/stack/glance', + 'horizon': '/opt/stack/horizon', + 'keystone': "/opt/stack/keystone", + 'nova': "/opt/stack/nova", + 'python_openstackclient': "/opt/stack/python-openstackclient", + 'python_novaclient': "/opt/stack/python-novaclient", + 'trove':"/opt/stack/trove", + 'python_troveclient':"/opt/stack/python-troveclient", + 'tempest':"/opt/stack/tempest" + } + + def __init__(self, **kwargs): + for name in Config.vm_paths.keys(): + if name not in kwargs: + raise RuntimeError('Missing configuration value "%s".' % name) + value = kwargs[name] + if value is not None and type(value) is not str \ + and type(value) is not unicode: + raise RuntimeError('Path "%s" must be a string or None but is ' + 'of type %s.' % (name, type(value))) + setattr(self, name, kwargs[name]) + self.vagrant_path = kwargs.get("vagrant_path", "Vagrantfile") + + @staticmethod + def load(file_path): + file_contents = open(file_path, "r").read() + dict = json.loads(file_contents); + return Config(**dict) + + def write_vagrant_file(self): + with open(self.vagrant_path, 'w') as file: + file.write(""" +Vagrant::Config.run do |global_config| + # Host config + global_config.vm.define :host do |config| + + config.vm.network "33.33.44.11" + + config.vm.box = "precise" + config.vm.host_name = "host" + + config.ssh.timeout = 3600 + config.vm.customize do |vm| + vm.memory_size = 2048 + end + + config.vm.share_folder "integration", "/integration", "../" + +""") + for key in Config.vm_paths.keys(): + local_path = getattr(self, key) + vm_path = Config.vm_paths[key] + if local_path is not None: + file.write('\tconfig.vm.share_folder "%s", "%s", "%s" \n' + % (key, vm_path, local_path)) + file.write(""" + + end +end + """) + +if __name__=="__main__": + conf = Config.load("conf.json") + conf.write_vagrant_file() diff --git a/integration/scripts/files/elements/apt-conf-dir/README.rst b/integration/scripts/files/elements/apt-conf-dir/README.rst new file mode 100644 index 0000000000..c94e00ea33 --- /dev/null +++ b/integration/scripts/files/elements/apt-conf-dir/README.rst @@ -0,0 +1,16 @@ +============ +apt-conf-dir +============ + +This element overrides the default apt.conf.d directory for APT based systems. + +Environment Variables +--------------------- + +DIB_APT_CONF_DIR: + :Required: No + :Default: None + :Description: To override `DIB_APT_CONF_DIR`, set it to the path to your + apt.conf.d. The new apt.conf.d will take effect at build time + and run time. + :Example: ``DIB_APT_CONF_DIR=/etc/apt/apt.conf`` diff --git a/integration/scripts/files/elements/apt-conf-dir/extra-data.d/99-use-host-apt-confd b/integration/scripts/files/elements/apt-conf-dir/extra-data.d/99-use-host-apt-confd new file mode 100755 index 0000000000..e286d684e1 --- /dev/null +++ b/integration/scripts/files/elements/apt-conf-dir/extra-data.d/99-use-host-apt-confd @@ -0,0 +1,21 @@ +#!/bin/bash +# Override the default /etc/apt/apt.conf.d directory with $DIB_APT_CONF_DIR + +if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then + set -x +fi +set -eu +set -o pipefail + +# exit directly if DIB_APT_CONF_DIR is not defined properly +if [ -z "${DIB_APT_CONF_DIR:-}" ] ; then + echo "DIB_APT_CONF_DIR is not set - no apt.conf.d will be copied in" + exit 0 +elif [ ! -d "$DIB_APT_CONF_DIR" ] ; then + echo "$DIB_APT_CONF_DIR is not a valid apt.conf.d directory." + echo "You should assign a proper apt.conf.d directory in DIB_APT_CONF_DIR" + exit 1 +fi + +# copy the apt.conf to cloudimg +sudo cp -L -f -R $DIB_APT_CONF_DIR $TMP_MOUNT_PATH/etc/apt diff --git a/integration/scripts/files/elements/fedora-guest/extra-data.d/15-reddwarf-dep b/integration/scripts/files/elements/fedora-guest/extra-data.d/15-reddwarf-dep new file mode 100755 index 0000000000..97a5e438bc --- /dev/null +++ b/integration/scripts/files/elements/fedora-guest/extra-data.d/15-reddwarf-dep @@ -0,0 +1,48 @@ +#!/bin/bash + +set -e +set -o xtrace + +# CONTEXT: HOST prior to IMAGE BUILD as SCRIPT USER +# PURPOSE: Setup the requirements file for use by 15-reddwarf-dep + +source $_LIB/die + +BRANCH_OVERRIDE=${BRANCH_OVERRIDE:-default} +ADD_BRANCH=$(basename ${BRANCH_OVERRIDE}) +REQUIREMENTS_FILE=${REDSTACK_SCRIPTS}/files/requirements/fedora-requirements-${ADD_BRANCH}.txt + +[ -n "$TMP_HOOKS_PATH" ] || die "Temp hook path not set" +[ -e ${REQUIREMENTS_FILE} ] || die "Requirements not found" +[ -n "$HOST_USERNAME" ] || die "HOST_USERNAME not set" + +sudo -Hiu ${HOST_USERNAME} dd if=${REQUIREMENTS_FILE} of=${TMP_HOOKS_PATH}/requirements.txt + +# Grab the upper constraints file, but don't fail if we can't find it. +# If we are running in the CI environment, $DEST will be set and stackrc +# will use $DEST/requirements as the location for the requirements repo. +# Use that as it will help us chain a job with something that is changing UC. + +UC_FILE=upper-constraints.txt + +if [ -f "${DEST}/requirements/${UC_FILE}" ]; then + echo "Found ${DEST}/requirements/${UC_FILE}, using that" + sudo -Hiu ${HOST_USERNAME} dd if="${DEST}/requirements/${UC_FILE}" \ + of="${TMP_HOOKS_PATH}/${UC_FILE}" +else + UC_DIR=$(pwd) + UC_BRANCH=${BRANCH_OVERRIDE} + if [ "${ADD_BRANCH}" == "default" ]; then + UC_BRANCH=master + fi + + set +e + curl -o "${UC_DIR}/${UC_FILE}" \ + https://git.openstack.org/cgit/openstack/requirements/plain/${UC_FILE}?h=${UC_BRANCH} + set -e + + if [ -f "${UC_DIR}/${UC_FILE}" ]; then + sudo -Hiu ${HOST_USERNAME} dd if="${UC_DIR}/${UC_FILE}" of=${TMP_HOOKS_PATH}/${UC_FILE} + rm -f "${UC_DIR}/${UC_FILE}" + fi +fi diff --git a/integration/scripts/files/elements/fedora-guest/extra-data.d/20-guest-systemd b/integration/scripts/files/elements/fedora-guest/extra-data.d/20-guest-systemd new file mode 100755 index 0000000000..960342e350 --- /dev/null +++ b/integration/scripts/files/elements/fedora-guest/extra-data.d/20-guest-systemd @@ -0,0 +1,21 @@ +#!/bin/bash + +set -e +set -o xtrace + +# CONTEXT: HOST prior to IMAGE BUILD as SCRIPT USER +# PURPOSE: stages the bootstrap file and upstart conf file while replacing variables so that guest image is properly +# configured + +source $_LIB/die + +[ -n "$TMP_HOOKS_PATH" ] || die "Temp hook path not set" + +[ -n "${GUEST_USERNAME}" ] || die "GUEST_USERNAME needs to be set to the user for the guest image" +[ -n "${HOST_SCP_USERNAME}" ] || die "HOST_SCP_USERNAME needs to be set to the user for the host instance" +[ -n "${CONTROLLER_IP}" ] || die "CONTROLLER_IP needs to be set to the ip address that guests will use to contact the controller" +[ -n "${ESCAPED_PATH_TROVE}" ] || die "ESCAPED_PATH_TROVE needs to be set to the path to the trove directory on the redstack host" +[ -n "${REDSTACK_SCRIPTS}" ] || die "REDSTACK_SCRIPTS needs to be set to the trove-integration scripts dir" +[ -n "${ESCAPED_GUEST_LOGDIR}" ] || die "ESCAPED_GUEST_LOGDIR must be set to the escaped guest log dir" + +sed "s/GUEST_USERNAME/${GUEST_USERNAME}/g;s/GUEST_LOGDIR/${ESCAPED_GUEST_LOGDIR}/g;s/HOST_SCP_USERNAME/${HOST_SCP_USERNAME}/g;s/CONTROLLER_IP/${CONTROLLER_IP}/g;s/PATH_TROVE/${ESCAPED_PATH_TROVE}/g" ${REDSTACK_SCRIPTS}/files/trove-guest.systemd.conf > ${TMP_HOOKS_PATH}/trove-guest.service diff --git a/integration/scripts/files/elements/fedora-guest/extra-data.d/62-ssh-key b/integration/scripts/files/elements/fedora-guest/extra-data.d/62-ssh-key new file mode 100755 index 0000000000..63453a75c6 --- /dev/null +++ b/integration/scripts/files/elements/fedora-guest/extra-data.d/62-ssh-key @@ -0,0 +1,31 @@ +#!/bin/bash + +set -e +set -o xtrace + +# CONTEXT: HOST prior to IMAGE BUILD as SCRIPT USER +# PURPOSE: creates the SSH key on the host if it doesn't exist. Then this copies the keys over to a staging area where +# they will be duplicated in the guest VM. +# This process allows the host to log into the guest but more importantly the guest phones home to get the trove +# source + +source $_LIB/die + +[ -n "$TMP_HOOKS_PATH" ] || die "Temp hook path not set" + +[ -n "${HOST_USERNAME}" ] || die "HOST_USERNAME needs to be set to the user for the current user on the host" + +if [ `whoami` = "root" ]; then + die "This should not be run as root" +fi + +# copy files over the "staging" area for the guest image (they'll later be put in the correct location by the guest user +# not these keys should not be overridden otherwise a) you won't be able to ssh in and b) the guest won't be able to +# rsync the files +if [ -e ${SSH_DIR}/authorized_keys ]; then + sudo -Hiu ${HOST_USERNAME} dd if=${SSH_DIR}/authorized_keys of=${TMP_HOOKS_PATH}/ssh-authorized-keys + sudo -Hiu ${HOST_USERNAME} dd if=${SSH_DIR}/id_rsa of=${TMP_HOOKS_PATH}/id_rsa + sudo -Hiu ${HOST_USERNAME} dd if=${SSH_DIR}/id_rsa.pub of=${TMP_HOOKS_PATH}/id_rsa.pub +else + die "SSH Authorized Keys file must exist along with pub and private key" +fi diff --git a/integration/scripts/files/elements/fedora-guest/install.d/15-reddwarf-dep b/integration/scripts/files/elements/fedora-guest/install.d/15-reddwarf-dep new file mode 100755 index 0000000000..98fb24ba18 --- /dev/null +++ b/integration/scripts/files/elements/fedora-guest/install.d/15-reddwarf-dep @@ -0,0 +1,30 @@ +#!/bin/bash + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install trove guest python dependencies - see redstack functions_qemu + +set -e +set -o xtrace + +dnf install -y python-devel libxml2-devel libxslt-devel python-setuptools \ + python-sqlalchemy python-lxml \ + python-routes python-eventlet python-webob \ + python-kombu python-paste-deploy python-paste python-netaddr \ + python-httplib2 python-iso8601 python-pip python-mysql \ + python-migrate python-anyjson gcc python-pexpect + +# pick up the requirements file left for us by +# extra-data.d/15-reddwarf-dep + +TMP_HOOKS_DIR="/tmp/in_target.d" + +UPPER_CONSTRAINTS= +if [ -f ${TMP_HOOKS_DIR}/upper-constraints.txt ]; then + UPPER_CONSTRAINTS=" -c ${TMP_HOOKS_DIR}/upper-constraints.txt" +fi + +pip install -q --upgrade -r ${TMP_HOOKS_DIR}/requirements.txt ${UPPER_CONSTRAINTS} + +echo "diagnostic pip freeze output follows" +pip freeze +echo "diagnostic pip freeze output above" diff --git a/integration/scripts/files/elements/fedora-guest/install.d/20-etc b/integration/scripts/files/elements/fedora-guest/install.d/20-etc new file mode 100755 index 0000000000..bec065ef32 --- /dev/null +++ b/integration/scripts/files/elements/fedora-guest/install.d/20-etc @@ -0,0 +1,8 @@ +#!/bin/sh + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: take "staged" trove-guest.conf file and put it in the init directory on guest image + +dd if=/tmp/in_target.d/trove-guest.service of=/usr/lib/systemd/system/trove-guest.service + +systemctl enable trove-guest.service diff --git a/integration/scripts/files/elements/fedora-guest/install.d/50-user b/integration/scripts/files/elements/fedora-guest/install.d/50-user new file mode 100755 index 0000000000..a4b666bf3b --- /dev/null +++ b/integration/scripts/files/elements/fedora-guest/install.d/50-user @@ -0,0 +1,17 @@ +#!/bin/bash + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Add the guest image user that will own the trove agent source...if the user does not already exist + +set -e +set -o xtrace + +# Difference from apt, -G admin option +if ! id -u ${GUEST_USERNAME} >/dev/null 2>&1; then + echo "Adding ${GUEST_USERNAME} user" + useradd -m ${GUEST_USERNAME} -s /bin/bash + passwd ${GUEST_USERNAME} <<_EOF_ +${GUEST_USERNAME} +${GUEST_USERNAME} +_EOF_ +fi \ No newline at end of file diff --git a/integration/scripts/files/elements/fedora-guest/install.d/62-ssh-key b/integration/scripts/files/elements/fedora-guest/install.d/62-ssh-key new file mode 100755 index 0000000000..80c1e65c50 --- /dev/null +++ b/integration/scripts/files/elements/fedora-guest/install.d/62-ssh-key @@ -0,0 +1,29 @@ +#!/bin/bash + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: take "staged" ssh keys (see extra-data.d/62-ssh-key) and put them in the GUEST_USERS home directory + +set -e +set -o xtrace + +SSH_DIR="/home/${GUEST_USERNAME}/.ssh" +TMP_HOOKS_DIR="/tmp/in_target.d" + +if [ -e "${TMP_HOOKS_DIR}/ssh-authorized-keys" ]; then + if [ ! -e ${SSH_DIR} ]; then + # this method worked more reliable in vmware fusion over doing sudo -Hiu ${GUEST_USERNAME} + mkdir ${SSH_DIR} + chown ${GUEST_USERNAME}:${GUEST_USERNAME} ${SSH_DIR} + fi + sudo -Hiu ${GUEST_USERNAME} dd of=${SSH_DIR}/authorized_keys conv=notrunc if=${TMP_HOOKS_DIR}/ssh-authorized-keys + sudo -Hiu ${GUEST_USERNAME} chmod 600 ${SSH_DIR}/authorized_keys + if [ ! -e "${SSH_DIR}/id_rsa" ]; then + sudo -Hiu ${GUEST_USERNAME} dd of=${SSH_DIR}/id_rsa if=${TMP_HOOKS_DIR}/id_rsa + # perms have to be right on this file for ssh to work + sudo -Hiu ${GUEST_USERNAME} chmod 600 ${SSH_DIR}/id_rsa + sudo -Hiu ${GUEST_USERNAME} dd of=${SSH_DIR}/id_rsa.pub if=${TMP_HOOKS_DIR}/id_rsa.pub + fi +else + echo "SSH Keys were not staged by host" + exit -1 +fi diff --git a/integration/scripts/files/elements/fedora-guest/post-install.d/05-ipforwarding b/integration/scripts/files/elements/fedora-guest/post-install.d/05-ipforwarding new file mode 100755 index 0000000000..4824cfcf7b --- /dev/null +++ b/integration/scripts/files/elements/fedora-guest/post-install.d/05-ipforwarding @@ -0,0 +1,5 @@ +#!/bin/bash +set -e +set -o xtrace +sed -i -r 's/^\s*#(net\.ipv4\.ip_forward=1.*)/\1/' /etc/sysctl.conf +echo 1 > /proc/sys/net/ipv4/ip_forward diff --git a/integration/scripts/files/elements/fedora-guest/post-install.d/62-trove-guest-sudoers b/integration/scripts/files/elements/fedora-guest/post-install.d/62-trove-guest-sudoers new file mode 100755 index 0000000000..0581fd2b56 --- /dev/null +++ b/integration/scripts/files/elements/fedora-guest/post-install.d/62-trove-guest-sudoers @@ -0,0 +1,15 @@ +#!/bin/bash +set -e +set -o xtrace + +# CONTEXT: HOST after IMAGE BUILD as SCRIPT USER +# PURPOSE: add the guest user account to the /etc/sudoers files with NOPASSWD + +# Adds user to the sudoers file so they can do everything w/o a pass +# Some binaries might be under /sbin or /usr/sbin, so make sure sudo will +# see them by forcing PATH +TEMPFILE=`mktemp` +echo "${GUEST_USERNAME} ALL=(ALL) NOPASSWD:ALL" > $TEMPFILE +chmod 0440 $TEMPFILE +sudo chown root:root $TEMPFILE +sudo mv $TEMPFILE /etc/sudoers.d/60_trove_guest diff --git a/integration/scripts/files/elements/fedora-guest/post-install.d/90-yum-update b/integration/scripts/files/elements/fedora-guest/post-install.d/90-yum-update new file mode 100755 index 0000000000..cd2992c107 --- /dev/null +++ b/integration/scripts/files/elements/fedora-guest/post-install.d/90-yum-update @@ -0,0 +1,9 @@ +#!/bin/bash + +# CONTEXT: GUEST after packages installed +# PURPOSE: do dnf update to save each instance having to do all the work + +set -e +set -o xtrace + +dnf -y update diff --git a/integration/scripts/files/elements/fedora-mariadb/README.md b/integration/scripts/files/elements/fedora-mariadb/README.md new file mode 100644 index 0000000000..757f00b864 --- /dev/null +++ b/integration/scripts/files/elements/fedora-mariadb/README.md @@ -0,0 +1,3 @@ +Sets up a MariaDB server install in the image. + +TODO: auto-tune settings based on host resources or metadata service. diff --git a/integration/scripts/files/elements/fedora-mariadb/install.d/10-mariadb b/integration/scripts/files/elements/fedora-mariadb/install.d/10-mariadb new file mode 100755 index 0000000000..a5cc2c0e67 --- /dev/null +++ b/integration/scripts/files/elements/fedora-mariadb/install.d/10-mariadb @@ -0,0 +1,9 @@ +#!/bin/sh + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install controller base required packages + +set -e +set -o xtrace + +dnf -y install mariadb-server percona-xtrabackup diff --git a/integration/scripts/files/elements/fedora-mariadb/pre-install.d/10-percona-copr b/integration/scripts/files/elements/fedora-mariadb/pre-install.d/10-percona-copr new file mode 100755 index 0000000000..bcc552052a --- /dev/null +++ b/integration/scripts/files/elements/fedora-mariadb/pre-install.d/10-percona-copr @@ -0,0 +1,10 @@ +#!/bin/sh + +# CONTEXT: GUEST during PRE-CONSTRUCTION as ROOT +# PURPOSE: Setup COPR Percona repository + +set -e +set -o xtrace + +# install from Fedora repos +dnf -y install percona-xtrabackup diff --git a/integration/scripts/files/elements/fedora-mongodb/README.md b/integration/scripts/files/elements/fedora-mongodb/README.md new file mode 100644 index 0000000000..2518abf234 --- /dev/null +++ b/integration/scripts/files/elements/fedora-mongodb/README.md @@ -0,0 +1 @@ +Sets up a MongoDB install in the image. \ No newline at end of file diff --git a/integration/scripts/files/elements/fedora-mongodb/install.d/10-mongodb b/integration/scripts/files/elements/fedora-mongodb/install.d/10-mongodb new file mode 100755 index 0000000000..3b53a18b6c --- /dev/null +++ b/integration/scripts/files/elements/fedora-mongodb/install.d/10-mongodb @@ -0,0 +1,24 @@ +#!/bin/sh + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install controller base required packages + +set -e +set -o xtrace + +cat > "/etc/rc.local" << _EOF_ +# Make sure to disable Linux kernel feature transparent huge pages, +# it will affect greatly both memory usage and latency in a negative way. +# See: http://docs.mongodb.org/manual/tutorial/transparent-huge-pages/ +if test -f /sys/kernel/mm/redhat_transparent_hugepage/defrag; then + echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag +fi +if test -f /sys/kernel/mm/redhat_transparent_hugepage/enabled; then + echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled +fi + +exit \$? + +_EOF_ + +dnf -y install mongodb-server diff --git a/integration/scripts/files/elements/fedora-mongodb/install.d/25-trove-mongo-dep b/integration/scripts/files/elements/fedora-mongodb/install.d/25-trove-mongo-dep new file mode 100755 index 0000000000..a3e2633b22 --- /dev/null +++ b/integration/scripts/files/elements/fedora-mongodb/install.d/25-trove-mongo-dep @@ -0,0 +1,9 @@ +#!/bin/bash + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install trove guest python dependencies - see redstack functions_qemu + +set -e +set -o xtrace + +pip install pymongo>=3.0.2,!=3.1 diff --git a/integration/scripts/files/elements/fedora-mysql/README.md b/integration/scripts/files/elements/fedora-mysql/README.md new file mode 100644 index 0000000000..39a6ab8c66 --- /dev/null +++ b/integration/scripts/files/elements/fedora-mysql/README.md @@ -0,0 +1,3 @@ +Sets up a MySQL server install in the image. + +TODO: auto-tune settings based on host resources or metadata service. diff --git a/integration/scripts/files/elements/fedora-mysql/install.d/10-mysql b/integration/scripts/files/elements/fedora-mysql/install.d/10-mysql new file mode 100755 index 0000000000..ff7efa365e --- /dev/null +++ b/integration/scripts/files/elements/fedora-mysql/install.d/10-mysql @@ -0,0 +1,16 @@ +#!/bin/sh + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install controller base required packages + +set -e +set -o xtrace + +dnf -y install https://repo.mysql.com/mysql-community-release-fc22.rpm +dnf -y install mysql-community-server + +# move the config dir for now but leave /etc/my.cnf alone +# ln -s creates problems for the systemd script +mkdir /etc/mysql +mv /etc/my.cnf.d /etc/mysql/conf.d +chown -R mysql:mysql /etc/mysql diff --git a/integration/scripts/files/elements/fedora-mysql/install.d/40-xtrabackup b/integration/scripts/files/elements/fedora-mysql/install.d/40-xtrabackup new file mode 100755 index 0000000000..9c9709ca41 --- /dev/null +++ b/integration/scripts/files/elements/fedora-mysql/install.d/40-xtrabackup @@ -0,0 +1,10 @@ +#!/bin/sh + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install controller base required packages + +set -e +set -o xtrace + +dnf -y install percona-xtrabackup + diff --git a/integration/scripts/files/elements/fedora-mysql/post-install.d/30-register-mysql-service b/integration/scripts/files/elements/fedora-mysql/post-install.d/30-register-mysql-service new file mode 100644 index 0000000000..a7db5d926e --- /dev/null +++ b/integration/scripts/files/elements/fedora-mysql/post-install.d/30-register-mysql-service @@ -0,0 +1,6 @@ +#!/bin/bash + +set -e +set -o xtrace + +# DO NOT enable or start mysqld for systemd, let the guestagent coordinate startup diff --git a/integration/scripts/files/elements/fedora-percona/install.d/05-percona-server b/integration/scripts/files/elements/fedora-percona/install.d/05-percona-server new file mode 100755 index 0000000000..9c43c6ef7a --- /dev/null +++ b/integration/scripts/files/elements/fedora-percona/install.d/05-percona-server @@ -0,0 +1,17 @@ +#!/bin/sh + +# CONTEXT: GUEST during PRE-CONSTRUCTION as ROOT +# PURPOSE: Setup apt-repo list so that we can connect to Percona's repo + +set -e +set -o xtrace + +curl -o /etc/pki/rpm-gpg/RPM-GPG-KEY-percona http://www.percona.com/downloads/RPM-GPG-KEY-percona +cat < /etc/yum.repos.d/Percona.repo +[percona] +name = CentOS \$releasever - Percona +baseurl=http://repo.percona.com/centos/latest/os/\$basearch/ +enabled = 1 +gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-percona +gpgcheck = 1 +EOL \ No newline at end of file diff --git a/integration/scripts/files/elements/fedora-percona/install.d/10-mysql b/integration/scripts/files/elements/fedora-percona/install.d/10-mysql new file mode 100755 index 0000000000..284e81ac1f --- /dev/null +++ b/integration/scripts/files/elements/fedora-percona/install.d/10-mysql @@ -0,0 +1,16 @@ +#!/bin/sh + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install controller base required packages + +set -e +set -o xtrace + +# The fix to make versions of percona-xtrabackup > v2.2 work with Trove +# was put into the mysql guestagent code for Mitaka. There are no current +# plans to backport so we need to make sure the guest generated when the +# tests are run for Kilo or Liberty get the 2.2 verson of PXB +if [[ $BRANCH_OVERRIDE == "stable/kilo" || $BRANCH_OVERRIDE == "stable/liberty" ]]; then + PXB_VERSION_OVERRIDE="-22" +fi +dnf -y install percona-toolkit Percona-Server-shared-55 Percona-Server-server-55 Percona-Server-test-55 Percona-Server-client-55 percona-xtrabackup${PXB_VERSION_OVERRIDE} diff --git a/integration/scripts/files/elements/fedora-postgresql/install.d/10-postgresql b/integration/scripts/files/elements/fedora-postgresql/install.d/10-postgresql new file mode 100755 index 0000000000..a2ee463390 --- /dev/null +++ b/integration/scripts/files/elements/fedora-postgresql/install.d/10-postgresql @@ -0,0 +1,83 @@ +#!/bin/sh + +set -e +set -o xtrace + +cat > "/etc/sysctl.d/10-postgresql-performance.conf" << _EOF_ +# See 'http://www.postgresql.org/docs/9.3/static/kernel-resources.html' +# for best practices. +# It is recommended to disable memory overcommit, +# but the Python interpreter may require it on smaller flavors. +# We therefore stick with the heuristic overcommit setting. +vm.overcommit_memory=0 + +_EOF_ + +cat > "/etc/rc.local" << _EOF_ +# See 'http://www.postgresql.org/docs/9.3/static/kernel-resources.html' +# Disable Linux kernel transparent huge pages. This feature is not supported by +# by Postgres 9.3 and may negatively impact performance of the database. +if test -f /sys/kernel/mm/redhat_transparent_hugepage/defrag; then + echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag +fi +if test -f /sys/kernel/mm/redhat_transparent_hugepage/enabled; then + echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled +fi + +exit \$? + +_EOF_ + +dnf install -y http://yum.postgresql.org/9.4/fedora/fedora-22-x86_64/pgdg-fedora94-9.4-4.noarch.rpm + +dnf install -y postgresql94-server postgresql94-contrib postgresql94-devel gcc + +########################################### +# Hack alert: +# For Postgresql 9.4, pg_rewind is not in the main source tree and +# no packages exist in the repos, so it must be compiled manually +# and installed on the image until we can move to 9.5 +# See README at +# https://github.com/vmware/pg_rewind/tree/REL9_4_STABLE + +dev_pkgs="readline-devel zlib-devel krb5-devel openssl-devel pam-devel libxml2-devel libxslt-devel" + +yum install -y $dev_pkgs + +# We need pg_config to be accessible on the path +mkdir -p /tmp/build +cd /tmp/build +git clone https://github.com/vmware/pg_rewind.git --branch REL9_4_STABLE +git clone https://github.com/postgres/postgres.git --branch REL9_4_STABLE + +ln -s /usr/pgsql-9.4/bin/pg_config /usr/bin/pg_config + +cd pg_rewind +make USE_PGXS=1 top_srcdir=/tmp/build/postgres install +ln -s /usr/pgsql-9.4/bin/pg_rewind /usr/bin/pg_rewind + +# Cleanup + +cd +rm -rf /tmp/build +yum remove -y $dev_pkgs + + +# Though /var/lib/pgsql is the preferred directory, need to move it as +# this is where the volume will be mounted +su - postgres -c "/usr/pgsql-9.4/bin/initdb /var/lib/pgsql/9.4/data" +mv /var/lib/pgsql /var/lib/postgresql + +mv /lib/systemd/system/postgresql-9.4.service /lib/systemd/system/postgresql.service + +sed -i 's/PGDATA=\/var\/lib\/pgsql\/9.4\/data/PGDATA=\/var\/lib\/postgresql\/9.4\/data/' /lib/systemd/system/postgresql.service + +# Create a volatile directory for runtime files. +echo "d /var/run/postgresql/ 0755 postgres postgres" > /lib/tmpfiles.d/postgresql.conf + +# Install the native Python client. +dnf install -y postgresql-devel python-devel +pip install psycopg2 + +systemctl enable postgresql.service +systemctl start postgresql.service diff --git a/integration/scripts/files/elements/fedora-redis/README.md b/integration/scripts/files/elements/fedora-redis/README.md new file mode 100644 index 0000000000..426072cfb7 --- /dev/null +++ b/integration/scripts/files/elements/fedora-redis/README.md @@ -0,0 +1 @@ +Sets up a redis server install in the image. diff --git a/integration/scripts/files/elements/fedora-redis/install.d/10-redis b/integration/scripts/files/elements/fedora-redis/install.d/10-redis new file mode 100755 index 0000000000..8c23b5f63f --- /dev/null +++ b/integration/scripts/files/elements/fedora-redis/install.d/10-redis @@ -0,0 +1,9 @@ +#!/bin/sh + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install controller base required packages + +set -ex + +dnf -y install redis +service redis start diff --git a/integration/scripts/files/elements/ubuntu-cassandra/install.d/10-cassandra b/integration/scripts/files/elements/ubuntu-cassandra/install.d/10-cassandra new file mode 100755 index 0000000000..2bd40181c1 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-cassandra/install.d/10-cassandra @@ -0,0 +1,25 @@ +#!/bin/bash + +set -ex +set -o xtrace + +export DEBIAN_FRONTEND=noninteractive +apt-get install -qy curl +echo "deb http://debian.datastax.com/community stable main" >> /etc/apt/sources.list.d/cassandra.sources.list +curl -L http://debian.datastax.com/debian/repo_key | apt-key add - +apt-get update +apt-get install -qy openjdk-7-jdk expect python-dev +apt-get install -qy libxml2-dev ntp mc +apt-get install -qy libxslt1-dev python-pexpect +apt-get install -qy python-migrate build-essential + +apt-get install dsc21=2.1.* cassandra=2.1.* -qy + +# The Python Driver 2.0 for Apache Cassandra. +pip install cassandra-driver +# Sorted sets support for the Python driver. +pip install blist + +service cassandra stop +rm -rf /var/lib/cassandra/data/system/* +service cassandra start diff --git a/integration/scripts/files/elements/ubuntu-couchbase/install.d/10-couchbase b/integration/scripts/files/elements/ubuntu-couchbase/install.d/10-couchbase new file mode 100755 index 0000000000..0dc67d8bd8 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-couchbase/install.d/10-couchbase @@ -0,0 +1,8 @@ +set -e +set -o xtrace + +export DEBIAN_FRONTEND=noninteractive +apt-get install -qy curl +apt-get install -qy libssl0.9.8 +curl -O http://packages.couchbase.com/releases/2.2.0/couchbase-server-community_2.2.0_x86_64.deb +INSTALL_DONT_START_SERVER=1 dpkg -i couchbase-server-community_2.2.0_x86_64.deb diff --git a/integration/scripts/files/elements/ubuntu-couchdb/install.d/10-couchdb b/integration/scripts/files/elements/ubuntu-couchdb/install.d/10-couchdb new file mode 100755 index 0000000000..77871d3f33 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-couchdb/install.d/10-couchdb @@ -0,0 +1,19 @@ +#!/bin/bash + +set -e +set -o xtrace + +export DEBIAN_FRONTEND=noninteractive + +# install the ppa-finding tool for ubuntu 12.0.4 release +apt-get install -y python-software-properties +add-apt-repository -y ppa:couchdb/stable +# update cached list of packages +apt-get update -y +# remove any existing couchdb binaries +apt-get remove -yf couchdb couchdb-bin couchdb-common +# install couchdb +apt-get install -yV couchdb +# install curl to provide a way to interact with CouchDB +# over HTTP REST API +apt-get install -qy curl diff --git a/integration/scripts/files/elements/ubuntu-db2/README.md b/integration/scripts/files/elements/ubuntu-db2/README.md new file mode 100644 index 0000000000..b488f2dbe9 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-db2/README.md @@ -0,0 +1,36 @@ + Creates an image for DB2 Express-C v10.5 + + The packages for DB2 Express-C can be downloaded from: + http://www-01.ibm.com/software/data/db2/express-c/download.html + and click on the link "DB2 Express-C for Linux 64-bit". + New users can either get an IBM ID or click on the "Proceed without an + IBM ID". User is provided with a registration form which needs to be + completed in order to proceed further to download the DB2 Express-C + packages. After accepting the license agreement, user can download the + the DB2 Express-C package (.tar.gz file). + + There are 2 options for making the DB2 Express-C package accessible to + the Trove disk-image building process: + - place the package in a private repository and set the environment + variable DATASTORE_PKG_LOCATION with the url to this private + repository. + e.g. export DATASTORE_PKG_LOCATION="http://www.foo.com/db2/v10.5_linuxx64_expc.tar.gz" + + - download the package and place it in any directory on the local + filesystem that the trove-integration scripts can access. Set the + environment variable DATASTORE_PKG_LOCATION with the full path to + the downloaded package. + e.g. export DATASTORE_PKG_LOCATION="/home/stack/db2/v10.5_linuxx64_expc.tar.gz" + + The environment variables used are as follows: + + DATASTORE_PKG_LOCATION - is the place where user stores the DB2 + Express-C package after registration. This can either be a + url to a private repository or the full path to the + downloaded package on a local filesystem. + DATASTORE_DOWNLOAD_OPTS - defines any wget options user wants to specify + like user,password, etc. This is an optional variable and is + needed only if specifying a private repository to download + the packages from. + e.g. export DATASTORE_DOWNLOAD_OPTS="--user=foo --password='secret'" + diff --git a/integration/scripts/files/elements/ubuntu-db2/extra-data.d/20-copy-db2-pkgs b/integration/scripts/files/elements/ubuntu-db2/extra-data.d/20-copy-db2-pkgs new file mode 100755 index 0000000000..f82fd32ad5 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-db2/extra-data.d/20-copy-db2-pkgs @@ -0,0 +1,25 @@ +#!/bin/bash + +set -e +set -o xtrace + +# CONTEXT: HOST prior to IMAGE BUILD as SCRIPT USER +# PURPOSE: Download the DB2 Express-C v10.5 packages to a directory on the local filesystem or +# to a private repository. The download location is specified using the env variable: +# DATASTORE_PKG_LOCATION + +[ -n "${TMP_HOOKS_PATH}" ] || die "Temp hook path not set" +[ -n "${DATASTORE_PKG_LOCATION}" ] || die "DATASTORE_PKG_LOCATION not set" + +# First check if the package is available on the local filesystem. +if [ -f "${DATASTORE_PKG_LOCATION}" ]; then + echo "Found the DB2 Express-C packages in ${DATASTORE_PKG_LOCATION}." + dd if="${DATASTORE_PKG_LOCATION}" of=${TMP_HOOKS_PATH}/db2.tar.gz +# else, check if the package is available for download in a private repository. +elif wget ${DATASTORE_DOWNLOAD_OPTS} "${DATASTORE_PKG_LOCATION}" -O ${TMP_HOOKS_PATH}/db2.tar.gz; then + echo "Downloaded the DB2 Express-C package from the private repository" +else + echo "Unable to find the DB2 package at ${DATASTORE_PKG_LOCATION}" + echo "Please register and download the DB2 Express-C packages to a private repository or local filesystem." + exit -1 +fi diff --git a/integration/scripts/files/elements/ubuntu-db2/install.d/10-db2 b/integration/scripts/files/elements/ubuntu-db2/install.d/10-db2 new file mode 100755 index 0000000000..63bc3bf739 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-db2/install.d/10-db2 @@ -0,0 +1,52 @@ +#!/bin/bash + +set -e +set -o xtrace + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Uncompress the DB2 packages and install and configure DB2 on Ubuntu. + +# DB2_PKG_LOCATION points to the directory where the DB2 packages +# are located to install. +DB2_PKG_LOCATION="/db2" +mkdir ${DB2_PKG_LOCATION} +cd ${DB2_PKG_LOCATION} + +# DB2 install requires the hostname to be resolved correctly +host_name=`hostname` +echo "127.0.0.1 ${host_name}" >> /etc/hosts + +tar -xvzf /tmp/in_target.d/db2.tar.gz + +# installing dependencies +apt-get install libaio1 +apt-get install libstdc++6 + +# start the installation process. Accepts the default installation directory '/opt/ibm/db2/V10.5' +${DB2_PKG_LOCATION}/expc/db2_install -b /opt/ibm/db2/V10.5 -f sysreq -l ${DB2_PKG_LOCATION}/db2_install.log + +# create the DB2 users. +# DB2 instance owner - db2inst1 +# DB2 fence user - db2fenc1 +# DB2 admin user - db2das1 +useradd -m db2inst1 +useradd -m db2fenc1 +useradd -m db2das1 + +# Create the DB2 server instance +/opt/ibm/db2/V10.5/instance/db2icrt -a server -u db2fenc1 db2inst1 +/opt/ibm/db2/V10.5/cfg/db2ln + +# Configure DB2 server instance to communicate via TCP/IP on a particulat port. +echo 'db2c_db2inst1 50000/tcp # DB2 connection service port' >> /etc/services + +# Configure DB2 to use the TCP/IP settings defined above. +su - db2inst1 -c "db2 update database manager configuration using svcename db2c_db2inst1" + +# Start the actual TCP/IP communication. +su - db2inst1 -c "db2set DB2COMM=tcpip" + +# DB2 requires the hostname to be resolved correctly. Delete this entry from the +# /etc/hosts since this is the hostname of the instance where the image is being +# built. The correct hostname will be set in the guest agent. +sed -i "/127.0.0.1[[:space:]]*${host_name}/d" /etc/hosts diff --git a/integration/scripts/files/elements/ubuntu-guest/extra-data.d/15-reddwarf-dep b/integration/scripts/files/elements/ubuntu-guest/extra-data.d/15-reddwarf-dep new file mode 100755 index 0000000000..33b42306de --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-guest/extra-data.d/15-reddwarf-dep @@ -0,0 +1,48 @@ +#!/bin/bash + +set -e +set -o xtrace + +# CONTEXT: HOST prior to IMAGE BUILD as SCRIPT USER +# PURPOSE: Setup the requirements file for use by 15-reddwarf-dep + +source $_LIB/die + +BRANCH_OVERRIDE=${BRANCH_OVERRIDE:-default} +ADD_BRANCH=$(basename ${BRANCH_OVERRIDE}) +REQUIREMENTS_FILE=${REDSTACK_SCRIPTS}/files/requirements/ubuntu-requirements-${ADD_BRANCH}.txt + +[ -n "$TMP_HOOKS_PATH" ] || die "Temp hook path not set" +[ -e ${REQUIREMENTS_FILE} ] || die "Requirements not found" +[ -n "$HOST_USERNAME" ] || die "HOST_USERNAME not set" + +sudo -Hiu ${HOST_USERNAME} dd if=${REQUIREMENTS_FILE} of=${TMP_HOOKS_PATH}/requirements.txt + +# Grab the upper constraints file, but don't fail if we can't find it. +# If we are running in the CI environment, $DEST will be set and stackrc +# will use $DEST/requirements as the location for the requirements repo. +# Use that as it will help us chain a job with something that is changing UC. + +UC_FILE=upper-constraints.txt + +if [ -f "${DEST}/requirements/${UC_FILE}" ]; then + echo "Found ${DEST}/requirements/${UC_FILE}, using that" + sudo -Hiu ${HOST_USERNAME} dd if="${DEST}/requirements/${UC_FILE}" \ + of="${TMP_HOOKS_PATH}/${UC_FILE}" +else + UC_DIR=$(pwd) + UC_BRANCH=${BRANCH_OVERRIDE} + if [ "${ADD_BRANCH}" == "default" ]; then + UC_BRANCH=master + fi + + set +e + curl -o "${UC_DIR}/${UC_FILE}" \ + https://git.openstack.org/cgit/openstack/requirements/plain/${UC_FILE}?h=${UC_BRANCH} + set -e + + if [ -f "${UC_DIR}/${UC_FILE}" ]; then + sudo -Hiu ${HOST_USERNAME} dd if="${UC_DIR}/${UC_FILE}" of=${TMP_HOOKS_PATH}/${UC_FILE} + rm -f "${UC_DIR}/${UC_FILE}" + fi +fi diff --git a/integration/scripts/files/elements/ubuntu-guest/extra-data.d/20-guest-upstart b/integration/scripts/files/elements/ubuntu-guest/extra-data.d/20-guest-upstart new file mode 100755 index 0000000000..2b2215920d --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-guest/extra-data.d/20-guest-upstart @@ -0,0 +1,21 @@ +#!/bin/bash + +set -e +set -o xtrace + +# CONTEXT: HOST prior to IMAGE BUILD as SCRIPT USER +# PURPOSE: stages the bootstrap file and upstart conf file while replacing variables so that guest image is properly +# configured + +source $_LIB/die + +[ -n "$TMP_HOOKS_PATH" ] || die "Temp hook path not set" + +[ -n "${GUEST_USERNAME}" ] || die "GUEST_USERNAME needs to be set to the user for the guest image" +[ -n "${HOST_SCP_USERNAME}" ] || die "HOST_SCP_USERNAME needs to be set to the user for the host instance" +[ -n "${CONTROLLER_IP}" ] || die "CONTROLLER_IP needs to be set to the ip address that guests will use to contact the controller" +[ -n "${ESCAPED_PATH_TROVE}" ] || die "ESCAPED_PATH_TROVE needs to be set to the path to the trove directory on the redstack host" +[ -n "${REDSTACK_SCRIPTS}" ] || die "REDSTACK_SCRIPTS needs to be set to the trove-integration scripts dir" +[ -n "${ESCAPED_GUEST_LOGDIR}" ] || die "ESCAPED_GUEST_LOGDIR must be set to the escaped guest log dir" + +sed "s/GUEST_USERNAME/${GUEST_USERNAME}/g;s/GUEST_LOGDIR/${ESCAPED_GUEST_LOGDIR}/g;s/HOST_SCP_USERNAME/${HOST_SCP_USERNAME}/g;s/CONTROLLER_IP/${CONTROLLER_IP}/g;s/PATH_TROVE/${ESCAPED_PATH_TROVE}/g" ${REDSTACK_SCRIPTS}/files/trove-guest.upstart.conf > ${TMP_HOOKS_PATH}/trove-guest.conf diff --git a/integration/scripts/files/elements/ubuntu-guest/extra-data.d/62-ssh-key b/integration/scripts/files/elements/ubuntu-guest/extra-data.d/62-ssh-key new file mode 100755 index 0000000000..63453a75c6 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-guest/extra-data.d/62-ssh-key @@ -0,0 +1,31 @@ +#!/bin/bash + +set -e +set -o xtrace + +# CONTEXT: HOST prior to IMAGE BUILD as SCRIPT USER +# PURPOSE: creates the SSH key on the host if it doesn't exist. Then this copies the keys over to a staging area where +# they will be duplicated in the guest VM. +# This process allows the host to log into the guest but more importantly the guest phones home to get the trove +# source + +source $_LIB/die + +[ -n "$TMP_HOOKS_PATH" ] || die "Temp hook path not set" + +[ -n "${HOST_USERNAME}" ] || die "HOST_USERNAME needs to be set to the user for the current user on the host" + +if [ `whoami` = "root" ]; then + die "This should not be run as root" +fi + +# copy files over the "staging" area for the guest image (they'll later be put in the correct location by the guest user +# not these keys should not be overridden otherwise a) you won't be able to ssh in and b) the guest won't be able to +# rsync the files +if [ -e ${SSH_DIR}/authorized_keys ]; then + sudo -Hiu ${HOST_USERNAME} dd if=${SSH_DIR}/authorized_keys of=${TMP_HOOKS_PATH}/ssh-authorized-keys + sudo -Hiu ${HOST_USERNAME} dd if=${SSH_DIR}/id_rsa of=${TMP_HOOKS_PATH}/id_rsa + sudo -Hiu ${HOST_USERNAME} dd if=${SSH_DIR}/id_rsa.pub of=${TMP_HOOKS_PATH}/id_rsa.pub +else + die "SSH Authorized Keys file must exist along with pub and private key" +fi diff --git a/integration/scripts/files/elements/ubuntu-guest/install.d/05-base-apps b/integration/scripts/files/elements/ubuntu-guest/install.d/05-base-apps new file mode 100755 index 0000000000..5cd392b2c4 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-guest/install.d/05-base-apps @@ -0,0 +1,10 @@ +#!/bin/bash + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install basic services and applications + +set -e +set -o xtrace + +export DEBIAN_FRONTEND=noninteractive +apt-get -y install ntp apparmor-utils diff --git a/integration/scripts/files/elements/ubuntu-guest/install.d/15-reddwarf-dep b/integration/scripts/files/elements/ubuntu-guest/install.d/15-reddwarf-dep new file mode 100755 index 0000000000..8d989fe0cd --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-guest/install.d/15-reddwarf-dep @@ -0,0 +1,31 @@ +#!/bin/bash + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install trove guest python dependencies - see redstack functions_qemu + +set -e +set -o xtrace + +export DEBIAN_FRONTEND=noninteractive +apt-get -y install python-dev libxml2-dev libxslt1-dev python-setuptools \ + python-pip python-sqlalchemy python-lxml \ + python-routes python-eventlet python-webob \ + python-pastedeploy python-paste python-netaddr \ + python-httplib2 python-iso8601 python-pexpect python-mysqldb python-migrate + + +# pick up the requirements file left for us by +# extra-data.d/15-reddwarf-dep + +TMP_HOOKS_DIR="/tmp/in_target.d" + +UPPER_CONSTRAINTS= +if [ -f ${TMP_HOOKS_DIR}/upper-constraints.txt ]; then + UPPER_CONSTRAINTS=" -c ${TMP_HOOKS_DIR}/upper-constraints.txt" +fi + +pip install -q --upgrade -r ${TMP_HOOKS_DIR}/requirements.txt ${UPPER_CONSTRAINTS} + +echo "diagnostic pip freeze output follows" +pip freeze +echo "diagnostic pip freeze output above" diff --git a/integration/scripts/files/elements/ubuntu-guest/install.d/20-etc b/integration/scripts/files/elements/ubuntu-guest/install.d/20-etc new file mode 100755 index 0000000000..8ac19f7cdd --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-guest/install.d/20-etc @@ -0,0 +1,8 @@ +#!/bin/sh + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: take "staged" trove-guest.conf file and put it in the init directory on guest image + +dd if=/tmp/in_target.d/trove-guest.conf of=/etc/init/trove-guest.conf + + diff --git a/integration/scripts/files/elements/ubuntu-guest/install.d/50-user b/integration/scripts/files/elements/ubuntu-guest/install.d/50-user new file mode 100755 index 0000000000..99f68966b6 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-guest/install.d/50-user @@ -0,0 +1,18 @@ +#!/bin/bash + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Add the guest image user that will own the trove agent source...if the user does not already exist + +set -e +set -o xtrace + +if ! id -u ${GUEST_USERNAME} >/dev/null 2>&1; then + echo "Adding ${GUEST_USERNAME} user" + useradd -G sudo -m ${GUEST_USERNAME} -s /bin/bash + chown ${GUEST_USERNAME}:${GUEST_USERNAME} /home/${GUEST_USERNAME} + passwd ${GUEST_USERNAME} <<_EOF_ +${GUEST_USERNAME} +${GUEST_USERNAME} +_EOF_ +fi + diff --git a/integration/scripts/files/elements/ubuntu-guest/install.d/62-ssh-key b/integration/scripts/files/elements/ubuntu-guest/install.d/62-ssh-key new file mode 100755 index 0000000000..8085cbaaca --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-guest/install.d/62-ssh-key @@ -0,0 +1,28 @@ +#!/bin/bash + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: take "staged" ssh keys (see extra-data.d/62-ssh-key) and put them in the GUEST_USERS home directory + +set -e +set -o xtrace + +SSH_DIR="/home/${GUEST_USERNAME}/.ssh" +TMP_HOOKS_DIR="/tmp/in_target.d" + +if [ -e "${TMP_HOOKS_DIR}/ssh-authorized-keys" ]; then + if [ ! -e ${SSH_DIR} ]; then + # this method worked more reliable in vmware fusion over doing sudo -Hiu ${GUEST_USERNAME} + mkdir ${SSH_DIR} + chown ${GUEST_USERNAME}:${GUEST_USERNAME} ${SSH_DIR} + fi + sudo -Hiu ${GUEST_USERNAME} dd of=${SSH_DIR}/authorized_keys conv=notrunc if=${TMP_HOOKS_DIR}/ssh-authorized-keys + if [ ! -e "${SSH_DIR}/id_rsa" ]; then + sudo -Hiu ${GUEST_USERNAME} dd of=${SSH_DIR}/id_rsa if=${TMP_HOOKS_DIR}/id_rsa + # perms have to be right on this file for ssh to work + sudo -Hiu ${GUEST_USERNAME} chmod 600 ${SSH_DIR}/id_rsa + sudo -Hiu ${GUEST_USERNAME} dd of=${SSH_DIR}/id_rsa.pub if=${TMP_HOOKS_DIR}/id_rsa.pub + fi +else + echo "SSH Keys were not staged by host" + exit -1 +fi diff --git a/integration/scripts/files/elements/ubuntu-guest/install.d/98-ssh b/integration/scripts/files/elements/ubuntu-guest/install.d/98-ssh new file mode 100755 index 0000000000..2134c19889 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-guest/install.d/98-ssh @@ -0,0 +1,8 @@ +#!/bin/bash +# Regenerate host keys now. XXX: Really should be a cloud-init task, should get +# that working. + +set -e +set -o xtrace + +dpkg-reconfigure openssh-server diff --git a/integration/scripts/files/elements/ubuntu-guest/install.d/99-clean-apt b/integration/scripts/files/elements/ubuntu-guest/install.d/99-clean-apt new file mode 100755 index 0000000000..cc348c5cb9 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-guest/install.d/99-clean-apt @@ -0,0 +1,11 @@ +#!/bin/sh + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Delete contents of apt cache on guest (saves image disk space) + +set -e +set -o xtrace + +apt-get clean + + diff --git a/integration/scripts/files/elements/ubuntu-guest/post-install.d/05-ipforwarding b/integration/scripts/files/elements/ubuntu-guest/post-install.d/05-ipforwarding new file mode 100755 index 0000000000..48b7ad0d37 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-guest/post-install.d/05-ipforwarding @@ -0,0 +1,4 @@ +#!/bin/bash +set -e +set -o xtrace +sed -i -r 's/^\s*#(net\.ipv4\.ip_forward=1.*)/\1/' /etc/sysctl.conf diff --git a/integration/scripts/files/elements/ubuntu-guest/post-install.d/10-ntp b/integration/scripts/files/elements/ubuntu-guest/post-install.d/10-ntp new file mode 100755 index 0000000000..bc494ae356 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-guest/post-install.d/10-ntp @@ -0,0 +1,10 @@ +#!/bin/bash +ntpfile=`mktemp` +cat << EOF > $ntpfile +server ntp.ubuntu.com iburst +server 127.127.1.0 +fudge 127.127.1.0 stratum 10 +EOF + +mv /etc/ntp.conf /etc/ntp.conf.orig +mv $ntpfile /etc/ntp.conf diff --git a/integration/scripts/files/elements/ubuntu-guest/post-install.d/62-trove-guest-sudoers b/integration/scripts/files/elements/ubuntu-guest/post-install.d/62-trove-guest-sudoers new file mode 100755 index 0000000000..0581fd2b56 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-guest/post-install.d/62-trove-guest-sudoers @@ -0,0 +1,15 @@ +#!/bin/bash +set -e +set -o xtrace + +# CONTEXT: HOST after IMAGE BUILD as SCRIPT USER +# PURPOSE: add the guest user account to the /etc/sudoers files with NOPASSWD + +# Adds user to the sudoers file so they can do everything w/o a pass +# Some binaries might be under /sbin or /usr/sbin, so make sure sudo will +# see them by forcing PATH +TEMPFILE=`mktemp` +echo "${GUEST_USERNAME} ALL=(ALL) NOPASSWD:ALL" > $TEMPFILE +chmod 0440 $TEMPFILE +sudo chown root:root $TEMPFILE +sudo mv $TEMPFILE /etc/sudoers.d/60_trove_guest diff --git a/integration/scripts/files/elements/ubuntu-guest/post-install.d/90-apt-get-update b/integration/scripts/files/elements/ubuntu-guest/post-install.d/90-apt-get-update new file mode 100755 index 0000000000..6cc087a7db --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-guest/post-install.d/90-apt-get-update @@ -0,0 +1,9 @@ +#!/bin/bash + +# CONTEXT: GUEST after packages installed +# PURPOSE: do apt-get update to save each instance having to do all the work + +set -e +set -o xtrace + +apt-get update diff --git a/integration/scripts/files/elements/ubuntu-guest/pre-install.d/01-trim-pkgs b/integration/scripts/files/elements/ubuntu-guest/pre-install.d/01-trim-pkgs new file mode 100755 index 0000000000..8787df7b48 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-guest/pre-install.d/01-trim-pkgs @@ -0,0 +1,117 @@ +#!/bin/bash + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install basic services and applications + +set -e +set -o xtrace + +export DEBIAN_FRONTEND=noninteractive +apt-get -y purge acpid\ + apport\ + apport-symptoms\ + apt-transport-https\ + apt-xapian-index\ + aptitude\ + at\ + bash-completion\ + bc\ + bind9-host\ + bsdmainutils\ + busybox-static\ + byobu\ + command-not-found\ + command-not-found-data\ + curl\ + dbus\ + dmidecode\ + dosfstools\ + ed\ + fonts-ubuntu-font-family-console\ + friendly-recovery\ + ftp\ + fuse\ + geoip-database\ + groff-base\ + hdparm\ + info\ + install-info\ + iptables\ + iputils-tracepath\ + irqbalance\ + landscape-client\ + landscape-common\ + language-selector-common\ + laptop-detect\ + libaccountsservice0\ + libbind9-90\ + libclass-accessor-perl\ + libcwidget3\ + libdns100\ + libept1.4.12\ + libevent-2.0-5\ + libgc1c2\ + libgeoip1\ + libio-string-perl\ + libisc95\ + liblwres90\ + libnfnetlink0\ + libparse-debianchangelog-perl\ + libparted0debian1\ + libpcap0.8\ + libpci3\ + libpipeline1\ + libpolkit-gobject-1-0\ + libsasl2-modules\ + libsigc++-2.0-0c2a\ + libsub-name-perl\ + libusb-1.0-0\ + libxapian22\ + lshw\ + lsof\ + ltrace\ + man-db\ + mlocate\ + mtr-tiny\ + nano\ + ntfs-3g\ + parted\ + patch\ + plymouth-theme-ubuntu-text\ + popularity-contest\ + powermgmt-base\ + ppp\ + pppoeconf\ + python-debian\ + python-gdbm\ + python-pam\ + python-twisted-bin\ + python-xapian\ + python-zope.interface\ + screen\ + shared-mime-info\ + strace\ + tasksel\ + tcpdump\ + telnet\ + time\ + tmux\ + ubuntu-standard\ + ufw\ + update-manager-core\ + update-notifier-common\ + usbutils\ + uuid-runtime\ + w3m + +# The following packages cannot be removed as they cause cloud-init to be +# uninstalled in Ubuntu 14.04 +# gir1.2-glib-2.0 +# libdbus-glib-1-2 +# libgirepository-1.0-1 +# python-chardet +# python-serial +# xz-utils + +apt-get -y autoremove + diff --git a/integration/scripts/files/elements/ubuntu-guest/pre-install.d/04-baseline-tools b/integration/scripts/files/elements/ubuntu-guest/pre-install.d/04-baseline-tools new file mode 100755 index 0000000000..1a8647f510 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-guest/pre-install.d/04-baseline-tools @@ -0,0 +1,7 @@ +#!/bin/bash +# Install baseline packages and tools. + +set -e +set -o xtrace + +apt-get install -y language-pack-en python-software-properties \ No newline at end of file diff --git a/integration/scripts/files/elements/ubuntu-mariadb/README.md b/integration/scripts/files/elements/ubuntu-mariadb/README.md new file mode 100644 index 0000000000..757f00b864 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-mariadb/README.md @@ -0,0 +1,3 @@ +Sets up a MariaDB server install in the image. + +TODO: auto-tune settings based on host resources or metadata service. diff --git a/integration/scripts/files/elements/ubuntu-mariadb/install.d/30-mariadb b/integration/scripts/files/elements/ubuntu-mariadb/install.d/30-mariadb new file mode 100755 index 0000000000..065c2f9870 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-mariadb/install.d/30-mariadb @@ -0,0 +1,34 @@ +#!/bin/sh + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install controller base required packages + +set -e +set -o xtrace + +export DEBIAN_FRONTEND=noninteractive + +# NOTE(vkmc): Using MariaDB repositories is required +# https://mariadb.com/kb/en/mariadb/installing-mariadb-deb-files/ +apt-get -y install software-properties-common +apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xcbcb082a1bb943db +add-apt-repository 'deb http://ftp.osuosl.org/pub/mariadb/repo/10.1/ubuntu trusty main' + +# Pin MariaDB repository +sudo echo -e "Package: *\nPin: origin ftp.osuosl.org\nPin-Priority: 1000" > /etc/apt/preferences.d/mariadb.pref + +apt-get -y update +# The fix to make versions of percona-xtrabackup > v2.2 work with Trove +# was put into the mysql guestagent code for Mitaka. There are no current +# plans to backport so we need to make sure the guest generated when the +# tests are run for Kilo or Liberty get the 2.2 verson of PXB +if [[ $BRANCH_OVERRIDE == "stable/kilo" || $BRANCH_OVERRIDE == "stable/liberty" ]]; then + PXB_VERSION_OVERRIDE="-22" +fi +apt-get -y install socat percona-xtrabackup${PXB_VERSION_OVERRIDE} +apt-get -y install libmariadbclient18 mariadb-server + +cat >/etc/mysql/conf.d/no_perf_schema.cnf <<_EOF_ +[mysqld] +performance_schema = off +_EOF_ diff --git a/integration/scripts/files/elements/ubuntu-mariadb/pre-install.d/10-percona-apt-key b/integration/scripts/files/elements/ubuntu-mariadb/pre-install.d/10-percona-apt-key new file mode 100755 index 0000000000..ec1d89d561 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-mariadb/pre-install.d/10-percona-apt-key @@ -0,0 +1,40 @@ +#!/bin/sh + +# CONTEXT: GUEST during PRE-CONSTRUCTION as ROOT +# PURPOSE: Setup apt-repo list so that we can connect to Percona's repo + +set -e +set -o xtrace + +[ -n "${GUEST_USERNAME}" ] || die "GUEST_USERNAME needs to be set to the user for the guest image" +[ -n "${RELEASE}" ] || die "RELEASE must be set to either Trusty or Precise" + +# Add Percona GPG key +mkdir -p /home/${GUEST_USERNAME}/.gnupg + +# sometimes the primary key server is unavailable and we should try an +# alternate. see +# https://bugs.launchpad.net/percona-server/+bug/907789. Disable +# shell errexit so we can interrogate the exit code and take action +# based on the exit code. We will reenable it later. +set +e +apt-key adv --keyserver hkp://keys.gnupg.net --recv-keys 1C4CBDCDCD2EFD2A + +if [ "$?" -ne "0" ]; +then + echo "Trying alternate keyserver hkp://keyserver.ubuntu.com" + set -e + apt-key adv --keyserver hkp://keyserver.ubuntu.com --recv-keys 1C4CBDCDCD2EFD2A +fi + +set -e + +# Add Percona repo +# Creates the Percona sources list +cat < /etc/apt/sources.list.d/percona.list +deb http://repo.percona.com/apt $RELEASE main +deb-src http://repo.percona.com/apt $RELEASE main +EOL + +# Force an update +apt-get -y update diff --git a/integration/scripts/files/elements/ubuntu-mariadb/pre-install.d/20-apparmor-mysql-local b/integration/scripts/files/elements/ubuntu-mariadb/pre-install.d/20-apparmor-mysql-local new file mode 100755 index 0000000000..a3e1dc7c7d --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-mariadb/pre-install.d/20-apparmor-mysql-local @@ -0,0 +1,11 @@ +#!/bin/sh + +set -e + +#CONTEXT: chroot on host +#PURPOSE: Allows mysqld to create temporary files when restoring backups + +cat <>/etc/apparmor.d/local/usr.sbin.mysqld + /tmp/ rw, + /tmp/** rwk, +EOF diff --git a/integration/scripts/files/elements/ubuntu-mongodb/README.md b/integration/scripts/files/elements/ubuntu-mongodb/README.md new file mode 100644 index 0000000000..5b9e33bb6a --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-mongodb/README.md @@ -0,0 +1 @@ +Sets up a MongoDB install in the image. diff --git a/integration/scripts/files/elements/ubuntu-mongodb/install.d/10-mongodb-thp b/integration/scripts/files/elements/ubuntu-mongodb/install.d/10-mongodb-thp new file mode 100755 index 0000000000..48d6c8bf76 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-mongodb/install.d/10-mongodb-thp @@ -0,0 +1,42 @@ +#!/bin/sh + +set -e +set -o xtrace + +export DEBIAN_FRONTEND=noninteractive + +cat > /etc/init.d/disable-transparent-hugepages << '_EOF_' +#!/bin/sh +### BEGIN INIT INFO +# Provides: disable-transparent-hugepages +# Required-Start: $local_fs +# Required-Stop: +# X-Start-Before: mongod mongodb-mms-automation-agent +# Default-Start: 2 3 4 5 +# Default-Stop: 0 1 6 +# Short-Description: Disable Linux transparent huge pages +# Description: Disable Linux transparent huge pages, to improve +# database performance. +### END INIT INFO + +case $1 in + start) + if [ -d /sys/kernel/mm/transparent_hugepage ]; then + thp_path=/sys/kernel/mm/transparent_hugepage + elif [ -d /sys/kernel/mm/redhat_transparent_hugepage ]; then + thp_path=/sys/kernel/mm/redhat_transparent_hugepage + else + return 0 + fi + + echo 'never' > ${thp_path}/enabled + echo 'never' > ${thp_path}/defrag + + unset thp_path + ;; +esac +_EOF_ + +chmod 755 /etc/init.d/disable-transparent-hugepages + +update-rc.d disable-transparent-hugepages defaults diff --git a/integration/scripts/files/elements/ubuntu-mongodb/install.d/20-mongodb b/integration/scripts/files/elements/ubuntu-mongodb/install.d/20-mongodb new file mode 100755 index 0000000000..6a95d2f03d --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-mongodb/install.d/20-mongodb @@ -0,0 +1,8 @@ +#!/bin/sh + +set -e +set -o xtrace + +export DEBIAN_FRONTEND=noninteractive + +apt-get -y install mongodb-org=3.2.6 diff --git a/integration/scripts/files/elements/ubuntu-mongodb/install.d/25-trove-mongo-dep b/integration/scripts/files/elements/ubuntu-mongodb/install.d/25-trove-mongo-dep new file mode 100755 index 0000000000..a3e2633b22 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-mongodb/install.d/25-trove-mongo-dep @@ -0,0 +1,9 @@ +#!/bin/bash + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install trove guest python dependencies - see redstack functions_qemu + +set -e +set -o xtrace + +pip install pymongo>=3.0.2,!=3.1 diff --git a/integration/scripts/files/elements/ubuntu-mongodb/install.d/30-mongodb-conf b/integration/scripts/files/elements/ubuntu-mongodb/install.d/30-mongodb-conf new file mode 100755 index 0000000000..afb582847d --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-mongodb/install.d/30-mongodb-conf @@ -0,0 +1,26 @@ +#!/bin/sh + +set -e +set -o xtrace + +# Remove the default pid file +rm -f /var/run/mongodb.pid + + +cat > /etc/mongod.conf << '_EOF_' +storage.dbPath: /var/lib/mongodb +security.authorization: enabled +storage.engine: wiredTiger +storage.journal.enabled: true +systemLog.destination: file +systemLog.logAppend: true +systemLog.path: /var/log/mongodb/mongod.log +_EOF_ + + +cat > /etc/mongos.conf << '_EOF_' +security.authorization: enabled +systemLog.destination: file +systemLog.logAppend: true +systemLog.path: /var/log/mongodb/mongos.log +_EOF_ diff --git a/integration/scripts/files/elements/ubuntu-mongodb/install.d/41-mongod-init b/integration/scripts/files/elements/ubuntu-mongodb/install.d/41-mongod-init new file mode 100755 index 0000000000..77a1292723 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-mongodb/install.d/41-mongod-init @@ -0,0 +1,46 @@ +#!/bin/sh + +set -e +set -o xtrace + +cat > /etc/init/mongod.conf << '_EOF_' +limit fsize unlimited unlimited +limit cpu unlimited unlimited +limit as unlimited unlimited +limit nofile 64000 64000 +limit rss unlimited unlimited +limit nproc 64000 64000 + +kill timeout 300 # wait 300s between SIGTERM and SIGKILL. + +pre-start script + mkdir -p /var/run/mongodb/ + touch /var/run/mongodb/mongod.pid + chown mongodb -R /var/run/mongodb/ +end script + +start on runlevel [2345] +stop on runlevel [06] + +script + CONF=/etc/mongod.conf + + # Handle NUMA access to CPUs (SERVER-3574) + # This verifies the existence of numactl as well as testing that the command works + NUMACTL_ARGS="--interleave=all" + if which numactl >/dev/null 2>/dev/null && numactl $NUMACTL_ARGS ls / >/dev/null 2>/dev/null + then + NUMACTL="$(which numactl) -- $NUMACTL_ARGS" + DAEMON_OPTS=${DAEMON_OPTS:-"--config $CONF"} + else + NUMACTL="" + DAEMON_OPTS="-- "${DAEMON_OPTS:-"--config $CONF"} + fi + + exec start-stop-daemon --start \ + --chuid mongodb \ + --pidfile /var/run/mongod.pid \ + --make-pidfile \ + --exec $NUMACTL /usr/bin/mongod $DAEMON_OPTS +end script +_EOF_ diff --git a/integration/scripts/files/elements/ubuntu-mongodb/install.d/42-mongos-init b/integration/scripts/files/elements/ubuntu-mongodb/install.d/42-mongos-init new file mode 100755 index 0000000000..dc91fdbac8 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-mongodb/install.d/42-mongos-init @@ -0,0 +1,30 @@ +#!/bin/sh + +set -e +set -o xtrace + +cat > /etc/init/mongos.conf << '_EOF_' +limit fsize unlimited unlimited +limit cpu unlimited unlimited +limit as unlimited unlimited +limit nofile 64000 64000 +limit rss unlimited unlimited +limit nproc 64000 64000 + +pre-start script + mkdir -p /var/run/mongodb/ + touch /var/run/mongodb/mongos.pid + chown mongodb -R /var/run/mongodb/ +end script + +start on runlevel [2345] +stop on runlevel [06] + +script + exec start-stop-daemon --start \ + --chuid mongodb \ + --pidfile /var/run/mongos.pid \ + --make-pidfile \ + --exec /usr/bin/mongos -- --config /etc/mongos.conf +end script +_EOF_ diff --git a/integration/scripts/files/elements/ubuntu-mongodb/pre-install.d/10-mongodb-apt-key b/integration/scripts/files/elements/ubuntu-mongodb/pre-install.d/10-mongodb-apt-key new file mode 100755 index 0000000000..1538d61cb0 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-mongodb/pre-install.d/10-mongodb-apt-key @@ -0,0 +1,14 @@ +#!/bin/sh + +set -e +set -o xtrace + +[ -n "${RELEASE}" ] || die "RELEASE must be set to either Precise or Quantal" + +apt-get -y install software-properties-common + +apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927 + +echo "deb http://repo.mongodb.org/apt/ubuntu "$(lsb_release -sc)"/mongodb-org/3.2 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.2.list + +apt-get -y update diff --git a/integration/scripts/files/elements/ubuntu-mysql/README.md b/integration/scripts/files/elements/ubuntu-mysql/README.md new file mode 100644 index 0000000000..39a6ab8c66 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-mysql/README.md @@ -0,0 +1,3 @@ +Sets up a MySQL server install in the image. + +TODO: auto-tune settings based on host resources or metadata service. diff --git a/integration/scripts/files/elements/ubuntu-mysql/install.d/30-mysql b/integration/scripts/files/elements/ubuntu-mysql/install.d/30-mysql new file mode 100755 index 0000000000..d31292eca0 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-mysql/install.d/30-mysql @@ -0,0 +1,23 @@ +#!/bin/bash + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install controller base required packages + +set -e +set -o xtrace + +export DEBIAN_FRONTEND=noninteractive + +# The fix to make versions of percona-xtrabackup > v2.2 work with Trove +# was put into the mysql guestagent code for Mitaka. There are no current +# plans to backport so we need to make sure the guest generated when the +# tests are run for Kilo or Liberty get the 2.2 verson of PXB +if [[ $BRANCH_OVERRIDE == "stable/kilo" || $BRANCH_OVERRIDE == "stable/liberty" ]]; then + PXB_VERSION_OVERRIDE="-22" +fi +apt-get -y install libmysqlclient18 mysql-server-5.6 percona-xtrabackup${PXB_VERSION_OVERRIDE} + +cat >/etc/mysql/conf.d/no_perf_schema.cnf <<_EOF_ +[mysqld] +performance_schema = off +_EOF_ diff --git a/integration/scripts/files/elements/ubuntu-mysql/pre-install.d/10-percona-apt-key b/integration/scripts/files/elements/ubuntu-mysql/pre-install.d/10-percona-apt-key new file mode 100755 index 0000000000..2a03ad5054 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-mysql/pre-install.d/10-percona-apt-key @@ -0,0 +1,40 @@ +#!/bin/sh + +# CONTEXT: GUEST during PRE-CONSTRUCTION as ROOT +# PURPOSE: Setup apt-repo list so that we can connect to Percona's repo + +set -e +set -o xtrace + +[ -n "${GUEST_USERNAME}" ] || die "GUEST_USERNAME needs to be set to the user for the guest image" +[ -n "${RELEASE}" ] || die "RELEASE must be set to either Precise or Quantal" + +# Add Percona GPG key +mkdir -p /home/${GUEST_USERNAME}/.gnupg + +# sometimes the primary key server is unavailable and we should try an +# alternate. see +# https://bugs.launchpad.net/percona-server/+bug/907789. Disable +# shell errexit so we can interrogate the exit code and take action +# based on the exit code. We will reenable it later. +set +e +apt-key adv --keyserver hkp://keys.gnupg.net --recv-keys 1C4CBDCDCD2EFD2A + +if [ "$?" -ne "0" ]; +then + echo "Trying alternate keyserver hkp://keyserver.ubuntu.com" + set -e + apt-key adv --keyserver hkp://keyserver.ubuntu.com --recv-keys 1C4CBDCDCD2EFD2A +fi + +set -e + +# Add Percona repo +# Creates the percona sources list +cat < /etc/apt/sources.list.d/percona.list +deb http://repo.percona.com/apt $RELEASE main +deb-src http://repo.percona.com/apt $RELEASE main +EOL + +# Force an update +apt-get update diff --git a/integration/scripts/files/elements/ubuntu-mysql/pre-install.d/20-apparmor-mysql-local b/integration/scripts/files/elements/ubuntu-mysql/pre-install.d/20-apparmor-mysql-local new file mode 100755 index 0000000000..a3e1dc7c7d --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-mysql/pre-install.d/20-apparmor-mysql-local @@ -0,0 +1,11 @@ +#!/bin/sh + +set -e + +#CONTEXT: chroot on host +#PURPOSE: Allows mysqld to create temporary files when restoring backups + +cat <>/etc/apparmor.d/local/usr.sbin.mysqld + /tmp/ rw, + /tmp/** rwk, +EOF diff --git a/integration/scripts/files/elements/ubuntu-percona/install.d/30-mysql b/integration/scripts/files/elements/ubuntu-percona/install.d/30-mysql new file mode 100755 index 0000000000..5d5b4265a6 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-percona/install.d/30-mysql @@ -0,0 +1,17 @@ +#!/bin/sh + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install controller base required packages + +set -e +set -o xtrace + +export DEBIAN_FRONTEND=noninteractive +# The fix to make versions of percona-xtrabackup > v2.2 work with Trove +# was put into the mysql guestagent code for Mitaka. There are no current +# plans to backport so we need to make sure the guest generated when the +# tests are run for Kilo or Liberty get the 2.2 verson of PXB +if [[ $BRANCH_OVERRIDE == "stable/kilo" || $BRANCH_OVERRIDE == "stable/liberty" ]]; then + PXB_VERSION_OVERRIDE="-22" +fi +apt-get -y install percona-toolkit percona-server-common-5.6 percona-server-server-5.6 percona-server-test-5.6 percona-server-client-5.6 percona-xtrabackup${PXB_VERSION_OVERRIDE} diff --git a/integration/scripts/files/elements/ubuntu-percona/pre-install.d/10-percona-apt-key b/integration/scripts/files/elements/ubuntu-percona/pre-install.d/10-percona-apt-key new file mode 100755 index 0000000000..c2b686c4c0 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-percona/pre-install.d/10-percona-apt-key @@ -0,0 +1,42 @@ +#!/bin/sh + +# CONTEXT: GUEST during PRE-CONSTRUCTION as ROOT +# PURPOSE: Setup apt-repo list so that we can connect to Percona's repo + +set -e +set -o xtrace + +[ -n "${GUEST_USERNAME}" ] || die "GUEST_USERNAME needs to be set to the user for the guest image" +[ -n "${RELEASE}" ] || die "RELEASE must be set to either Precise or Quantal" + +#5 add Percona GPG key +if [ ! -e /home/${GUEST_USERNAME}/.gnupg ]; then + mkdir -p /home/${GUEST_USERNAME}/.gnupg +fi + +# sometimes the primary key server is unavailable and we should try an +# alternate. see +# https://bugs.launchpad.net/percona-server/+bug/907789. Disable +# shell errexit so we can interrogate the exit code and take action +# based on the exit code. We will reenable it later. +set +e +apt-key adv --keyserver hkp://keys.gnupg.net --recv-keys 1C4CBDCDCD2EFD2A + +if [ "$?" -ne "0" ]; +then + echo "Trying alternate keyserver hkp://keyserver.ubuntu.com" + set -e + apt-key adv --keyserver hkp://keyserver.ubuntu.com --recv-keys 1C4CBDCDCD2EFD2A +fi + +set -e + +# add Percona repo +# creates the percona sources list +cat < /etc/apt/sources.list.d/percona.list +deb http://repo.percona.com/apt $RELEASE main +deb-src http://repo.percona.com/apt $RELEASE main +EOL + +# force an update +apt-get update diff --git a/integration/scripts/files/elements/ubuntu-percona/pre-install.d/20-apparmor-mysql-local b/integration/scripts/files/elements/ubuntu-percona/pre-install.d/20-apparmor-mysql-local new file mode 100755 index 0000000000..a3e1dc7c7d --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-percona/pre-install.d/20-apparmor-mysql-local @@ -0,0 +1,11 @@ +#!/bin/sh + +set -e + +#CONTEXT: chroot on host +#PURPOSE: Allows mysqld to create temporary files when restoring backups + +cat <>/etc/apparmor.d/local/usr.sbin.mysqld + /tmp/ rw, + /tmp/** rwk, +EOF diff --git a/integration/scripts/files/elements/ubuntu-postgresql/install.d/10-postgresql b/integration/scripts/files/elements/ubuntu-postgresql/install.d/10-postgresql new file mode 100755 index 0000000000..cd85325667 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-postgresql/install.d/10-postgresql @@ -0,0 +1,79 @@ +#!/bin/sh + +set -e +set -o xtrace + +export DEBIAN_FRONTEND=noninteractive + +cat > "/etc/sysctl.d/10-postgresql-performance.conf" << _EOF_ +# See 'http://www.postgresql.org/docs/9.4/static/kernel-resources.html' +# for best practices. +# It is recommended to disable memory overcommit, +# but the Python interpreter may require it on smaller flavors. +# We therefore stick with the heuristic overcommit setting. +vm.overcommit_memory=0 +vm.nr_hugepages=64 + +_EOF_ + +cat > "/etc/rc.local" << _EOF_ +# See 'http://www.postgresql.org/docs/9.4/static/kernel-resources.html' +# Postgres 9.4 added support for THP. Using huge pages reduces overhead when +# using large contiguous chunks of memory, like PostgreSQL does. +if test -f /sys/kernel/mm/transparent_hugepage/defrag; then + echo never > /sys/kernel/mm/transparent_hugepage/defrag +fi +if test -f /sys/kernel/mm/transparent_hugepage/enabled; then + echo always > /sys/kernel/mm/transparent_hugepage/enabled +fi + +exit \$? + +_EOF_ + +apt-get -y install postgresql-9.4 postgresql-contrib-9.4 postgresql-server-dev-9.4 + +########################################### +# Hack alert: +# For Postgresql 9.4, pg_rewind is not in the main source tree and +# no packages exist in the repos, so it must be compiled manually +# and installed on the image until we can move to 9.5 +# See README at +# https://github.com/vmware/pg_rewind/tree/REL9_4_STABLE + +tmpdir=/tmp/build +mkdir -p $tmpdir +cd $tmpdir +git clone https://github.com/postgres/postgres.git --branch REL9_4_STABLE +cd postgres/contrib +git clone https://github.com/vmware/pg_rewind.git --branch REL9_4_STABLE + +dev_pkgs="libreadline-dev libkrb5-dev libssl-dev libpam-dev libxml2-dev libxslt-dev libedit-dev libselinux1-dev bison flex" + +apt-get install $dev_pkgs -y + +# Unfortunately, on ubuntu, was not able to get pg_rewind to build +# outside of the pgsql source tree. Configure and compile postgres +# but only call make install against the contrib/pg_rewind directory +# so that support library is accessible to the server +cd $tmpdir/postgres +./configure +make +cd contrib/pg_rewind +make install + +# Make the pg_rewind binary and the library used by the +# pg_rewind stored procedures accessible +ln -s /usr/local/pgsql/bin/pg_rewind /usr/bin/pg_rewind +ln -s /usr/local/pgsql/lib/pg_rewind_support.so /usr/lib/postgresql/9.4/lib/pg_rewind_support.so + +cd +rm -rf $tmpdir +apt-get remove -y $dev_pkgs + +# End hack +################################ + +# Install the native Python client. +apt-get -y install libpq-dev +pip install psycopg2 diff --git a/integration/scripts/files/elements/ubuntu-postgresql/pre-install.d/10-postgresql-repo b/integration/scripts/files/elements/ubuntu-postgresql/pre-install.d/10-postgresql-repo new file mode 100755 index 0000000000..48a25d1d37 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-postgresql/pre-install.d/10-postgresql-repo @@ -0,0 +1,12 @@ +#!/bin/sh + +set -e +set -o xtrace + +cat < /etc/apt/sources.list.d/postgresql.list +deb http://apt.postgresql.org/pub/repos/apt/ trusty-pgdg main +EOL + +wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - + +apt-get update diff --git a/integration/scripts/files/elements/ubuntu-pxc/install.d/30-mysql b/integration/scripts/files/elements/ubuntu-pxc/install.d/30-mysql new file mode 100755 index 0000000000..ae658957e2 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-pxc/install.d/30-mysql @@ -0,0 +1,14 @@ +#!/bin/sh + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install controller base required packages + +set -e +set -o xtrace + +export DEBIAN_FRONTEND=noninteractive +apt-get -y install percona-xtradb-cluster-server-5.6 percona-xtradb-cluster-client-5.6 percona-xtrabackup + +# Don't auto start mysql (we'll start it up in guest) +update-rc.d mysql defaults +update-rc.d mysql disable diff --git a/integration/scripts/files/elements/ubuntu-pxc/pre-install.d/10-percona-apt-key b/integration/scripts/files/elements/ubuntu-pxc/pre-install.d/10-percona-apt-key new file mode 100755 index 0000000000..c2b686c4c0 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-pxc/pre-install.d/10-percona-apt-key @@ -0,0 +1,42 @@ +#!/bin/sh + +# CONTEXT: GUEST during PRE-CONSTRUCTION as ROOT +# PURPOSE: Setup apt-repo list so that we can connect to Percona's repo + +set -e +set -o xtrace + +[ -n "${GUEST_USERNAME}" ] || die "GUEST_USERNAME needs to be set to the user for the guest image" +[ -n "${RELEASE}" ] || die "RELEASE must be set to either Precise or Quantal" + +#5 add Percona GPG key +if [ ! -e /home/${GUEST_USERNAME}/.gnupg ]; then + mkdir -p /home/${GUEST_USERNAME}/.gnupg +fi + +# sometimes the primary key server is unavailable and we should try an +# alternate. see +# https://bugs.launchpad.net/percona-server/+bug/907789. Disable +# shell errexit so we can interrogate the exit code and take action +# based on the exit code. We will reenable it later. +set +e +apt-key adv --keyserver hkp://keys.gnupg.net --recv-keys 1C4CBDCDCD2EFD2A + +if [ "$?" -ne "0" ]; +then + echo "Trying alternate keyserver hkp://keyserver.ubuntu.com" + set -e + apt-key adv --keyserver hkp://keyserver.ubuntu.com --recv-keys 1C4CBDCDCD2EFD2A +fi + +set -e + +# add Percona repo +# creates the percona sources list +cat < /etc/apt/sources.list.d/percona.list +deb http://repo.percona.com/apt $RELEASE main +deb-src http://repo.percona.com/apt $RELEASE main +EOL + +# force an update +apt-get update diff --git a/integration/scripts/files/elements/ubuntu-pxc/pre-install.d/20-apparmor-mysql-local b/integration/scripts/files/elements/ubuntu-pxc/pre-install.d/20-apparmor-mysql-local new file mode 100755 index 0000000000..a3e1dc7c7d --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-pxc/pre-install.d/20-apparmor-mysql-local @@ -0,0 +1,11 @@ +#!/bin/sh + +set -e + +#CONTEXT: chroot on host +#PURPOSE: Allows mysqld to create temporary files when restoring backups + +cat <>/etc/apparmor.d/local/usr.sbin.mysqld + /tmp/ rw, + /tmp/** rwk, +EOF diff --git a/integration/scripts/files/elements/ubuntu-redis/README.md b/integration/scripts/files/elements/ubuntu-redis/README.md new file mode 100644 index 0000000000..426072cfb7 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-redis/README.md @@ -0,0 +1 @@ +Sets up a redis server install in the image. diff --git a/integration/scripts/files/elements/ubuntu-redis/install.d/10-redis b/integration/scripts/files/elements/ubuntu-redis/install.d/10-redis new file mode 100755 index 0000000000..57fcdf9035 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-redis/install.d/10-redis @@ -0,0 +1,53 @@ +#!/bin/sh + +# CONTEXT: GUEST during CONSTRUCTION as ROOT +# PURPOSE: Install controller base required packages + +set -ex + +export DEBIAN_FRONTEND=noninteractive + +cat > "/etc/sysctl.d/10-redis-performance.conf" << _EOF_ +# See 'http://redis.io/topics/admin' for best practices. +# Make sure to set the Linux kernel overcommit memory setting to 1. +vm.overcommit_memory=1 + +# Linux kernel will silently truncate 'tcp-backlog' to the value of +# '/proc/sys/net/core/somaxconn' so make sure to raise both the value of +# 'somaxconn' and 'tcp_max_syn_backlog' in order to get the desired effect. +net.ipv4.tcp_max_syn_backlog=1024 +net.core.somaxconn=1024 + +_EOF_ + +cat > "/etc/rc.local" << _EOF_ +# Make sure to disable Linux kernel feature transparent huge pages, +# it will affect greatly both memory usage and latency in a negative way. +if test -f /sys/kernel/mm/transparent_hugepage/defrag; then + echo never > /sys/kernel/mm/transparent_hugepage/defrag +fi +if test -f /sys/kernel/mm/transparent_hugepage/enabled; then + echo never > /sys/kernel/mm/transparent_hugepage/enabled +fi + +_EOF_ + +add-apt-repository -y ppa:chris-lea/redis-server +apt-get -y update +apt-get install -y redis-server + +cat > "/etc/default/redis-server" << _EOF_ +# Call ulimit -n with this argument prior to invoking Redis itself. +# This may be required for high-concurrency environments. Redis itself cannot +# alter its limits as it is not being run as root. +ULIMIT=65536 + +_EOF_ + +# Install Python driver for Redis ('redis-py'). +pip install redis + +# By default, redis-py will attempt to use the HiredisParser if installed. +# Using Hiredis can provide up to a 10x speed improvement in parsing responses +# from the Redis server. +pip install hiredis diff --git a/integration/scripts/files/elements/ubuntu-vertica/README.md b/integration/scripts/files/elements/ubuntu-vertica/README.md new file mode 100644 index 0000000000..8620237967 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-vertica/README.md @@ -0,0 +1 @@ +Sets up a Vertica CE 7.1 debian package and other dependencies install in the image. diff --git a/integration/scripts/files/elements/ubuntu-vertica/extra-data.d/93-copy-vertica-deb b/integration/scripts/files/elements/ubuntu-vertica/extra-data.d/93-copy-vertica-deb new file mode 100755 index 0000000000..56c02cda9e --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-vertica/extra-data.d/93-copy-vertica-deb @@ -0,0 +1,14 @@ +#!/bin/bash + +# CONTEXT: HOST prior to IMAGE BUILD as SCRIPT USER +# PURPOSE: Stages the package installer file from DATASTORE_PKG_LOCATION, +# so that guest image has the package file. + +set -e +set -o xtrace + +source $_LIB/die + +[ -n "$TMP_HOOKS_PATH" ] || die "Temp hook path not set" +[ -f "$DATASTORE_PKG_LOCATION" ] || die "Datastore package installer file not found at:" $DATASTORE_PKG_LOCATION +dd if=${DATASTORE_PKG_LOCATION} of=${TMP_HOOKS_PATH}/vertica.deb diff --git a/integration/scripts/files/elements/ubuntu-vertica/install.d/97-vertica b/integration/scripts/files/elements/ubuntu-vertica/install.d/97-vertica new file mode 100755 index 0000000000..2d50bc6a78 --- /dev/null +++ b/integration/scripts/files/elements/ubuntu-vertica/install.d/97-vertica @@ -0,0 +1,54 @@ +#!/bin/sh + +# CONTEXT: GUEST during CONSTRUCTION as ROOT - install.d +# PURPOSE: Install controller base required packages + +set -e +set -o xtrace +export DEBIAN_FRONTEND=noninteractive + + +# Copy the package file to the image, +# as it needs to be used later during configuration. +dd if=/tmp/in_target.d/vertica.deb of=/vertica.deb + +# Install base packages +apt-get install -qy build-essential bc iptables +apt-get install -qy curl sysstat pstack mcelog +apt-get install -qy python-dev g++ unixODBC unixODBC-dev dialog +apt-get install -qy dialog libbz2-dev libboost-all-dev libcurl4-gnutls-dev +apt-get install -qy openjdk-7-jdk + +# Install Vertica package +dpkg -i /vertica.deb + +# Creating dbadmin user and verticadba group +groupadd verticadba +useradd -g verticadba -d /home/dbadmin -s /bin/bash -m dbadmin +echo "export PATH=/opt/vertica/bin:\$PATH" >> ~dbadmin/.profile +echo "export TZ=`date +%Z`" >> ~dbadmin/.profile + +# Create base directory for to be used for database creation +mkdir /var/lib/vertica +chown dbadmin:verticadba /var/lib/vertica + +# Backup /etc/hosts +cp -p /etc/hosts /etc/hosts.bkp + +# Compile the SDK examples - the supplied UDFs can then be loaded +cd /opt/vertica/sdk/examples +TMPDIR=/tmp JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 make +cd + +cat > "/etc/rc.local" << _EOF_ +# Vertica requires THP to be turned off +if test -f /sys/kernel/mm/transparent_hugepage/defrag; then + echo never > /sys/kernel/mm/transparent_hugepage/defrag +fi +if test -f /sys/kernel/mm/transparent_hugepage/enabled; then + echo never > /sys/kernel/mm/transparent_hugepage/enabled +fi + +exit \$? + +_EOF_ diff --git a/integration/scripts/files/keys/authorized_keys b/integration/scripts/files/keys/authorized_keys new file mode 100644 index 0000000000..4093a7145c --- /dev/null +++ b/integration/scripts/files/keys/authorized_keys @@ -0,0 +1 @@ +ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmQeA/uyEyFf9DsmwR+OztWb7Hb/uTC+R3xG1QgBvRwhSbpBnyBESGMZZ07bIw5Ib7BUSDzwoeryUqNAhAhir2KLeIYODS39UmTwOIl+rIvhlTxhsIoQHV90pewD2qw0T8KgVMPUDsQ0Bd98E6e5dbxciZp67ihVD0r7srhdSRo8PIc56hJWrD52j5FeiIGEmLXHXiZLOyma1M7j/EmiV81wHAzgql6sihWSZHm3xPZZ712JtXbmHhe3RLFIK13u9PSb3XbuEIdGwkZdzP+vYNE0CsYqwjXjVRrY/APsiEkbSNVzHI5p2W1L7ZMtSOMUqZ1Ve+sytVb+YcIJ9L8y07 trove@devstack \ No newline at end of file diff --git a/integration/scripts/files/keys/id_rsa b/integration/scripts/files/keys/id_rsa new file mode 100644 index 0000000000..041e61fdfd --- /dev/null +++ b/integration/scripts/files/keys/id_rsa @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEpAIBAAKCAQEA5kHgP7shMhX/Q7JsEfjs7Vm+x2/7kwvkd8RtUIAb0cIUm6QZ +8gREhjGWdO2yMOSG+wVEg88KHq8lKjQIQIYq9ii3iGDg0t/VJk8DiJfqyL4ZU8Yb +CKEB1fdKXsA9qsNE/CoFTD1A7ENAXffBOnuXW8XImaeu4oVQ9K+7K4XUkaPDyHOe +oSVqw+do+RXoiBhJi1x14mSzspmtTO4/xJolfNcBwM4KperIoVkmR5t8T2We9dib +V25h4Xt0SxSCtd7vT0m9127hCHRsJGXcz/r2DRNArGKsI141Ua2PwD7IhJG0jVcx +yOadltS+2TLUjjFKmdVXvrMrVW/mHCCfS/MtOwIDAQABAoIBAQCTAQHbjmwuHu8e +2F24x7Tz//UWPT9fbRtK/6RO3ctlCsS/bXCHHARnrGcDdfHq1yv6PS21/UvXtThE +Dn4qO75X9DzgnAFNgEwELjPyVBM4YG2pF3SQ+MJESaI4hgGY8Rws5eMF/qFhdbo1 +hATggqFqnQZqWy0DP9wkq8ESk1nYNICehj0d6Mo5uW190TDMD6QrfHg4rDYbgITf +SCmsRdybCASlgOHCrYgjweG9czNoFimhaG8WwP59yfCX0A1TrDJ0toryyopupIre +A+5HHBM2Dk3KylrtVBAPRsM9eGUo4bmz8p6hRkuw7mr321d416MHdIXcHK38EWR0 +ZvUWM8QJAoGBAPlPe3ggR9xhEbQJQkOohnZ1hkogE/uxZrcRiUI3pnDNcCXy9Ogt +SYfwxYnHLa5kuynbmCCzSLOtq2DN9QwJ5o+zgRjX9T8DAWub6KFdEnTHhq/ZLF/w +PWPg3Oe8dYDEEcrPLvN25aetY2LrWKsRPFL8//WLJc1+LFRG6Vc1ATftAoGBAOxv +hGxNGrcD5c8g0ZcyeKVbCCRGfp0+mwonnQ25mDyIXNH+PEHa7SPs5dVexA5r0/Ky +lM/jQgs756EvslwA2oukqVz7ehDhJI4RE92OPjpYrAu7HF6eN/fAhUMghs/vAZ6c +YCM6i9emHYHM4mU6H/yLIr+0e2JNf+479bB5hQTHAoGBAOGnFUQXQ7OukE16C+Yd +RQc3PIMfIbcwTJ3qW2f54sY6zAUtMIptYx6NyN35z9kHB3jNb5Y5b9ZhnLqT7/Yj +h/INMQ4BedK8r65sgVR8X1YfukKzuLxlP8uFHa0KIPiZftkoSYDH3vmzsD86cRj1 +ErqykCH4/hBO4WSugkkSirXJAoGARXvQcvOF8lsW4nRGpCSVCCNklSSSeSu47JcP +tMTiVIfOn3gTxVbNck1gjgA3pfVSaHTK/v1On3aPb/NQe3FUyM0vaMAO8372+zlR +mT8AUq1Ugm4OvE/LKuhNQZkBhYI7+50BM9k0179d1JOdxRn75IAPSj+EMzOLcTv1 +zFMqIGkCgYBm5xT3Gu8fJh/8ztelzrDkGga6UpYKKYjHGFHpaqmDn9sjjCu8X2pG +JUGgyUVj1NkJAtHMS45Ud3upQwxpy2aNmMaQbwzHybvX7EYZHHVuCwsSzaXRtwj2 +Q6mG1Ghi0UQ76SPKQr0Vu8Uu+0CAzYAK4IEKeH6BCRjrzHggSpdNzQ== +-----END RSA PRIVATE KEY----- \ No newline at end of file diff --git a/integration/scripts/files/keys/id_rsa.pub b/integration/scripts/files/keys/id_rsa.pub new file mode 100644 index 0000000000..4093a7145c --- /dev/null +++ b/integration/scripts/files/keys/id_rsa.pub @@ -0,0 +1 @@ +ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmQeA/uyEyFf9DsmwR+OztWb7Hb/uTC+R3xG1QgBvRwhSbpBnyBESGMZZ07bIw5Ib7BUSDzwoeryUqNAhAhir2KLeIYODS39UmTwOIl+rIvhlTxhsIoQHV90pewD2qw0T8KgVMPUDsQ0Bd98E6e5dbxciZp67ihVD0r7srhdSRo8PIc56hJWrD52j5FeiIGEmLXHXiZLOyma1M7j/EmiV81wHAzgql6sihWSZHm3xPZZ712JtXbmHhe3RLFIK13u9PSb3XbuEIdGwkZdzP+vYNE0CsYqwjXjVRrY/APsiEkbSNVzHI5p2W1L7ZMtSOMUqZ1Ve+sytVb+YcIJ9L8y07 trove@devstack \ No newline at end of file diff --git a/integration/scripts/files/requirements/fedora-requirements-default.txt b/integration/scripts/files/requirements/fedora-requirements-default.txt new file mode 100644 index 0000000000..c976bfac52 --- /dev/null +++ b/integration/scripts/files/requirements/fedora-requirements-default.txt @@ -0,0 +1,31 @@ +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +unittest2 +testtools +extras +python-novaclient>=2.22.0 +python-swiftclient>=2.2.0 +python-cinderclient>=1.1.0 +python-keystoneclient>=2.0.0,!=2.1.0 # Apache-2.0 +kombu>=2.5.0 +babel>=1.3 +python-heatclient>=0.3.0 +passlib +jinja2>=2.6 +PyMySQL>=0.6.2 # MIT License +python-neutronclient>=2.3.11,<3 +netifaces>=0.10.4 +oslo.config>=1.9.3 # Apache-2.0 +oslo.messaging>=1.8.0 # Apache-2.0 +oslo.i18n>=1.5.0 # Apache-2.0 +oslo.serialization>=1.4.0 # Apache-2.0 +oslo.service>=0.1.0 # Apache-2.0 +oslo.utils>=1.4.0 # Apache-2.0 +oslo.log>=1.8.0 # Apache-2.0 +osprofiler>=0.3.0 +oslo.concurrency>=1.8.0 # Apache-2.0 +pexpect>=3.1,!=3.3 +enum34;python_version=='2.7' or python_version=='2.6' or python_version=='3.3' # BSD +pycrypto>=2.6 # Public Domain +xmltodict>=0.10.1 # MIT diff --git a/integration/scripts/files/requirements/fedora-requirements-juno.txt b/integration/scripts/files/requirements/fedora-requirements-juno.txt new file mode 100644 index 0000000000..a3ddea3a0f --- /dev/null +++ b/integration/scripts/files/requirements/fedora-requirements-juno.txt @@ -0,0 +1,19 @@ +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +unittest2 +testtools>=0.9.34,!=1.4.0,<=1.5.0 +extras +python-novaclient>=2.18.0,<=2.20.0 +python-swiftclient>=2.2.0,<=2.3.1 +python-cinderclient>=1.1.0,<=1.1.1 +kombu>=2.5.0,<=3.0.7 +six>=1.7.0,<=1.9.0 +Babel>=1.3,<=1.3 +python-heatclient>=0.2.9,<0.3.0 +passlib<=1.6.2 +Jinja2<=2.7.2 +python-neutronclient>=2.3.6,<2.4.0 +netifaces>=0.10.4,<=0.10.4 +oslo.config>=1.4.0,<=1.6.0 # Apache-2.0 +oslo.utils>=1.4.0,<1.5.0 # Apache-2.0 diff --git a/integration/scripts/files/requirements/fedora-requirements-kilo.txt b/integration/scripts/files/requirements/fedora-requirements-kilo.txt new file mode 100644 index 0000000000..48bda11ba9 --- /dev/null +++ b/integration/scripts/files/requirements/fedora-requirements-kilo.txt @@ -0,0 +1,24 @@ +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +unittest2 +testtools>=0.9.36,!=1.2.0 +extras +python-novaclient>=2.22.0,<2.24.0 +python-swiftclient>=2.2.0,<2.5.0 +python-cinderclient>=1.1.0,<1.2.0 +kombu>=2.5.0 +six>=1.9.0 +Babel>=1.3 +python-heatclient>=0.3.0,<0.5.0 +passlib +Jinja2>=2.6 # BSD License (3 clause) +python-neutronclient>=2.3.11,<2.5.0 +netifaces>=0.10.4 +oslo.config>=1.9.3,<1.10.0 # Apache-2.0 +oslo.i18n>=1.5.0,<1.6.0 # Apache-2.0 +oslo.serialization>=1.4.0,<1.5.0 # Apache-2.0 +oslo.utils>=1.4.0,<1.5.0 # Apache-2.0 +oslo.concurrency>=1.8.0,<1.9.0 # Apache-2.0 +oslo.messaging>=1.8.0,<1.9.0 # Apache-2.0 +osprofiler>=0.3.0 # Apache-2.0 diff --git a/integration/scripts/files/requirements/fedora-requirements-liberty.txt b/integration/scripts/files/requirements/fedora-requirements-liberty.txt new file mode 100644 index 0000000000..f47cbadab5 --- /dev/null +++ b/integration/scripts/files/requirements/fedora-requirements-liberty.txt @@ -0,0 +1,27 @@ +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +unittest2 +testtools +extras +python-novaclient>=2.22.0 +python-swiftclient>=2.2.0 +python-cinderclient>=1.1.0 +kombu>=2.5.0 +babel>=1.3 +python-heatclient>=0.3.0 +passlib +jinja2>=2.6 +python-neutronclient>=2.3.11,<3 +netifaces>=0.10.4 +oslo.context>=0.2.0,<=1.0.0 +oslo.config>=1.9.3,<1.10.0 # Apache-2.0 +oslo.messaging>=1.8.0 # Apache-2.0 +oslo.i18n>=1.5.0,<1.6.0 # Apache-2.0 +oslo.serialization>=1.4.0,<1.5.0 # Apache-2.0 +oslo.service>=0.1.0 # Apache-2.0 +oslo.utils>=1.4.0,<1.5.0 # Apache-2.0 +oslo.log>=1.8.0 # Apache-2.0 +osprofiler>=0.3.0 +oslo.concurrency>=1.8.0,<1.9.0 # Apache-2.0 +pexpect>=3.1,!=3.3 diff --git a/integration/scripts/files/requirements/fedora-requirements-mitaka.txt b/integration/scripts/files/requirements/fedora-requirements-mitaka.txt new file mode 100644 index 0000000000..6f8c478cd6 --- /dev/null +++ b/integration/scripts/files/requirements/fedora-requirements-mitaka.txt @@ -0,0 +1,28 @@ +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +unittest2 +testtools +extras +python-novaclient>=2.22.0 +python-swiftclient>=2.2.0 +python-cinderclient>=1.1.0 +kombu>=2.5.0 +babel>=1.3 +python-heatclient>=0.3.0 +passlib +jinja2>=2.6 +python-neutronclient>=2.3.11,<3 +netifaces>=0.10.4 +oslo.config>=1.9.3 # Apache-2.0 +oslo.messaging>=1.8.0 # Apache-2.0 +oslo.i18n>=1.5.0 # Apache-2.0 +oslo.serialization>=1.4.0 # Apache-2.0 +oslo.service>=0.1.0 # Apache-2.0 +oslo.utils>=1.4.0 # Apache-2.0 +oslo.log>=1.8.0 # Apache-2.0 +osprofiler>=0.3.0 +oslo.concurrency>=1.8.0 # Apache-2.0 +pexpect>=3.1,!=3.3 +enum34;python_version=='2.7' or python_version=='2.6' or python_version=='3.3' # BSD +pycrypto>=2.6 # Public Domain diff --git a/integration/scripts/files/requirements/fedora-requirements-newton.txt b/integration/scripts/files/requirements/fedora-requirements-newton.txt new file mode 100644 index 0000000000..c976bfac52 --- /dev/null +++ b/integration/scripts/files/requirements/fedora-requirements-newton.txt @@ -0,0 +1,31 @@ +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +unittest2 +testtools +extras +python-novaclient>=2.22.0 +python-swiftclient>=2.2.0 +python-cinderclient>=1.1.0 +python-keystoneclient>=2.0.0,!=2.1.0 # Apache-2.0 +kombu>=2.5.0 +babel>=1.3 +python-heatclient>=0.3.0 +passlib +jinja2>=2.6 +PyMySQL>=0.6.2 # MIT License +python-neutronclient>=2.3.11,<3 +netifaces>=0.10.4 +oslo.config>=1.9.3 # Apache-2.0 +oslo.messaging>=1.8.0 # Apache-2.0 +oslo.i18n>=1.5.0 # Apache-2.0 +oslo.serialization>=1.4.0 # Apache-2.0 +oslo.service>=0.1.0 # Apache-2.0 +oslo.utils>=1.4.0 # Apache-2.0 +oslo.log>=1.8.0 # Apache-2.0 +osprofiler>=0.3.0 +oslo.concurrency>=1.8.0 # Apache-2.0 +pexpect>=3.1,!=3.3 +enum34;python_version=='2.7' or python_version=='2.6' or python_version=='3.3' # BSD +pycrypto>=2.6 # Public Domain +xmltodict>=0.10.1 # MIT diff --git a/integration/scripts/files/requirements/ubuntu-requirements-default.txt b/integration/scripts/files/requirements/ubuntu-requirements-default.txt new file mode 100644 index 0000000000..9607060b4b --- /dev/null +++ b/integration/scripts/files/requirements/ubuntu-requirements-default.txt @@ -0,0 +1,30 @@ +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +unittest2 +testtools +extras +python-novaclient>=2.18.0 +python-swiftclient>=2.2.0 +python-cinderclient>=1.1.0 +python-keystoneclient>=2.0.0,!=2.1.0 # Apache-2.0 +kombu>=2.5.0 +six>=1.7.0 +babel +python-heatclient>=0.2.9 +passlib +jinja2 +PyMySQL>=0.6.2 # MIT License +python-neutronclient>=2.3.6,<3 +netifaces>=0.10.4 +oslo.config>=1.4.0 # Apache-2.0 +oslo.messaging>=1.4.0,!=1.5.0 +oslo.i18n>=1.0.0 +oslo.serialization>=1.0.0 +oslo.service>=0.1.0 # Apache-2.0 +oslo.utils>=1.1.0 +osprofiler>=0.3.0 +oslo.concurrency>=0.3.0 +enum34;python_version=='2.7' or python_version=='2.6' or python_version=='3.3' # BSD +pycrypto>=2.6 # Public Domain +xmltodict>=0.10.1 # MIT diff --git a/integration/scripts/files/requirements/ubuntu-requirements-juno.txt b/integration/scripts/files/requirements/ubuntu-requirements-juno.txt new file mode 100644 index 0000000000..a3ddea3a0f --- /dev/null +++ b/integration/scripts/files/requirements/ubuntu-requirements-juno.txt @@ -0,0 +1,19 @@ +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +unittest2 +testtools>=0.9.34,!=1.4.0,<=1.5.0 +extras +python-novaclient>=2.18.0,<=2.20.0 +python-swiftclient>=2.2.0,<=2.3.1 +python-cinderclient>=1.1.0,<=1.1.1 +kombu>=2.5.0,<=3.0.7 +six>=1.7.0,<=1.9.0 +Babel>=1.3,<=1.3 +python-heatclient>=0.2.9,<0.3.0 +passlib<=1.6.2 +Jinja2<=2.7.2 +python-neutronclient>=2.3.6,<2.4.0 +netifaces>=0.10.4,<=0.10.4 +oslo.config>=1.4.0,<=1.6.0 # Apache-2.0 +oslo.utils>=1.4.0,<1.5.0 # Apache-2.0 diff --git a/integration/scripts/files/requirements/ubuntu-requirements-kilo.txt b/integration/scripts/files/requirements/ubuntu-requirements-kilo.txt new file mode 100644 index 0000000000..48bda11ba9 --- /dev/null +++ b/integration/scripts/files/requirements/ubuntu-requirements-kilo.txt @@ -0,0 +1,24 @@ +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +unittest2 +testtools>=0.9.36,!=1.2.0 +extras +python-novaclient>=2.22.0,<2.24.0 +python-swiftclient>=2.2.0,<2.5.0 +python-cinderclient>=1.1.0,<1.2.0 +kombu>=2.5.0 +six>=1.9.0 +Babel>=1.3 +python-heatclient>=0.3.0,<0.5.0 +passlib +Jinja2>=2.6 # BSD License (3 clause) +python-neutronclient>=2.3.11,<2.5.0 +netifaces>=0.10.4 +oslo.config>=1.9.3,<1.10.0 # Apache-2.0 +oslo.i18n>=1.5.0,<1.6.0 # Apache-2.0 +oslo.serialization>=1.4.0,<1.5.0 # Apache-2.0 +oslo.utils>=1.4.0,<1.5.0 # Apache-2.0 +oslo.concurrency>=1.8.0,<1.9.0 # Apache-2.0 +oslo.messaging>=1.8.0,<1.9.0 # Apache-2.0 +osprofiler>=0.3.0 # Apache-2.0 diff --git a/integration/scripts/files/requirements/ubuntu-requirements-liberty.txt b/integration/scripts/files/requirements/ubuntu-requirements-liberty.txt new file mode 100644 index 0000000000..1f3d0d7b61 --- /dev/null +++ b/integration/scripts/files/requirements/ubuntu-requirements-liberty.txt @@ -0,0 +1,26 @@ +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +unittest2 +testtools +extras +python-novaclient>=2.18.0 +python-swiftclient>=2.2.0 +python-cinderclient>=1.1.0 +kombu>=2.5.0 +six>=1.7.0 +babel +python-heatclient>=0.2.9 +passlib +jinja2 +python-neutronclient>=2.3.6,<3 +netifaces>=0.10.4 +oslo.context>=0.2.0,<=1.0.0 +oslo.config>=1.4.0 # Apache-2.0 +oslo.messaging>=1.4.0,!=1.5.0 +oslo.i18n>=1.0.0 +oslo.serialization>=1.0.0 +oslo.service>=0.1.0 # Apache-2.0 +oslo.utils>=1.1.0 +osprofiler>=0.3.0 +oslo.concurrency>=0.3.0 diff --git a/integration/scripts/files/requirements/ubuntu-requirements-mitaka.txt b/integration/scripts/files/requirements/ubuntu-requirements-mitaka.txt new file mode 100644 index 0000000000..8724d163fb --- /dev/null +++ b/integration/scripts/files/requirements/ubuntu-requirements-mitaka.txt @@ -0,0 +1,27 @@ +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +unittest2 +testtools +extras +python-novaclient>=2.18.0 +python-swiftclient>=2.2.0 +python-cinderclient>=1.1.0 +kombu>=2.5.0 +six>=1.7.0 +babel +python-heatclient>=0.2.9 +passlib +jinja2 +python-neutronclient>=2.3.6,<3 +netifaces>=0.10.4 +oslo.config>=1.4.0 # Apache-2.0 +oslo.messaging>=1.4.0,!=1.5.0 +oslo.i18n>=1.0.0 +oslo.serialization>=1.0.0 +oslo.service>=0.1.0 # Apache-2.0 +oslo.utils>=1.1.0 +osprofiler>=0.3.0 +oslo.concurrency>=0.3.0 +enum34;python_version=='2.7' or python_version=='2.6' or python_version=='3.3' # BSD +pycrypto>=2.6 # Public Domain diff --git a/integration/scripts/files/requirements/ubuntu-requirements-newton.txt b/integration/scripts/files/requirements/ubuntu-requirements-newton.txt new file mode 100644 index 0000000000..9607060b4b --- /dev/null +++ b/integration/scripts/files/requirements/ubuntu-requirements-newton.txt @@ -0,0 +1,30 @@ +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +unittest2 +testtools +extras +python-novaclient>=2.18.0 +python-swiftclient>=2.2.0 +python-cinderclient>=1.1.0 +python-keystoneclient>=2.0.0,!=2.1.0 # Apache-2.0 +kombu>=2.5.0 +six>=1.7.0 +babel +python-heatclient>=0.2.9 +passlib +jinja2 +PyMySQL>=0.6.2 # MIT License +python-neutronclient>=2.3.6,<3 +netifaces>=0.10.4 +oslo.config>=1.4.0 # Apache-2.0 +oslo.messaging>=1.4.0,!=1.5.0 +oslo.i18n>=1.0.0 +oslo.serialization>=1.0.0 +oslo.service>=0.1.0 # Apache-2.0 +oslo.utils>=1.1.0 +osprofiler>=0.3.0 +oslo.concurrency>=0.3.0 +enum34;python_version=='2.7' or python_version=='2.6' or python_version=='3.3' # BSD +pycrypto>=2.6 # Public Domain +xmltodict>=0.10.1 # MIT diff --git a/integration/scripts/files/trove-guest.systemd.conf b/integration/scripts/files/trove-guest.systemd.conf new file mode 100644 index 0000000000..47bc2cf82d --- /dev/null +++ b/integration/scripts/files/trove-guest.systemd.conf @@ -0,0 +1,32 @@ +[Unit] +Description=Trove Guest +After=syslog.target +After=network.target + +[Service] +Type=simple +User=GUEST_USERNAME +Group=GUEST_USERNAME + +ExecStartPre=/bin/bash -c "sudo mkdir -p GUEST_LOGDIR ; sudo chown GUEST_USERNAME:root GUEST_LOGDIR" + +# If ~/trove-installed does not exist, copy the trove source from +# the user's development environment, then touch the sentinel file +ExecStartPre=/bin/bash -c "test -d /home/GUEST_USERNAME/trove-installed || sudo -u GUEST_USERNAME rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -avz --exclude='.*' HOST_SCP_USERNAME@CONTROLLER_IP:PATH_TROVE/ /home/GUEST_USERNAME/trove && touch /home/GUEST_USERNAME/trove-installed" + +# If /etc/trove does not exist, create it and then copy the trove-guestagent.conf +# from /etc/trove on the user's development environment, +ExecStartPre=/bin/bash -c "test -d /etc/trove/conf.d || sudo mkdir -p /etc/trove/conf.d && sudo -u GUEST_USERNAME rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -avz --exclude='.*' HOST_SCP_USERNAME@CONTROLLER_IP:/etc/trove/trove-guestagent.conf ~GUEST_USERNAME/ && sudo mv ~GUEST_USERNAME/trove-guestagent.conf /etc/trove/conf.d/trove-guestagent.conf" + +ExecStartPre=/bin/bash -c "sudo chown -R GUEST_USERNAME:root /etc/trove" + +ExecStart=/home/GUEST_USERNAME/trove/contrib/trove-guestagent --config-dir=/etc/trove/conf.d + +# Give a reasonable amount of time for the server to start up/shut down +TimeoutSec=300 + +# PgSql doesn't play nice with PrivateTmp +PrivateTmp=false + +[Install] +WantedBy=multi-user.target diff --git a/integration/scripts/files/trove-guest.upstart.conf b/integration/scripts/files/trove-guest.upstart.conf new file mode 100644 index 0000000000..2e20d6cfb9 --- /dev/null +++ b/integration/scripts/files/trove-guest.upstart.conf @@ -0,0 +1,40 @@ +description "Trove Guest" +author "Auto-Gen" + +start on (filesystem and net-device-up IFACE!=lo) +stop on runlevel [016] +chdir /var/run +pre-start script + mkdir -p /var/run/trove + chown GUEST_USERNAME:root /var/run/trove/ + + mkdir -p /var/lock/trove + chown GUEST_USERNAME:root /var/lock/trove/ + + mkdir -p GUEST_LOGDIR + chown GUEST_USERNAME:root GUEST_LOGDIR + + # Copy the trove source from the user's development environment + if [ ! -d /home/GUEST_USERNAME/trove ]; then + sudo -u GUEST_USERNAME rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -avz --exclude='.*' HOST_SCP_USERNAME@CONTROLLER_IP:PATH_TROVE/ /home/GUEST_USERNAME/trove + fi + + # Ensure conf dir exists and is readable + mkdir -p /etc/trove/conf.d + chmod -R +r /etc/trove +end script + +script + # For backwards compatibility until https://review.openstack.org/#/c/100381 merges + TROVE_CONFIG="--config-dir=/etc/trove/conf.d" + if [ ! -f /etc/trove/conf.d/guest_info ] && [ ! -f /etc/trove/conf.d/trove-guestagent.conf ]; then + + chmod +r /etc/guest_info + sudo -u GUEST_USERNAME rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -avz --exclude='.*' HOST_SCP_USERNAME@CONTROLLER_IP:/etc/trove/trove-guestagent.conf ~GUEST_USERNAME/ + mv ~GUEST_USERNAME/trove-guestagent.conf /etc/trove/trove-guestagent.conf + TROVE_CONFIG="--config-file=/etc/guest_info --config-file=/etc/trove/trove-guestagent.conf" + + fi + + exec su -c "/home/GUEST_USERNAME/trove/contrib/trove-guestagent $TROVE_CONFIG" GUEST_USERNAME +end script diff --git a/integration/scripts/functions b/integration/scripts/functions new file mode 100644 index 0000000000..1a09f55201 --- /dev/null +++ b/integration/scripts/functions @@ -0,0 +1,324 @@ +#!/bin/bash +# This file format was stolen from devstack <3 + +# This method was stolen from devstack +# git clone only if directory doesn't exist already. Since ``DEST`` might not +# be owned by the installation user, we create the directory and change the +# ownership to the proper user. +# Set global RECLONE=yes to simulate a clone when dest-dir exists +# git_clone remote dest-dir branch +function git_clone { + [[ "$OFFLINE" = "True" ]] && return + + GIT_REMOTE=$1 + GIT_DEST=$2 + GIT_BRANCH=$3 + + if echo $GIT_BRANCH | egrep -q "^refs"; then + # If our branch name is a gerrit style refs/changes/... + if [[ ! -d $GIT_DEST ]]; then + git_timed clone $GIT_REMOTE $GIT_DEST + fi + cd $GIT_DEST + git_timed fetch $GIT_REMOTE $GIT_BRANCH && git_timed checkout FETCH_HEAD + else + # do a full clone only if the directory doesn't exist + if [[ ! -d $GIT_DEST ]]; then + git_timed clone $GIT_REMOTE $GIT_DEST + cd $GIT_DEST + # This checkout syntax works for both branches and tags + git_timed checkout $GIT_BRANCH + elif [[ "$RECLONE" == "yes" ]]; then + # if it does exist then simulate what clone does if asked to RECLONE + cd $GIT_DEST + # set the url to pull from and fetch + git_timed remote set-url origin $GIT_REMOTE + git_timed fetch origin + # remove the existing ignored files (like pyc) as they cause breakage + # (due to the py files having older timestamps than our pyc, so python + # thinks the pyc files are correct using them) + find $GIT_DEST -name '*.pyc' -delete + git_timed checkout -f origin/$GIT_BRANCH + # a local branch might not exist + git_timed branch -D $GIT_BRANCH || true + git_timed checkout -b $GIT_BRANCH + fi + fi +} + +# Determinate is the given option present in the INI file +# ini_has_option config-file section option +function ini_has_option() { + local file=$1 + local section=$2 + local option=$3 + local line + line=$(sed -ne "/^\[$section\]/,/^\[.*\]/ { /^$option[ \t]*=/ p; }" "$file") + [ -n "$line" ] +} + +# Get an option from an INI file +# iniget config-file section option +function iniget() { + local file=$1 + local section=$2 + local option=$3 + local line + line=$(sed -ne "/^\[$section\]/,/^\[.*\]/ { /^$option[ \t]*=/ p; }" "$file") + echo ${line#*=} +} + +# Set an option in an INI file +# iniset config-file section option value +function iniset() { + local file=$1 + local section=$2 + local option=$3 + local value=$4 + if ! grep -q "^\[$section\]" "$file"; then + # Add section at the end + echo -e "\n[$section]" >>"$file" + fi + if ! ini_has_option "$file" "$section" "$option"; then + # Add it + sed -i -e "/^\[$section\]/ a\\ +$option = $value +" "$file" + else + # Replace it + sed -i -e "/^\[$section\]/,/^\[.*\]/ s|^\($option[ \t]*=[ \t]*\).*$|\1$value|" "$file" + fi +} + +# Determine OS Vendor, Release and Update +# Tested with OS/X, Ubuntu, RedHat, CentOS, Fedora +# Returns results in global variables: +# os_VENDOR - vendor name +# os_RELEASE - release +# os_UPDATE - update +# os_PACKAGE - package type +# os_CODENAME - vendor's codename for release +# GetOSVersion +GetOSVersion() { + # Figure out which vendor we are + if [[ -x "`which sw_vers 2>/dev/null`" ]]; then + # OS/X + os_VENDOR=`sw_vers -productName` + os_RELEASE=`sw_vers -productVersion` + os_UPDATE=${os_RELEASE##*.} + os_RELEASE=${os_RELEASE%.*} + os_PACKAGE="" + if [[ "$os_RELEASE" =~ "10.7" ]]; then + os_CODENAME="lion" + elif [[ "$os_RELEASE" =~ "10.6" ]]; then + os_CODENAME="snow leopard" + elif [[ "$os_RELEASE" =~ "10.5" ]]; then + os_CODENAME="leopard" + elif [[ "$os_RELEASE" =~ "10.4" ]]; then + os_CODENAME="tiger" + elif [[ "$os_RELEASE" =~ "10.3" ]]; then + os_CODENAME="panther" + else + os_CODENAME="" + fi + elif [[ -x $(which lsb_release 2>/dev/null) ]]; then + os_VENDOR=$(lsb_release -i -s) + os_RELEASE=$(lsb_release -r -s) + os_UPDATE="" + os_PACKAGE="rpm" + if [[ "Debian,Ubuntu,LinuxMint" =~ $os_VENDOR ]]; then + os_PACKAGE="deb" + elif [[ "SUSE LINUX" =~ $os_VENDOR ]]; then + lsb_release -d -s | grep -q openSUSE + if [[ $? -eq 0 ]]; then + os_VENDOR="openSUSE" + fi + elif [[ $os_VENDOR == "openSUSE project" ]]; then + os_VENDOR="openSUSE" + elif [[ $os_VENDOR =~ Red.*Hat ]]; then + os_VENDOR="Red Hat" + fi + os_CODENAME=$(lsb_release -c -s) + elif [[ -r /etc/redhat-release ]]; then + # Red Hat Enterprise Linux Server release 5.5 (Tikanga) + # Red Hat Enterprise Linux Server release 7.0 Beta (Maipo) + # CentOS release 5.5 (Final) + # CentOS Linux release 6.0 (Final) + # Fedora release 16 (Verne) + # XenServer release 6.2.0-70446c (xenenterprise) + os_CODENAME="" + for r in "Red Hat" CentOS Fedora XenServer; do + os_VENDOR=$r + if [[ -n "`grep \"$r\" /etc/redhat-release`" ]]; then + ver=`sed -e 's/^.* \([0-9].*\) (\(.*\)).*$/\1\|\2/' /etc/redhat-release` + os_CODENAME=${ver#*|} + os_RELEASE=${ver%|*} + os_UPDATE=${os_RELEASE##*.} + os_RELEASE=${os_RELEASE%.*} + break + fi + os_VENDOR="" + done + os_PACKAGE="rpm" + elif [[ -r /etc/SuSE-release ]]; then + for r in openSUSE "SUSE Linux"; do + if [[ "$r" = "SUSE Linux" ]]; then + os_VENDOR="SUSE LINUX" + else + os_VENDOR=$r + fi + + if [[ -n "`grep \"$r\" /etc/SuSE-release`" ]]; then + os_CODENAME=`grep "CODENAME = " /etc/SuSE-release | sed 's:.* = ::g'` + os_RELEASE=`grep "VERSION = " /etc/SuSE-release | sed 's:.* = ::g'` + os_UPDATE=`grep "PATCHLEVEL = " /etc/SuSE-release | sed 's:.* = ::g'` + break + fi + os_VENDOR="" + done + os_PACKAGE="rpm" + # If lsb_release is not installed, we should be able to detect Debian OS + elif [[ -f /etc/debian_version ]] && [[ $(cat /proc/version) =~ "Debian" ]]; then + os_VENDOR="Debian" + os_PACKAGE="deb" + os_CODENAME=$(awk '/VERSION=/' /etc/os-release | sed 's/VERSION=//' | sed -r 's/\"|\(|\)//g' | awk '{print $2}') + os_RELEASE=$(awk '/VERSION_ID=/' /etc/os-release | sed 's/VERSION_ID=//' | sed 's/\"//g') + fi + export os_VENDOR os_RELEASE os_UPDATE os_PACKAGE os_CODENAME +} + + +# Translate the OS version values into common nomenclature +# Sets ``DISTRO`` from the ``os_*`` values +function GetDistro() { + GetOSVersion + if [[ "$os_VENDOR" =~ (Ubuntu) || "$os_VENDOR" =~ (Debian) ]]; then + # 'Everyone' refers to Ubuntu / Debian releases by the code name adjective + DISTRO=$os_CODENAME + elif [[ "$os_VENDOR" =~ (Fedora) ]]; then + # For Fedora, just use 'f' and the release + DISTRO="f$os_RELEASE" + elif [[ "$os_VENDOR" =~ (openSUSE) ]]; then + DISTRO="opensuse-$os_RELEASE" + elif [[ "$os_VENDOR" =~ (SUSE LINUX) ]]; then + # For SLE, also use the service pack + if [[ -z "$os_UPDATE" ]]; then + DISTRO="sle${os_RELEASE}" + else + DISTRO="sle${os_RELEASE}sp${os_UPDATE}" + fi + elif [[ "$os_VENDOR" =~ (Red Hat) || "$os_VENDOR" =~ (CentOS) ]]; then + # Drop the . release as we assume it's compatible + DISTRO="rhel${os_RELEASE::1}" + elif [[ "$os_VENDOR" =~ (XenServer) ]]; then + DISTRO="xs$os_RELEASE" + else + # Catch-all for now is Vendor + Release + Update + DISTRO="$os_VENDOR-$os_RELEASE.$os_UPDATE" + fi + export DISTRO +} + + +# Determine if current distribution is a Fedora-based distribution +# (Fedora, RHEL, CentOS, etc). +# is_fedora +function is_fedora { + if [[ -z "$os_VENDOR" ]]; then + GetOSVersion + fi + + [ "$os_VENDOR" = "Fedora" ] || [ "$os_VENDOR" = "Red Hat" ] || [ "$os_VENDOR" = "CentOS" ] +} + + +# Determine if current distribution is a SUSE-based distribution +# (openSUSE, SLE). +# is_suse +function is_suse { + if [[ -z "$os_VENDOR" ]]; then + GetOSVersion + fi + + [ "$os_VENDOR" = "openSUSE" ] || [ "$os_VENDOR" = "SUSE LINUX" ] +} + +# Get the path to the direcotry where python executables are installed. +# get_python_exec_prefix +function get_python_exec_prefix() { + if is_fedora || is_suse; then + echo "/usr/bin" + else + echo "/usr/local/bin" + fi +} + +# Returns 'true' if 'true', 'yes', 'on' or '1' - false, otherwise. +# Converts values to lower case first. +# If no default is provided, assumes false. +function get_bool() { + local VARIABLE="$1" + local DEFAULT=${2:-false} + + VALUE=${!VARIABLE:-$DEFAULT} + + VALUE=$(eval echo "$VALUE" | tr '[:upper:]' '[:lower:]') + if [[ "1 yes true on" =~ "$VALUE" ]]; then + VALUE=true + else + VALUE=false + fi + + echo $VALUE +} + +# Get the project branch to switch to. Uses PROJECT_BRANCH_NAME, +# then BRANCH_DEFAULT, then 'master' +function get_project_branch() { + local PROJECT_BRANCH_NAME=$1 + local BRANCH_DEFAULT=${2:-master} + + PROJECT_BRANCH_OVERRIDE=${!PROJECT_BRANCH_NAME} + + BRANCH=${PROJECT_BRANCH_OVERRIDE:-$BRANCH_DEFAULT} + + echo "$BRANCH" +} + +# Try to switch to a branch or commit in a repo +# Fails if the branch/commit doesn't exist +function git_checkout() { + local PROJECT=$1 + local REPO_DIR=$2 + local REPO_BRANCH=$3 + local REPO_BRANCH_VAR_NAME=$4 + + if [ -n "$REPO_BRANCH" ]; then + pushd "$REPO_DIR" + if [ $PROJECT == "diskimage-builder" ] || [ $PROJECT == "tripleo-image-elements" ]; then + REPO_BRANCH=master + fi + CURRENT_BRANCH=$(git branch | grep "\*" | awk '{print $2}') + GIT_STATUS=$(git checkout "$REPO_BRANCH" &> /dev/null || echo "failed") + if [[ "$GIT_STATUS" = "failed" ]]; then + exclaim "${COLOR_RED}Could not switch to branch/commit '$REPO_BRANCH' in $PROJECT, exiting${COLOR_NONE}" + echo "Please set '$REPO_BRANCH_VAR_NAME' to a valid branch/commit and try again." + if [[ "$CURRENT_BRANCH" != "master" ]]; then + echo "Note: This repo is currently on branch ${CURRENT_BRANCH} - if this is correct," + echo "you should set $REPO_BRANCH_VAR_NAME=${CURRENT_BRANCH} and re-run your command." + else + echo "Note: This error may also mean that there are modified files in $PROJECT." + echo " If that is the case, please stash them and re-run your command." + fi + exit 1 + else + if [[ "$REPO_BRANCH" != "$CURRENT_BRANCH" ]]; then + exclaim "${COLOR_BLUE}Switched to $PROJECT branch '$REPO_BRANCH'${COLOR_NONE}" + else + echo "Using $PROJECT branch '$REPO_BRANCH'" + fi + fi + popd + fi +} + diff --git a/integration/scripts/functions_qemu b/integration/scripts/functions_qemu new file mode 100644 index 0000000000..62875eec0a --- /dev/null +++ b/integration/scripts/functions_qemu @@ -0,0 +1,174 @@ +#!/bin/bash +# +# Additional functions that would mostly just pertain to a Ubuntu + Qemu setup +# + +function build_vm() { + exclaim "Actually building the image, this can take up to 15 minutes" + + # set variables here and ensure they are not changed during the duration of this script + readonly HOMEDIR=$1 + readonly HOST_USERNAME=$2 + GUEST_USERNAME=${GUEST_USERNAME:-$2} + HOST_SCP_USERNAME=${HOST_SCP_USERNAME:-$2} + VM=$3 + DISTRO=$4 + SERVICE_TYPE=$5 + + readonly SSH_DIR=${KEY_DIR:-${HOMEDIR}/.ssh} + manage_ssh_keys + + if [ $DISTRO == 'ubuntu' ]; then + export RELEASE=trusty + export DIB_RELEASE=$RELEASE + export DIB_CLOUD_IMAGES=cloud-images.ubuntu.com + # Use the apt sources.list on the build host, its almost always preferred + if [ -f /etc/apt/sources.list ]; then + export DIB_APT_SOURCES=/etc/apt/sources.list + EXTRA_ELEMENTS="apt-sources apt-conf-dir" + fi + fi + if [ $DISTRO == 'fedora' ]; then + EXTRA_ELEMENTS=selinux-permissive + fi + + export HOST_USERNAME + export HOST_SCP_USERNAME + export GUEST_USERNAME + export CONTROLLER_IP + export REDSTACK_SCRIPTS + export SERVICE_TYPE + export PATH_TROVE + export ESCAPED_PATH_TROVE + export SSH_DIR + export GUEST_LOGDIR + export ESCAPED_GUEST_LOGDIR + export ELEMENTS_PATH=$REDSTACK_SCRIPTS/files/elements + export ELEMENTS_PATH+=:$PATH_DISKIMAGEBUILDER/elements + export ELEMENTS_PATH+=:$PATH_TRIPLEO_ELEMENTS/elements + export DIB_CLOUD_INIT_DATASOURCES="ConfigDrive" + export DATASTORE_PKG_LOCATION + export BRANCH_OVERRIDE + export DIB_APT_CONF_DIR=/etc/apt/apt.conf.d + export DIB_CLOUD_INIT_ETC_HOSTS=true + local QEMU_IMG_OPTIONS=$(! $(qemu-img | grep -q 'version 1') && echo "--qemu-img-options compat=0.10") + ${PATH_DISKIMAGEBUILDER}/bin/disk-image-create -a amd64 -o "${VM}" \ + -x ${QEMU_IMG_OPTIONS} ${DISTRO} ${EXTRA_ELEMENTS} vm heat-cfntools \ + cloud-init-datasources ${DISTRO}-guest ${DISTRO}-${SERVICE_TYPE} +} + +function build_guest_image() { + exclaim "Building an image for use with development and integration tests." + if [ -z "$1" ] + then + echo "You must pass an image type to build, like mysql" + exit 1 + fi + SERVICE_TYPE=$1 + VALID_SERVICES='mysql percona mariadb redis cassandra couchbase mongodb postgresql couchdb vertica db2 pxc' + if ! [[ " $VALID_SERVICES " =~ " $SERVICE_TYPE " ]]; then + exclaim "You did not pass in a valid image type. Valid types are:" $VALID_SERVICES + exit 1 + fi + + GUEST_LOGDIR=$(iniget $PATH_TROVE/etc/trove/trove-guestagent.conf.sample DEFAULT log_dir) + GUEST_LOGFILE=$(iniget $PATH_TROVE/etc/trove/trove-guestagent.conf.sample DEFAULT log_file) + + if [ -z $GUEST_LOGDIR ] || [ -z $GUEST_LOGFILE ] + then + exclaim "error: log_dir and log_file are required in: " $PATH_TROVE/etc/trove/trove-guestagent.conf.sample + exit 1 + fi + + ESCAPED_GUEST_LOGDIR=`echo $GUEST_LOGDIR | sed 's/\//\\\\\//g'` + + USERNAME=`whoami` + # To change the distro, edit the redstack.rc file + readonly IMAGENAME=${DISTRO}_${SERVICE_TYPE} + readonly VM_PATH=$USERHOME/images/${IMAGENAME} + readonly VM_PATH_NAME=${VM_PATH}/${IMAGENAME} + mkdir -p $VM_PATH + + # If the path does not exist, build it, otherwise just upload it + # (unless we're explicitly told to rebuild it) + REBUILD_IMAGE=$(echo "${REBUILD_IMAGE}" | tr '[:upper:]' '[:lower:]') + if [ "${REBUILD_IMAGE}" = "true" ] || [ ! -d $VM_PATH ] || [ `ls -1 $VM_PATH | wc -l` -eq '0' ] + then + if [ "${REBUILD_IMAGE}" = "true" ] + then + exclaim "Rebuilding image" + rm -rf "${VM_PATH}" + fi + build_vm $USERHOME $USERNAME $VM_PATH_NAME $DISTRO $SERVICE_TYPE + touch -c "${VM_PATH}" + else + exclaim "Found image in $VM_PATH - using the qcow2 image found here..." + ELEMENTS_DIR="files/elements/${DISTRO}-${SERVICE_TYPE}" + ELEMENTS_DIR_GUEST="files/elements/${DISTRO}-guest" + # Print out a warning on all the elements files that are newer than the image. + # Directories are not excluded as that is the only way to determine if a file + # has been removed. + # The rebuild is not automatically triggered as there are valid reasons for a + # new file to be present (rollback of change, inadvertent .log files present, + # feature half implemented, etc.). + IMAGE_OLD= + while IFS= read -r -d '' ELEMENT_FILE + do + if [ "${ELEMENT_FILE}" -nt "${VM_PATH}" ] + then + IMAGE_OLD=true + exclaim "${COLOR_RED}WARNING: Element file '${ELEMENT_FILE}' is newer than cached image${COLOR_NONE}" + fi + done < <(find "${ELEMENTS_DIR}" "${ELEMENTS_DIR_GUEST}" -depth -print0) + if [ "${IMAGE_OLD}" = "true" ] + then + exclaim "${COLOR_RED}Use ${COLOR_NONE}REBUILD_IMAGE=True${COLOR_RED} to rebuild image${COLOR_NONE}" + fi + + fi +} + +function clean_instances() { + LIST=`virsh -q list|awk '{print $1}'` + for i in $LIST; do sudo virsh destroy $i; done +} + +function manage_ssh_keys() { + if [ -e ${SSH_DIR} ]; then + echo "${SSH_DIR} already exists" + else + echo "Creating ${SSH_DIR} for ${HOST_USERNAME}" + sudo -Hiu ${HOST_USERNAME} mkdir -m go-w -p ${SSH_DIR} + fi + + if [ ! -f ${SSH_DIR}/id_rsa.pub ]; then + sudo ${PKG_MGR} ${PKG_GET_ARGS} install expect + generate_empty_passphrase_ssh_key ${HOST_USERNAME} + fi + + add_host_key_to_authorizedkeys +} + +function generate_empty_passphrase_ssh_key () { + echo "generating a empty passphrase DEV ONLY rsa key" + expect -c " +spawn sudo -Hiu ${HOST_USERNAME} /usr/bin/ssh-keygen -f ${SSH_DIR}/id_rsa -q +expect \"empty for no passphrase\" +send \n +expect assphrase +send \n +expect eof" +} + +function add_host_key_to_authorizedkeys () { + # test to see if the host key is already in its own authorized_keys file - if not then add it. This is then later copied + # to the guest image + is_in_keyfile=`cat ${SSH_DIR}/id_rsa.pub | grep -f - ${SSH_DIR}/authorized_keys | wc -l` + if [ $is_in_keyfile == 0 ]; then + echo "Adding keyfile to authorized_keys, it does not yet exist" + cat ${SSH_DIR}/id_rsa.pub >> ${SSH_DIR}/authorized_keys + chmod 600 ${SSH_DIR}/authorized_keys + else + echo "Keyfile already exists in authorized_keys - skipping" + fi +} diff --git a/integration/scripts/image-projects-list b/integration/scripts/image-projects-list new file mode 100644 index 0000000000..76d0da45b8 --- /dev/null +++ b/integration/scripts/image-projects-list @@ -0,0 +1,3 @@ +diskimage-builder +tripleo-image-elements +trove diff --git a/integration/scripts/local.conf.d/ceilometer_cinder.conf.rc b/integration/scripts/local.conf.d/ceilometer_cinder.conf.rc new file mode 100644 index 0000000000..9b80b6c690 --- /dev/null +++ b/integration/scripts/local.conf.d/ceilometer_cinder.conf.rc @@ -0,0 +1,3 @@ +[[post-config|\$CINDER_CONF]] +[DEFAULT] +notification_driver = messagingv2 diff --git a/integration/scripts/local.conf.d/ceilometer_nova.conf.rc b/integration/scripts/local.conf.d/ceilometer_nova.conf.rc new file mode 100644 index 0000000000..fe48b0224b --- /dev/null +++ b/integration/scripts/local.conf.d/ceilometer_nova.conf.rc @@ -0,0 +1,3 @@ +[[post-config|\$NOVA_CONF]] +[DEFAULT] +instance_usage_audit = True diff --git a/integration/scripts/local.conf.d/ceilometer_services.conf.rc b/integration/scripts/local.conf.d/ceilometer_services.conf.rc new file mode 100644 index 0000000000..ce33948ef2 --- /dev/null +++ b/integration/scripts/local.conf.d/ceilometer_services.conf.rc @@ -0,0 +1,3 @@ +[[post-config|\$CEILOMETER_CONF]] +[notification] +store_events = True diff --git a/integration/scripts/local.conf.d/sample.rc b/integration/scripts/local.conf.d/sample.rc new file mode 100644 index 0000000000..78753bc7d7 --- /dev/null +++ b/integration/scripts/local.conf.d/sample.rc @@ -0,0 +1,42 @@ +# +# Files in this directory are automatically added to the devstack +# local.conf file, between a specific set of tags. +# +# Filenames must end with '.rc' to be recognized; sample.rc is +# ignored. +# +# A '\' is required in front of any devstack variables since all +# .rc files are parsed first (using eval). +# +# Meta section headings must be included in each file, such as: +# [[local|localrc]] +# as the order of inserting the files is not guaranteed. +# +# All files are inherently included by default - to exclude a file, +# add a variable 'FILENAME_IN_UPPERCASE_MINUS_RC=false' in redstack.rc +# For Example: USING_VAGRANT=false (for the using_vagrant.rc file). +# +# Symbolic links are followed, so additional files can be loaded +# by placing them in an external directory and linking it in +# local.conf.d (this should allow complete flexibility in setting +# up testing options). +# For Example: +# cd /path/to/trove-integration/scripts/local.conf.d +# ln -s $HOME/local.conf.d local.conf.d +# cp /path/to/my_conf.rc $HOME/local.conf.d + + +[[local|localrc]] +# Put regular devstack variables under this meta section heading. +# This section is written out to a file and sourced by devstack, +# so it can contain logic as well. + +# The following section types should only contain ini file style +# section headings and name=value pairs +[[post-config|\$TROVE_CONF]] + +[[post-config|\$TROVE_TASKMANAGER_CONF]] + +[[post-config|\$TROVE_CONDUCTOR_CONF]] + +[[post-config|\$TROVE_API_PASTE_INI]] diff --git a/integration/scripts/local.conf.d/trove_services.conf.rc b/integration/scripts/local.conf.d/trove_services.conf.rc new file mode 100644 index 0000000000..6eedc1c30b --- /dev/null +++ b/integration/scripts/local.conf.d/trove_services.conf.rc @@ -0,0 +1,24 @@ +[[post-config|\$TROVE_CONF]] +[profiler] +enabled = $ENABLE_PROFILER +trace_sqlalchemy = $PROFILER_TRACE_SQL + +[[post-config|\$TROVE_TASKMANAGER_CONF]] +[profiler] +enabled = $ENABLE_PROFILER +trace_sqlalchemy = $PROFILER_TRACE_SQL + +[[post-config|\$TROVE_CONDUCTOR_CONF]] +[profiler] +enabled = $ENABLE_PROFILER +trace_sqlalchemy = $PROFILER_TRACE_SQL + +[[post-config|\$TROVE_GUESTAGENT_CONF]] +[profiler] +enabled = $ENABLE_PROFILER +trace_sqlalchemy = $PROFILER_TRACE_SQL + +[[post-config|\$TROVE_API_PASTE_INI]] +[filter:osprofiler] +enabled = $ENABLE_PROFILER +hmac_keys = $PROFILER_HMAC_KEYS diff --git a/integration/scripts/local.conf.d/use_kvm.rc b/integration/scripts/local.conf.d/use_kvm.rc new file mode 100644 index 0000000000..06bc2ebcb2 --- /dev/null +++ b/integration/scripts/local.conf.d/use_kvm.rc @@ -0,0 +1,4 @@ +[[local|localrc]] + +# force kvm as the libvirt type. +LIBVIRT_TYPE=kvm diff --git a/integration/scripts/local.conf.d/use_uuid_token.rc b/integration/scripts/local.conf.d/use_uuid_token.rc new file mode 100644 index 0000000000..587a4064c8 --- /dev/null +++ b/integration/scripts/local.conf.d/use_uuid_token.rc @@ -0,0 +1,3 @@ +[[local|localrc]] + +KEYSTONE_TOKEN_FORMAT=UUID diff --git a/integration/scripts/local.conf.d/using_vagrant.rc b/integration/scripts/local.conf.d/using_vagrant.rc new file mode 100644 index 0000000000..7333cd466b --- /dev/null +++ b/integration/scripts/local.conf.d/using_vagrant.rc @@ -0,0 +1,9 @@ +[[local|localrc]] + +# This is similar to code found at +# https://github.com/bcwaldon/vagrant_devstack/blob/master/Vagrantfile +# and seems to make instances ping'able in VirtualBox. +FLAT_INTERFACE=eth1 +PUBLIC_INTERFACE=eth1 +FLOATING_RANGE=`ip_chunk eth0 1`.`ip_chunk eth0 2`.`ip_chunk eth0 3`.128/28 +HOST_IP=`ip_chunk eth0 1`.`ip_chunk eth0 2`.`ip_chunk eth0 3`.`ip_chunk eth0 4` diff --git a/integration/scripts/local.conf.rc b/integration/scripts/local.conf.rc new file mode 100644 index 0000000000..6b9eb0ad75 --- /dev/null +++ b/integration/scripts/local.conf.rc @@ -0,0 +1,37 @@ +$TROVE_PRESENT_TAG +# Set some arguments for devstack. +# +# Note: This file contains autogenerated parts. +# All lines are removed from between the tag/end of tag +# markers (lines with '$MARKER_TOKEN' at beginning and end) and +# are replaced by trove-integration. +# Edits to these sections will not persist. +# +# See the '$USER_OPTS_TAG' section +# for ways to insert user args into this file. +# + +# +# This section is for things that belong in localrc +# It comes from $DEFAULT_LOCALRC +# +[[local|localrc]] + +$LOCALRC_OPTS_TAG +$LOCALRC_OPTS_TAG_END + +# +# User options here were inserted from the file USER_LOCAL_CONF +# (defaults to $USERHOME/.$LOCAL_CONF) +# + +$USER_OPTS_TAG +$USER_OPTS_TAG_END + +# +# Additional options here were inserted by trove-integration +# automatically from files in $LOCAL_CONF_D +# + +$ADD_OPTS_TAG +$ADD_OPTS_TAG_END diff --git a/integration/scripts/localrc.rc b/integration/scripts/localrc.rc new file mode 100644 index 0000000000..94c598ee6e --- /dev/null +++ b/integration/scripts/localrc.rc @@ -0,0 +1,100 @@ +# These passwords originally come from redstack.rc. +MYSQL_PASSWORD=$MYSQL_PASSWORD +RABBIT_PASSWORD=$RABBIT_PASSWORD +SERVICE_TOKEN=$SERVICE_TOKEN +ADMIN_PASSWORD=$ADMIN_PASSWORD +SERVICE_PASSWORD=$SERVICE_PASSWORD + +PUBLIC_INTERFACE=eth0 +TROVE_LOGDIR=$TROVE_LOGDIR +TROVE_AUTH_CACHE_DIR=$TROVE_AUTH_CACHE_DIR + +# Enable the Trove plugin for devstack +if [[ $USE_DEVSTACK_TROVE_PLUGIN = true ]]; then + enable_plugin trove $TROVE_REPO $TROVE_BRANCH +fi + +# Enable Trove, Swift, and Heat +ENABLED_SERVICES+=,trove,tr-api,tr-tmgr,tr-cond +ENABLED_SERVICES+=,s-proxy,s-object,s-container,s-account +ENABLED_SERVICES+=,heat,h-api,h-api-cfn,h-api-cw,h-eng +ENABLED_SERVICES+=,-n-novnc,-n-xvnc +if [[ $ENABLE_NEUTRON = true ]]; then + ENABLED_SERVICES+=,neutron,q-svc,q-agt,q-dhcp,q-l3,q-meta + disable_service n-net +else + enable_service n-net + disable_service neutron q-svc q-agt q-dhcp q-l3 q-meta +fi +# OSprofiler depends on Ceilometer +if [[ $ENABLE_CEILOMETER = true ]] || [[ $ENABLE_PROFILER = true ]]; then + CEILOMETER_BACKEND=mysql + CEILOMETER_NOTIFICATION_TOPICS=notifications,profiler + PROJ_BRANCH=$(get_project_branch CEILOMETER_BRANCH $PROJECT_BRANCH) + enable_plugin ceilometer ${GIT_OPENSTACK}/ceilometer.git $PROJ_BRANCH +fi + +# Enable Mistral, if configured +if [[ $ENABLE_MISTRAL = true ]]; then + PROJ_BRANCH=$(get_project_branch MISTRAL_BRANCH $PROJECT_BRANCH) + enable_plugin mistral ${GIT_OPENSTACK}/mistral.git $PROJ_BRANCH +fi + +# Use Git repositories for client components +LIBS_FROM_GIT=python-troveclient +if [[ $LIBS_FROM_GIT_ALL_CLIENTS = true ]]; then + LIBS_FROM_GIT+=,python-ceilometerclient + LIBS_FROM_GIT+=,python-cinderclient + LIBS_FROM_GIT+=,python-glanceclient + LIBS_FROM_GIT+=,python-heatclient + LIBS_FROM_GIT+=,python-keystoneclient + LIBS_FROM_GIT+=,python-mistralclient + LIBS_FROM_GIT+=,python-neutronclient + LIBS_FROM_GIT+=,python-novaclient + LIBS_FROM_GIT+=,python-openstackclient + LIBS_FROM_GIT+=,python-swiftclient +else + if [[ $ENABLE_NEUTRON = true ]]; then + LIBS_FROM_GIT+=,python-neutronclient + fi + if [[ $ENABLE_MISTRAL = true ]]; then + LIBS_FROM_GIT+=,python-mistralclient + fi + if [[ $ENABLE_CEILOMETER = true ]]; then + LIBS_FROM_GIT+=,python-ceilometerclient + fi +fi +# Add Git repositories for libraries +if [[ $LIBS_FROM_GIT_ALL_OSLO = true ]]; then + LIBS_FROM_GIT+=,cliff,futurist + LIBS_FROM_GIT+=,debtcollector,automaton + LIBS_FROM_GIT+=,oslo.cache,oslo.concurrency + LIBS_FROM_GIT+=,oslo.config,oslo.context + LIBS_FROM_GIT+=,oslo.db,oslo.i18n + LIBS_FROM_GIT+=,oslo.log,oslo.messaging + LIBS_FROM_GIT+=,oslo.middleware,oslo.policy + LIBS_FROM_GIT+=,oslo.reports,oslo.rootwrap + LIBS_FROM_GIT+=,oslo.serialization,oslo.service + LIBS_FROM_GIT+=,oslo.utils,oslo.versionedobjects + LIBS_FROM_GIT+=,oslo.vmware + LIBS_FROM_GIT+=,pycadf,stevedore + LIBS_FROM_GIT+=,taskflow,tooz + LIBS_FROM_GIT+=,pbr +fi + +NOVNC_FROM_PACKAGE=false +SWIFT_HASH=$SWIFT_HASH +# Set Cinder Volume from Redstack so that later Redstack can help manage +# reconnecting Volume Group to Backing File +DEST=$DEST +DATA_DIR=$DATA_DIR +SERVICE_DIR=$SERVICE_DIR +VOLUME_GROUP=${VOLUME_GROUP} +VOLUME_BACKING_FILE=${VOLUME_BACKING_FILE} +VOLUME_BACKING_FILE_SIZE=${VOLUME_BACKING_FILE_SIZE} +# The lock_path is by default /opt/stack/nova; if this path is a shared +# folder in VirtualBox things seem to break. We fix it by setting EXTRA_OPS +# to force lock_path to /tmp. +EXTRA_OPTS=(lock_path=$USERHOME/nova_locks rescan_timeout=180 resizefs_timeout=240 force_dhcp_release=False) +UNDO_REQUIREMENTS=False + diff --git a/integration/scripts/projects-list b/integration/scripts/projects-list new file mode 100644 index 0000000000..eacd4d8e8b --- /dev/null +++ b/integration/scripts/projects-list @@ -0,0 +1,12 @@ +keystone +nova +glance +horizon +swift +neutron +heat +python-openstackclient +python-novaclient +python-troveclient +python-neutronclient +python-heatclient diff --git a/integration/scripts/redstack b/integration/scripts/redstack new file mode 100755 index 0000000000..105aceb97c --- /dev/null +++ b/integration/scripts/redstack @@ -0,0 +1,1433 @@ +#!/usr/bin/env bash +############################################################################### +# RedStack, the Trove Dev Machine Controller # +############################################################################### +# # +# This script provides all the functionality to run all the steps from # +# setting up the environment, resetting the nova database to running the # +# test. # +# # +############################################################################### + +REDSTACK_SCRIPTS=${REDSTACK_SCRIPTS:-`pwd`} +REDSTACK_TESTS=$REDSTACK_SCRIPTS/../tests/ + +DEFAULT_LOCAL_CONF=local.conf.rc +DEFAULT_LOCALRC=localrc.rc +LOCAL_CONF=local.conf +LOCALRC=localrc +LOCALRC_AUTO=.localrc.auto +USER_LOCAL_CONF_NAME=.devstack.$LOCAL_CONF + +USERHOME=$HOME +# Load options not checked into VCS. +if [ -f $USERHOME/.redstack.options.rc ]; then + . $USERHOME/.redstack.options.rc +fi +if [ -f $REDSTACK_SCRIPTS/options.rc ]; then + . $REDSTACK_SCRIPTS/options.rc +fi + +# NOTE(mriedem): The gate-trove-functional-dsvm-* job config in project-config +# sets this value for Jenkins runs. +BRANCH_OVERRIDE=${BRANCH_OVERRIDE:-default} +if [[ $BRANCH_OVERRIDE == "default" && $OVERRIDE_ZUUL_BRANCH != "master" ]]; then + BRANCH_OVERRIDE=$OVERRIDE_ZUUL_BRANCH +fi + +# Bail on errors. +set -e + +# Get default host ip from interface +function get_default_host_ip() { + host_iface=$(ip route | sed -n '/^default/{ s/.*dev \(\w\+\)\s\+.*/\1/; p; }' | head -1) + echo `LC_ALL=C ip -f inet addr show ${host_iface} | awk '/inet/ {split($2,parts,"/"); print parts[1]}' | head -1` +} + +# Load functions devstack style +. $REDSTACK_SCRIPTS/functions +. $REDSTACK_SCRIPTS/functions_qemu + +# Load global configuration variables. +. $REDSTACK_SCRIPTS/redstack.rc +. $REDSTACK_SCRIPTS/reviews.rc + +# allow overrides from devstack if already set +[[ -f $PATH_DEVSTACK_SRC/functions-common ]] && source $PATH_DEVSTACK_SRC/functions-common +[[ -f $PATH_DEVSTACK_SRC/functions ]] && source $PATH_DEVSTACK_SRC/functions + +# Source the old-style localrc, or new-style .local.auto - only one should exist. +# Note: The devstack localrc's have references to 'enable_plugin' which causes +# errors when sourcing them in the stable/juno and stable/kilo branches. +# These errors are safe to ignore when sourcing these files. +set +e +[[ -f $PATH_DEVSTACK_SRC/$LOCALRC ]] && source $PATH_DEVSTACK_SRC/$LOCALRC +[[ -f $PATH_DEVSTACK_SRC/$LOCALRC_AUTO ]] && source $PATH_DEVSTACK_SRC/$LOCALRC_AUTO +set -e + +# Set up variables for the CONF files - this has to happen after loading redstack.rc, since +# TROVE_CONF_DIR is defined there - these will be used by devstack too +export TROVE_CONF=$TROVE_CONF_DIR/trove.conf +export TROVE_TASKMANAGER_CONF=$TROVE_CONF_DIR/trove-taskmanager.conf +export TROVE_CONDUCTOR_CONF=$TROVE_CONF_DIR/trove-conductor.conf +export TROVE_GUESTAGENT_CONF=$TROVE_CONF_DIR/trove-guestagent.conf +export TROVE_API_PASTE_INI=$TROVE_CONF_DIR/api-paste.ini +export TEST_CONF=$TROVE_CONF_DIR/test.conf + +# Public facing bits +SERVICE_PROTOCOL=${SERVICE_PROTOCOL:-http} +NETWORK_INTERFACE=${NETWORK_INTERFACE:-eth0} +NETWORK_SUBNET=${NETWORK_SUBNET:-10.0.0.0/24} +NETWORK_GATEWAY=${NETWORK_GATEWAY:-10.0.0.1} +BRIDGE_IP=${BRIDGE_IP:-172.24.4.1} +KEYSTONE_AUTH_HOST=${KEYSTONE_AUTH_HOST:-$SERVICE_HOST} +KEYSTONE_AUTH_PROTOCOL=${KEYSTONE_AUTH_PROTOCOL:-$SERVICE_PROTOCOL} +KEYSTONE_AUTH_PORT=${KEYSTONE_AUTH_PORT:-35357} +GLANCE_HOSTPORT=${GLANCE_HOSTPORT:-$SERVICE_HOST:9292} +GLANCE_SERVICE_PROTOCOL=${GLANCE_SERVICE_PROTOCOL:-http} +TROVE_AUTH_ENDPOINT=$KEYSTONE_AUTH_PROTOCOL://$KEYSTONE_AUTH_HOST:$KEYSTONE_AUTH_PORT/v2.0 + +# The following depends on whether neutron is used or nova-network +# neutron uses a bridge, nova-network does not +[[ $ENABLE_NEUTRON = true ]] && CONTROLLER_IP=$BRIDGE_IP || CONTROLLER_IP=$NETWORK_GATEWAY + +# PATH_TROVE more than likely has file separators, which sed does not like +# This will escape them +ESCAPED_PATH_TROVE=$(echo $PATH_TROVE | sed 's/\//\\\//g') +ESCAPED_REDSTACK_SCRIPTS=$(echo $REDSTACK_SCRIPTS | sed 's/\//\\\//g') +TROVE_AUTH_CACHE_DIR=${TROVE_AUTH_CACHE_DIR:-/var/cache/trove} +TROVE_LOGDIR=${TROVE_LOGDIR:-$DEST/logs} +# DATASTORE_PKG_LOCATION defines the location from where the datastore packages +# can be accessed by the DIB elements. This is applicable only for datastores +# that do not have a public repository from where their packages can be accessed. +# This can either be a url to a private repository or a location on the local +# filesystem that contains the datastore packages. +DATASTORE_PKG_LOCATION=${DATASTORE_PKG_LOCATION:-} + +# Support entry points installation of console scripts +if [[ -d $PATH_TROVE/bin ]]; then + TROVE_BIN_DIR=$PATH_TROVE/bin +else + TROVE_BIN_DIR=$(get_python_exec_prefix) +fi + +# Allow sourcing config values from env.rc for overrides +if [ -f /tmp/integration/env.rc ]; then + source /tmp/integration/env.rc +fi + +# set up respective package managers +if is_fedora; then + PKG_INSTALL_OPTS="" + PKG_MGR=dnf + PKG_GET_ARGS="-y" +else + PKG_INSTALL_OPTS="DEBIAN_FRONTEND=noninteractive" + PKG_MGR=apt-get + PKG_GET_ARGS="-y --allow-unauthenticated --force-yes" +fi +PKG_INSTALL_ARG="install" +PKG_UPDATE_ARG="update" + +############################################################################### +# Utility functions +############################################################################### + +# Colors that can be used in 'exclaim' +COLOR_RED='\033[0;31m' +COLOR_GREEN='\033[0;32m' +COLOR_BLUE='\033[0;34m' +COLOR_NONE='\033[0m' + +function exclaim () { + echo "*******************************************************************************" + echo -e "$@" + echo "*******************************************************************************" +} + +# Set the location of the Trove setup commands file for devstack - either in +# the devstack repo, or the Trove one. Also sets the flag which activates the +# devstack trove plugin, if required. We'll use the devstack version if it +# exists, otherwise we assume the plugin method. +function set_trove_plugin_vars () { + FAIL_IF_MISSING=${1:-true} + + TROVE_SETUP_CMD_FILE="$PATH_DEVSTACK_SRC/lib/trove" + if [ -f "$TROVE_SETUP_CMD_FILE" ]; then + USE_DEVSTACK_TROVE_PLUGIN=false + else + TROVE_SETUP_CMD_FILE="$DEST/trove/devstack/plugin.sh" + USE_DEVSTACK_TROVE_PLUGIN=true + fi + # Only complain if we have a devstack directory and are told to + if [ "$FAIL_IF_MISSING" = "true" ] && [ -d "$PATH_DEVSTACK_SRC" ] && [ ! -f "$TROVE_SETUP_CMD_FILE" ]; then + exclaim "${COLOR_RED}Trove setup file '${TROVE_SETUP_CMD_FILE}' not found!${COLOR_NONE}" + exit 1 + fi +} + +function pkg_install () { + echo Installing $@... + sudo -E $PKG_INSTALL_OPTS $HTTP_PROXY $PKG_MGR $PKG_GET_ARGS $PKG_INSTALL_ARG $@ +} + +function pkg_update () { + echo Updating $@... + sudo -E $PKG_INSTALL_OPTS $HTTP_PROXY $PKG_MGR $PKG_GET_ARGS $PKG_UPDATE_ARG $@ +} + +function set_home_dir() { + exclaim "set_home_dir has been nooped." + exit 1 +} + +function set_http_proxy() { + if [ ! "${http_proxy}" = '' ]; then + HTTP_PROXY="http_proxy=$http_proxy https_proxy=$https_proxy" + fi +} + +function get_ip_for_device() { + /sbin/ifconfig $1 | awk '/inet addr/{gsub(/addr:/,"");print $2}' +} + +function ip_chunk() { + # Given 1-4 returns a bit of where the ip range starts. + # Full IP= `ip_chunk 1`.`ip_chunk 2`.`ip_chunk 3`.`ip_chunk 4` + get_ip_for_device $1 | cut -d. -f$2 +} + +# Add a flavor and a corresponding flavor.resize +# (flavor.resize adds 16 to the memory and one more vcpu) +function add_flavor() { + local FLAVOR_NAME=$1 + local FLAVOR_ID=$2 + local FLAVOR_MEMORY_MB=$3 + local FLAVOR_ROOT_GB=$4 + local FLAVOR_VCPUS=$5 + + credentials="--os-username=admin --os-password=$ADMIN_PASSWORD --os-tenant-name=admin --os-auth-url=$TROVE_AUTH_ENDPOINT" + if [[ -z "$FLAVOR_LIST_FOR_ADD" ]]; then + FLAVOR_LIST_FOR_ADD=$(nova $credentials flavor-list | cut -d'|' -f3 | sed -e's/ /,/g') + fi + + base_id=${FLAVOR_ID} + base_name_prefix=test + ephemeral_name_prefix=${base_name_prefix}.eph + for name_prefix in $base_name_prefix $ephemeral_name_prefix; do + reg_name=${name_prefix}.${FLAVOR_NAME}-${FLAVOR_ROOT_GB} + resize_name=${reg_name}.resize + ephemeral=0 + if [[ $name_prefix == $ephemeral_name_prefix ]]; then + ephemeral=1 + fi + for name in ${reg_name} ${resize_name}; do + id=$base_id + memory=${FLAVOR_MEMORY_MB} + vcpus=${FLAVOR_VCPUS} + if [[ $ephemeral != 0 ]]; then + if [[ $BRANCH_OVERRIDE == "stable/juno" || $BRANCH_OVERRIDE == "stable/kilo" ]]; then + id=1${id} + else + id=${id}e + fi + fi + if [[ $name == ${resize_name} ]]; then + if [[ $BRANCH_OVERRIDE == "stable/juno" || $BRANCH_OVERRIDE == "stable/kilo" ]]; then + id=2${id} + else + id=${id}r + fi + memory=$((${FLAVOR_MEMORY_MB} + 16)) + vcpus=$((${FLAVOR_VCPUS} + 1)) + fi + if [[ $FLAVOR_LIST_FOR_ADD != *",$name,"* ]]; then + nova $credentials flavor-create $name $id $memory $FLAVOR_ROOT_GB $vcpus --ephemeral $ephemeral + fi + done + done +} + +function get_attribute_id() { + openstack --os-auth-url=$TROVE_AUTH_ENDPOINT --os-username=admin --os-password=$ADMIN_PASSWORD --os-project-name=admin $1 list | grep " $2" | get_field $3 +} + + +############################################################################### +# Install all the required dependencies +############################################################################### + +function install_prep_packages() { + # Called before devstack + exclaim 'Updating dependencies (part 1a)...' + pkg_update + exclaim 'Installing dependencies (part 1b)...' + pkg_install python-pip + if is_fedora; then + pkg_install git gettext + else + pkg_install git-core kvm-ipxe gettext + fi + sudo -H $HTTP_PROXY pip install --upgrade pip dib-utils +} + +function install_devstack_code() { + exclaim "Installing devstack..." + # Installs devstack (if needed). + if [ ! -d $PATH_DEVSTACK_SRC ]; then + echo "DevStack not in a shared folder, cloning from git." + mkdir -p $PATH_DEVSTACK_SRC + git clone $DEVSTACK_REPO $PATH_DEVSTACK_SRC + fi + + source $PATH_DEVSTACK_SRC/functions-common + source $PATH_DEVSTACK_SRC/functions + + # Switch to a branch if specified. The order the variables are checked is: + # DEVSTACK_BRANCH then PROJECT_BRANCH + BRANCH_SPECIFIED=$(test -z "${DEVSTACK_BRANCH}${PROJECT_BRANCH}" || echo 'True') + if [[ "${BRANCH_SPECIFIED}" = "True" ]]; then + PROJ_BRANCH=$(get_project_branch DEVSTACK_BRANCH $PROJECT_BRANCH) + ENV_VARS="DEVSTACK_BRANCH' or 'PROJECT_BRANCH" + git_checkout "devstack" "$PATH_DEVSTACK_SRC" "$PROJ_BRANCH" "$ENV_VARS" + fi + + exclaim "Installing devstack projects..." + # Ensures present user can get to the devstack dirs + sudo mkdir -p $PATH_DEVSTACK_OUTPUT + if [ ! -w $PATH_DEVSTACK_OUTPUT ]; then + sudo chown `whoami` $PATH_DEVSTACK_OUTPUT + fi + # Clones all of the code to where devstack expects it to be + pushd $PATH_DEVSTACK_OUTPUT + cmd_clone_projects do_not_force_update $REDSTACK_SCRIPTS/projects-list \ + $REDSTACK_SCRIPTS/image-projects-list + popd +} + +function install_reviews_on_top_of_devstack() { + exclaim "Putting gerrit review code on top of the existing devstack code" + run_review_for nova $PATH_NOVA $REVIEW_NOVA + run_review_for python-novaclient $PATH_PYTHON_NOVACLIENT $REVIEW_PYTHON_NOVACLIENT + run_review_for keystone $PATH_KEYSTONE $REVIEW_KEYSTONE + run_review_for python-keystoneclient $PATH_KEYSTONECLIENT $REVIEW_PYTHON_KEYSTONECLIENT + run_review_for python-openstackclient $PATH_OPENSTACKCLIENT $REVIEW_PYTHON_OPENSTACKCLIENT + run_review_for glance $PATH_GLANCE $REVIEW_GLANCE + run_review_for swift $PATH_SWIFT $REVIEW_SWIFT + run_review_for python-swiftclient $PATH_PYTHON_SWIFTCLIENT $REVIEW_PYTHON_SWIFTCLIENT + run_review_for trove $PATH_TROVE $REVIEW_TROVE + run_review_for python-troveclient $PATH_PYTHON_TROVECLIENT $REVIEW_PYTHON_TROVECLIENT +} + +function run_review_for() { + # Splits based on colon in the REVIEW_ARG and pulls from + GIT_NAME=$1 + PATH_ARG=$2 + REVIEW_ARG=$3 + for review in `echo $REVIEW_ARG| tr ":" "\n"` + do + # This should be the ref spec for what we pull + pushd $PATH_ARG + git_timed pull https://review.openstack.org/p/openstack/$GIT_NAME refs/changes/$review + popd + done +} + +function fixup_broken_devstack() { + # Nothing to do here, devstack is working + : +} + +# Delete all the lines from FILE_NAME between START_TAG and END_TAG +# Tags must appear at the beginning of a line +function clear_file_lines() { + local FILE_NAME=$1 + local START_TAG=$2 + local END_TAG=$3 + + sed -i "/^$START_TAG$/,/^$END_TAG$/{/^$START_TAG/!{/^$END_TAG/!d;}}" "$FILE_NAME" +} + +# Checks to see if a variable with the same name as FILE_NAME exists. +# Returns 'true' if no varable exists or if the value of the variable +# is set to 'true' - returns the VAR_NAME to set otherwise. +# FILE_NAME is first converted to uppercase, the extension is removed +# and all remaining '.' and spaces are replaced with '_' +function check_filename_var() { + local FILE_NAME=$1 + + DEREF_VALUE=false + if [ -f "$FILE_NAME" ]; then + VAR_NAME=$(basename "$FILE_NAME" ".rc" | tr '[:lower:][:blank:][:punct:]' '[:upper:]__') + DEREF_VALUE=$(get_bool "$VAR_NAME" "true") + if [ "$DEREF_VALUE" != "true" ]; then + DEREF_VALUE=$VAR_NAME + fi + fi + echo "$DEREF_VALUE" +} + +# Add the contents of one file to another, after the given tag +# Run through 'eval' if PARSE_FILE is true (defaults to true) +# Start with a blank line if BLANK_LINE_TO_START is true (defaults to false) +function add_file_contents() { + local FILE_NAME=$1 + local FILE_TO_ADD=$2 + local TAG=$3 + local PARSE_FILE=${4:-true} + local BLANK_LINE_TO_START=${5:-false} + + TEMP_FILE=".redstack.$$" + rm -f "$TEMP_FILE" + if [ "$BLANK_LINE_TO_START" = "true" ]; then + echo "" > "$TEMP_FILE" + fi + if [ -f "$FILE_TO_ADD" ]; then + echo "Adding $FILE_TO_ADD to $FILE_NAME" + echo "# Contents from $FILE_TO_ADD" >> "$TEMP_FILE" + if [ "$PARSE_FILE" = "true" ]; then + eval echo "\"$(cat "$FILE_TO_ADD")\"" >> "$TEMP_FILE" + else + cat "$FILE_TO_ADD" >> "$TEMP_FILE" + fi + echo "# End Of Contents from $FILE_TO_ADD" >> "$TEMP_FILE" + fi + echo "" >> "$TEMP_FILE" + sed -i "/^$TAG/r $TEMP_FILE" "$FILE_NAME" + rm -f "$TEMP_FILE" +} + +function run_devstack() { + exclaim "Running devstack..." + + # (Re)Creating this lock directory seems sure-fire. + rm -rf "$USERHOME/nova_locks" + mkdir -p "$USERHOME/nova_locks" + + TROVE_PRESENT_TAG="# Trove-integration" + LOCAL_CONF_D=local.conf.d + CONF_MATCH="*.rc" + MARKER_TOKEN="#####" + USER_LOCAL_CONF=$(readlink -f "${USER_LOCAL_CONF:-$USERHOME/$USER_LOCAL_CONF_NAME}") + LOCALRC_OPTS_TAG="$MARKER_TOKEN Redstack Localrc Options $MARKER_TOKEN" + LOCALRC_OPTS_TAG_END="$MARKER_TOKEN End Of Redstack Localrc Options $MARKER_TOKEN" + USER_OPTS_TAG="$MARKER_TOKEN User Specified Options $MARKER_TOKEN" + USER_OPTS_TAG_END="$MARKER_TOKEN End Of User Specified Options $MARKER_TOKEN" + ADD_OPTS_TAG="$MARKER_TOKEN Additional Options $MARKER_TOKEN" + ADD_OPTS_TAG_END="$MARKER_TOKEN End Of Additional Options $MARKER_TOKEN" + set_trove_plugin_vars false + + pushd "$PATH_DEVSTACK_SRC" + DEVSTACK_LOCAL_CONF=$LOCAL_CONF + # remain backwards compatible with existing localrc files + if [ -f "$LOCALRC" ]; then + DEVSTACK_LOCAL_CONF=$LOCALRC + echo "Old-style devstack config file $PATH_DEVSTACK_SRC/$DEVSTACK_LOCAL_CONF found." + echo "Consider removing to generate the preferred-sytle config file $LOCAL_CONF." + fi + if [ -f "$DEVSTACK_LOCAL_CONF" ]; then + # Check if we have already configured the devstack config file + already_in_conf=$(grep "$TROVE_PRESENT_TAG" "$DEVSTACK_LOCAL_CONF" | wc -l) + if [ "$already_in_conf" == 0 ]; then + # We can no longer append to an existing old-style localrc file + if [ "$DEVSTACK_LOCAL_CONF" == "$LOCALRC" ]; then + echo "The devstack config file $PATH_DEVSTACK_SRC/$DEVSTACK_LOCAL_CONF is too old to append to." + echo "Please remove and try again." + exit 1 + fi + # Otherwise append the redstack version to the existing file + eval echo "\"$(cat "$REDSTACK_SCRIPTS/$DEFAULT_LOCALRC")\"" >> "$DEVSTACK_LOCAL_CONF" + fi + else + # If a devstack config file doesn't exist, create it + eval echo "\"$(cat "$REDSTACK_SCRIPTS/$DEFAULT_LOCAL_CONF")\"" > "$DEVSTACK_LOCAL_CONF" + fi + + # We can only replace sections from the LOCAL_CONF style files + if [ "$DEVSTACK_LOCAL_CONF" == "$LOCAL_CONF" ]; then + # Clear out all the options + clear_file_lines "$DEVSTACK_LOCAL_CONF" "$LOCALRC_OPTS_TAG" "$LOCALRC_OPTS_TAG_END" + clear_file_lines "$DEVSTACK_LOCAL_CONF" "$USER_OPTS_TAG" "$USER_OPTS_TAG_END" + clear_file_lines "$DEVSTACK_LOCAL_CONF" "$ADD_OPTS_TAG" "$ADD_OPTS_TAG_END" + + # Add the main localrc file + PARSE_FILE="true" + BLANK_LINE_TO_START="true" + if [ -f "$REDSTACK_SCRIPTS/$DEFAULT_LOCALRC" ]; then + add_file_contents "$DEVSTACK_LOCAL_CONF" "$REDSTACK_SCRIPTS/$DEFAULT_LOCALRC" "$LOCALRC_OPTS_TAG" "$PARSE_FILE" "$BLANK_LINE_TO_START" + fi + + # Add any user options + PARSE_FILE="false" + BLANK_LINE_TO_START="true" + if [ -f "$USER_LOCAL_CONF" ]; then + add_file_contents "$DEVSTACK_LOCAL_CONF" "$USER_LOCAL_CONF" "$USER_OPTS_TAG" "$PARSE_FILE" "$BLANK_LINE_TO_START" + fi + + # Add all the files in the LOCAL_CONF_D directory that match CONF_MATCH (except for sample files) + # and that aren't excluded. Files are excluded by having a variable + # 'FILENAME_IN_UPPERCASE_MINUS_RC=false' in redstack.rc + # For Example: USING_VAGRANT=false (for the using_vagrant.rc file). + PARSE_FILE="true" + BLANK_LINE_TO_START="false" + while IFS= read -r -d '' CONF_FILE + do + FILE_NAME_VAR=$(check_filename_var "$CONF_FILE") + if [ "$FILE_NAME_VAR" = "true" ]; then + add_file_contents "$DEVSTACK_LOCAL_CONF" "$CONF_FILE" "$ADD_OPTS_TAG" "$PARSE_FILE" "$BLANK_LINE_TO_START" + else + echo "Skipping $CONF_FILE" + echo "Use $FILE_NAME_VAR=true to include" + fi + done < <(find "$REDSTACK_SCRIPTS/${LOCAL_CONF_D}" -name "${CONF_MATCH}" -follow -not -name "sample*.rc" -type f -print0) + # this is to add a blank line for readability + add_file_contents "$DEVSTACK_LOCAL_CONF" "" "$ADD_OPTS_TAG" + fi + ./stack.sh + popd +} + + +function cmd_install() { + sudo mkdir -p $TROVE_LOGDIR # Creates TROVE_LOGDIR if it does not exist + if [ ! -w $TROVE_LOGDIR ]; then + sudo chown `whoami` $TROVE_LOGDIR + fi + install_prep_packages + install_devstack_code + install_reviews_on_top_of_devstack + fixup_broken_devstack + run_devstack + exclaim "${COLOR_GREEN}FINISHED INSTALL${COLOR_NONE}" +} + + +############################################################################### +# Build the image +# see functions_qemu +############################################################################### + +# Grab a numbered field from python prettytable output +# Fields are numbered starting with 1 +# Reverse syntax is supported: -1 is the last field, -2 is second to last, etc. +# get_field field-number +function get_field() { + while read data; do + if [ "$1" -lt 0 ]; then + field="(\$(NF$1))" + else + field="\$$(($1 + 1))" + fi + echo "$data" | awk -F'[ \t]*\\|[ \t]*' "{print $field}" + done +} + +function get_glance_id () { + echo `$@ | grep ' id ' | get_field 2` +} + +function set_bin_path() { + if is_fedora; then + sed -i "s|%bin_path%|/usr/bin|g" $TEST_CONF + else + sed -i "s|%bin_path%|/usr/local/bin|g" $TEST_CONF + fi +} + +function set_mysql_pkg() { + if is_fedora; then + MYSQL_PKG="mysql-community-server" + else + MYSQL_PKG="mysql-server-5.6" + fi +} + + +function cmd_set_datastore() { + IMAGEID=$1 + DATASTORE_TYPE=$2 + # rd_manage datastore_update + rd_manage datastore_update "$DATASTORE_TYPE" "" + PACKAGES=${PACKAGES:-""} + if [ "$DATASTORE_TYPE" == "mysql" ]; then + set_mysql_pkg + PACKAGES=${PACKAGES:-$MYSQL_PKG} + VERSION="5.6" + elif [ "$DATASTORE_TYPE" == "percona" ]; then + PACKAGES=${PACKAGES:-"percona-server-server-5.6"} + VERSION="5.6" + elif [ "$DATASTORE_TYPE" == "pxc" ]; then + PACKAGES=${PACKAGES:-"percona-xtradb-cluster-server-5.6"} + VERSION="5.6" + elif [ "$DATASTORE_TYPE" == "mariadb" ]; then + PACKAGES=${PACKAGES:-"mariadb-server"} + VERSION="10.1" + elif [ "$DATASTORE_TYPE" == "mongodb" ]; then + PACKAGES=${PACKAGES:-"mongodb-org"} + VERSION="3.2" + elif [ "$DATASTORE_TYPE" == "redis" ]; then + PACKAGES=${PACKAGES:-"redis-server"} + VERSION="3.0" + elif [ "$DATASTORE_TYPE" == "cassandra" ]; then + PACKAGES=${PACKAGES:-"cassandra"} + VERSION="2.1.0" + elif [ "$DATASTORE_TYPE" == "couchbase" ]; then + PACKAGES=${PACKAGES:-"couchbase-server"} + VERSION="2.2.0" + elif [ "$DATASTORE_TYPE" == "postgresql" ]; then + PACKAGES=${PACKAGES:-"postgresql-9.4"} + VERSION="9.4" + elif [ "$DATASTORE_TYPE" == "couchdb" ]; then + PACKAGES=${PACKAGES:-"couchdb"} + VERSION="1.6.1" + elif [ "$DATASTORE_TYPE" == "vertica" ]; then + PACKAGES=${PACKAGES:-"vertica"} + VERSION="7.1" + elif [ "$DATASTORE_TYPE" == "db2" ]; then + PACKAGES=${PACKAGES:-""} + VERSION="10.5" + else + echo "Unrecognized datastore type. ($DATASTORE_TYPE)" + exit 1 + fi + + sed -i "s/%datastore_type%/$DATASTORE_TYPE/g" $TEST_CONF + sed -i "s/%datastore_version%/$VERSION/g" $TEST_CONF + + #rd_manage datastore_version_update + rd_manage datastore_version_update "$DATASTORE_TYPE" "$VERSION" "$DATASTORE_TYPE" $IMAGEID "$PACKAGES" 1 + rd_manage datastore_version_update "$DATASTORE_TYPE" "inactive_version" "manager1" $IMAGEID "" 0 + rd_manage datastore_update "$DATASTORE_TYPE" "$VERSION" + rd_manage datastore_update Test_Datastore_1 "" + + if [ -f "$PATH_TROVE"/trove/templates/$DATASTORE_TYPE/validation-rules.json ]; then + # add the configuration parameters to the database for the kick-start datastore + rd_manage db_load_datastore_config_parameters "$DATASTORE_TYPE" "$VERSION" "$PATH_TROVE"/trove/templates/$DATASTORE_TYPE/validation-rules.json + fi + + cmd_stop + iniset $TROVE_CONF DEFAULT default_datastore "$DATASTORE_TYPE" + sleep 1.5 + cmd_start +} + +############################################################################### +# Run Unit Tests +############################################################################### + +function cmd_unit_tests() { + exclaim "Running Trove Unit Tests..." + $PATH_TROVE/run_tests.sh -N +} + +############################################################################### +# Start various OpenStack daemons interactively in a screen session +############################################################################### + +function cmd_start_deps() { + if ! sudo vgs $VOLUME_GROUP; then + exclaim "Reconnecting Volume Group to Backing File" + sudo losetup -f --show ${VOLUME_BACKING_FILE} + fi + + if ! egrep -q ${SWIFT_DATA_DIR}/drives/sdb1 /proc/mounts; then + exclaim "Re-mounting Swift Disk Image" + sudo mount -t xfs -o loop,noatime,nodiratime,nobarrier,logbufs=8 ${SWIFT_DISK_IMAGE} ${SWIFT_DATA_DIR}/drives/sdb1 || true + fi + + if [[ -e $PATH_DEVSTACK_SRC/stack-screenrc ]]; then + screen -dmS stack -c $PATH_DEVSTACK_SRC/stack-screenrc + fi +} + +function cmd_stop_deps() { + if screen -ls | grep -q stack; then + screen -S stack -X quit + rm -f $DEST/status/stack/* + fi +} + + +############################################################################### +# Initialize Trove +############################################################################### + +function rd_manage() { + pushd $PATH_TROVE + $TROVE_BIN_DIR/trove-manage --config-file=$TROVE_CONF "$@" + popd +} + +function install_test_packages() { + sudo -H $HTTP_PROXY pip install openstack.nose_plugin proboscis pexpect +} + +function mod_confs() { + DATASTORE_TYPE=$1 + + sudo install -b --mode 0664 $REDSTACK_SCRIPTS/conf/test_begin.conf $TEST_CONF + TROVE_REPORT_DIR=$REDSTACK_SCRIPTS/../report/ + EXTRA_CONF=$REDSTACK_SCRIPTS/conf/test.extra.conf + if [ -e $EXTRA_CONF ]; then + cat $EXTRA_CONF >> $TEST_CONF + fi + # Append datastore specific configuration file + DATASTORE_CONF=$REDSTACK_SCRIPTS/conf/$DATASTORE_TYPE.conf + if [ ! -f $DATASTORE_CONF ]; then + exclaim "Datastore configuration file ${DATASTORE_CONF} not found" + exit 1 + fi + cat $DATASTORE_CONF | sudo tee -a $TEST_CONF > /dev/null + cat $REDSTACK_SCRIPTS/conf/test_end.conf | sudo tee -a $TEST_CONF > /dev/null + + #Add the paths to the test conf + sed -i "s,%report_directory%,$TROVE_REPORT_DIR,g" $TEST_CONF + sed -i "s,%keystone_path%,$PATH_KEYSTONE,g" $TEST_CONF + sed -i "s,%nova_path%,$PATH_NOVA,g" $TEST_CONF + sed -i "s,%glance_path%,$PATH_GLANCE,g" $TEST_CONF + sed -i "s,%trove_path%,$PATH_TROVE,g" $TEST_CONF + sed -i "s,%service_host%,$SERVICE_HOST,g" $TEST_CONF + sed -i "s,%swifth_path%,$PATH_SWIFT,g" $TEST_CONF + + # Add the region name into test.conf + sed -i "s/%region_name%/${REGION_NAME}/g" $TEST_CONF + + # Add the tenant id's into test.conf + sed -i "s/%admin_tenant_id%/$(get_attribute_id project admin 1)/g" $TEST_CONF + sed -i "s/%alt_demo_tenant_id%/$(get_attribute_id project alt_demo 1)/g" $TEST_CONF + sed -i "s/%demo_tenant_id%/$(get_attribute_id project demo 1)/g" $TEST_CONF + sed -i "s/%admin_password%/$ADMIN_PASSWORD/g" $TEST_CONF + + # Enable neutron tests if needed + sed -i "s/%neutron_enabled%/$ENABLE_NEUTRON/g" $TEST_CONF + + # If neutron is enabled, we create a shared network and write this info to the + # confs so that the integration tests can use it. + if [[ $ENABLE_NEUTRON = true ]]; then + management_network_id=$(neutron --os-username=admin --os-password=$ADMIN_PASSWORD --os-tenant-name=admin --os-auth-url=$TROVE_AUTH_ENDPOINT net-list | awk '/ alt-private / {print $2}') + management_subnet=$(neutron --os-username=admin --os-password=$ADMIN_PASSWORD --os-tenant-name=admin --os-auth-url=$TROVE_AUTH_ENDPOINT subnet-list | awk '/ alt-private-subnet / {print $2}') + echo "Using neutron network $management_network_id and subnet $management_subnet" + sed -i "s,%shared_network%,$management_network_id,g" $TEST_CONF + sed -i "s,%shared_network_subnet%,$management_subnet,g" $TEST_CONF + else + # do not leave invalid keys in the configuration when using Nova for networking + sed -i "/%shared_network%/d" $TEST_CONF + sed -i "/%shared_network_subnet%/d" $TEST_CONF + fi + + if [ "$DATASTORE_TYPE" = "vertica" ]; then + # Vertica needs more time than mysql for its boot/start/stop operations. + setup_cluster_configs cluster_member_count 3 + elif [ "$DATASTORE_TYPE" = "pxc" ]; then + setup_cluster_configs min_cluster_member_count 2 + elif [ "$DATASTORE_TYPE" = "cassandra" ]; then + setup_cluster_configs cluster_member_count 2 + elif [ "$DATASTORE_TYPE" = "mongodb" ]; then + setup_cluster_configs cluster_member_count 2 + # Decrease the number of required config servers per cluster to save resources. + iniset $TROVE_CONF $DATASTORE_TYPE num_config_servers_per_cluster 1 + fi + + set_bin_path + +} + +function setup_cluster_configs() { + # Setting cluster_member_count to 2 to decrease cluster spawn time. + iniset $TROVE_CONF $DATASTORE_TYPE $1 $2 +} + +# Add useful flavors for testing (with corresponding *.resize flavors) +function add_test_flavors() { + # name id ram root_vol vcpu + # the ram and vcpu for name.resize are automatically calculated + # eph and non-eph flavors are created for each entry + add_flavor 'tiny' 10 512 3 1 + + add_flavor 'small' 15 768 3 1 + add_flavor 'small' 16 768 4 1 + add_flavor 'small' 17 768 5 1 + + add_flavor 'medium' 20 1024 4 1 + add_flavor 'medium' 21 1024 5 1 + + add_flavor 'large' 25 2048 5 1 + add_flavor 'large' 26 2048 10 1 + add_flavor 'large' 27 2048 15 1 + + # This will allow Nova to create an instance, but not enough disk to boot the image + add_flavor 'fault_1' 30 512 1 1 + # This should be enough memory to cause Nova to fail entirely due to too much allocation + add_flavor 'fault_2' 31 131072 5 1 +} + +function cmd_test_init() { + exclaim 'Initializing Configuration for Running Tests...' + + exclaim "Installing python test packages." + install_test_packages + + exclaim "Modifying test.conf and guest.conf with appropriate values." + mod_confs $1 + + exclaim "Creating Test Flavors." + add_test_flavors + + if [[ -n $KEY_DIR ]]; then + exclaim "Installing the SSH key from $KEY_DIR to the test environment." + mkdir -m 700 -p $USERHOME/.ssh + install -b --mode 0400 $KEY_DIR/id_rsa $USERHOME/.ssh + cat $KEY_DIR/authorized_keys >> $USERHOME/.ssh/authorized_keys + chmod 600 $USERHOME/.ssh/authorized_keys + fi +} + +function cmd_build_image() { + IMAGE_DATASTORE_TYPE=${1:-'mysql'} + ESCAPED_PATH_TROVE=${2:-'\/opt\/stack\/trove'} + HOST_SCP_USERNAME=${3:-'ubuntu'} + GUEST_USERNAME=${4:-'ubuntu'} + + exclaim "Ensuring we have all packages needed to build image." + sudo $HTTP_PROXY $PKG_MGR $PKG_GET_ARGS update + sudo $HTTP_PROXY $PKG_MGR $PKG_GET_ARGS install qemu + sudo -H $HTTP_PROXY pip install --upgrade pip dib-utils + + install_devstack_code + + cmd_clone_projects do_not_force_update $REDSTACK_SCRIPTS/image-projects-list + + exclaim "Use tripleo-diskimagebuilder to actually build the Trove Guest Agent Image." + build_guest_image $IMAGE_DATASTORE_TYPE +} + +function cmd_build_and_upload_image() { + local IMAGE_URL="" + # Use /tmp as file_cache + FILES=/tmp + if [[ -n $IMAGE_DOWNLOAD_URL ]]; then + exclaim "Downloading and using cached image" + IMAGE_URL=$IMAGE_DOWNLOAD_URL + else + exclaim "Trying to build image" + build_guest_image $1 + QCOW_IMAGE=`find $VM_PATH -name '*.qcow2'` + IMAGE_URL="file://$QCOW_IMAGE" + fi + + # The devstack openrc has references to 'enable_plugin' which causes errors + # in the stable/juno and stable/kilo branches. These are safe to ignore. + set +e; source $PATH_DEVSTACK_SRC/openrc admin admin; set -e + TOKEN=$(openstack token issue | grep ' id ' | get_field 2) + GLANCE_IMAGEIDS=$(glance image-list | grep $(basename $IMAGE_URL .qcow2) | get_field 1) + if [[ -n $GLANCE_IMAGEIDS ]]; then + glance image-delete $GLANCE_IMAGEIDS + fi + GLANCE_IMAGEID=`get_glance_id upload_image $IMAGE_URL $TOKEN` + set +e; source $PATH_DEVSTACK_SRC/openrc demo demo; set -e + [[ -z "$GLANCE_IMAGEID" ]] && echo "Glance upload failed!" && exit 1 + echo "IMAGE ID: $GLANCE_IMAGEID" + + exclaim "Updating Datastores" + cmd_set_datastore $GLANCE_IMAGEID $1 +} + + +function cmd_initialize() { + exclaim '(Re)Initializing Trove...' + pushd $PATH_DEVSTACK_SRC + ./unstack.sh + ./stack.sh + popd +} + + +############################################################################### +# Start Trove specific daemons interactively in a screen session +############################################################################### + +function tr_screen_it { + if screen -ls | grep -q stack; then + echo "Starting $@..." + screen -S stack -p $1 -X stuff "$2"$'\015' + fi +} + +function init_fake_mode() { + # Create a test conf which, unlike the conf which runs on a user's machine, + # takes advantage of the running keystone service we have in our VM. + # You could think of this fake mode, which runs in the VM as being + # slightly less fake than the default one which runs outside of it. + CONF_FILE=/tmp/trove.conf.test + cp $PATH_TROVE/etc/trove/trove.conf.test $CONF_FILE + # Switch keystone from the fake class to the real one. + sed -i \ + "s/trove.tests.fakes.keystone/keystone.middleware.auth_token/g" \ + $CONF_FILE + sed -i "s/log_file = rdtest.log//g" $CONF_FILE + sed -i "s/use_stderr = False/use_stderr = True/g" $CONF_FILE + cd $PATH_TROVE + set -e + rm -f trove_test.sqlite + set +e + $TROVE_BIN_DIR/trove-manage --config-file=$CONF_FILE db_sync + sqlite3 trove_test.sqlite \ + "INSERT INTO datastores VALUES ('a00000a0-00a0-0a00-00a0-000a000000aa', \ + 'mysql', 'b00000b0-00b0-0b00-00b0-000b000000bb'); \ + INSERT INTO datastores values ('e00000e0-00e0-0e00-00e0-000e000000ee', \ + 'Test_Datastore_1', ''); \ + INSERT INTO datastore_versions VALUES ('b00000b0-00b0-0b00-00b0-000b000000bb', \ + 'a00000a0-00a0-0a00-00a0-000a000000aa', '5.6', \ + 'c00000c0-00c0-0c00-00c0-000c000000cc', $MYSQL_PKG, 1, 'mysql'); \ + INSERT INTO datastore_versions VALUES ('d00000d0-00d0-0d00-00d0-000d000000dd', \ + 'a00000a0-00a0-0a00-00a0-000a000000aa', 'inactive_version', \ + '', '', 0, 'manager1'); \ + INSERT INTO datastore_configuration_parameters VALUES \ + ('00000000-0000-0000-0000-000000000001', \ + 'key_buffer_size', 'b00000b0-00b0-0b00-00b0-000b000000bb', \ + 0, 4294967296, 0, 'integer', 0, NULL); \ + INSERT INTO datastore_configuration_parameters VALUES \ + ('00000000-0000-0000-0000-000000000002', \ + 'connect_timeout', 'b00000b0-00b0-0b00-00b0-000b000000bb', \ + 0, 65535, 1, 'integer', 0, NULL); \ + INSERT INTO datastore_configuration_parameters VALUES \ + ('00000000-0000-0000-0000-000000000003', \ + 'join_buffer_size', 'b00000b0-00b0-0b00-00b0-000b000000bb', \ + 0, 4294967296, 0, 'integer', 0, NULL); \ + INSERT INTO datastore_configuration_parameters VALUES \ + ('00000000-0000-0000-0000-000000000004', \ + 'local_infile', 'b00000b0-00b0-0b00-00b0-000b000000bb', \ + 0, 1, 0, 'integer', 0, NULL); \ + INSERT INTO datastore_configuration_parameters VALUES \ + ('00000000-0000-0000-0000-000000000005', \ + 'collation_server', 'b00000b0-00b0-0b00-00b0-000b000000bb', \ + 0, NULL, NULL, 'string', 0, NULL); \ + " +} + +function cmd_start() { + if screen -ls | grep -q stack; then + set_trove_plugin_vars + USE_SCREEN=True + TOP_DIR=$PATH_DEVSTACK_SRC + LOGDIR=$TROVE_LOGDIR + RUNNING=$(screen -S stack -Q windows) + if [[ "$RUNNING" =~ " tr-" ]]; then + exclaim "${COLOR_RED}WARNING: Trove services appear to be running. Please run 'stop' or 'restart'${COLOR_NONE}" + else + source /dev/stdin < <(sed -n '/^function start_trove\(\)/,/^}/p' "$TROVE_SETUP_CMD_FILE") + start_trove + fi + else + echo "WARNING: Could not start Trove services because there is no stack screen session running" + fi +} + +function cmd_start_fake() { + init_fake_mode + CONF_FILE=/tmp/trove.conf.test + tr_screen_it tr-fake "cd $PATH_TROVE; $TROVE_BIN_DIR/trove-fake-mode --config-file=$CONF_FILE $@" +} + +function cmd_run() { + cd $PATH_TROVE; $TROVE_BIN_DIR/trove-api \ + --config-file=$TROVE_CONF $@ +} + +function cmd_run_fake() { + init_fake_mode + CONF_FILE=/tmp/trove.conf.test + $TROVE_BIN_DIR/trove-fake-mode --config-file=$CONF_FILE $@ +} + +############################################################################### +# Stop any active Trove screen session +############################################################################### + +function cmd_stop() { + if screen -ls | grep -q stack; then + set_trove_plugin_vars + rm -f $DEST/status/stack/tr-* + USE_SCREEN=True + source /dev/stdin < <(sed -n '/^function stop_trove\(\)/,/^}/p' "$TROVE_SETUP_CMD_FILE") + MAX_RETRY=5 + COUNT=1 + while true; do + RUNNING=$(screen -S stack -Q windows) + if [[ "$RUNNING" =~ " tr-" ]]; then + stop_trove + else + break + fi + ((COUNT++)) + if [ "$COUNT" -gt "$MAX_RETRY" ]; then + exclaim "${COLOR_RED}WARNING: Could not stop Trove services after ${MAX_RETRY} attempts${COLOR_NONE}" + break + fi + done + else + echo "WARNING: Could not stop Trove services because there is no stack screen session running" + fi +} + + +############################################################################### +# Run Integration Tests +############################################################################### + +function cmd_int_tests() { + exclaim "Running Trove Integration Tests..." + if [ ! $USAGE_ENDPOINT ]; then + export USAGE_ENDPOINT=trove.tests.util.usage.FakeVerifier + fi + cd $REDSTACK_SCRIPTS + if [ $# -lt 1 ]; then + args="--group=blackbox" + else + args="$@" + fi + + # -- verbose makes it prettier. + # -- logging-clear-handlers keeps the novaclient and other things from + # spewing logs to stdout. + args="$INT_TEST_OPTIONS -B $REDSTACK_TESTS/integration/int_tests.py --verbose --logging-clear-handlers $args" + echo "python $args" + python $args +} + +function cmd_int_tests_simple() { + exclaim "Running Trove Simple Integration Tests..." + cd $REDSTACK_SCRIPTS + if [ $# -lt 1 ]; then + args="--group=simple_blackbox" + else + args="$@" + fi + + # -- verbose makes it prettier. + # -- logging-clear-handlers keeps the novaclient and other things from + # spewing logs to stdout. + args="$INT_TEST_OPTIONS -B $REDSTACK_TESTS/integration/int_tests.py --verbose --logging-clear-handlers $args" + echo "python $args" + python $args +} + +function cmd_int_tests_white_box() { + export PYTHONPATH=$PYTHONPATH:$PATH_TROVE + export PYTHONPATH=$PYTHONPATH:$PATH_NOVA + cmd_int_tests --test-config white_box=True \ + --config-file=$TROVE_CONF \ + --nova-flags=/etc/nova/nova.conf $@ +} + +function cmd_example_tests() { + set +e + cmd_stop + set -e + cmd_start_fake + sleep 3 + echo " +{ + \"directory\": \"$REDSTACK_TESTS/../apidocs/src/resources/samples/\", + \"auth_url\":\"http://$KEYSTONE_AUTH_HOST:35357/v2.0/tokens\", + \"api_url\":\"http://$SERVICE_HOST:8779\", + \"replace_host\":\"https://ord.databases.api.rackspacecloud.com\", + \"replace_dns_hostname\": \"e09ad9a3f73309469cf1f43d11e79549caf9acf2.rackspaceclouddb.com\", + \"username\":\"examples\", + \"password\":\"examples\", + \"tenant\":\"trove\" +}" > /tmp/example-tests.conf + python $REDSTACK_TESTS/examples/examples/example_generation.py /tmp/example-tests.conf + pushd $REDSTACK_TESTS/../apidocs + mvn clean + mvn generate-sources + popd + cmd_stop +} + + +############################################################################### +# Misc. tools +############################################################################### + +function mysql_nova() { + echo mysql nova --execute "$@" + mysql -u root -p$MYSQL_PASSWORD nova --execute "$@" +} + +function mysql_trove() { + echo mysql trove --execute "$@" + mysql -u root -p$MYSQL_PASSWORD trove --execute "$@" +} + +function cmd_wipe_logs() { + for file in `ls $TROVE_LOGDIR/*.log` + do + echo "Reseting log file $file..." + echo "Reset at `date`" > $file + done +} + +function cmd_rd_sql() { + mysql -u root -p$MYSQL_PASSWORD trove +} + +function cmd_fake_sql() { + pushd $PATH_TROVE + sqlite3 trove_test.sqlite $@ + popd +} + +function cmd_vagrant_ssh() { + # Runs a command on a vagrant VM from the host machine. + VHOST=`vagrant ssh_config host | awk '/HostName/{print $2}'` + VUSER=`vagrant ssh_config host | awk '/User /{print $2}'` + VPORT=`vagrant ssh_config host | awk '/Port/{print $2}'` + VIDFILE=`vagrant ssh_config host | awk '/IdentityFile/{print $2}'` + echo ssh ${VUSER}@${VHOST} -p ${VPORT} -i ${VIDFILE} -o NoHostAuthenticationForLocalhost=yes "$@" + ssh ${VUSER}@${VHOST} -p ${VPORT} -i ${VIDFILE} -o NoHostAuthenticationForLocalhost=yes "$@" +} + + +function cmd_run_ci() { + exclaim "Running CI suite..." + set +e + cmd_stop_deps + cmd_stop + set -e + cmd_install + cmd_test_init $1 + # The arg will be the image type + cmd_build_and_upload_image $1 + + # Test in fake mode. + exclaim "Testing in fake mode." + cmd_start_fake + FAKE_MODE=True cmd_int_tests + cmd_stop + + # Test in real mode. + exclaim "Testing in real mode." + cmd_start + FAKE_MODE=False cmd_int_tests +} + +function cmd_wipe_queues() { + # Obliterate rabbit. + for i in stop_app reset start_app "change_password guest $RABBIT_PASSWORD"; \ + do sudo rabbitmqctl $i; done +} + +function cmd_clear() { + cmd_int_tests --group=dbaas.api.instances.delete + clean_instances + mysql_nova "DELETE FROM instance_info_caches;" + mysql_nova "DELETE FROM instances;" + mysql_trove "DELETE FROM instances;" + mysql_trove "DELETE FROM service_statuses;" + cmd_wipe_queues +} + +function exec_cmd_on_output() { + local output_cmd=$1 + local exec_cmd=$2 + local delete_sleep_time=${3:-0} + local skip_pattern=${4:-""} + + echo "Cleaning up objects from '${output_cmd}'" + local skip_cmd="cat" + if [[ -n "${skip_pattern}" ]]; then + local temp_skip_cmd=(grep -v "${skip_pattern}") + skip_cmd=${temp_skip_cmd[*]} + fi + local max_retry=10 + local count=1 + local again= + while true; do + ids=$($output_cmd | ${skip_cmd} | grep -v -e'---' | grep -iv ' id ' | cut -d'|' -f2) + if [[ -n $ids ]]; then + for id in $ids; do + echo -e "Executing: ${exec_cmd} ${id} ${again}" + # don't stop if we get an error executing the delete, and don't print + # out anything from stderr + set +e + ${exec_cmd} "${id}" &> /dev/null + set -e + done + sleep "${delete_sleep_time}" + else + break + fi + ((count++)) + if [[ "$count" -gt "$max_retry" ]]; then + exclaim "${COLOR_RED}WARNING: '$output_cmd' still returning output after ${max_retry} delete attempts${COLOR_NONE}" + break + fi + again="${COLOR_BLUE}(again)${COLOR_NONE}" + done +} + +function cmd_clean() { + echo "Cleaning up project '${OS_PROJECT_NAME}'" + # reset any stuck backups + mysql_trove "update backups set state='COMPLETED'" + # clear out any DS version metadata + mysql_trove "delete from datastore_version_metadata" + # reset any stuck instances, and clear all replicas + mysql_trove "update instances set task_id=2, slave_of_id=null" + # reset any stuck clusters + mysql_trove "update clusters set task_id=1" + # get rid of any extraneous quota usage + mysql_trove "delete from quota_usages" + # mark all instance modules as deleted + mysql_trove "update instance_modules set deleted=1" + + source "${PATH_DEVSTACK_SRC}"/openrc admin "${OS_PROJECT_NAME}" + # delete any trove clusters + exec_cmd_on_output "trove cluster-list" "trove cluster-delete" 20 + # delete any trove instances + exec_cmd_on_output "trove list" "trove delete" 10 + # delete any backups + exec_cmd_on_output "trove backup-list" "trove backup-delete" + # clean up any remaining nova instances or cinder volumes + exec_cmd_on_output "nova list" "nova delete" 5 + exec_cmd_on_output "cinder list" "cinder delete" 1 + # delete any config groups since all instances should be gone now + exec_cmd_on_output "trove configuration-list" "trove configuration-delete" + # delete any modules too + exec_cmd_on_output "trove module-list" "trove module-delete" + # make sure that security groups are also gone, except the default + exec_cmd_on_output "openstack security group list" "nova security group delete" 0 "default" + # delete server groups + exec_cmd_on_output "nova server-group-list" "nova server-group-delete" +} + +function cmd_kick_start() { + cmd_test_init $1 + cmd_build_and_upload_image $1 +} + +function cmd_dsvm_gate_tests() { + DATASTORE_TYPE=${1:-'mysql'} + TEST_GROUP=${2:-${DATASTORE_TYPE}} + HOST_SCP_USERNAME=${3:-'jenkins'} + GUEST_USERNAME=${4:-'ubuntu'} + CONTROLLER_IP=${5:-'10.1.0.1'} + ESCAPED_PATH_TROVE=${6:-'\/opt\/stack\/new\/trove'} + + if [[ $BRANCH_OVERRIDE == "stable/liberty" ]]; then + # Devstack in liberty doesn't copy the clouds.yaml file to /etc so we need to + # ensure we have access to the clouds.yaml file set up by devstack-gate + sudo mkdir -p ~/.config/openstack + sudo ln -s $DEST/.config/openstack/clouds.yaml ~/.config/openstack/clouds.yaml + sudo chown -R $(whoami) ~/.config + fi + + # Devstack vm-gate runs as the jenkins user, but needs to connect to the guest image as ubuntu + echo "User=ubuntu" >> /home/jenkins/.ssh/config + + # Fix iptables rules that prevent amqp connections from the devstack box to the guests + sudo iptables -D openstack-INPUT -j REJECT --reject-with icmp-host-prohibited || true + + sudo chown -R $(whoami) /etc/trove + sudo chown -R $(whoami) $DEST/trove-integration + iniset $TROVE_GUESTAGENT_CONF DEFAULT rabbit_host $CONTROLLER_IP + iniset $TROVE_GUESTAGENT_CONF oslo_messaging_rabbit rabbit_hosts $CONTROLLER_IP + cd $DEST/trove-integration/scripts + sudo -H $HTTP_PROXY pip install --upgrade pip dib-utils + + cmd_kick_start $DATASTORE_TYPE + + # Update the local swift endpoint in the catalog to use the CONTROLLER_IP instead of 127.0.0.1 + source $DEST/devstack/accrc/admin/admin + # NOTE(mriedem): We have to treat stable branches before liberty special + # due to constraints with older versions of python-openstackclient. + if [[ $BRANCH_OVERRIDE == "stable/juno" || $BRANCH_OVERRIDE == "stable/kilo" ]]; then + SWIFT_ENDPOINT=$(openstack endpoint list | grep 'swift' | get_field 1) + openstack endpoint create swift --region RegionOne --publicurl 'http://'$CONTROLLER_IP':8080/v1/AUTH_$(tenant_id)s' \ + --internalurl 'http://'$CONTROLLER_IP':8080/v1/AUTH_$(tenant_id)s' --adminurl 'http://'$CONTROLLER_IP':8080' + openstack endpoint delete $SWIFT_ENDPOINT + else + OS_CLIENT_ARGS="--os-auth-type v3password --os-auth-url $KEYSTONE_AUTH_PROTOCOL://$KEYSTONE_AUTH_HOST:5000/v3 --os-identity-api-version=3" + SWIFT_ENDPOINTS=$(openstack endpoint list $OS_CLIENT_ARGS --service swift -c ID -f value) + openstack endpoint create $OS_CLIENT_ARGS swift public 'http://'$CONTROLLER_IP':8080/v1/AUTH_$(tenant_id)s' --region RegionOne + openstack endpoint create $OS_CLIENT_ARGS swift internal 'http://'$CONTROLLER_IP':8080/v1/AUTH_$(tenant_id)s' --region RegionOne + openstack endpoint create $OS_CLIENT_ARGS swift admin 'http://'$CONTROLLER_IP':8080' --region RegionOne + echo $SWIFT_ENDPOINTS | xargs -n 1 openstack endpoint delete $OS_CLIENT_ARGS + fi + + cmd_int_tests --group=$TEST_GROUP +} + +function cmd_reset_task() { + mysql_trove "UPDATE instances SET task_id=1 WHERE id='$1'" +} + +function cmd_clone_projects() { + UPDATE_PROJECTS=$1 + PROJECT_LIST_FILES=${@:2} + for project in $(cat $PROJECT_LIST_FILES); do + if [ ! -d $PATH_DEVSTACK_OUTPUT/$project ]; then + echo "Creating a new clone of $project..." + git_clone $GIT_OPENSTACK/"$project".git ${PATH_DEVSTACK_OUTPUT}/$project master + else + if [ $UPDATE_PROJECTS != "force_update" ]; then + echo "$project was already cloned or exists in a shared folder. Ignoring..." + else + echo "$project was already cloned. Pulling changes to update." + cd $PATH_DEVSTACK_OUTPUT/$project + git pull + fi + fi + # Switch to a branch if specified. The order the variables are checked is: + # _BRANCH then PROJECT_CLIENT_BRANCH (if a client) then PROJECT_BRANCH + # Note: For the Trove project, only TROVE_BRANCH and PYTHON_TROVECLIENT_BRANCH are used + PROJECT_BRANCH_NAME=$(eval echo "${project}_BRANCH" | tr '[:lower:]-' '[:upper:]_') + PROJECT_BRANCH_VALUE=${!PROJECT_BRANCH_NAME} + # TROVE_BRANCH is defaulted to master if not set, so use the original value here + if [[ "$project" = "trove" ]]; then + PROJECT_BRANCH_VALUE=${TROVE_BRANCH_ORIG} + fi + BRANCH_SPECIFIED=$(test -z "${PROJECT_BRANCH_VALUE}${PROJECT_CLIENT_BRANCH}${PROJECT_BRANCH}" || echo 'True') + if [[ "${BRANCH_SPECIFIED}" = "True" ]]; then + # Set up the default branch and env var names for the project + DEFAULT_BRANCH="$PROJECT_BRANCH" + ENV_VARS="$PROJECT_BRANCH_NAME' or 'PROJECT_BRANCH" + # Don't use 'PROJECT_BRANCH' or 'PROJECT_CLIENT_BRANCH' for the Trove project + if [[ "$project" =~ "trove" ]]; then + DEFAULT_BRANCH=master + ENV_VARS="$PROJECT_BRANCH_NAME" + # Use 'PROJECT_CLIENT_BRANCH' first for clients + elif [[ "$project" =~ "client" ]]; then + DEFAULT_BRANCH="${PROJECT_CLIENT_BRANCH:-$PROJECT_BRANCH}" + ENV_VARS="$PROJECT_BRANCH_NAME' or 'PROJECT_CLIENT_BRANCH' or 'PROJECT_BRANCH" + fi + PROJ_BRANCH=$(get_project_branch $PROJECT_BRANCH_NAME $DEFAULT_BRANCH) + git_checkout "$project" "$PATH_DEVSTACK_OUTPUT/$project" "$PROJ_BRANCH" "$ENV_VARS" + fi + done +} + +function cmd_repl() { + INT_TEST_OPTIONS=-i cmd_int_tests_white_box --repl --group=_does_not_exist_ $@ +} + + +############################################################################### +# Process the user provided command and run the appropriate command +############################################################################### + +# Let's not run this as the root user +if [ $EUID -eq 0 ]; then + echo "You are running this script as root. You need to run as a regular user" + exit 1 +fi + +# Set this to exit immediately on error +set -o errexit + +# set_home_dir +set_http_proxy +set_trove_plugin_vars false + +function print_usage() { + echo "Usage: $0 [command]" + echo " + Commands : + --setup environment-- + install - Install all the required dependencies and bring up tr-api and tr-tmgr + - devstack config can be altered by using a USER_LOCAL_CONF file + which will be copied into devstack/local.conf on each 'install' run + (defaults to \$HOME/$USER_LOCAL_CONF_NAME) + - Set DEVSTACK_BRANCH to switch the branch/commit of devstack + (i.e. 'stable/kilo' or '7ef2462') + test-init - Configure the test configuration files and add keystone test users + build-image - Builds the vm image for the trove guest + initialize - Reinitialize the trove database, users, services, and test config + + --helper for environment-- + kick-start - kick start the setup of trove. + (redstack test-init/build-image in one step) + [mysql no-clean] no clean avoids rebuilding packages from scratch + - Set REBUILD_IMAGE=True to force rebuild (won't use cached image) + + --trove dependency services-- + start-deps - Start or resume daemons Trove depends on. + stop-deps - Kill daemons Trove depends on. + + --trove services-- + start - Start or resume daemons Trove depends on. + stop - Kill daemons Trove depends on. + restart - Runs stop then start for Trove services. + + --tests-- + unit-tests - Run the unit tests.dependencies + int-tests - Runs the integration tests (requires all daemons). + simple-tests - Runs the simple integration tests (requires all daemons). + dsvm-gate-tests - Configures and runs the int-tests in a devstack vm-gate environment. + + --tools-- + debug - Debug this script (shows all commands). + wipe-logs - Resets all log files. + rd-sql - Opens the Trove MySQL database. + vagrant-ssh - Runs a command from the host on the server. + clear - Destroy instances and rabbit queues. + clean - Clean up resources created by a failed test run. + run - Starts RD but not in a screen. + run-fake - Runs the server in fake mode. + update-projects - Git pull on all the daemons trove dependencies. + reset-task - Sets an instance task to NONE. + wipe-queues - Resets RabbitMQ queues. + " + exit 1 +} + +function run_command() { + # Print the available commands + if [ $# -lt 1 ]; then + print_usage + fi + + case "$1" in + "install" ) cmd_install;; + "test-init" ) cmd_test_init $@;; + "build-image" ) shift; cmd_build_image $@;; + "initialize" ) cmd_initialize;; + "unit-tests" ) cmd_unit_tests;; + "start-deps" ) cmd_start_deps;; + "stop-deps" ) cmd_stop_deps;; + "start" ) cmd_start;; + "int-tests" ) shift; cmd_int_tests $@;; + "int-tests-wb" ) shift; cmd_int_tests_white_box $@;; + "simple-tests") shift; cmd_int_tests_simple $@;; + "stop" ) cmd_stop;; + "restart" ) cmd_stop; cmd_start;; + "wipe-logs" ) cmd_wipe_logs;; + "rd-sql" ) shift; cmd_rd_sql $@;; + "fake-sql" ) shift; cmd_fake_sql $@;; + "run-ci" ) shift; cmd_run_ci $@;; + "vagrant-ssh" ) shift; cmd_vagrant_ssh $@;; + "debug" ) shift; echo "Enabling debugging."; \ + set -o xtrace; run_command $@;; + "clear" ) shift; cmd_clear $@;; + "clean" ) shift; cmd_clean $@;; + "run" ) shift; cmd_run $@;; + "kick-start" ) shift; cmd_kick_start $@;; + "dsvm-gate-tests" ) shift; cmd_dsvm_gate_tests $@;; + "run-fake" ) shift; cmd_run_fake $@;; + "start-fake" ) shift; cmd_start_fake $@;; + "update-projects" ) cmd_clone_projects force_update \ + $REDSTACK_SCRIPTS/projects-list \ + $REDSTACK_SCRIPTS/image-projects-list;; + "reset-task" ) shift; cmd_reset_task $@;; + "wipe-queues" ) shift; cmd_wipe_queues $@;; + "example-tests" ) shift; cmd_example_tests $@;; + "repl" ) shift; cmd_repl $@;; + "help" ) print_usage;; + * ) + echo "'$1' not a valid command" + exit 1 + esac +} + +run_command $@ diff --git a/integration/scripts/redstack.rc b/integration/scripts/redstack.rc new file mode 100644 index 0000000000..b4036ba18a --- /dev/null +++ b/integration/scripts/redstack.rc @@ -0,0 +1,120 @@ +# Paths inside the VM. +[ -z $SERVICE_HOST ] && SERVICE_HOST=`get_default_host_ip` +[ -z $DEST ] && DEST=/opt/stack +[ -z $BRIDGE_IP ] && BRIDGE_IP=172.24.4.1 +[ -z $PATH_DEVSTACK_SRC ] && PATH_DEVSTACK_SRC=~/devstack +[ -z $TROVE_CONF_DIR ] && TROVE_CONF_DIR=/etc/trove +[ -z $MYSQL_HOST ] && MYSQL_HOST=$SERVICE_HOST + +# Set up the region name +# Try REGION_NAME then OS_REGION_NAME then RegionOne (the devstack default) +REGION_NAME=${REGION_NAME:-${OS_REGION_NAME:-RegionOne}} + +# Enable neutron instead of nova-network +# Note: Until a few key changesets land, we can't enable Neutron properly. +# See: https://review.openstack.org/#/c/356026 +# https://review.openstack.org/#/c/356763 +# https://review.openstack.org/#/c/356701 +# NEUTRON_DEFAULT=true +NEUTRON_DEFAULT=false +if [[ $BRANCH_OVERRIDE == "stable/liberty" || $BRANCH_OVERRIDE == "stable/mitaka" ]]; then + NEUTRON_DEFAULT=false +fi +ENABLE_NEUTRON=$(get_bool ENABLE_NEUTRON $NEUTRON_DEFAULT) + +# Enable osprofiler - note: Enables Ceilometer as well +ENABLE_PROFILER=$(get_bool ENABLE_PROFILER false) +PROFILER_TRACE_SQL=$(get_bool PROFILER_TRACE_SQL false) +[ -z $PROFILER_HMAC_KEYS ] && PROFILER_HMAC_KEYS=SECRET_KEY + +# Enable ceilometer +ENABLE_CEILOMETER=$(get_bool ENABLE_CEILOMETER $ENABLE_PROFILER) + +# Enable Mistral +ENABLE_MISTRAL=$(get_bool ENABLE_MISTRAL false) + +# Enable LIBS_FROM_GIT +LIBS_FROM_GIT_ALL_CLIENTS=$(get_bool LIBS_FROM_GIT_ALL_CLIENTS false) +LIBS_FROM_GIT_ALL_OSLO=$(get_bool LIBS_FROM_GIT_ALL_OSLO false) + +# Don't include certain .rc files in local.conf.d by default +USING_VAGRANT=$(get_bool USING_VAGRANT false) +USE_KVM=$(get_bool USE_KVM false) +USE_UUID_TOKEN=$(get_bool USE_UUID_TOKEN false) + +# Specify configuration for Ceilometer +CEILOMETER_SERVICES_CONF=$(get_bool CEILOMETER_SERVICES_CONF $ENABLE_CEILOMETER) +CEILOMETER_CINDER_CONF=$(get_bool CEILOMETER_CINDER_CONF false) +CEILOMETER_NOVA_CONF=$(get_bool CEILOMETER_NOVA_CONF false) + +# Paths for various OpenStack components +PATH_DEVSTACK_OUTPUT=$DEST +PATH_NOVA=$DEST/nova +PATH_KEYSTONE=$DEST/keystone +PATH_GLANCE=$DEST/glance +PATH_SWIFT=$DEST/swift +PATH_TROVE=$DEST/trove +PATH_PYTHON_NOVACLIENT=$DEST/python-novaclient +PATH_KEYSTONECLIENT=$DEST/python-keystoneclient +PATH_OPENSTACKCLIENT=$DEST/python-openstackclient +PATH_PYTHON_SWIFTCLIENT=$DEST/python-swiftclient +PATH_PYTHON_TROVECLIENT=$DEST/python-troveclient +PATH_TROVE_DASHBOARD=$DEST/trove-dashboard +PATH_DISKIMAGEBUILDER=$DEST/diskimage-builder +PATH_TRIPLEO_ELEMENTS=$DEST/tripleo-image-elements + +# Save the state of TROVE_BRANCH first, since it's used in redstack +TROVE_BRANCH_ORIG=${TROVE_BRANCH} +# Devstack and OpenStack git repo source paths, etc. +GIT_BASE=${GIT_BASE:-git://git.openstack.org} +GIT_OPENSTACK=${GIT_OPENSTACK:-${GIT_BASE}/openstack} +DEVSTACK_REPO=${DEVSTACK_REPO:-${GIT_BASE}/openstack-dev/devstack.git} +TROVE_REPO=${TROVE_REPO:-${GIT_OPENSTACK}/trove.git} +TROVE_DIR=${TROVE_DIR:-${PATH_TROVE}} +TROVE_BRANCH=${TROVE_BRANCH:-master} +TROVE_CLIENT_REPO=${TROVE_CLIENT_REPO:-${TROVECLIENT_REPO:-${GIT_OPENSTACK}/python-troveclient.git}} +TROVE_CLIENT_DIR=${TROVE_CLIENT_DIR:-${TROVECLIENT_DIR:-${PATH_PYTHON_TROVECLIENT}}} +TROVE_CLIENT_BRANCH=${TROVE_CLIENT_BRANCH:-${TROVECLIENT_BRANCH:-master}} +TROVE_DASHBOARD_REPO=${TROVE_DASHBOARD_REPO:-${TROVEDASHBOARD_REPO:-${GIT_OPENSTACK}/trove-dashboard.git}} +TROVE_DASHBOARD_DIR=${TROVE_DASHBOARD_DIR:-${TROVEDASHBOARD_DIR:-${PATH_TROVE_DASHBOARD}}} +TROVE_DASHBOARD_BRANCH=${TROVE_DASHBOARD_BRANCH:-${TROVEDASHBOARD_BRANCH:-master}} + +# Destination for working data +DATA_DIR=${DEST}/data +# Destination for status files +SERVICE_DIR=${DEST}/status + +# Cinder Volume Group Name +VOLUME_GROUP=${VOLUME_GROUP:-stack-volumes} +VOLUME_BACKING_FILE=${VOLUME_BACKING_FILE:-${DATA_DIR}/${VOLUME_GROUP}-backing-file} +VOLUME_BACKING_FILE_SIZE=${VOLUME_BACKING_FILE_SIZE:-51200M} + +# Passwords used by devstack. +MYSQL_PASSWORD=e1a2c042c828d3566d0a +RABBIT_PASSWORD=f7999d1955c5014aa32c +SERVICE_TOKEN=be19c524ddc92109a224 +ADMIN_PASSWORD=3de4922d8b6ac5a1aad9 +SERVICE_PASSWORD=7de4162d826bc5a11ad9 + +# Swift hash used by devstack. +SWIFT_HASH=12go358snjw24501 + +# Swift Disk Image +SWIFT_DATA_DIR=${DATA_DIR}/swift +SWIFT_DISK_IMAGE=${SWIFT_DATA_DIR}/drives/images/swift.img + +DISTRO=${DISTRO:-ubuntu} +#DISTRO=fedora + +# The following values can be used to tweak how devstack sets +# up Trove. If not explicitly set, the defaults in the code are used. +# To make changes without modifying the repo, add these variables +# to options.rc or ~/redstack.options.rc +#export TROVE_MAX_ACCEPTED_VOLUME_SIZE=10 +#export TROVE_MAX_INSTANCES_PER_TENANT=10 +#export TROVE_MAX_VOLUMES_PER_TENANT=40 +#export TROVE_AGENT_CALL_LOW_TIMEOUT=15 +#export TROVE_AGENT_CALL_HIGH_TIMEOUT=300 +#export TROVE_RESIZE_TIME_OUT=900 +#export TROVE_USAGE_TIMEOUT=1500 +#export TROVE_STATE_CHANGE_WAIT_TIME=180 diff --git a/integration/scripts/reviews.rc b/integration/scripts/reviews.rc new file mode 100644 index 0000000000..accccdcdc3 --- /dev/null +++ b/integration/scripts/reviews.rc @@ -0,0 +1,5 @@ +# This file will contain variables such as below, uncommented. +# There will be a : separator between multiple inflight reviews +# The path comes from the gerrit review system, and is the only +# unique portion of each gerrit review. +#REVIEW_PYTHON_NOVACLIENT=18/5018/2 diff --git a/integration/tests/examples/README b/integration/tests/examples/README new file mode 100644 index 0000000000..f7ebb0b87c --- /dev/null +++ b/integration/tests/examples/README @@ -0,0 +1,13 @@ +Example Generator + +Settings for the example generator are in examples/local.conf + +After customizing examples/local.conf, run: +./example_gen.sh + +2012-06-12 + - Updated to work with trove + - All XML calls are commented out + - Management calls are also commented out +2012-06-14 + - Renabled XML calls diff --git a/integration/tests/examples/example_gen.sh b/integration/tests/examples/example_gen.sh new file mode 100755 index 0000000000..cbbe153fa3 --- /dev/null +++ b/integration/tests/examples/example_gen.sh @@ -0,0 +1,4 @@ +#!/bin/bash + +python ./examples/example_generation.py ./examples/local.conf + diff --git a/integration/tests/examples/examples/client.py b/integration/tests/examples/examples/client.py new file mode 100644 index 0000000000..e52d1833d6 --- /dev/null +++ b/integration/tests/examples/examples/client.py @@ -0,0 +1,235 @@ +import httplib2 +import json +import os +import re +import sys +import time +from urlparse import urlparse +import xml.dom.minidom + +from proboscis.asserts import * +from troveclient.compat.client import TroveHTTPClient +from troveclient.compat.xml import TroveXmlClient + + +print_req = True + + +class ConfigFile(object): + + def __init__(self, config_file): + if not os.path.exists(config_file): + raise RuntimeError("Could not find Example CONF at %s." % + config_file) + file_contents = open(config_file, "r").read() + try: + config = json.loads(file_contents) + except Exception as exception: + msg = 'Error loading config file "%s".' % config_file + raise RuntimeError(msg, exception) + + self.directory = config.get("directory", None) + if not self.directory.endswith('/'): + self.directory += '/' + self.api_url = config.get("api_url", None) + self.auth_url = config.get("auth_url", None) + self.username = config.get("username", None) + self.password = config.get("password", None) + self.tenant = config.get("tenant", None) + self.replace_host = config.get("replace_host", None) + self.replace_dns_hostname = config.get("replace_dns_hostname", None) + if self.auth_url: + auth_id, tenant_id = self.get_auth_token_id_tenant_id(self.auth_url, + self.username, + self.password) + else: + auth_id = self.tenant + tenant_id = self.tenant + + print "id = %s" % auth_id + self.headers = { + 'X-Auth-Token': str(auth_id) + } + print "tenantID = %s" % tenant_id + self.tenantID = tenant_id + self.dbaas_url = "%s/v1.0/%s" % (self.api_url, self.tenantID) + + +def shorten_url(url): + parsed = urlparse(url) + if parsed.query: + method_url = parsed.path + '?' + parsed.query + else: + method_url = parsed.path + return method_url + + +class SnippetWriter(object): + + def __init__(self, conf): + self.conf = conf + + def _indent_xml(self, my_string): + my_string = my_string.encode("utf-8") + # convert to plain string without indents and spaces + my_re = re.compile('>\s+([^\s])', re.DOTALL) + my_string = myre.sub('>\g<1>', my_string) + my_string = xml.dom.minidom.parseString(my_string).toprettyxml() + # remove line breaks + my_re = re.compile('>\n\s+([^<>\s].*?)\n\s+\g<1>\s+([^\s])', re.DOTALL) + my_string = myre.sub('>\g<1>', my_string) + my_string = xml.dom.minidom.parseString(my_string).toprettyxml() + # remove line breaks + my_re = re.compile('>\n\s+([^<>\s].*?)\n\s+\g<1>= len(sys.argv) - 1: + print('Expected an argument to follow "--test-conf".') + sys.exit() + conf_line = sys.argv[index + 1] + extra_test_conf_lines.append(conf_line) + elif arg[:11] == "--flagfile=": + pass + elif arg[:14] == "--config-file=": + rdl_config_file = arg[14:] + elif arg[:13] == "--nova-flags=": + nova_flag_file = arg[13:] + elif arg.startswith('--hide-elapsed'): + show_elapsed = False + else: + nose_args.append(arg) + index += 1 + + # Many of the test decorators depend on configuration values, so before + # start importing modules we have to load the test config followed by the + # flag files. + from trove.tests.config import CONFIG + + # Find config file. + if not "TEST_CONF" in os.environ: + raise RuntimeError("Please define an environment variable named " + + "TEST_CONF with the location to a conf file.") + file_path = os.path.expanduser(os.environ["TEST_CONF"]) + if not os.path.exists(file_path): + raise RuntimeError("Could not find TEST_CONF at " + file_path + ".") + # Load config file and then any lines we read from the arguments. + CONFIG.load_from_file(file_path) + for line in extra_test_conf_lines: + CONFIG.load_from_line(line) + + if CONFIG.white_box: # If white-box testing, set up the flags. + # Handle loading up RDL's config file madness. + initialize_rdl_config(rdl_config_file) + + # Set up the report, and print out how we're running the tests. + from tests.util import report + from datetime import datetime + report.log("Trove Integration Tests, %s" % datetime.now()) + report.log("Invoked via command: " + str(sys.argv)) + report.log("Groups = " + str(groups)) + report.log("Test conf file = %s" % os.environ["TEST_CONF"]) + if CONFIG.white_box: + report.log("") + report.log("Test config file = %s" % rdl_config_file) + report.log("") + report.log("sys.path:") + for path in sys.path: + report.log("\t%s" % path) + + # Now that all configurations are loaded its time to import everything + test_importer() + + atexit.register(_clean_up) + + c = config.Config(stream=sys.stdout, + env=os.environ, + verbosity=3, + plugins=core.DefaultPluginManager()) + runner = NovaTestRunner(stream=c.stream, + verbosity=c.verbosity, + config=c, + show_elapsed=show_elapsed, + known_bugs=CONFIG.known_bugs) + MAIN_RUNNER = runner + + if repl: + # Turn off the following "feature" of the unittest module in case + # we want to start a REPL. + sys.exit = lambda x: None + + proboscis.TestProgram(argv=nose_args, groups=groups, config=c, + testRunner=MAIN_RUNNER).run_and_exit() + sys.stdout = sys.__stdout__ + sys.stderr = sys.__stderr__ + + +if __name__ == "__main__": + run_main(import_tests) diff --git a/integration/tests/integration/localhost.test.conf b/integration/tests/integration/localhost.test.conf new file mode 100644 index 0000000000..6459935cae --- /dev/null +++ b/integration/tests/integration/localhost.test.conf @@ -0,0 +1,95 @@ +{ + "include-files":["core.test.conf"], + + "fake_mode": true, + "dbaas_url":"http://localhost:8779/v1.0", + "version_url":"http://localhost:8779", + "nova_auth_url":"http://localhost:8779/v1.0/auth", + "trove_auth_url":"http://localhost:8779/v1.0/auth", + "trove_client_insecure":false, + "auth_strategy":"fake", + + "trove_version":"v1.0", + "trove_api_updated":"2012-08-01T00:00:00Z", + + "trove_dns_support":false, + "trove_ip_support":false, + + "nova_client": null, + + + "users": [ + { + "auth_user":"admin", + "auth_key":"password", + "tenant":"admin-1000", + "requirements": { + "is_admin":true, + "services": ["trove"] + } + }, + { + "auth_user":"jsmith", + "auth_key":"password", + "tenant":"2500", + "requirements": { + "is_admin":false, + "services": ["trove"] + } + }, + { + "auth_user":"hub_cap", + "auth_key":"password", + "tenant":"3000", + "requirements": { + "is_admin":false, + "services": ["trove"] + } + } + ], + + "flavors": [ + { + "id": 1, + "name": "m1.tiny", + "ram": 512 + }, + { + "id": 2, + "name": "m1.small", + "ram": 2048 + }, + { + "id": 3, + "name": "m1.medium", + "ram": 4096 + }, + { + "id": 4, + "name": "m1.large", + "ram": 8192 + }, + { + "id": 5, + "name": "m1.xlarge", + "ram": 16384 + }, + { + "id": 6, + "name": "tinier", + "ram": 506 + }, + { + "id": 7, + "name": "m1.rd-tiny", + "ram": 512 + }, + { + "id": 8, + "name": "m1.rd-smaller", + "ram": 768 + } + + ], + "sentinel": null +} diff --git a/integration/tests/integration/run_local.sh b/integration/tests/integration/run_local.sh new file mode 100755 index 0000000000..83d313dc88 --- /dev/null +++ b/integration/tests/integration/run_local.sh @@ -0,0 +1,57 @@ +#!/usr/bin/env bash +# Specify the path to the Trove repo as argument one. +# This script will create a .pid file and report in the current directory. + +set -e +if [ $# -lt 1 ]; then + echo "Please give the path to the Trove repo as argument one." + exit 5 +else + TROVE_PATH=$1 +fi +if [ $# -lt 2 ]; then + echo "Please give the path to the Trove Client as argument two." + exit 5 +else + TROVECLIENT_PATH=$2 +fi +shift; +shift; + + +PID_FILE="`pwd`.pid" + +function start_server() { + pushd $TROVE_PATH + bin/start_server.sh --pid_file=$PID_FILE + popd +} + +function stop_server() { + if [ -f $PID_FILE ]; + then + pushd $TROVE_PATH + bin/stop_server.sh $PID_FILE + popd + else + echo "The pid file did not exist, so not stopping server." + fi +} +function on_error() { + echo "Something went wrong!" + stop_server +} + +trap on_error EXIT # Proceed to trap - END in event of failure. + +TROVE_CLIENT_PATH=$TROVECLIENT_PATH tox -e py26 +start_server +.tox/py26/bin/pip install -U $TROVECLIENT_PATH +PYTHONPATH=$PYTHONPATH:$TROVECLIENT_PATH .tox/py26/bin/python int_tests.py \ + --conf=localhost.test.conf -- $@ +stop_server + + +trap - EXIT +echo "Ran tests successfully. :)" +exit 0 diff --git a/integration/tests/integration/setup.py b/integration/tests/integration/setup.py new file mode 100644 index 0000000000..52216f6157 --- /dev/null +++ b/integration/tests/integration/setup.py @@ -0,0 +1,30 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +import os +from setuptools import setup + + +def read(fname): + return open(os.path.join(os.path.dirname(__file__), fname)).read() + + +setup( + name="Trove Integration Tests", + version="0.0.9.9", + author='OpenStack', + description="Runs integration tests on Ridley.", + license='Apache', + py_modules=[], + packages=['tests'], + scripts=[] +) diff --git a/integration/tests/integration/tests/README b/integration/tests/integration/tests/README new file mode 100644 index 0000000000..05e1db677b --- /dev/null +++ b/integration/tests/integration/tests/README @@ -0,0 +1 @@ +Integration tests. diff --git a/integration/tests/integration/tests/__init__.py b/integration/tests/integration/tests/__init__.py new file mode 100644 index 0000000000..65f633d2a2 --- /dev/null +++ b/integration/tests/integration/tests/__init__.py @@ -0,0 +1,27 @@ +# vim: tabstop=4 shiftwidth=4 softtabstop=4 +# +# Copyright (c) 2011 OpenStack, LLC. +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +""" +:mod:`tests` -- Integration / Functional Tests for Nova +=================================== + +.. automodule:: tests + :platform: Unix + :synopsis: Tests for Nova. +.. moduleauthor:: Nirmal Ranganathan +.. moduleauthor:: Tim Simpson +""" diff --git a/integration/tests/integration/tests/api/__init__.py b/integration/tests/integration/tests/api/__init__.py new file mode 100644 index 0000000000..40d014dd8b --- /dev/null +++ b/integration/tests/integration/tests/api/__init__.py @@ -0,0 +1,13 @@ +# Copyright 2011 OpenStack LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. diff --git a/integration/tests/integration/tests/api/delete_all.py b/integration/tests/integration/tests/api/delete_all.py new file mode 100644 index 0000000000..98c67aba52 --- /dev/null +++ b/integration/tests/integration/tests/api/delete_all.py @@ -0,0 +1,32 @@ +# Copyright 2011 OpenStack LLC. +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +from proboscis import test + +from trove.tests.config import CONFIG +from trove.tests.util import create_dbaas_client +from trove.tests.util.users import Requirements + +GROUP = "dbaas.api.instances.delete" + + +@test(groups=[GROUP]) +def delete_all(): + """Delete every single one.""" + user = CONFIG.users.find_user(Requirements(is_admin=False)) + dbaas = create_dbaas_client(user) + instances = dbaas.instances.list() + for instance in instances: + instance.delete() diff --git a/integration/tests/integration/tests/api/instances_pagination.py b/integration/tests/integration/tests/api/instances_pagination.py new file mode 100644 index 0000000000..a21aadd77c --- /dev/null +++ b/integration/tests/integration/tests/api/instances_pagination.py @@ -0,0 +1,219 @@ + +from proboscis import after_class +from proboscis import before_class +from proboscis import test +from proboscis.asserts import assert_equal +from proboscis.asserts import assert_is_not +from proboscis.asserts import assert_is_none +from proboscis.asserts import assert_true + + +from troveclient.compat import exceptions +from trove.tests.config import CONFIG +from trove.tests.util import create_dbaas_client +from trove.tests.util.users import Requirements + + +class TestBase(object): + + def set_up(self): + """Create a ton of instances.""" + reqs = Requirements(is_admin=False) + self.user = CONFIG.users.find_user(reqs) + self.dbaas = create_dbaas_client(self.user) + + def delete_instances(self): + chunk = 0 + while True: + chunk += 1 + attempts = 0 + instances = self.dbaas.instances.list() + if len(instances) == 0: + break + # Sit around and try to delete this chunk. + while True: + instance_results = [] + attempts += 1 + deleted_count = 0 + for instance in instances: + try: + instance.delete() + result = "[w]" + except exceptions.UnprocessableEntity: + result = "[W]" + except exceptions.NotFound: + result = "[O]" + deleted_count += 1 + except Exception: + result = "[X]" + instance_results.append(result) + print("Chunk %d, attempt %d : %s" + % (chunk, attempts, ",".join(instance_results))) + if deleted_count == len(instances): + break + + def create_instances(self): + self.ids = [] + for index in range(self.max): + name = "multi-%03d" % index + result = self.dbaas.instances.create(name, 1, + {'size': 1}, [], []) + self.ids.append(result.id) + # Sort the list of IDs in order, so we can confirm the lists pagination + # returns is also sorted correctly. + self.ids.sort() + + @staticmethod + def assert_instances_sorted_by_ids(instances): + # Assert that the strings are always increasing. + last_id = "" + for instance in instances: + assert_true(last_id < instance.id) + + def print_list(self, instances): + print("Length = %d" % len(instances)) + print(",".join([instance.id for instance in instances])) + + def test_pagination(self, requested_limit, requested_marker, + expected_length, expected_marker, expected_last_item): + instances = self.dbaas.instances.list(limit=requested_limit, + marker=requested_marker) + marker = instances.next + + self.print_list(instances) + + # Better get as many as we asked for. + assert_equal(len(instances), expected_length) + # The last one should be roughly this one in the list. + assert_equal(instances[-1].id, expected_last_item) + # Because limit < count, the marker must be something. + if expected_marker: + assert_is_not(marker, None) + assert_equal(marker, expected_marker) + else: + assert_is_none(marker) + self.assert_instances_sorted_by_ids(instances) + + +@test(runs_after_groups=["dbaas.guest.shutdown"], + groups=['dbaas.api.instances.pagination']) +class SimpleCreateAndDestroy(TestBase): + """ + It turns out a big part of guaranteeing pagination works is to make sure + we can create a big batch of instances and delete them without problems. + Even in fake mode though its worth it to check this is the case. + """ + + max = 5 + + @before_class + def set_up(self): + """Create a ton of instances.""" + super(SimpleCreateAndDestroy, self).set_up() + self.delete_instances() + + @test + def spin_up(self): + self.create_instances() + + @after_class(always_run=True) + def tear_down(self): + self.delete_instances() + + +@test(runs_after_groups=["dbaas.guest.shutdown"], + groups=['dbaas.api.instances.pagination']) +class InstancePagination50(TestBase): + + max = 50 + + @before_class + def set_up(self): + """Create a ton of instances.""" + super(InstancePagination50, self).set_up() + self.delete_instances() + self.create_instances() + + @after_class(always_run=True) + def tear_down(self): + """Tear down all instances.""" + self.delete_instances() + + @test + def pagination_short(self): + self.test_pagination(requested_limit=10, requested_marker=None, + expected_length=10, expected_marker=self.ids[9], + expected_last_item=self.ids[9]) + + @test + def pagination_default(self): + self.test_pagination(requested_limit=None, requested_marker=None, + expected_length=20, expected_marker=self.ids[19], + expected_last_item=self.ids[19]) + + @test + def pagination_full(self): + self.test_pagination(requested_limit=50, requested_marker=None, + expected_length=20, expected_marker=self.ids[19], + expected_last_item=self.ids[19]) + + +@test(runs_after_groups=["dbaas.guest.shutdown"], + groups=['dbaas.api.instances.pagination']) +class InstancePagination20(TestBase): + + max = 20 + + @before_class + def set_up(self): + """Create a ton of instances.""" + super(InstancePagination20, self).set_up() + self.delete_instances() + self.create_instances() + + @after_class(always_run=True) + def tear_down(self): + """Tear down all instances.""" + self.delete_instances() + + @test + def pagination_short(self): + self.test_pagination(requested_limit=10, requested_marker=None, + expected_length=10, expected_marker=self.ids[9], + expected_last_item=self.ids[9]) + + @test + def pagination_default(self): + self.test_pagination(requested_limit=None, requested_marker=None, + expected_length=20, expected_marker=None, + expected_last_item=self.ids[19]) + + @test + def pagination_full(self): + self.test_pagination(requested_limit=20, requested_marker=None, + expected_length=20, expected_marker=None, + expected_last_item=self.ids[19]) + + @test + def pagination_overkill(self): + self.test_pagination(requested_limit=30, requested_marker=None, + expected_length=20, expected_marker=None, + expected_last_item=self.ids[19]) + + @test + def pagination_last_half(self): + self.test_pagination(requested_limit=10, requested_marker=self.ids[9], + expected_length=10, expected_marker=None, + expected_last_item=self.ids[19]) + + @test + def pagination_third_quarter(self): + self.test_pagination(requested_limit=5, requested_marker=self.ids[9], + expected_length=5, expected_marker=self.ids[14], + expected_last_item=self.ids[14]) + + @test + def pagination_fourth_quarter(self): + self.test_pagination(requested_limit=20, requested_marker=self.ids[14], + expected_length=5, expected_marker=None, + expected_last_item=self.ids[19]) diff --git a/integration/tests/integration/tests/api/instances_quotas.py b/integration/tests/integration/tests/api/instances_quotas.py new file mode 100644 index 0000000000..3a1c2de637 --- /dev/null +++ b/integration/tests/integration/tests/api/instances_quotas.py @@ -0,0 +1,47 @@ +from proboscis import before_class +from proboscis import test +from proboscis.asserts import assert_raises + +from troveclient.compat import exceptions +from trove.tests.config import CONFIG +from trove.tests.util import create_client + + +@test(groups=['dbaas.api.instances.quotas']) +class InstanceQuotas(object): + + created_instances = [] + + @before_class + def setup(self): + self.client = create_client(is_admin=False) + + @test + def test_too_many_instances(self): + self.created_instances = [] + if 'trove_max_instances_per_user' in CONFIG.values: + too_many = CONFIG.values['trove_max_instances_per_user'] + already_there = len(self.client.instances.list()) + flavor = 1 + for i in range(too_many - already_there): + response = self.client.instances.create('too_many_%d' % i, + flavor, + {'size': 1}) + self.created_instances.append(response) + # This one better fail, because we just reached our quota. + assert_raises(exceptions.OverLimit, + self.client.instances.create, + "too_many", flavor, + {'size': 1}) + + @test(runs_after=[test_too_many_instances]) + def delete_excessive_entries(self): + # Delete all the instances called too_many*. + for id in self.created_instances: + while True: + try: + self.client.instances.delete(id) + except exceptions.UnprocessableEntity: + continue + except exceptions.NotFound: + break diff --git a/integration/tests/integration/tests/api/instances_states.py b/integration/tests/integration/tests/api/instances_states.py new file mode 100644 index 0000000000..965d8e7703 --- /dev/null +++ b/integration/tests/integration/tests/api/instances_states.py @@ -0,0 +1,76 @@ +# Copyright 2012 OpenStack LLC. +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + + +GROUP = "dbaas.api.instances.status" + +from proboscis import before_class +from proboscis import test +from proboscis.asserts import assert_equal + +from trove.tests.config import CONFIG +from trove.tests.util import create_dbaas_client +from trove.tests.util.users import Requirements +from trove.common.utils import poll_until + + +@test(groups=[GROUP]) +class InstanceStatusTests(object): + + @before_class + def set_up(self): + reqs = Requirements(is_admin=False) + self.user = CONFIG.users.find_user(reqs) + self.dbaas = create_dbaas_client(self.user) + + @test + def test_create_failure_on_volume_prov_failure(self): + # Fake nova will fail a volume of size 9. + response = self.dbaas.instances.create('volume_fail', 1, + {'size': 9}, []) + poll_until(lambda: self.dbaas.instances.get(response.id), + lambda instance: instance.status == 'ERROR', + time_out=10) + instance = self.dbaas.instances.get(response.id) + print "Status: %s" % instance.status + assert_equal(instance.status, "ERROR", + "Instance did not drop to error after volume prov failure.") + + @test + def test_create_failure_on_server_failure(self): + # Fake nova will fail a server ending with 'SERVER_ERROR'." + response = self.dbaas.instances.create('test_SERVER_ERROR', 1, + {'size': 1}, []) + poll_until(lambda: self.dbaas.instances.get(response.id), + lambda instance: instance.status == 'ERROR', + time_out=10) + instance = self.dbaas.instances.get(response.id) + print "Status: %s" % instance.status + assert_equal(instance.status, "ERROR", + "Instance did not drop to error after server prov failure.") + + ###TODO(ed-): We don't at present have a way to test DNS in FAKE_MODE. + @test(enabled=False) + def test_create_failure_on_dns_failure(self): + #TODO(ed-): Throw DNS-specific monkeywrench into works + response = self.dbaas.instances.create('test_DNS_ERROR', 1, + {'size': 1}, []) + poll_until(lambda: self.dbaas.instances.get(response.id), + lambda instance: instance.status == 'ERROR', + time_out=10) + instance = self.dbaas.instances.get(response.id) + print "Status: %s" % instance.status + assert_equal(instance.status, "ERROR", + "Instance did not drop to error after DNS prov failure.") diff --git a/integration/tests/integration/tests/colorizer.py b/integration/tests/integration/tests/colorizer.py new file mode 100644 index 0000000000..4dd797dc18 --- /dev/null +++ b/integration/tests/integration/tests/colorizer.py @@ -0,0 +1,446 @@ +#!/usr/bin/env python +# vim: tabstop=4 shiftwidth=4 softtabstop=4 + +# Copyright 2010 United States Government as represented by the +# Administrator of the National Aeronautics and Space Administration. +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Colorizer Code is borrowed from Twisted: +# Copyright (c) 2001-2010 Twisted Matrix Laboratories. +# +# Permission is hereby granted, free of charge, to any person obtaining +# a copy of this software and associated documentation files (the +# "Software"), to deal in the Software without restriction, including +# without limitation the rights to use, copy, modify, merge, publish, +# distribute, sublicense, and/or sell copies of the Software, and to +# permit persons to whom the Software is furnished to do so, subject to +# the following conditions: +# +# The above copyright notice and this permission notice shall be +# included in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +"""Unittest runner for Nova. + +To run all tests + python run_tests.py + +To run a single test: + python run_tests.py test_compute:ComputeTestCase.test_run_terminate + +To run a single test module: + python run_tests.py test_compute + + or + + python run_tests.py api.test_wsgi + +""" + +import gettext +import heapq +import logging +import os +import unittest +import sys +import time + +gettext.install('nova', unicode=1) + +from nose import config +from nose import core +from nose import result +from proboscis import case +from proboscis import SkipTest + +class _AnsiColorizer(object): + """ + A colorizer is an object that loosely wraps around a stream, allowing + callers to write text to the stream in a particular color. + + Colorizer classes must implement C{supported()} and C{write(text, color)}. + """ + _colors = dict(black=30, red=31, green=32, yellow=33, + blue=34, magenta=35, cyan=36, white=37) + + def __init__(self, stream): + self.stream = stream + + def supported(cls, stream=sys.stdout): + """ + A class method that returns True if the current platform supports + coloring terminal output using this method. Returns False otherwise. + """ + if not stream.isatty(): + return False # auto color only on TTYs + try: + import curses + except ImportError: + return False + else: + try: + try: + return curses.tigetnum("colors") > 2 + except curses.error: + curses.setupterm() + return curses.tigetnum("colors") > 2 + except: + raise + # guess false in case of error + return False + supported = classmethod(supported) + + def write(self, text, color): + """ + Write the given text to the stream in the given color. + + @param text: Text to be written to the stream. + + @param color: A string label for a color. e.g. 'red', 'white'. + """ + color = self._colors[color] + self.stream.write('\x1b[%s;1m%s\x1b[0m' % (color, text)) + + +class _Win32Colorizer(object): + """ + See _AnsiColorizer docstring. + """ + def __init__(self, stream): + from win32console import GetStdHandle, STD_OUT_HANDLE, \ + FOREGROUND_RED, FOREGROUND_BLUE, FOREGROUND_GREEN, \ + FOREGROUND_INTENSITY + red, green, blue, bold = (FOREGROUND_RED, FOREGROUND_GREEN, + FOREGROUND_BLUE, FOREGROUND_INTENSITY) + self.stream = stream + self.screenBuffer = GetStdHandle(STD_OUT_HANDLE) + self._colors = { + 'normal': red | green | blue, + 'red': red | bold, + 'green': green | bold, + 'blue': blue | bold, + 'yellow': red | green | bold, + 'magenta': red | blue | bold, + 'cyan': green | blue | bold, + 'white': red | green | blue | bold + } + + def supported(cls, stream=sys.stdout): + try: + import win32console + screenBuffer = win32console.GetStdHandle( + win32console.STD_OUT_HANDLE) + except ImportError: + return False + import pywintypes + try: + screenBuffer.SetConsoleTextAttribute( + win32console.FOREGROUND_RED | + win32console.FOREGROUND_GREEN | + win32console.FOREGROUND_BLUE) + except pywintypes.error: + return False + else: + return True + supported = classmethod(supported) + + def write(self, text, color): + color = self._colors[color] + self.screenBuffer.SetConsoleTextAttribute(color) + self.stream.write(text) + self.screenBuffer.SetConsoleTextAttribute(self._colors['normal']) + + +class _NullColorizer(object): + """ + See _AnsiColorizer docstring. + """ + def __init__(self, stream): + self.stream = stream + + def supported(cls, stream=sys.stdout): + return True + supported = classmethod(supported) + + def write(self, text, color): + self.stream.write(text) + + +def get_elapsed_time_color(elapsed_time): + if elapsed_time > 1.0: + return 'yellow' + elif elapsed_time > 0.25: + return 'cyan' + else: + return 'green' + + +class NovaTestResult(case.TestResult): + def __init__(self, *args, **kw): + self.show_elapsed = kw.pop('show_elapsed') + self.known_bugs = kw.pop('known_bugs', {}) + super(NovaTestResult, self).__init__(*args, **kw) + self.num_slow_tests = 5 + self.slow_tests = [] # this is a fixed-sized heap + self._last_case = None + self.colorizer = None + # NOTE(vish): reset stdout for the terminal check + stdout = sys.stdout + sys.stdout = sys.__stdout__ + for colorizer in [_Win32Colorizer, _AnsiColorizer, _NullColorizer]: + if colorizer.supported(): + self.colorizer = colorizer(self.stream) + break + sys.stdout = stdout + + # NOTE(lorinh): Initialize start_time in case a sqlalchemy-migrate + # error results in it failing to be initialized later. Otherwise, + # _handleElapsedTime will fail, causing the wrong error message to + # be outputted. + self.start_time = time.time() + + def _intercept_known_bugs(self, test, err): + name = str(test) + excuse = self.known_bugs.get(name, None) + if excuse: + tracker_id, error_string = excuse + if error_string in str(err[1]): + skip = SkipTest("KNOWN BUG: %s\n%s" + % (tracker_id, str(err[1]))) + self.onError(test) + super(NovaTestResult, self).addSkip(test, skip) + else: + result = (RuntimeError, RuntimeError( + 'Test "%s" contains known bug %s.\n' + 'Expected the following error string:\n%s\n' + 'What was seen was the following:\n%s\n' + 'If the bug is no longer happening, please change ' + 'the test config.' + % (name, tracker_id, error_string, str(err))), None) + self.onError(test) + super(NovaTestResult, self).addError(test, result) + return True + return False + + def getDescription(self, test): + return str(test) + + def _handleElapsedTime(self, test): + self.elapsed_time = time.time() - self.start_time + item = (self.elapsed_time, test) + # Record only the n-slowest tests using heap + if len(self.slow_tests) >= self.num_slow_tests: + heapq.heappushpop(self.slow_tests, item) + else: + heapq.heappush(self.slow_tests, item) + + def _writeElapsedTime(self, test): + color = get_elapsed_time_color(self.elapsed_time) + self.colorizer.write(" %.2f" % self.elapsed_time, color) + + def _writeResult(self, test, long_result, color, short_result, success): + if self.showAll: + self.colorizer.write(long_result, color) + if self.show_elapsed and success: + self._writeElapsedTime(test) + self.stream.writeln() + elif self.dots: + self.stream.write(short_result) + self.stream.flush() + + # NOTE(vish): copied from unittest with edit to add color + def addSuccess(self, test): + if self._intercept_known_bugs(test, None): + return + unittest.TestResult.addSuccess(self, test) + self._handleElapsedTime(test) + self._writeResult(test, 'OK', 'green', '.', True) + + # NOTE(vish): copied from unittest with edit to add color + def addFailure(self, test, err): + if self._intercept_known_bugs(test, err): + return + self.onError(test) + unittest.TestResult.addFailure(self, test, err) + self._handleElapsedTime(test) + self._writeResult(test, 'FAIL', 'red', 'F', False) + + # NOTE(vish): copied from nose with edit to add color + def addError(self, test, err): + """Overrides normal addError to add support for + errorClasses. If the exception is a registered class, the + error will be added to the list for that class, not errors. + """ + if self._intercept_known_bugs(test, err): + return + self.onError(test) + self._handleElapsedTime(test) + stream = getattr(self, 'stream', None) + ec, ev, tb = err + try: + exc_info = self._exc_info_to_string(err, test) + except TypeError: + # 2.3 compat + exc_info = self._exc_info_to_string(err) + for cls, (storage, label, isfail) in self.errorClasses.items(): + if result.isclass(ec) and issubclass(ec, cls): + if isfail: + test.passed = False + storage.append((test, exc_info)) + # Might get patched into a streamless result + if stream is not None: + if self.showAll: + message = [label] + detail = result._exception_detail(err[1]) + if detail: + message.append(detail) + stream.writeln(": ".join(message)) + elif self.dots: + stream.write(label[:1]) + return + self.errors.append((test, exc_info)) + test.passed = False + if stream is not None: + self._writeResult(test, 'ERROR', 'red', 'E', False) + + @staticmethod + def get_doc(cls_or_func): + """Grabs the doc abbreviated doc string.""" + try: + return cls_or_func.__doc__.split("\n")[0].strip() + except (AttributeError, IndexError): + return None + + def startTest(self, test): + unittest.TestResult.startTest(self, test) + self.start_time = time.time() + test_name = None + try: + entry = test.test.__proboscis_case__.entry + if entry.method: + current_class = entry.method.im_class + test_name = self.get_doc(entry.home) or entry.home.__name__ + else: + current_class = entry.home + except AttributeError: + current_class = test.test.__class__ + + if self.showAll: + if current_class.__name__ != self._last_case: + self.stream.writeln(current_class.__name__) + self._last_case = current_class.__name__ + try: + doc = self.get_doc(current_class) + except (AttributeError, IndexError): + doc = None + if doc: + self.stream.writeln(' ' + doc) + + if not test_name: + if hasattr(test.test, 'shortDescription'): + test_name = test.test.shortDescription() + if not test_name: + test_name = test.test._testMethodName + self.stream.write('\t%s' % str(test_name).ljust(60)) + self.stream.flush() + + +class NovaTestRunner(core.TextTestRunner): + def __init__(self, *args, **kwargs): + self.show_elapsed = kwargs.pop('show_elapsed') + self.known_bugs = kwargs.pop('known_bugs', {}) + self.__result = None + self.__finished = False + self.__start_time = None + super(NovaTestRunner, self).__init__(*args, **kwargs) + + def _makeResult(self): + self.__result = NovaTestResult( + self.stream, + self.descriptions, + self.verbosity, + self.config, + show_elapsed=self.show_elapsed, + known_bugs=self.known_bugs) + self.__start_time = time.time() + return self.__result + + def _writeSlowTests(self, result_): + # Pare out 'fast' tests + slow_tests = [item for item in result_.slow_tests + if get_elapsed_time_color(item[0]) != 'green'] + if slow_tests: + slow_total_time = sum(item[0] for item in slow_tests) + self.stream.writeln("Slowest %i tests took %.2f secs:" + % (len(slow_tests), slow_total_time)) + for elapsed_time, test in sorted(slow_tests, reverse=True): + time_str = "%.2f" % elapsed_time + self.stream.writeln(" %s %s" % (time_str.ljust(10), test)) + + def on_exit(self): + if self.__result is None: + print("Exiting before tests even started.") + else: + if not self.__finished: + msg = "Tests aborted, trying to print available results..." + print(msg) + stop_time = time.time() + self.__result.printErrors() + self.__result.printSummary(self.__start_time, stop_time) + self.config.plugins.finalize(self.__result) + if self.show_elapsed: + self._writeSlowTests(self.__result) + + def run(self, test): + result_ = super(NovaTestRunner, self).run(test) + if self.show_elapsed: + self._writeSlowTests(result_) + self.__finished = True + return result_ + + +if __name__ == '__main__': + logging.setup() + # If any argument looks like a test name but doesn't have "nova.tests" in + # front of it, automatically add that so we don't have to type as much + show_elapsed = True + argv = [] + test_fixture = os.getenv("UNITTEST_FIXTURE", "trove") + for x in sys.argv: + if x.startswith('test_'): + argv.append('%s.tests.%s' % (test_fixture, x)) + elif x.startswith('--hide-elapsed'): + show_elapsed = False + else: + argv.append(x) + + testdir = os.path.abspath(os.path.join(test_fixture, "tests")) + c = config.Config(stream=sys.stdout, + env=os.environ, + verbosity=3, + workingDir=testdir, + plugins=core.DefaultPluginManager()) + + runner = NovaTestRunner(stream=c.stream, + verbosity=c.verbosity, + config=c, + show_elapsed=show_elapsed) + sys.exit(not core.run(config=c, testRunner=runner, argv=argv)) diff --git a/integration/tests/integration/tests/dns/__init__.py b/integration/tests/integration/tests/dns/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/integration/tests/integration/tests/dns/check_domain.py b/integration/tests/integration/tests/dns/check_domain.py new file mode 100644 index 0000000000..61e5f6384a --- /dev/null +++ b/integration/tests/integration/tests/dns/check_domain.py @@ -0,0 +1,174 @@ +# Copyright 2011 OpenStack LLC. +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +"""Checks that the domain specified in the flag file exists and is valid. + +If you define the environment variable ADD_DOMAINS=True when running the tests, +they will create the domain if its not found (see below for details). + +""" +import os +import time +import unittest +from proboscis import test +from proboscis import before_class +from proboscis.asserts import assert_equal +from proboscis.asserts import assert_not_equal +from proboscis.decorators import expect_exception +from proboscis.decorators import time_out + +from trove.tests.config import CONFIG + +WHITE_BOX = CONFIG.white_box +RUN_DNS = CONFIG.values.get("trove_dns_support", False) + +if WHITE_BOX: + from nova import utils + from nova import flags + import rsdns + from trove.dns.rsdns.driver import create_client_with_flag_values + from trove.dns.driver import DnsEntry + from trove.dns.rsdns.driver import RsDnsInstanceEntryFactory + from trove.dns.rsdns.driver import RsDnsDriver + from trove.dns.rsdns.driver import RsDnsZone + from trove.utils import poll_until + FLAGS = flags.FLAGS + TEST_CONTENT = "126.1.1.1" + TEST_NAME = "hiwassup.%s" % FLAGS.dns_domain_name + DNS_DOMAIN_ID = None + + +@test(groups=["rsdns.domains", "rsdns.show_entries"], + enabled=WHITE_BOX and RUN_DNS) +class ClientTests(object): + + @before_class + def increase_logging(self): + import httplib2 + httplib2.debuglevel = 1 + + @test + def can_auth(self): + self.client = create_client_with_flag_values() + self.client.authenticate() + + @test(depends_on=[can_auth]) + def list_domains(self): + domains = self.client.domains.list() + print(domains) + + +@test(groups=["rsdns.domains"], depends_on=[ClientTests], + enabled=WHITE_BOX and RUN_DNS) +class RsDnsDriverTests(object): + """Tests the RS DNS Driver.""" + + def create_domain_if_needed(self): + """Adds the domain specified in the flags.""" + print("Creating domain %s" % self.driver.default_dns_zone.name) + future = self.driver.dns_client.domains.create( + self.driver.default_dns_zone.name) + while not future.ready: + time.sleep(2) + print("Got something: %s" % future.resource) + with open('/home/vagrant/dns_resource.txt', 'w') as f: + f.write('%r\n' % future.result[0].id) + global DNS_DOMAIN_ID + DNS_DOMAIN_ID = future.result[0].id + print("The domain should have been created with id=%s" % DNS_DOMAIN_ID) + + @test + @time_out(2 * 60) + def ensure_domain_specified_in_flags_exists(self): + """Make sure the domain in the FLAGS exists.""" + self.driver = RsDnsDriver(raise_if_zone_missing=False) + assert_not_equal(None, self.driver.default_dns_zone) + + def zone_found(): + zones = self.driver.get_dns_zones() + print("Retrieving zones.") + for zone in zones: + print("zone %s" % zone) + if zone.name == self.driver.default_dns_zone.name: + self.driver.default_dns_zone.id = zone.id + global DNS_DOMAIN_ID + DNS_DOMAIN_ID = zone.id + return True + return False + if zone_found(): + return + self.create_domain_if_needed() + for i in range(5): + if zone_found(): + return + self.fail("""Could not find default dns zone. + This happens when they clear the staging DNS service of data. + To fix it, manually run the tests as follows: + $ ADD_DOMAINS=True python int_tests.py + and if all goes well the tests will create a new domain + record.""") + + @test(depends_on=[ensure_domain_specified_in_flags_exists], + enabled=WHITE_BOX and FLAGS.dns_domain_name != "dbaas.rackspace.com") + def delete_all_entries(self): + """Deletes all entries under the default domain.""" + list = self.driver.get_entries() + for entry in list: + if entry.type == "A": + self.driver.delete_entry(name=entry.name, type=entry.type, + dns_zone=entry.dns_zone) + # It takes awhile for them to be deleted. + poll_until(lambda: self.driver.get_entries_by_name(TEST_NAME), + lambda list: len(list) == 0, + sleep_time=4, time_out=60) + + @test(depends_on=[delete_all_entries]) + def create_test_entry(self): + fullname = TEST_NAME + entry = DnsEntry(name=fullname, content=TEST_CONTENT, type="A", + ttl=3600) + self.driver.create_entry(entry) + list = None + for i in range(500): + list = self.driver.get_entries_by_name(name=fullname) + if len(list) > 0: + break + time.sleep(1) + print("This is the list: %r" % list) + assert_equal(1, len(list)) + list2 = self.driver.get_entries_by_content(content=TEST_CONTENT) + assert_equal(1, len(list2)) + + @test(depends_on=[delete_all_entries]) + def create_test_rsdns_entry(self): + """Create an entry using the RsDnsInstanceEntryFactory.""" + instance = {'uuid': '000136c0-effa-4711-a747-a5b9fbfcb3bd', 'id': '10'} + ip = "10.100.2.7" + factory = RsDnsInstanceEntryFactory(dns_domain_id=DNS_DOMAIN_ID) + entry = factory.create_entry(instance) + entry.content = ip + self.driver.create_entry(entry) + entries = self.driver.get_entries_by_name(name=entry.name) + assert_equal(1, len(entries)) + assert_equal(ip, entries[0].content) + assert_equal(FLAGS.dns_ttl, entries[0].ttl) + + @test(depends_on=[create_test_entry]) + def delete_test_entry(self): + fullname = TEST_NAME + self.driver.delete_entry(fullname, "A") + # It takes awhile for them to be deleted. + poll_until(lambda: self.driver.get_entries_by_name(TEST_NAME), + lambda list: len(list) == 0, + sleep_time=2, time_out=60) diff --git a/integration/tests/integration/tests/dns/concurrency.py b/integration/tests/integration/tests/dns/concurrency.py new file mode 100644 index 0000000000..4fe460b049 --- /dev/null +++ b/integration/tests/integration/tests/dns/concurrency.py @@ -0,0 +1,111 @@ +# Copyright 2011 OpenStack LLC. +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +""" +This test recreates an issue we had with eventlet. In the logs, we'd see that +the JSON response was malformed; instead of JSON, it contained the following +string: +Second simultaneous read on fileno 5 detected. Unless you really know what +you're doing, make sure that only one greenthread can read any particular +socket. Consider using a pools.Pool. If you do know what you're doing and want +to disable this error, call +eventlet.debug.hub_multiple_reader_prevention(False) + +It is perhaps the most helpful error message ever created. + +The root issue was that a subclass of httplib2.Http was created at program +started and used in all threads. + +Using the old (broken) RsDNS client code this test recreates the greatest error +message ever. +""" + +try: + import eventlet + CAN_USE_EVENTLET = True +except ImportError: + CAN_USE_EVENTLET = False +import uuid + +from proboscis import before_class +from proboscis import test +from proboscis.asserts import assert_true + +from trove.tests.config import CONFIG + +WHITE_BOX = CONFIG.white_box +RUN_DNS = CONFIG.values.get("trove_dns_support", False) + + +if CONFIG.white_box: + from trove.dns.rsdns.driver import RsDnsInstanceEntryFactory + from nova import flags + from nova import utils + FLAGS = flags.FLAGS + + +@test(groups=["rsdns.eventlet"], enabled=CAN_USE_EVENTLET) +class RsdnsEventletTests(object): + """Makes sure the RSDNS client can be used from multiple green threads.""" + + def assert_record_created(self, index): + msg = "Record %d wasn't created!" % index + assert_true(index in self.new_records, msg) + + @before_class(enabled=WHITE_BOX and RUN_DNS) + def create_driver(self): + """Creates the DNS Driver used in subsequent tests.""" + self.driver = utils.import_object(FLAGS.dns_driver) + self.entry_factory = RsDnsInstanceEntryFactory() + self.test_uuid = uuid.uuid4().hex + self.new_records = {} + + def make_record(self, index): + """Creates a record with the form 'eventlet-%s-%d'.""" + uuid = "eventlet-%s-%d" % (self.test_uuid, index) + instance = {'uuid': uuid} + entry = self.entry_factory.create_entry(instance) + entry.name = uuid + "." + self.entry_factory.default_dns_zone.name + entry.content = "123.123.123.123" + self.driver.create_entry(entry) + self.new_records[index] = True + + @test(enabled=WHITE_BOX and RUN_DNS) + def use_dns_from_a_single_thread(self): + """Add DNS records one at a time.""" + self.new_records = {} + for index in range(-1, -5, -1): + self.make_record(index) + self.assert_record_created(index) + + @test(enabled=WHITE_BOX and RUN_DNS) + def use_dns_from_multiple_greenthreads(self): + """Add multiple DNS records at once.""" + self.new_records = {} + + def make_record(index): + def __cb(): + self.make_record(index) + self.assert_record_created(index) + return index + return __cb + + pile = eventlet.GreenPile() + indices = range(1, 4) + for index in indices: + pile.spawn(make_record(index)) + + list(pile) # Wait for them to finish + for index in indices: + self.assert_record_created(index) diff --git a/integration/tests/integration/tests/dns/conversion.py b/integration/tests/integration/tests/dns/conversion.py new file mode 100644 index 0000000000..2af3b959ad --- /dev/null +++ b/integration/tests/integration/tests/dns/conversion.py @@ -0,0 +1,105 @@ +# Copyright 2011 OpenStack LLC. +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +"""Tests classes which convert RS style-entries to Nova DNS entries.""" + +import hashlib +import re +import unittest +from proboscis import test +from proboscis.decorators import expect_exception + +from trove.tests.config import CONFIG + + +if CONFIG.white_box: + from nova import flags + from rsdns.client.records import Record + from trove.dns.rsdns.driver import EntryToRecordConverter + from trove.dns.rsdns.driver import RsDnsInstanceEntryFactory + from trove.dns.rsdns.driver import RsDnsZone + FLAGS = flags.FLAGS + driver = None + DEFAULT_ZONE = RsDnsZone(1, "dbaas.rackspace.org") + TEST_CONTENT = "126.1.1.1" + TEST_NAME = "hiwassup.dbaas.rackspace.org" + + +@test(groups=["unit", "rsdns.conversion"], + enabled=CONFIG.white_box) +class ConvertingNovaEntryNamesToRecordNames(unittest.TestCase): + + def setUp(self): + self.converter = EntryToRecordConverter(DEFAULT_ZONE) + self.fake_zone = RsDnsZone(id=5, name="blah.org") + + def test_normal_name(self): + long_name = self.converter.name_to_long_name("hi", self.fake_zone) + self.assertEqual("hi.blah.org", long_name) + + def test_short_name(self): + long_name = self.converter.name_to_long_name("", self.fake_zone) + self.assertEqual("", long_name) + + def test_long_name(self): + long_name = self.converter.name_to_long_name("blah.org.", + self.fake_zone) + self.assertEqual("blah.org..blah.org", long_name) + + +@test(groups=["unit", "rsdns.conversion"], + enabled=CONFIG.white_box) +class ConvertingRecordsToEntries(unittest.TestCase): + + def setUp(self): + self.converter = EntryToRecordConverter(DEFAULT_ZONE) + self.fake_zone = RsDnsZone(id=5, name="blah.org") + + def test_normal_name(self): + record = Record(None, {"id": 5, "name": "hi.blah.org", + "data": "stacker.com blah@blah 13452378", + "ttl": 5, + "type": "SOA"}) + entry = self.converter.record_to_entry(record=record, + dns_zone=self.fake_zone) + self.assertEqual("stacker.com blah@blah 13452378", entry.content) + self.assertEqual("hi.blah.org", entry.name) + self.assertEqual("5", str(entry.ttl)) + self.assertEqual("SOA", entry.type) + + +@test(groups=["rsdns.conversion"], + enabled=CONFIG.white_box) +class WhenCreatingAnEntryForAnInstance(unittest.TestCase): + # This isn't a unit test because RsDnsInstanceEntryFactory connects to the + # service. + + def setUp(self): + self.creator = RsDnsInstanceEntryFactory() + + def test_should_concatanate_strings(self): + instance = {'id': '56', + 'uuid': '000136c0-effa-4711-a747-a5b9fbfcb3bd'} + entry = self.creator.create_entry(instance) + expected_name = "%s.%s" % (hashlib.sha1(instance['uuid']).hexdigest(), + FLAGS.dns_domain_name) + self.assertEqual(expected_name, entry.name, + msg="Entry name should match - %s" % entry.name) + self.assertIsNone(entry.content) + self.assertEqual("A", entry.type) + self.assertEqual(FLAGS.dns_ttl, entry.ttl) + self.assertIsNone(entry.priority) + self.assertEqual(FLAGS.dns_domain_name, entry.dns_zone.name) + if not entry.dns_zone.id: + self.fail(msg="DNS Zone Id should not be empty") diff --git a/integration/tests/integration/tests/dns/dns.py b/integration/tests/integration/tests/dns/dns.py new file mode 100644 index 0000000000..3734ad5475 --- /dev/null +++ b/integration/tests/integration/tests/dns/dns.py @@ -0,0 +1,104 @@ + +import unittest + +from proboscis import test + +from trove.tests.api.instances import instance_info +from trove.tests.api.instances import GROUP_START as INSTANCE_START +from trove.tests.api.instances import GROUP_TEST +from trove.tests.api.instances import GROUP_STOP as INSTANCE_STOP +from trove.tests.config import CONFIG +from trove.common.utils import import_object +from trove.common.utils import poll_until + +WHITE_BOX = CONFIG.white_box + +if WHITE_BOX: + # TODO(tim.simpson): Restore this once white box functionality can be + # added back to this test module. + pass + # import rsdns + # from nova import flags + # from nova import utils + + # from trove import exception + # from trove.utils import poll_until + + # FLAGS = flags.FLAGS + +dns_driver = None + +GROUP = "dbaas.guest.dns" + + +@test(groups=[GROUP, GROUP_TEST]) +class Setup(unittest.TestCase): + """Creates the DNS Driver and entry factory used in subsequent tests.""" + + def test_create_rs_dns_driver(self): + global dns_driver + dns_driver = import_object(FLAGS.dns_driver) + + +def expected_dns_entry(): + """Returns expected DNS entry for this instance. + + :rtype: Instance of :class:`DnsEntry`. + + """ + return create_dns_entry(instance_info.local_id, instance_info.id) + + +@test(depends_on_classes=[Setup], + depends_on_groups=[INSTANCE_START], + groups=[GROUP, GROUP_TEST]) +class WhenInstanceIsCreated(unittest.TestCase): + """Make sure the DNS name was provisioned. + + This class actually calls the DNS driver to confirm the entry that should + exist for the given instance does exist. + + """ + + def test_dns_entry_should_exist(self): + entry = expected_dns_entry() + if entry: + def get_entries(): + return dns_driver.get_entries_by_name(entry.name) + try: + poll_until(get_entries, lambda entries: len(entries) > 0, + sleep_time=2, time_out=60) + except exception.PollTimeOut: + self.fail("Did not find name " + entry.name + \ + " in the entries, which were as follows:" + + str(dns_driver.get_entries())) + + +@test(depends_on_classes=[Setup, WhenInstanceIsCreated], + depends_on_groups=[INSTANCE_STOP], + groups=[GROUP]) +class AfterInstanceIsDestroyed(unittest.TestCase): + """Make sure the DNS name is removed along with an instance. + + Because the compute manager calls the DNS manager with RPC cast, it can + take awhile. So we wait for 30 seconds for it to disappear. + + """ + + def test_dns_entry_exist_should_be_removed_shortly_thereafter(self): + entry = expected_dns_entry() + + if not entry: + return + + def get_entries(): + return dns_driver.get_entries_by_name(entry.name) + + try: + poll_until(get_entries, lambda entries: len(entries) == 0, + sleep_time=2, time_out=60) + except exception.PollTimeOut: + # Manually delete the rogue item + dns_driver.delete_entry(entry.name, entry.type, entry.dns_zone) + self.fail("The DNS entry was never deleted when the instance " + "was destroyed.") diff --git a/integration/tests/integration/tests/initialize.py b/integration/tests/integration/tests/initialize.py new file mode 100644 index 0000000000..ddd5aa86d1 --- /dev/null +++ b/integration/tests/integration/tests/initialize.py @@ -0,0 +1,176 @@ +# Copyright (c) 2011 OpenStack, LLC. +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +import unittest +import os +import time +import socket + +from nose.plugins.skip import SkipTest + +from proboscis import test +from proboscis.asserts import fail +from proboscis.decorators import time_out +from tests.util.services import Service +from tests.util.services import start_proc +from tests.util.services import WebService +from trove.tests.config import CONFIG + + +FAKE = CONFIG.fake_mode +START_SERVICES = (not FAKE) and CONFIG.values.get('start_services', False) +START_NOVA_NETWORK = (START_SERVICES and + not CONFIG.values.get('neutron_enabled', + False)) +KEYSTONE_ALL = CONFIG.values.get('keystone_use_combined', True) +USE_NOVA_VOLUME = CONFIG.values.get('use_nova_volume', False) + +dbaas_image = None +instance_name = None +success_statuses = ["build", "active"] + + +def dbaas_url(): + return str(CONFIG.values.get("dbaas_url")) + +def nova_url(): + return str(CONFIG.values.get("nova_client")['url']) + + + +class Daemon(object): + """Starts a daemon.""" + + def __init__(self, alternate_path=None, conf_file_name=None, + extra_cmds=None, service_path_root=None, service_path=None): + # The path to the daemon bin if the other one doesn't work. + self.alternate_path = alternate_path + self.extra_cmds = extra_cmds or [] + # The name of a test config value which points to a conf file. + self.conf_file_name = conf_file_name + # The name of a test config value, which is inserted into the service_path. + self.service_path_root = service_path_root + # The first path to the daemon bin we try. + self.service_path = service_path or "%s" + + def run(self): + # Print out everything to make it + print("Looking for config value %s..." % self.service_path_root) + print(CONFIG.values[self.service_path_root]) + path = self.service_path % CONFIG.values[self.service_path_root] + print("Path = %s" % path) + if not os.path.exists(path): + path = self.alternate_path + if path is None: + fail("Could not find path to %s" % self.service_path_root) + conf_path = str(CONFIG.values[self.conf_file_name]) + cmds = CONFIG.python_cmd_list() + [path] + self.extra_cmds + \ + [conf_path] + print("Running cmds: %s" % cmds) + self.service = Service(cmds) + if not self.service.is_service_alive(): + self.service.start() + +@test(groups=["services.initialize"], + enabled=START_SERVICES and (not KEYSTONE_ALL)) +def start_keystone_all(): + """Starts the Keystone API.""" + Daemon(service_path_root="usr_bin_dir", + service_path="%s/keystone-all", + extra_cmds=['--config-file'], + conf_file_name="keystone_conf").run() + + +@test(groups=["services.initialize", "services.initialize.glance"], + enabled=START_SERVICES) +def start_glance_registry(): + """Starts the Glance Registry.""" + Daemon(alternate_path="/usr/bin/glance-registry", + conf_file_name="glance_reg_conf", + service_path_root="usr_bin_dir", + service_path="%s/glance-registry").run() + + +@test(groups=["services.initialize", "services.initialize.glance"], + depends_on=[start_glance_registry], enabled=START_SERVICES) +def start_glance_api(): + """Starts the Glance API.""" + Daemon(alternate_path="/usr/bin/glance-api", + conf_file_name="glance_reg_conf", + service_path_root="usr_bin_dir", + service_path="%s/glance-api").run() + + +@test(groups=["services.initialize"], depends_on_classes=[start_glance_api], + enabled=START_NOVA_NETWORK) +def start_nova_network(): + """Starts the Nova Network Service.""" + Daemon(service_path_root="usr_bin_dir", + service_path="%s/nova-network", + extra_cmds=['--config-file='], + conf_file_name="nova_conf").run() + + +@test(groups=["services.initialize"], enabled=START_SERVICES) +def start_scheduler(): + """Starts the Scheduler Service.""" + Daemon(service_path_root="usr_bin_dir", + service_path="%s/nova-scheduler", + extra_cmds=['--config-file='], + conf_file_name="nova_conf").run() + + +@test(groups=["services.initialize"], + depends_on_classes=[start_glance_api], + enabled=START_SERVICES) +def start_compute(): + """Starts the Nova Compute Service.""" + Daemon(service_path_root="usr_bin_dir", + service_path="%s/nova-compute", + extra_cmds=['--config-file='], + conf_file_name="nova_conf").run() + + +@test(groups=["services.initialize"], depends_on_classes=[start_scheduler], + enabled=START_SERVICES and USE_NOVA_VOLUME) +def start_volume(): + """Starts the Nova Compute Service.""" + Daemon(service_path_root="usr_bin_dir", + service_path="%s/nova-volume", + extra_cmds=['--config-file='], + conf_file_name="nova_conf").run() + + +@test(groups=["services.initialize"], + depends_on_classes=[start_glance_api, start_nova_network, start_compute, + start_volume], + enabled=START_SERVICES) +def start_nova_api(): + """Starts the Nova Compute Service.""" + Daemon(service_path_root="usr_bin_dir", + service_path="%s/nova-api", + extra_cmds=['--config-file='], + conf_file_name="nova_conf").run() + + +@test(groups=["services.initialize"], + depends_on_classes=[start_nova_api], + enabled=START_SERVICES) +def start_trove_api(): + """Starts the Trove Service.""" + Daemon(service_path_root="usr_bin_dir", + service_path="%s/trove-api", + extra_cmds=['--config-file='], + conf_file_name="trove_conf").run() diff --git a/integration/tests/integration/tests/smoke/__init__.py b/integration/tests/integration/tests/smoke/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/integration/tests/integration/tests/smoke/instance.py b/integration/tests/integration/tests/smoke/instance.py new file mode 100644 index 0000000000..3488e5f2b9 --- /dev/null +++ b/integration/tests/integration/tests/smoke/instance.py @@ -0,0 +1,103 @@ + +from proboscis.asserts import assert_equal +from proboscis import test +from proboscis import before_class + +from trove.common.utils import poll_until +from trove.tests.util import create_client + + +class InstanceGenerator(object): + + def __init__(self, client, status=None, name=None, flavor=None, + account_id=None, created_at=None, databases=None, users=None, + volume_size=None): + self.client = client + self.status = status + self.name = name + self.flavor = flavor + self.account_id = account_id + self.databases = databases + self.users = users + self.volume_size = volume_size + self.id = None + + def create_instance(self): + #make the call to create the instance + instance = self.client.instances.create(self.name, self.flavor, + self.volume_size, self.databases, self.users) + self.client.assert_http_code(200) + + #verify we are in a build state + assert_equal(instance.status, "BUILD") + #pull out the ID + self.id = instance.id + + return instance + + def wait_for_build_to_finish(self): + poll_until(lambda: self.client.instance.get(self.id), + lambda instance: instance.status != "BUILD", + time_out=600) + + def get_active_instance(self): + instance = self.client.instance.get(self.id) + self.client.assert_http_code(200) + + #check the container name + assert_equal(instance.name, self.name) + + #pull out volume info and verify + assert_equal(str(instance.volume_size), str(self.volume_size)) + + #pull out the flavor and verify + assert_equal(str(instance.flavor), str(self.flavor)) + + return instance + + +@test(groups=['smoke', 'positive']) +class CreateInstance(object): + + @before_class + def set_up(self): + client = create_client(is_admin=False) + name = 'test_createInstance_container' + flavor = 1 + volume_size = 1 + db_name = 'test_db' + databases = [ + { + "name": db_name + } + ] + users = [ + { + "name": "lite", + "password": "litepass", + "databases": [{"name": db_name}] + } + ] + + #create the Instance + instance = InstanceGenerator(client, name=self.name, + flavor=flavor, + volume_size=self.volume_size, + databases=databases, users=users) + instance.create_instance() + + #wait for the instance + instance.wait_for_build_to_finish() + + #get the active instance + inst = instance.get_active_instance() + + #list out the databases for our instance and verify the db name + dbs = client.databases.list(inst.id) + client.assert_http_code(200) + + assert_equal(len(dbs), 1) + assert_equal(dbs[0].name, instance.db_name) + + client.instance.delete(inst.id) + client.assert_http_code(202) diff --git a/integration/tests/integration/tests/util/__init__.py b/integration/tests/integration/tests/util/__init__.py new file mode 100644 index 0000000000..671d3c173e --- /dev/null +++ b/integration/tests/integration/tests/util/__init__.py @@ -0,0 +1,16 @@ +# vim: tabstop=4 shiftwidth=4 softtabstop=4 + +# Copyright (c) 2011 OpenStack, LLC. +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. diff --git a/integration/tests/integration/tests/util/report.py b/integration/tests/integration/tests/util/report.py new file mode 100644 index 0000000000..c3c1007cf3 --- /dev/null +++ b/integration/tests/integration/tests/util/report.py @@ -0,0 +1,76 @@ +"""Creates a report for the test. +""" + +import os +import shutil +from os import path +from trove.tests.config import CONFIG + +USE_LOCAL_OVZ = CONFIG.use_local_ovz + + +class Reporter(object): + """Saves the logs from a test run.""" + + def __init__(self, root_path): + self.root_path = root_path + if not path.exists(self.root_path): + os.mkdir(self.root_path) + for file in os.listdir(self.root_path): + if file.endswith(".log"): + os.remove(path.join(self.root_path, file)) + + def _find_all_instance_ids(self): + instances = [] + if USE_LOCAL_OVZ: + for dir in os.listdir("/var/lib/vz/private"): + instances.append(dir) + return instances + + def log(self, msg): + with open("%s/report.log" % self.root_path, 'a') as file: + file.write(str(msg) + "\n") + + def _save_syslog(self): + try: + shutil.copyfile("/var/log/syslog", "host-syslog.log") + except (shutil.Error, IOError) as err: + self.log("ERROR logging syslog : %s" % (err)) + + def _update_instance(self, id): + root = "%s/%s" % (self.root_path, id) + + def save_file(path, short_name): + if USE_LOCAL_OVZ: + try: + shutil.copyfile("/var/lib/vz/private/%s/%s" % (id, path), + "%s-%s.log" % (root, short_name)) + except (shutil.Error, IOError) as err: + self.log("ERROR logging %s for instance id %s! : %s" + % (path, id, err)) + else: + #TODO: Can we somehow capture these (maybe SSH to the VM)? + pass + + save_file("/var/log/firstboot", "firstboot") + save_file("/var/log/syslog", "syslog") + save_file("/var/log/nova/guest.log", "nova-guest") + + def _update_instances(self): + for id in self._find_all_instance_ids(): + self._update_instance(id) + + def update(self): + self._update_instances() + self._save_syslog() + + +REPORTER = Reporter(CONFIG.report_directory) + + +def log(msg): + REPORTER.log(msg) + + +def update(): + REPORTER.update() diff --git a/integration/tests/integration/tests/util/rpc.py b/integration/tests/integration/tests/util/rpc.py new file mode 100644 index 0000000000..d534ff31a9 --- /dev/null +++ b/integration/tests/integration/tests/util/rpc.py @@ -0,0 +1,110 @@ +# Copyright (c) 2012 OpenStack, LLC. +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +""" +Test utility for RPC checks. + +Functionality to check for rabbit here depends on having rabbit running on +the same machine as the tests, so that the rabbitmqctl commands will function. +The functionality is turned on or off by the test config "rabbit_runs_locally". + +""" + +import re + +from trove.tests.config import CONFIG +from services import start_proc + + +if CONFIG.values.get('rabbit_runs_locally', False) == True: + + DIRECT_ACCESS = True + + class Rabbit(object): + + def declare_queue(self, topic): + """Call this to declare a queue from Python.""" + #from trove.rpc.impl_kombu import Connection + from trove.openstack.common.rpc import create_connection + with create_connection() as conn: + consumer = conn.declare_topic_consumer(topic=topic) + + def get_queue_items(self, queue_name): + """Determines if the queue exists and if so the message count. + + If the queue exists the return value is an integer, otherwise + its None. + + Be careful because queue_name is used in a regex and can't have + any unescaped characters. + + """ + proc = start_proc(["/usr/bin/sudo", "rabbitmqctl", "list_queues"], + shell=False) + for line in iter(proc.stdout.readline, ""): + print("LIST QUEUES:" + line) + m = re.search("""%s\s+([0-9]+)""" % queue_name, line) + if m: + return int(m.group(1)) + return None + + @property + def is_alive(self): + """Calls list_queues, should fail.""" + try: + stdout, stderr = self.run(0, "rabbitmqctl", "list_queues") + for lines in stdout, stderr: + for line in lines: + if "no_exists" in line: + return False + return True + except Exception: + return False + + def reset(self): + out, err = self.run(0, "rabbitmqctl", "reset") + print(out) + print(err) + + def run(self, check_exit_code, *cmd): + cmds = ["/usr/bin/sudo"] + list(cmd) + proc = start_proc(cmds) + lines = proc.stdout.readlines() + err_lines = proc.stderr.readlines() + return lines, err_lines + + def start(self): + print("Calling rabbitmqctl start_app") + out = self.run(0, "rabbitmqctl", "start_app") + print(out) + out, err = self.run(0, "rabbitmqctl", "change_password", "guest", + CONFIG.values['rabbit_password']) + print(out) + print(err) + + def stop(self): + print("Calling rabbitmqctl stop_app") + out = self.run(0, "rabbitmqctl", "stop_app") + print(out) + +else: + + DIRECT_ACCESS = False + + class Rabbit(object): + + def __init__(self): + raise RuntimeError("rabbit_runs_locally is set to False in the " + "test config, so this test cannot be run.") + diff --git a/integration/tests/integration/tests/util/services.py b/integration/tests/integration/tests/util/services.py new file mode 100644 index 0000000000..b54f92d6b8 --- /dev/null +++ b/integration/tests/integration/tests/util/services.py @@ -0,0 +1,280 @@ +# Copyright (c) 2011 OpenStack, LLC. +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +"""Functions to initiate and shut down services needed by the tests.""" + +import os +import re +import subprocess +import time + +from collections import namedtuple +from httplib2 import Http +from nose.plugins.skip import SkipTest + +from proboscis import decorators + + +def _is_web_service_alive(url): + """Does a HTTP GET request to see if the web service is up.""" + client = Http() + try: + resp = client.request(url, 'GET') + return resp != None + except Exception: + return False + + +_running_services = [] + + +def get_running_services(): + """ Returns the list of services which this program has started.""" + return _running_services + + +def start_proc(cmd, shell=False): + """Given a command, starts and returns a process.""" + env = os.environ.copy() + proc = subprocess.Popen( + cmd, + shell=shell, + stdin=subprocess.PIPE, + bufsize=0, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + env=env + ) + return proc + + +MemoryInfo = namedtuple("MemoryInfo", ['mapped', 'writeable', 'shared']) + + +class Service(object): + """Starts and stops a service under test. + + The methods to start and stop the service will not actually do anything + if they detect the service is already running on this machine. This is + because it may be useful for developers to start the services themselves + some other way. + + """ + + # TODO(tim.simpson): Hard to follow, consider renaming certain attributes. + + def __init__(self, cmd): + """Defines a service to run.""" + if not isinstance(cmd, list): + raise TypeError() + self.cmd = cmd + self.do_not_manage_proc = False + self.proc = None + + def __del__(self): + if self.is_running: + self.stop() + + def ensure_started(self): + """Starts the service if it is not running.""" + if not self.is_running: + self.start() + + def find_proc_id(self): + """Finds and returns the process id.""" + if not self.cmd: + return False + # The cmd[1] signifies the executable python script. It gets invoked + # as python /path/to/executable args, so the entry is + # /path/to/executable + actual_command = self.cmd[1].split("/")[-1] + proc_command = ["/usr/bin/pgrep", "-f", actual_command] + proc = start_proc(proc_command, shell=False) + # this is to make sure there is only one pid returned from the pgrep + has_two_lines = False + pid = None + for line in iter(proc.stdout.readline, ""): + if has_two_lines: + raise RuntimeError("Found PID twice.") + pid = int(line) + has_two_lines = True + return pid + + def get_memory_info(self): + """Returns how much memory the process is using according to pmap.""" + pid = self.find_proc_id() + if not pid: + raise RuntimeError("Can't find PID, so can't get memory.") + proc = start_proc(["/usr/bin/pmap", "-d", str(pid)], + shell=False) + for line in iter(proc.stdout.readline, ""): + m = re.search("""mapped\:\s([0-9]+)K\s+""" + """writeable/private:\s([0-9]+)K\s+""" + """shared:\s+([0-9]+)K""", line) + if m: + return MemoryInfo(int(m.group(1)), int(m.group(2)), + int(m.group(3))) + raise RuntimeError("Memory info not found.") + + def get_fd_count_from_proc_file(self): + """Returns file descriptors according to /proc//status.""" + pid = self.find_proc_id() + with open("/proc/%d/status" % pid) as status: + for line in status.readlines(): + index = line.find(":") + name = line[:index] + value = line[index + 1:] + if name == "FDSize": + return int(value) + raise RuntimeError("FDSize not found!") + + def get_fd_count(self): + """Returns file descriptors according to /proc//status.""" + pid = self.find_proc_id() + cmd = "Finding file descriptors..." + print("CMD" + cmd) + proc = start_proc(['ls', '-la', '/proc/%d/fd' % pid], shell=False) + count = -3 + has_two_lines = False + for line in iter(proc.stdout.readline, ""): + print("\t" + line) + count += 1 + if not count: + raise RuntimeError("Could not get file descriptors!") + return count + + + with open("/proc/%d/fd" % pid) as status: + for line in status.readlines(): + index = line.find(":") + name = line[:index] + value = line[index + 1:] + if name == "FDSize": + return int(value) + raise RuntimeError("FDSize not found!") + + def kill_proc(self): + """Kills the process, wherever it may be.""" + pid = self.find_proc_id() + if pid: + start_proc("sudo kill -9 " + str(pid), shell=True) + time.sleep(1) + if self.is_service_alive(): + raise RuntimeError('Cannot kill process, PID=' + + str(self.proc.pid)) + + def is_service_alive(self, proc_name_index=1): + """Searches for the process to see if its alive. + + This function will return true even if this class has not started + the service (searches using ps). + + """ + if not self.cmd: + return False + time.sleep(1) + # The cmd[1] signifies the executable python script. It gets invoked + # as python /path/to/executable args, so the entry is + # /path/to/executable + actual_command = self.cmd[proc_name_index].split("/")[-1] + print actual_command + proc_command = ["/usr/bin/pgrep", "-f", actual_command] + print proc_command + proc = start_proc(proc_command, shell=False) + line = proc.stdout.readline() + print line + # pgrep only returns a pid. if there is no pid, it'll return nothing + return len(line) != 0 + + @property + def is_running(self): + """Returns true if the service has already been started. + + Returns true if this program has started the service or if it + previously detected it had started. The main use of this property + is to know if the service was already begun by this program- + use is_service_alive for a more definitive answer. + + """ + return self.proc or self.do_not_manage_proc + + def restart(self, extra_args): + if self.do_not_manage_proc: + raise RuntimeError("Can't restart proc as the tests don't own it.") + self.stop() + time.sleep(2) + self.start(extra_args=extra_args) + + def start(self, time_out=30, extra_args=None): + """Starts the service if necessary.""" + extra_args = extra_args or [] + if self.is_running: + raise RuntimeError("Process is already running.") + if self.is_service_alive(): + self.do_not_manage_proc = True + return + self.proc = start_proc(self.cmd + extra_args, shell=False) + if not self._wait_for_start(time_out=time_out): + self.stop() + raise RuntimeError("Issued the command successfully but the " + "service (" + str(self.cmd + extra_args) + + ") never seemed to start.") + _running_services.append(self) + + def stop(self): + """Stops the service, but only if this program started it.""" + if self.do_not_manage_proc: + return + if not self.proc: + raise RuntimeError("Process was not started.") + self.proc.terminate() + self.proc.kill() + self.proc.wait() + self.proc.stdin.close() + self.kill_proc() + self.proc = None + global _running_services + _running_services = [svc for svc in _running_services if svc != self] + + def _wait_for_start(self, time_out): + """Waits until time_out (in seconds) for service to appear.""" + give_up_time = time.time() + time_out + while time.time() < give_up_time: + if self.is_service_alive(): + return True + return False + + +class NativeService(Service): + + def is_service_alive(self): + return super(NativeService, self).is_service_alive(proc_name_index=0) + + + +class WebService(Service): + """Starts and stops a web service under test.""" + + def __init__(self, cmd, url): + """Defines a service to run.""" + Service.__init__(self, cmd) + if not isinstance(url, (str, unicode)): + raise TypeError() + self.url = url + self.do_not_manage_proc = self.is_service_alive() + + def is_service_alive(self): + """Searches for the process to see if its alive.""" + return _is_web_service_alive(self.url) diff --git a/integration/tests/integration/tests/volumes/__init__.py b/integration/tests/integration/tests/volumes/__init__.py new file mode 100644 index 0000000000..09c4cfed1c --- /dev/null +++ b/integration/tests/integration/tests/volumes/__init__.py @@ -0,0 +1,25 @@ +# Copyright (c) 2011 OpenStack, LLC. +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +""" +:mod:`volumes` -- Tests for volumes. +=================================== +""" + +""""Tests for Volumes.""" + +# Is a set of tests written directly against the VolumeManager and VolumeClient +# classes which doesn't require standing up Nova daemons or anything. +VOLUMES_DRIVER = "trove.volumes.driver" diff --git a/integration/tests/integration/tests/volumes/driver.py b/integration/tests/integration/tests/volumes/driver.py new file mode 100644 index 0000000000..221b2a41f4 --- /dev/null +++ b/integration/tests/integration/tests/volumes/driver.py @@ -0,0 +1,546 @@ +# Copyright (c) 2012 OpenStack, LLC. +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +from numbers import Number +import os +import re +import shutil +import socket +import time +import unittest + +import pexpect + +from proboscis import test +from proboscis.asserts import assert_raises +from proboscis.decorators import expect_exception +from proboscis.decorators import time_out + +from trove.tests.config import CONFIG +from trove.common.utils import poll_until +from trove.tests.util import process +from trove.common.utils import import_class +from tests import initialize + + +WHITE_BOX = CONFIG.white_box +VOLUMES_DRIVER = "trove.volumes.driver" + +if WHITE_BOX: + # TODO(tim.simpson): Restore this once white box functionality can be + # added back to this test module. + pass + # from nova import context + # from nova import exception + # from nova import flags + # from nova import utils + # from trove import exception as trove_exception + # from trove.utils import poll_until + # from trove import volume + # from trove.tests.volume import driver as test_driver + + # FLAGS = flags.FLAGS + + +UUID_PATTERN = re.compile('^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-' + '[0-9a-f]{4}-[0-9a-f]{12}$') + +HUGE_VOLUME = 5000 + + +def is_uuid(text): + return UUID_PATTERN.search(text) is not None + + +class StoryDetails(object): + + def __init__(self): + self.api = volume.API() + self.client = volume.Client() + self.context = context.get_admin_context() + self.device_path = None + self.volume_desc = None + self.volume_id = None + self.volume_name = None + self.volume = None + self.host = socket.gethostname() + self.original_uuid = None + self.original_device_info = None + self.resize_volume_size = 2 + + def get_volume(self): + return self.api.get(self.context, self.volume_id) + + @property + def mount_point(self): + return "%s/%s" % (LOCAL_MOUNT_PATH, self.volume_id) + + @property + def test_mount_file_path(self): + return "%s/test.txt" % self.mount_point + + +story = None +storyFail = None + +LOCAL_MOUNT_PATH = "/testsmnt" + + +class VolumeTest(unittest.TestCase): + """This test tells the story of a volume, from cradle to grave.""" + + def __init__(self, *args, **kwargs): + unittest.TestCase.__init__(self, *args, **kwargs) + + def setUp(self): + global story, storyFail + self.story = story + self.storyFail = storyFail + + def assert_volume_as_expected(self, volume): + self.assertIsInstance(volume["id"], Number) + self.assertEqual(self.story.volume_name, volume["display_name"]) + self.assertEqual(self.story.volume_desc, volume["display_description"]) + self.assertEqual(1, volume["size"]) + self.assertEqual(self.story.context.user_id, volume["user_id"]) + self.assertEqual(self.story.context.project_id, volume["project_id"]) + + +@test(groups=[VOLUMES_DRIVER], depends_on_classes=[initialize.start_volume]) +class SetUp(VolumeTest): + + def test_05_create_story(self): + """Creating 'story' vars used by the rest of these tests.""" + global story, storyFail + story = StoryDetails() + storyFail = StoryDetails() + + @time_out(60) + def test_10_wait_for_topics(self): + """Wait until the volume topic is up before proceeding.""" + topics = ["volume"] + from tests.util.topics import hosts_up + while not all(hosts_up(topic) for topic in topics): + pass + + def test_20_refresh_local_folders(self): + """Delete the local folders used as mount locations if they exist.""" + if os.path.exists(LOCAL_MOUNT_PATH): + #TODO(rnirmal): Also need to remove any existing mounts. + shutil.rmtree(LOCAL_MOUNT_PATH) + os.mkdir(LOCAL_MOUNT_PATH) + # Give some time for the services to startup + time.sleep(10) + + @time_out(60) + def test_30_mgmt_volume_check(self): + """Get the volume information from the mgmt API""" + story_context = self.story.context + device_info = self.story.api.get_storage_device_info(story_context) + print("device_info : %r" % device_info) + self.assertNotEqual(device_info, None, + "the storage device information should exist") + self.story.original_device_info = device_info + + @time_out(60) + def test_31_mgmt_volume_info(self): + """Check the available space against the mgmt API info.""" + story_context = self.story.context + device_info = self.story.api.get_storage_device_info(story_context) + print("device_info : %r" % device_info) + info = {'spaceTotal': device_info['raw_total'], + 'spaceAvail': device_info['raw_avail']} + self._assert_available_space(info) + + def _assert_available_space(self, device_info, fail=False): + """ + Give the SAN device_info(fake or not) and get the asserts for free + """ + print("DEVICE_INFO on SAN : %r" % device_info) + # Calculate the GBs; Divide by 2 for the FLAGS.san_network_raid_factor + gbs = 1.0 / 1024 / 1024 / 1024 / 2 + total = int(device_info['spaceTotal']) * gbs + free = int(device_info['spaceAvail']) * gbs + used = total - free + usable = total * (FLAGS.san_max_provision_percent * 0.01) + real_free = float(int(usable - used)) + + print("total : %r" % total) + print("free : %r" % free) + print("used : %r" % used) + print("usable : %r" % usable) + print("real_free : %r" % real_free) + + check_space = self.story.api.check_for_available_space + self.assertFalse(check_space(self.story.context, HUGE_VOLUME)) + self.assertFalse(check_space(self.story.context, real_free + 1)) + + if fail: + self.assertFalse(check_space(self.story.context, real_free)) + self.assertFalse(check_space(self.story.context, real_free - 1)) + self.assertFalse(check_space(self.story.context, 1)) + else: + self.assertTrue(check_space(self.story.context, real_free)) + self.assertTrue(check_space(self.story.context, real_free - 1)) + self.assertTrue(check_space(self.story.context, 1)) + + +@test(groups=[VOLUMES_DRIVER], depends_on_classes=[SetUp]) +class AddVolumeFailure(VolumeTest): + + @time_out(60) + def test_add(self): + """ + Make call to FAIL a prov. volume and assert the return value is a + FAILURE. + """ + self.assertIsNone(self.storyFail.volume_id) + name = "TestVolume" + desc = "A volume that was created for testing." + self.storyFail.volume_name = name + self.storyFail.volume_desc = desc + volume = self.storyFail.api.create(self.storyFail.context, + size=HUGE_VOLUME, + snapshot_id=None, name=name, + description=desc) + self.assertEqual(HUGE_VOLUME, volume["size"]) + self.assertTrue("creating", volume["status"]) + self.assertTrue("detached", volume["attach_status"]) + self.storyFail.volume = volume + self.storyFail.volume_id = volume["id"] + + +@test(groups=[VOLUMES_DRIVER], depends_on_classes=[AddVolumeFailure]) +class AfterVolumeFailureIsAdded(VolumeTest): + """Check that the volume can be retrieved via the API, and setup. + + All we want to see returned is a list-like with an initial string. + + """ + + @time_out(120) + def test_api_get(self): + """Wait until the volume is a FAILURE.""" + volume = poll_until(lambda: self.storyFail.get_volume(), + lambda volume: volume["status"] != "creating") + self.assertEqual(volume["status"], "error") + self.assertTrue(volume["attach_status"], "detached") + + @time_out(60) + def test_mgmt_volume_check(self): + """Get the volume information from the mgmt API""" + info = self.story.api.get_storage_device_info(self.story.context) + print("device_info : %r" % info) + self.assertNotEqual(info, None, + "the storage device information should exist") + self.assertEqual(self.story.original_device_info['raw_total'], + info['raw_total']) + self.assertEqual(self.story.original_device_info['raw_avail'], + info['raw_avail']) + + +@test(groups=[VOLUMES_DRIVER], depends_on_classes=[SetUp]) +class AddVolume(VolumeTest): + + @time_out(60) + def test_add(self): + """Make call to prov. a volume and assert the return value is OK.""" + self.assertIsNone(self.story.volume_id) + name = "TestVolume" + desc = "A volume that was created for testing." + self.story.volume_name = name + self.story.volume_desc = desc + volume = self.story.api.create(self.story.context, size=1, + snapshot_id=None, name=name, + description=desc) + self.assert_volume_as_expected(volume) + self.assertTrue("creating", volume["status"]) + self.assertTrue("detached", volume["attach_status"]) + self.story.volume = volume + self.story.volume_id = volume["id"] + + +@test(groups=[VOLUMES_DRIVER], depends_on_classes=[AddVolume]) +class AfterVolumeIsAdded(VolumeTest): + """Check that the volume can be retrieved via the API, and setup. + + All we want to see returned is a list-like with an initial string. + + """ + + @time_out(120) + def test_api_get(self): + """Wait until the volume is finished provisioning.""" + volume = poll_until(lambda: self.story.get_volume(), + lambda volume: volume["status"] != "creating") + self.assertEqual(volume["status"], "available") + self.assert_volume_as_expected(volume) + self.assertTrue(volume["attach_status"], "detached") + + @time_out(60) + def test_mgmt_volume_check(self): + """Get the volume information from the mgmt API""" + print("self.story.original_device_info : %r" % + self.story.original_device_info) + info = self.story.api.get_storage_device_info(self.story.context) + print("device_info : %r" % info) + self.assertNotEqual(info, None, + "the storage device information should exist") + self.assertEqual(self.story.original_device_info['raw_total'], + info['raw_total']) + volume_size = int(self.story.volume['size']) * (1024 ** 3) * 2 + print("volume_size: %r" % volume_size) + print("self.story.volume['size']: %r" % self.story.volume['size']) + avail = int(self.story.original_device_info['raw_avail']) - volume_size + print("avail space: %r" % avail) + self.assertEqual(int(info['raw_avail']), avail) + + +@test(groups=[VOLUMES_DRIVER], depends_on_classes=[AfterVolumeIsAdded]) +class SetupVolume(VolumeTest): + + @time_out(60) + def test_assign_volume(self): + """Tell the volume it belongs to this host node.""" + #TODO(tim.simpson) If this is important, could we add a test to + # make sure some kind of exception is thrown if it + # isn't added to certain drivers? + self.assertNotEqual(None, self.story.volume_id) + self.story.api.assign_to_compute(self.story.context, + self.story.volume_id, + self.story.host) + + @time_out(60) + def test_setup_volume(self): + """Set up the volume on this host. AKA discovery.""" + self.assertNotEqual(None, self.story.volume_id) + device = self.story.client._setup_volume(self.story.context, + self.story.volume_id, + self.story.host) + if not isinstance(device, basestring): + self.fail("Expected device to be a string, but instead it was " + + str(type(device)) + ".") + self.story.device_path = device + + +@test(groups=[VOLUMES_DRIVER], depends_on_classes=[SetupVolume]) +class FormatVolume(VolumeTest): + + @expect_exception(IOError) + @time_out(60) + def test_10_should_raise_IOError_if_format_fails(self): + """ + + Tests that if the driver's _format method fails, its + public format method will perform an assertion properly, discover + it failed, and raise an exception. + + """ + + volume_driver_cls = import_class(FLAGS.volume_driver) + + class BadFormatter(volume_driver_cls): + + def _format(self, device_path): + pass + + bad_client = volume.Client(volume_driver=BadFormatter()) + bad_client._format(self.story.device_path) + + @time_out(60) + def test_20_format(self): + self.assertNotEqual(None, self.story.device_path) + self.story.client._format(self.story.device_path) + + def test_30_check_options(self): + cmd = ("sudo dumpe2fs -h %s 2> /dev/null | " + "awk -F ':' '{ if($1 == \"Reserved block count\") " + "{ rescnt=$2 } } { if($1 == \"Block count\") " + "{ blkcnt=$2 } } END { print (rescnt/blkcnt)*100 }'") + cmd = cmd % self.story.device_path + out, err = process(cmd) + self.assertEqual(float(5), round(float(out)), msg=out) + + +@test(groups=[VOLUMES_DRIVER], depends_on_classes=[FormatVolume]) +class MountVolume(VolumeTest): + + @time_out(60) + def test_mount(self): + self.story.client._mount(self.story.device_path, + self.story.mount_point) + with open(self.story.test_mount_file_path, 'w') as file: + file.write("Yep, it's mounted alright.") + self.assertTrue(os.path.exists(self.story.test_mount_file_path)) + + def test_mount_options(self): + cmd = "mount -l | awk '/%s.*noatime/ { print $1 }'" + cmd %= LOCAL_MOUNT_PATH.replace('/', '') + out, err = process(cmd) + self.assertEqual(os.path.realpath(self.story.device_path), out.strip(), + msg=out) + + +@test(groups=[VOLUMES_DRIVER], depends_on_classes=[MountVolume]) +class ResizeVolume(VolumeTest): + + @time_out(300) + def test_resize(self): + self.story.api.resize(self.story.context, self.story.volume_id, + self.story.resize_volume_size) + + volume = poll_until(lambda: self.story.get_volume(), + lambda volume: volume["status"] == "resized") + self.assertEqual(volume["status"], "resized") + self.assertTrue(volume["attach_status"], "attached") + self.assertTrue(volume['size'], self.story.resize_volume_size) + + @time_out(300) + def test_resizefs_rescan(self): + self.story.client.resize_fs(self.story.context, + self.story.volume_id) + expected = "trove.tests.volume.driver.ISCSITestDriver" + if FLAGS.volume_driver is expected: + size = self.story.resize_volume_size * \ + test_driver.TESTS_VOLUME_SIZE_MULTIPLIER * 1024 * 1024 + else: + size = self.story.resize_volume_size * 1024 * 1024 + out, err = process('sudo blockdev --getsize64 %s' % + os.path.realpath(self.story.device_path)) + if int(out) < (size * 0.8): + self.fail("Size %s is not more or less %s" % (out, size)) + + # Reset the volume status to available + self.story.api.update(self.story.context, self.story.volume_id, + {'status': 'available'}) + + +@test(groups=[VOLUMES_DRIVER], depends_on_classes=[MountVolume]) +class UnmountVolume(VolumeTest): + + @time_out(60) + def test_unmount(self): + self.story.client._unmount(self.story.mount_point) + child = pexpect.spawn("sudo mount %s" % self.story.mount_point) + child.expect("mount: can't find %s in" % self.story.mount_point) + + +@test(groups=[VOLUMES_DRIVER], depends_on_classes=[UnmountVolume]) +class GrabUuid(VolumeTest): + + @time_out(60) + def test_uuid_must_match_pattern(self): + """UUID must be hex chars in the form 8-4-4-4-12.""" + client = self.story.client # volume.Client() + device_path = self.story.device_path # '/dev/sda5' + uuid = client.get_uuid(device_path) + self.story.original_uuid = uuid + self.assertTrue(is_uuid(uuid), "uuid must match regex") + + @time_out(60) + def test_get_invalid_uuid(self): + """DevicePathInvalidForUuid is raised if device_path is wrong.""" + client = self.story.client + device_path = "gdfjghsfjkhggrsyiyerreygghdsghsdfjhf" + self.assertRaises(trove_exception.DevicePathInvalidForUuid, + client.get_uuid, device_path) + + +@test(groups=[VOLUMES_DRIVER], depends_on_classes=[GrabUuid]) +class RemoveVolume(VolumeTest): + + @time_out(60) + def test_remove(self): + self.story.client.remove_volume(self.story.context, + self.story.volume_id, + self.story.host) + self.assertRaises(Exception, + self.story.client._format, self.story.device_path) + + +@test(groups=[VOLUMES_DRIVER], depends_on_classes=[GrabUuid]) +class Initialize(VolumeTest): + + @time_out(300) + def test_10_initialize_will_format(self): + """initialize will setup, format, and store the UUID of a volume""" + self.assertTrue(self.story.get_volume()['uuid'] is None) + self.story.client.initialize(self.story.context, self.story.volume_id, + self.story.host) + volume = self.story.get_volume() + self.assertTrue(is_uuid(volume['uuid']), "uuid must match regex") + self.assertNotEqual(self.story.original_uuid, volume['uuid'], + "Validate our assumption that the volume UUID " + "will change when the volume is formatted.") + self.story.client.remove_volume(self.story.context, + self.story.volume_id, + self.story.host) + + @time_out(60) + def test_20_initialize_the_second_time_will_not_format(self): + """If initialize is called but a UUID exists, it should not format.""" + old_uuid = self.story.get_volume()['uuid'] + self.assertTrue(old_uuid is not None) + + class VolumeClientNoFmt(volume.Client): + + def _format(self, device_path): + raise RuntimeError("_format should not be called!") + + no_fmt_client = VolumeClientNoFmt() + no_fmt_client.initialize(self.story.context, self.story.volume_id, + self.story.host) + self.assertEqual(old_uuid, self.story.get_volume()['uuid'], + "UUID should be the same as no formatting occurred.") + self.story.client.remove_volume(self.story.context, + self.story.volume_id, + self.story.host) + + def test_30_check_device_exists(self): + assert_raises(exception.InvalidDevicePath, self.story.client._format, + self.story.device_path) + + +@test(groups=[VOLUMES_DRIVER], depends_on_classes=[Initialize]) +class DeleteVolume(VolumeTest): + + @time_out(60) + def test_delete(self): + self.story.api.delete(self.story.context, self.story.volume_id) + + +@test(groups=[VOLUMES_DRIVER], depends_on_classes=[DeleteVolume]) +class ConfirmMissing(VolumeTest): + + @time_out(60) + def test_discover_should_fail(self): + try: + self.story.client.driver.discover_volume(self.story.context, + self.story.volume) + self.fail("Expecting an error but did not get one.") + except exception.Error: + pass + except trove_exception.ISCSITargetNotDiscoverable: + pass + + @time_out(60) + def test_get_missing_volume(self): + try: + volume = poll_until(lambda: self.story.api.get(self.story.context, + self.story.volume_id), + lambda volume: volume["status"] != "deleted") + self.assertEqual(volume["deleted"], False) + except exception.VolumeNotFound: + pass diff --git a/integration/tests/integration/tox.ini b/integration/tests/integration/tox.ini new file mode 100644 index 0000000000..8105149305 --- /dev/null +++ b/integration/tests/integration/tox.ini @@ -0,0 +1,28 @@ +# Examples: +# Run tests against Trove running locally in fake mode: +# TROVE_CLIENT_PATH=../some_path tox -e local -- --group=blackbox +[tox] +envlist = py26 + +[testenv] +deps = + coverage + nose + pexpect + proboscis + sqlalchemy + {env:TROVE_PATH} + {env:TROVE_CLIENT_PATH} + +[testenv:py26] + +[testenv:local] +deps = + nose + pexpect + proboscis + sqlalchemy + {env:TROVE_PATH} + {env:TROVE_CLIENT_PATH} +commands = + {envpython} int_tests.py --conf=localhost.test.conf {posargs:DEFAULTS} diff --git a/integration/xsd/common.ent b/integration/xsd/common.ent new file mode 100644 index 0000000000..b8972274d6 --- /dev/null +++ b/integration/xsd/common.ent @@ -0,0 +1,72 @@ + + + + +GET'> +PUT'> +POST'> +DELETE'> + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + '> + + + + + + '> + + + + + + '> diff --git a/integration/xsd/dbaas.wadl b/integration/xsd/dbaas.wadl new file mode 100644 index 0000000000..91953198a9 --- /dev/null +++ b/integration/xsd/dbaas.wadl @@ -0,0 +1,1177 @@ + + + %common; + + +]> + + + + + + + + + + + + + + + The account ID of the owner of the specified instance. + + + + + + + The instance ID for the specified database instance. + + + + + + + + + + + + + The name for the specified database. + + + + + + + + + The name for the specified user. + + + + + + + + + + + + + + + The flavor ID for the specified flavor. + + + + + + + + + + + + + Returns detailed information about the specified version of the API. + + Reviewer: in the DNS project, we have been requested by the customer to provide a table of parameters (should be pulled automatically if parms defined in wadl) and a table of attributes (for calls that allow detailed info about the object created to be specified. No doubt our DB customers will want this too. + Reviewer: These tables probably need 4 columns: name; parameter type: e.g. template, query, etc.; data type: string, etc.; required?; description. + This operation returns detailed information about the specified version of the API. + + + + + The following examples show the List Version Details requests: + + + + + + + + + + + + + The following examples show the List Version Details responses: + + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Lists information about all versions of the API. + + This operation lists information about all versions of the API. + + + + + The following examples show the List Versions requests: + + + + + + + + + + + + + The following examples show the List Versions responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Creates a new database instance. + This operation asynchronously provisions a new database instance. This call requires the user to specify a flavor and a volume size. The service then provisions the instance with the requested flavor and sets up a volume of the specified size, which is the storage for the database instance. + Notes + + You can create only one database instance per POST request. + + + You can create a database instance with one or more databases, and users associated to those databases. + + + The default binding for the MySQL instance is port 3306. + + + Database instances are directly accessible only on the internal ServiceNet network and using a Cloud resource within the same regional datacenter. For example, a database instance in DFW can only be accessed by a Cloud Server in DFW. For details and information about using a public Cloud Load Balancer to allow access to your database instance, refer to + for details. + + + The following table lists the required and optional attributes for Create Instance: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Required and Optional Attributes for Create Instance
Applies To Name DescriptionRequired
InstanceflavorRefReference (href) to a flavor as specified in the response from the List Flavors API call. + This is the actual URI as specified by the href field in the link. For example, in the following List Flavors response, the link to flavor id 1 is specified as + "https://ord.databases.api.rackspacecloud.com/v1.0/1234/flavors/1". + Rather than the flavor URI, you can also pass the flavor id (integer) as the value for flavorRef. For example, the flavor id for the flavor URI shown above is "1". + Refer to for details.Yes
(volume) sizeSpecifies the volume size in gigabytes (GB). The value specified must be between 1 and 50.Yes
nameName of the instance to create. The length of the name is limited to 255 characters and any characters are permitted.No
DatabasenameSpecifies database names for creating databases on instance creation. Refer to for the required xml/json format.No
character_setSet of symbols and encodings. The default character set is utf8.No
collateSet of rules for comparing characters in a character set. The default value for collate is utf8_general_ci.No
UsernameSpecifies user name for the database on instance creation. Refer to for the required xml/json format.No
passwordSpecifies password for those users on instance creation. Refer to for the required xml/json format.No
(database) nameSpecifies names of databases that those users can access on instance creation. Refer to for the required xml/json format.No
+ + Refer to for a list of possible database instance statuses that may be returned. + + + + + The following examples show the Create Database Instance requests and responses: + + + + + + + + + + + + + + + + + + + For convenience, notice in the response examples above that resources contain links to themselves. This allows a client to easily obtain + resource URIs rather than to construct them. There are two kinds of link relations + associated with resources. A self link contains a versioned link to the resource. These + links should be used in cases where the link will be followed immediately. A bookmark + link provides a permanent link to a resource that is appropriate for long term storage. + + + + &commonFaults; + &getFaults; + + + + + Deletes the specified database instance. + This operation deletes the specified database instance, including any associated data. + Refer to for a list of possible database instance statuses that may be returned. + This operation does not delete any read slaves. + Reviewer: please provide a description of read slaves that I can add to the previous note. + This operation is not allowed when the instance state is either REBUILDING or BUILDING. + + + + + The following examples show the Delete Database Instance requests: + + + + + + + + + + + &commonFaults; + + + + The following examples show the Delete Database Instance responses: + + + + + + + + + + &getFaults; + + + + + Lists the status and information for all database instances. + This operation lists the status and information for all database instances. + Refer to for a list of possible database instance statuses that may be returned. + + + + + The following examples show the List All Database Instances Detail requests: + + + + + + + + + + + + + + The following examples show the List All Database Instances responses: + + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Lists status and details for a specified database instance. + This operation lists the status and details of the specified database instance. + This operation lists the volume size in gigabytes (GB) and the approximate GB used. + After instance creation, the used size of your volume will be + greater than 0. This is expected and due to the automatic creation of + non-empty transaction logs for mysql optimization. The used attribute is not returned + in the response when the status for the instance is BUILD, REBOOT, or RESIZE. + Refer to for a list of possible database instance statuses that may be returned. + The list operations return a DNS-resolvable hostname associated with the database instance instead of an IP address. Since the hostname always resolves to the correct IP address of the database instance, this relieves the user from the task of maintaining the mapping. Note that although the IP address may likely change on resizing, migrating, and so forth, the hostname always resolves to the correct database instance. + + + + + The following examples show the List Database Instance Status and Details requests: + + + + + + + + + + + + + The following examples show the List Database Instance Status and Details responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Restart the database service on the instance. + The restart operation will restart only the MySQL Instance. Restarting MySQL will erase any dynamic configuration settings that you have made within MySQL. + The MySQL service will be unavailable until the instance restarts. + This operation returns a 202 Accepted response. + + + + + The following examples show the Restart Instance requests: + + + + + + + + + + + + + The following examples show the Restart Instance responses: + + + + + + + + + + &commonFaults; + &getFaults; + &postPutFaults; + + + + + Resize the memory of the instance. + This operation changes the memory size of the instance, assuming a valid flavorRef is provided. Restarts MySQL in the process. + + + + + The following examples show the Resize Instance requests: + + + + + + + + + + + + + The following examples show the Resize Instance responses: + + + + + + + + + + &commonFaults; + &getFaults; + &postPutFaults; + + + + + Resize the volume attached to the Instance. + This operation supports resizing the attached volume for an instance. It supports only increasing the volume size and does not support decreasing the size. + The volume size is in gigabytes (GB) and must be an integer. + You cannot increase the volume to a size larger than the API volume size limit specifies. + This operation returns a 202 Accepted response. + + + + + The following examples show the Resize Instance Volume requests: + + + + + + + + + + + + + The following examples show the Resize Instance Volume responses: + + + + + + + + + + &commonFaults; + &getFaults; + &postPutFaults; + + + + + + Creates a new database within the specified instance. + This operation creates a new database within the specified instance. + The name of the database is a required attribute. + The following additional attributes can be specified for each database: collate and character_set. + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Required and Optional Attributes for Create Database
Name DescriptionRequired
nameSpecifies the database name for creating the database. Refer to the request examples for the required xml/json format.Yes
character_setSet of symbols and encodings. The default character set is utf8.No
collateSet of rules for comparing characters in a character set. The default value for collate is utf8_general_ci.No
+ + See the MySQL documentation for information about supported character sets and collations at http://dev.mysql.com/doc/refman/5.1/en/charset-mysql.html. + + + The following database names are reserved and cannot be used for creating databases: lost+found, information_schema, and mysql. + Refer to the following tables for information about characters that are valid/invalid for creating database names. + + + + + + + + + + + + + + + + + + + + + + + +
Valid Characters That Can Be Used in a Database Name
Character
Letters (upper and lower cases allowed)
Numbers
'@', '?', '#', and spaces are allowed, but not at the beginning and end of the database name
'_' is allowed anywhere in the database name
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Characters That Cannot Be Used in a Database Name
Character
Single quotes
Double quotes
Back quotes
Semicolons
Commas
Backslashes
Forwardslashes
+ + + + + + + + + + + + + + + +
Length Restrictions for Database Name
RestrictionValue
Database-name maximum length64
+ + + + + The following examples show the Create Database requests: + + + + + + + + + + + + + The following examples show the Create Database responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Lists databases for the specified instance. + This operation lists the databases for the specified instance. + This operation returns only the user-defined databases, not the system databases. The system databases (mysql, information_schema, lost+found) can only be viewed by a database administrator. + + + + + The following examples show the List Databases for Instance requests: + + + + + + + + + + + + + The following examples show the List Databases for Instance responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + Deletes the specified database. + This operation deletes the requested database within the specified database instance. Note that all data associated with the database is also deleted. + + + + + The following examples show the Delete Database requests: + + + + + + + + + + + + + The following examples show the Delete Database responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Creates a user for the specified database instance. + This operation asynchronously provisions a new user for the specified database instance based on the configuration defined in the request object. Once the request is validated and progress has started on the provisioning process, a 202 Accepted response object is returned. + Writer: please add the following note back into the doc once the List User Details call is added back into the API: Using the identifier, the caller can check on the progress of the operation by performing a GET on users/name (for more details on this operation see the "List User Details" section of this document). + If the corresponding request cannot be fulfilled due to insufficient or invalid data, an HTTP 400 "Bad Request" error response is returned with information regarding the nature of the failure. Failures in the validation process are non-recoverable and require the caller to correct the cause of the failure and POST the request again. + + The following table lists the required attributes for Create User. Refer to the request examples for the required xml/json format: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Required Attributes for Create User
Applies ToName DescriptionRequired
UsernameName of the user for the + database.Yes
passwordUser password for database + access.Yes
(database) nameName of the database that the user can access. One or more database names must be specified.No
+ + Notes + + A user is granted all privileges on the specified databases. + + + The following user name is reserved and cannot be used for creating users: root. + + + + Refer to the following tables for information about characters that are valid/invalid for creating database names, user names, and passwords. + + + + + + + + + + + + + + + + + + + + + + + +
Valid Characters That Can Be Used in a Database Name, User Name, and Password
Character
Letters (upper and lower cases allowed)
Numbers
'@', '?', '#', and spaces are allowed, but not at the beginning and end of the database name, user name, and password
"_" is allowed anywhere in the database name, user name, and password
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Characters That Cannot Be Used in a Database Name, User Name, and Password
Character
Single quotes
Double quotes
Back quotes
Semicolons
Commas
Backslashes
Forwardslashes
Spaces at the front or end of the user name or password
+ + + + + + + + + + + + + + + + + + + + + + + +
Length Restrictions for Database Name, User Name, and Password
RestrictionValue
Database name maximum length64
User name maximum length16
Password maximum lengthunlimited (no restrictions)
+ + + + + The following examples show the Create User requests: + + + + + + + + + + + + + The following examples show the Create User responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Lists the users in the specified database instance. + This operation lists the users in the specified database instance, along with the associated databases for that user. + This operation does not return the system users (database administrators that administer the health of the database). Also, this operation returns the "root" user only if "root" user has been enabled. + + The following notes apply to MySQL users: + + + User names can be up to 16 characters long. + + + When you create accounts with INSERT, you must use FLUSH PRIVILEGES to tell the server to reload the grant tables. + + + For additional information, refer to: http://dev.mysql.com/doc/refman/5.1/en/user-account-management.html + + + + + + + The following examples show the List Users in Database Instance requests: + + + + + + + + + + + + + + The following examples show the List Users in Database Instance responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + Deletes the user identified by {name} for the specified database instance. + This operation deletes the specified user for the specified database instance. + There is a bug in a python library that Rackspace is using that may cause incorrect user deletions to occur + if a period (.) is used in the user name. In this case, the user name is truncated to remove the portion of the + name from the period to the end, leaving only the portion from the beginning up to the period. For example, for a + user named "my.userA", the bug would truncate the user name to "my", and if the user "my" exists, that user will + be incorrectly deleted. To avoid the problem, do not use periods in user names. + + + + + The following examples show the Delete User requests: + + + + + + + + + + + + + The following examples show the Delete User responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Enables the root user for the specified database instance and returns the root password. + This operation enables login from any host for the root user and provides the user with a generated root password. + Changes you make as a root user may cause detrimental effects to the database instance and unpredictable behavior for API operations. When you enable the root user, you accept the possibility that we will not be able to support your database instance. While enabling root does not prevent us from a “best effort” approach to helping you if something goes wrong with your instance, we cannot ensure that we will be able to assist you if you change core MySQL settings. These changes can be (but are not limited to) turning off binlogs, removing users that we use to access your instance, and so forth. + + + + + The following examples show the Enable Root User requests: + + + + + + + + + + + + + The following examples show the Enable Root User responses: + + + + + + + + + + + &commonFaults; + &getFaults; + + + + + Returns true if root user is enabled for the specified database instance or false otherwise. + This operation checks an active specified database instance to see if root access is enabled. It returns True if root user is enabled for the specified database instance or False otherwise. + + + + + The following examples show the Check Root User Access requests: + + + + + + + + + + + + + The following examples show the Check Root User Access responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Lists information for all available flavors. + This operation lists information for all available flavors. + This resource is identical to the flavors found in the OpenStack Nova API, but without the disk property. + Reviewer: please check that the xml example below is now correct. Previously it was reported to be incorrect. + + + + + The following examples show the List Flavors requests: + + + + + + + + + + + + + The following examples show the List Flavors responses: + + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + + Lists all flavor information about the specified flavor ID. + This operation lists all information for the specified flavor ID with details of the RAM. + This resource is identical to the flavors found in the OpenStack Nova API, but without the disk property. + The flavorId parameter should be an integer. If a floating point value is used for the flavorId parameter, the decimal portion is truncated and the integer portion is used as the value of the flavorId. + Writer: need to confirm that this behavior is not changed in subsequent releases, and if it is prevented, remove the Note above. + + + + + The following examples show the List Flavor By ID requests: + + + + + + + + + + + + + The following examples show the List Flavor By ID responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + diff --git a/integration/xsd/dbaas.xsd b/integration/xsd/dbaas.xsd new file mode 100644 index 0000000000..3e1876a538 --- /dev/null +++ b/integration/xsd/dbaas.xsd @@ -0,0 +1,613 @@ + + + + + + DBaaS + + +

+ This is the main index XML Schema document + for DBaaS API Schema Types Version 1.0. +

+ + + + + + +

A MySQL User.

+ + + + + + +

A MySQL User.

+ + + + + + +

A Database Instance.

+ + + + + + +

Database Instances.

+ + + + + + +

Settings such as memory used to create a database instance.

+ + + + + + +

A list of flavors.

+ + + + + + +

A Database.

+ + + + + + +

List of Databases.

+ + + + + + +

Restart the Database

+ + + + + + +

Resize an Instance

+ + + + + + + + + + +

A list of database user names.

+ + + + + + + + + + + +

A list of databases.

+ + + + + + + +

The name for the user.

+ + + + + + +

The password for the user.

+ + + + + + +

The status of the user.

+ + + + + + + + + + +

A list of database instances.

+ + + + + + + + + + + +

A list of optional databases.

+ + + + + + +

A list of optional users.

+ + + + + + +

The links for the type of instance.

+ + + + + + +

The volume attached to the instance.

+ + + + + + + +

A unique database instance id.

+ + + + + + +

Date/Time the instance was created.

+ + + + + + +

Date/Time the instance was last updated.

+ + + + + + +

The name for the instance.

+ + + + + + +

The flavor reference of the instance.

+ + + + + + +

The status of the instance.

+ + + + + + +

The hostname attached to the instance.

+ + + + + + +

Whether or not root is enabled for the instance.

+ + + + + + + + + +

The type of link.

+ + + + + + +

The URL.

+ + + + + + +

The type of link.

+ + + + + + + + + + +

A list of flavors.

+ + + + + + + + + + + +

A list of links.

+ + + + + + + +

A unique flavor id.

+ + + + + + +

The name for the instance.

+ + + + + + +

The RAM in megabytes.

+ + + + + + + + + + +

A list of links.

+ + + + + + + + + + + +

A list of databases.

+ + + + + + + + + + +

The Database character set.

+ + + + + + +

The name for the instance.

+ + + + + + +

The Collation type of the database.

+ + + + + + + + + +

Whether or not root is enabled for the given instance.

+ + + + + + + + + +

Size of the volume in GBs.

+ + + + + + +

Used space on the attached volume in GBs.

+ + + + + + + + + +

A unique flavor id.

+ + + + + + + + + + +

The volume attached to the instance.

+ + + + + + +

New flavorRef to size the instance.

+ + + + + + + + + + +

The instance status.

+ + + + + + +

Healthy status.

+ + + + + + +

Down status.

+ + + + + + +

unavailable status.

+ + + + + + + + + +

The mysql user status.

+ + + + + + +

Enabled status.

+ + + + + + + + + + + + + + + + + + + + + + + + + +

+ A human readable message that is appropriate for display + to the end user. +

+ + + + + + +

+ The optional <details> element may contain useful + information for tracking down errors (e.g a stack + trace). This information may or may not be appropriate + for display to an end user. +

+ + + + + + + + +

+ The HTTP status code associated with the current fault. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

+ An optional dateTime denoting when an operation should + be retried. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/integration/xsd/management.wadl b/integration/xsd/management.wadl new file mode 100644 index 0000000000..2271219252 --- /dev/null +++ b/integration/xsd/management.wadl @@ -0,0 +1,625 @@ + + + %common; +]> + + + + + + + + + + + + + + The host ID for the specified host. + + + + + + + + + + + + + The instance ID for the specified database instance. + + + + + + + + + + + + + + + + + + + + + + + The account ID for the specified account. + + + + + + + + + + + + Lists all of the accounts owning at least one instance that is not deleted. + + This operation returns a list of all the accounts across all locations that own at least one instance that has not been deleted, that is, all accounts of current users. + + + + + The following examples show the List All Active Accounts requests: + + + + + + + + + + + + + The following examples show the List All Active Accounts responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Lists all of the compute hosts. + + Reviewer: in the DNS project, we have been requested by the customer to provide a table of parameters (should be pulled automatically if parms defined in wadl) and a table of attributes (for calls that allow detailed info about the object created to be specified. No doubt our DB customers will want this too. + Reviewer: These tables probably need 4 columns: name; parameter type: e.g. template, query, etc.; data type: string, etc.; required?; description. + This operation returns a list of all the hosts from the database that are running the binary 'nova-compute'. + + + + + The following examples show the List All Compute Hosts requests: + + + + + + + + + + + + + The following examples show the List All Compute Hosts responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Lists all of the instances for the specified host. + + Reviewer: in the DNS project, we have been requested by the customer to provide a table of parameters (should be pulled automatically if parms defined in wadl) and a table of attributes (for calls that allow detailed info about the object created to be specified. No doubt our DB customers will want this too. + Reviewer: These tables probably need 4 columns: name; parameter type: e.g. template, query, etc.; data type: string, etc.; required?; description. + This operation returns the following information: + + + host name + + + percent of RAM used + + + total RAM on host + + + used RAM on host + + + a list of the compute instances running on the given host + + + If the host name does not exist, a 404 not found error is returned. + + + + + The following examples show the List All Instances for a Host requests: + + + + + + + + + + + + + The following examples show the List All Instances for a Host responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Lists all of the database instances, optionally filtered by deleted status. + + + + + + This query parameter specifies whether or not to list deleted instances. + If true, deleted instances are listed. If false, deleted instances are not listed. + + + + + + The following examples show the List All Instances requests: + + + + + + + + + + + + + The following examples show the List All Instances response: + + + + + + + By default, all instances (both deleted and not deleted) are displayed. + Use the deleted query parameter to list only the deleted or not deleted instances. + + + + &commonFaults; + &getFaults; + + + + + + Returns detailed information for the specified Database Instance. + + Reviewer: in the DNS project, we have been requested by the customer to provide a table of parameters (should be pulled automatically if parms defined in wadl) and a table of attributes (for calls that allow detailed info about the object created to be specified. No doubt our DB customers will want this too. + Reviewer: These tables probably need 4 columns: name; parameter type: e.g. template, query, etc.; data type: string, etc.; required?; description. + This operation returns detailed information about the status and details for the specified database instance. + + + + + The following examples show the List Database Instance Status and Details requests: + + + + + + + + + + + + + The following examples show the List Database Instance Status and Details responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Perform a reboot of the instance. + This operation will reboot the underlying OS along with the mysql instance. + This operation returns a 202 Accepted response. + + + + + The following examples show the Reboot Instance requests: + + + + + + + + + + + + + The following examples show the Reboot Instance responses: + + + + + + + + + + &commonFaults; + &getFaults; + &postPutFaults; + + + + + Update all instances on a specified host. + This operation will call on all guest agents in a specified host to install new versions of itself. + This operation returns a 202 Accepted response. + + + + + The following examples show the Update all Instances on Host requests: + + + + + + + + + + + + + The following examples show the Update all Instances on Host responses: + + + + + + + + + + &commonFaults; + &getFaults; + &postPutFaults; + + + + + + Returns detailed information for the storage device. + + Reviewer: in the DNS project, we have been requested by the customer to provide a table of parameters (should be pulled automatically if parms defined in wadl) and a table of attributes (for calls that allow detailed info about the object created to be specified. No doubt our DB customers will want this too. + Reviewer: These tables probably need 4 columns: name; parameter type: e.g. template, query, etc.; data type: string, etc.; required?; description. + This operation returns detailed information about the details for the storage device. + + + + + The following examples show the List Storage Device Details requests: + + + + + + + + + + + + + The following examples show the List Storage Device Details responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Returns detailed information for the specified account. + + Reviewer: This section needs examples. + This operation returns: + + + account name + + + id + + + list of hosts that have the id + + + instance name + + + status of each instance on the hosts + + + + + + + The following examples show the List Account Details requests: + + + + + + + + + + + + + The following examples show the List Account Details responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Gets the root flag details for the specified instance. + + Reviewer: Need the description for the summary table above and the detailed description for the next paragraph. + This operation shows the root flag details for the specified instance. + + + + + The following examples show the Get Root Details requests: + + + + + + + + + + + + + The following examples show the Get Root Details responses: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Get the instance agent and other instance diagnostics info. + + This operation provides the following info for the guest agent managing the specified instance. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Return Attributes for Get Diagnostics Info
NameDescription
versionThe version of the guest installed on the instance.
fdSizeNumber of file descriptor slots currently allocated.
vmSizeVirtual memory size.
vmPeakPeak virtual memory size.
vmRssResident set size.
vmHwmPeak resident set size ("high water mark").
threadsNumber of threads in process containing this thread.
+ + + + + The following examples show the Get Diagnostics Info requests: + + + + + + + + + + + + + The following examples show the Get Diagnostics Info requests: + + + + + + + + + + &commonFaults; + &getFaults; + + + + + + Get the instance hardware info from the guest agent. + + This operation provides the following info from the guest agent managing the specified instance. + + + + + + + + + + + + + + + + + + +
Return Attributes for Get Hardware Info
NameDescription
mem_totalTotal memory an instance sees.
num_cpusNumber of CPUs an instance sees.
+ + + + + The following examples show the Get Hardware Info requests: + + + + + + + + + + + + + The following examples show the Get Hardware Info responses: + + + + + + + + + + &commonFaults; + &getFaults; + + diff --git a/integration/xsd/management.xsd b/integration/xsd/management.xsd new file mode 100644 index 0000000000..4b69123f5b --- /dev/null +++ b/integration/xsd/management.xsd @@ -0,0 +1,398 @@ + + + + + + + + Dbaas Management + + +

This schema defines entity contains entities related to the dbaas Management API.

+ + + + + + +

Information about a Host.

+ + + + + + +

List of Hosts.

+ + + + + + + +

Information about storage devices.

+ + + + + + + +

Information about Account.

+ + + + + + +

Details of a configuration item

+ + + + + + +

List of Configs.

+ + + + + + + +

Details of a configuration item

+ + + + + + + +

Diagnostic details of a guest.

+ + + + + + + +

Reboot an Instance

+ + + + + + + + + + +

A list of Host names.

+ + + + + + + + + + + +

Timestamp of the first time MySQL root was enabled for the instance.

+ + + + + + +

ID of the first user to enable MySQL root for the instance.

+ + + + + + + + + + +

A list of database instances, with additional management information.

+ + + + + + + + + + + + + + + + + + + + + + +

A single database instance, with some additional management information.

+ + + + + + + + + + + + + + + + + + +

A list of database index instances, with some additional management information.

+ + + + + + + + + + + +

A list of database instances on the Host.

+ + + + + + + +

The name of the Host.

+ + + + + + +

The percent of the used RAM on the Host.

+ + + + + + +

The total amount of RAM on the Host.

+ + + + + + +

The used amount of RAM on the Host.

+ + + + + + + + + + +

A storage device on the Host.

+ + + + + + + + + + +

The id of the storage device.

+ + + + + + +

The name of the storage device.

+ + + + + + +

The type of the storage device.

+ + + + + + +

The available size of the storage device.

+ + + + + + +

The total size of the storage device.

+ + + + + + + + + +

The name of the account.

+ + + + + + + +

List of hosts that the Account has instances running on.

+ + + + + + + + + + + +

A configuration entry

+ + + + + + + + + + +

The configuration entry key

+ + + + + + +

The value of the configuration entry

+ + + + + + +

The description of the configuration entry

+ + + + + + + + + +

The instance id of that is checked by the call.

+ + + + + + +

The date timestamp that a user enabled root on the instance.

+ + + + + + +

The user that enabled root on the instance.

+ + + + + + + + + +

The version of the guest installed on the instance.

+ + + + + + +

Number of file descriptor slots currently allocated.

+ + + + + + +

Virtual memory size.

+ + + + + + +

Peak virtual memory size.

+ + + + + + +

Resident set size.

+ + + + + + +

Peak resident set size ("high water mark").

+ + + + + + +

Number of threads in process containing this thread.

+ + + + + + diff --git a/tox.ini b/tox.ini index 0d1a00bb3f..73e4b114cb 100644 --- a/tox.ini +++ b/tox.ini @@ -73,7 +73,7 @@ show-source = True # The rest of the ignores are TODOs. ignore = F821,H301,H404,H405,H501 builtins = _ -exclude=.venv,.tox,.git,dist,doc,*egg,tools,etc,build,*.po,*.pot +exclude=.venv,.tox,.git,dist,doc,*egg,tools,etc,build,*.po,*.pot,integration filename=*.py,trove-* [testenv:api-ref]