Removes openstack-object-storage-admin directory, moving content.
Moves config info to Config Ref Moves Admin CLI task to Admin User Guide Moves CLI tasks to User Guide Moves Monitoring section to Cloud Admin Guide Removes anything that was a copy/paste from dev docs completely, refers to developer docs instead Change-Id: If201d68c1e72a24c57a61775bc22d1784beea324 Closes-bug: 1216037
This commit is contained in:
annegentle
parent
4f6cb137c1
commit
82fa21b9d0
@ -44,6 +44,16 @@
|
||||
</abstract>
|
||||
<revhistory>
|
||||
<!-- ... continue adding more revisions here as you change this document using the markup shown below... -->
|
||||
<revision>
|
||||
<date>2013-09-05</date>
|
||||
<revdescription>
|
||||
<itemizedlist spacing="compact">
|
||||
<listitem>
|
||||
<para>Moves object storage monitoring section to this guide.</para></listitem>
|
||||
<listitem><para>Removes redundant object storage info.</para></listitem>
|
||||
</itemizedlist>
|
||||
</revdescription>
|
||||
</revision>
|
||||
<revision>
|
||||
<date>2013-09-03</date>
|
||||
<revdescription>
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,17 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<section
|
||||
xmlns="http://docbook.org/ns/docbook"
|
||||
xmlns:xi="http://www.w3.org/2001/XInclude"
|
||||
xmlns:xlink="http://www.w3.org/1999/xlink"
|
||||
version="5.0"
|
||||
xml:id="ch_running-openstack-object-storage">
|
||||
<title>System Administration for OpenStack Object Storage</title>
|
||||
<para>By understanding the concepts inherent to the Object Storage
|
||||
system you can better monitor and administer your storage
|
||||
solution. The majority of the administration information is maintained
|
||||
in developer documentation at
|
||||
<link xlink:href="http://docs.openstack.org/developer/swift/">docs.openstack.org/developer/swift/</link>.</para>
|
||||
<para>Refer to the <citetitle>OpenStack Configuration Reference</citetitle>
|
||||
for a listing of all configuration options for OpenStack Object
|
||||
Storage.</para>
|
||||
</section>
|
||||
@ -1,7 +1,9 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<chapter xmlns="http://docbook.org/ns/docbook"
|
||||
<section xmlns="http://docbook.org/ns/docbook"
|
||||
xmlns:xi="http://www.w3.org/2001/XInclude"
|
||||
xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0" xml:id="ch_introduction-to-openstack-object-storage-monitoring">
|
||||
xmlns:xlink="http://www.w3.org/1999/xlink"
|
||||
version="5.0"
|
||||
xml:id="ch_introduction-to-openstack-object-storage-monitoring">
|
||||
<title>OpenStack Object Storage Monitoring</title>
|
||||
<?dbhtml stop-chunking?>
|
||||
<para>Excerpted from a blog post by <link
|
||||
@ -228,5 +230,4 @@ def process_container(self, dbfile):
|
||||
benefit of not introducing another external library
|
||||
dependency into Object Storage.</para>
|
||||
</section>
|
||||
|
||||
</chapter>
|
||||
</section>
|
||||
13
doc/src/docbkx/common/section_about-object-storage.xml
Normal file
13
doc/src/docbkx/common/section_about-object-storage.xml
Normal file
@ -0,0 +1,13 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<section xmlns="http://docbook.org/ns/docbook"
|
||||
xmlns:xi="http://www.w3.org/2001/XInclude"
|
||||
xmlns:xlink="http://www.w3.org/1999/xlink"
|
||||
version="5.0"
|
||||
xml:id="ch_introduction-to-openstack-object-storage">
|
||||
<title>Introduction to OpenStack Object Storage</title>
|
||||
<para>OpenStack Object Storage is a scalable object storage system - it is not a file system in the
|
||||
traditional sense. You will not be able to mount this system like traditional SAN or NAS volumes.
|
||||
Since OpenStack Object Storage is a different way of thinking when it comes to storage, take a few
|
||||
moments to review the key concepts in the developer documentation at
|
||||
<link xlink:href="http://docs.openstack.org/developer/swift/">docs.openstack.org/developer/swift/</link>.</para>
|
||||
</section>
|
||||
@ -8,26 +8,8 @@
|
||||
background daemons, and paste.deploy to manage server configurations. Default configuration
|
||||
options are set in the <code>[DEFAULT]</code> section, and any options specified there can
|
||||
be overridden in any of the other sections.</para>
|
||||
<xi:include href="../common/section_about-object-storage.xml"/>
|
||||
<xi:include href="object-storage/section_object-storage-general-service-conf.xml"/>
|
||||
<!--these need to be placed under appropriate sections
|
||||
<section xml:id="memcache-conf">
|
||||
<title>Sample Memcache configuration file</title>
|
||||
<programlisting language="ini"><xi:include parse="text" href="https://raw.github.com/openstack/swift/master/etc/memcache.conf-sample"/></programlisting></section>
|
||||
<section xml:id="mime-types-conf">
|
||||
<title>Sample Mime.Types configuration file</title>
|
||||
<programlisting language="ini"><xi:include parse="text" href="https://raw.github.com/openstack/swift/master/etc/mime.types-sample"/></programlisting></section>
|
||||
<section xml:id="object-expirer-conf">
|
||||
<title>Sample Object Expirer configuration file</title>
|
||||
<programlisting><xi:include parse="text" href="https://raw.github.com/openstack/swift/master/etc/object-expirer.conf-sample"/></programlisting></section>
|
||||
<section xml:id="rsyncd-conf">
|
||||
<title>Sample Rsyncd configuration file</title>
|
||||
<programlisting language="ini"><xi:include parse="text" href="https://raw.github.com/openstack/swift/master/etc/rsyncd.conf-sample"/></programlisting></section>
|
||||
<section xml:id="swift-bench-conf">
|
||||
<title>Sample Swift bench configuration file</title>
|
||||
<programlisting language="ini"><xi:include parse="text" href="https://raw.github.com/openstack/swift/master/etc/swift-bench.conf-sample"/></programlisting></section>
|
||||
<section xml:id="swift-conf">
|
||||
<title>Sample Swift configuration file</title>
|
||||
<programlisting><xi:include parse="text" href="https://raw.github.com/openstack/swift/master/etc/swift.conf-sample"/></programlisting></section>-->
|
||||
<section xml:id="object-server-configuration">
|
||||
<title>Object Server Configuration</title>
|
||||
<para>An example Object Server configuration can be found at
|
||||
|
||||
@ -1,134 +0,0 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<chapter xmlns="http://docbook.org/ns/docbook"
|
||||
xmlns:xi="http://www.w3.org/2001/XInclude"
|
||||
xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0" xml:id="ch_introduction-to-openstack-object-storage">
|
||||
<title>Introduction to OpenStack Object Storage</title>
|
||||
<para>OpenStack Object Storage is a scalable object storage system - it is not a file system in
|
||||
the traditional sense. You will not be able to mount this system like traditional SAN or NAS
|
||||
volumes. Since OpenStack Object Storage is a different way of thinking when it comes to
|
||||
storage, take a few moments to review the key concepts listed below.</para>
|
||||
<section xml:id="accounts-and-account-servers">
|
||||
<title>Accounts and Account Servers</title>
|
||||
<para>The OpenStack Object Storage system is designed to be used by many different storage
|
||||
consumers or customers. Each user must identify themselves using an authentication
|
||||
system.</para>
|
||||
<para>Nodes that run the Account service are a separate concept from individual accounts.
|
||||
Account servers are part of the storage system and must be configured along with
|
||||
Container servers and Object servers.</para>
|
||||
</section>
|
||||
<section xml:id="authentication-and-access-permissions">
|
||||
<title>Authentication and Access Permissions</title>
|
||||
<para>You must authenticate against an Authentication service to receive OpenStack Object
|
||||
Storage connection parameters and an authentication token. The token must be passed in
|
||||
for all subsequent container/object operations. One authentication service that you can
|
||||
use as a middleware example is called swauth and you can download it from
|
||||
<link xlink:href="https://github.com/gholt/swauth">https://github.com/gholt/swauth</link>. You can also integrate with the OpenStack Identity Service, code-named Keystone, which you can download from <link xlink:href="https://github.com/openstack/keystone">https://github.com/openstack/keystone</link>.</para>
|
||||
<note>
|
||||
<para>Typically the language-specific APIs handle authentication, token passing, and HTTPS
|
||||
request/response communication.</para>
|
||||
</note>
|
||||
|
||||
<para>You can implement access control for objects either for users or accounts using
|
||||
X-Container-Read: accountname and X-Container-Write: accountname:username, which allows
|
||||
any user from the accountname account to read but only allows the username user from the
|
||||
accountname account to write. You can also grant public access to objects stored in
|
||||
OpenStack Object Storage but also limit public access using the Referer header to
|
||||
prevent site-based content theft such as hot-linking (for example, linking to an image
|
||||
file from off-site and therefore using other's bandwidth). The public container settings
|
||||
are used as the default authorization over access control lists. For example, using
|
||||
X-Container-Read: referer:any allows anyone to read from the container regardless of
|
||||
other authorization settings.</para>
|
||||
<para>Generally speaking, each user has their own storage account and has full access to
|
||||
that account. Users must authenticate with their credentials as described above, but
|
||||
once authenticated they can create/delete containers and objects within that account.
|
||||
The only way a user can access the content from another account is if they share an API
|
||||
access key or a session token provided by your authentication system.</para>
|
||||
</section>
|
||||
<section xml:id="containers-and-objects">
|
||||
<title>Containers and Objects</title>
|
||||
<para>A container is a storage compartment for your data and provides a way for you to
|
||||
organize your data. You can think of a container as a folder in Windows® or a directory in
|
||||
UNIX®. The primary difference between a container and these other file system concepts is
|
||||
that containers cannot be nested. You can, however, create an unlimited number of containers
|
||||
within your account. Data must be stored in a container so you must have at least one
|
||||
container defined in your account prior to uploading data.</para>
|
||||
<para>The only restrictions on container names is that they cannot contain a forward slash
|
||||
(<code>/</code>) or an ascii null (<code>%00</code>) and must be less than 257 bytes in length.
|
||||
Please note that the length
|
||||
restriction applies to the name after it has been URL encoded. For example, a container name
|
||||
of <code>Course Docs</code> would be URL encoded as <code>Course%20Docs</code> and therefore
|
||||
be 13 bytes in length rather than the expected 11.</para>
|
||||
|
||||
<para>An object is the basic storage entity and any optional metadata that represents the
|
||||
files you store in the OpenStack Object Storage system. When you upload data to OpenStack Object Storage, the data is
|
||||
stored as-is (no compression or encryption) and consists of a location (container), the
|
||||
object's name, and any metadata consisting of key/value pairs. For instance, you may chose
|
||||
to store a backup of your digital photos and organize them into albums. In this case, each
|
||||
object could be tagged with metadata such as <code>Album : Caribbean Cruise</code> or
|
||||
<code>Album : Aspen Ski Trip</code>.</para>
|
||||
<para>The only restriction on object names is that they must be less than 1024 bytes in length
|
||||
after URL encoding. For example, an object name of <code>C++final(v2).txt</code> should be
|
||||
URL encoded as <code>C%2B%2Bfinal%28v2%29.txt</code> and therefore be 24 bytes in length
|
||||
rather than the expected 16.</para>
|
||||
<para>The maximum allowable size for a storage object upon upload is 5 gigabytes (GB) and
|
||||
the minimum is zero bytes. You can use the built-in large object support and the swift
|
||||
utility to retrieve objects larger than 5 GB.</para>
|
||||
<para>For metadata, you should not exceed 90 individual key/value pairs for any one object
|
||||
and the total byte length of all key/value pairs should not exceed 4KB (4096 bytes).</para>
|
||||
<note><title>Warning</title><para>The Object Storage API uses HTTP which specifies that header
|
||||
fields, such as those used to send metadata are case insensitive. Please do not expect
|
||||
the case of metadata items being preserved.</para></note>
|
||||
</section>
|
||||
<section xml:id="operations">
|
||||
<title>Operations</title>
|
||||
<para>Operations are the actions you perform within an OpenStack Object Storage system such
|
||||
as creating or deleting containers, uploading or downloading objects, and so on. The
|
||||
full list of operations is documented in the Developer Guide. Operations may be
|
||||
performed via the REST web service API or a language-specific API; currently, we support
|
||||
Python, PHP, Java, Ruby, and C#/.NET.</para>
|
||||
<important>
|
||||
<para>All operations must include a valid authorization token from your authorization system.</para>
|
||||
</important>
|
||||
</section>
|
||||
|
||||
<section xml:id="language-specific-api-bindings">
|
||||
<title>Language-Specific API Bindings</title>
|
||||
<para>A set of supported API bindings in several popular languages are available from the
|
||||
Rackspace Cloud Files product, which uses OpenStack Object Storage code for its
|
||||
implementation. These bindings provide a layer of abstraction on top of the base REST
|
||||
API, allowing programmers to work with a container and object model instead of working
|
||||
directly with HTTP requests and responses. These bindings are free (as in beer and as in
|
||||
speech) to download, use, and modify. They are all licensed under the MIT License as
|
||||
described in the COPYING file packaged with each binding. If you do make any
|
||||
improvements to an API, you are encouraged (but not required) to submit those changes
|
||||
back to us.</para>
|
||||
<para>The API bindings for Rackspace Cloud Files are hosted at <link
|
||||
xlink:href="http://github.com/rackspace">http://github.com/rackspace</link>. Feel
|
||||
free to coordinate your changes through github or, if you prefer, send your changes to
|
||||
cloudfiles@rackspacecloud.com. Just make sure to indicate which language and version you
|
||||
modified and send a unified diff.</para>
|
||||
<para>Each binding includes its own documentation (either HTML, PDF, or CHM). They also
|
||||
include code snippets and examples to help you get started. The currently supported API
|
||||
binding for OpenStack Object Storage are:</para>
|
||||
<itemizedlist>
|
||||
<listitem>
|
||||
<para>PHP (requires 5.x and the modules: cURL, FileInfo, mbstring)</para>
|
||||
</listitem>
|
||||
<listitem>
|
||||
<para>Python (requires 2.4 or newer)</para>
|
||||
</listitem>
|
||||
<listitem>
|
||||
<para>Java (requires JRE v1.5 or newer)</para>
|
||||
</listitem>
|
||||
<listitem>
|
||||
<para>C#/.NET (requires .NET Framework v3.5)</para>
|
||||
</listitem>
|
||||
<listitem>
|
||||
<para>Ruby (requires 1.8 or newer and mime-tools module)</para>
|
||||
</listitem>
|
||||
</itemizedlist>
|
||||
<para>There are no other supported language-specific bindings at this time. You are welcome
|
||||
to create your own language API bindings and we can help answer any questions during
|
||||
development, host your code if you like, and give you full credit for your work.</para>
|
||||
</section>
|
||||
</chapter>
|
||||
@ -1,171 +0,0 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<book xmlns="http://docbook.org/ns/docbook"
|
||||
xmlns:xlink="http://www.w3.org/1999/xlink"
|
||||
xmlns:xi="http://www.w3.org/2001/XInclude"
|
||||
xmlns:svg="http://www.w3.org/2000/svg"
|
||||
xmlns:html="http://www.w3.org/1999/xhtml" version="5.0"
|
||||
xml:id="openstack-object-storage-admin-manual-grizzly">
|
||||
<title>OpenStack Object Storage Administration Guide</title>
|
||||
<?rax title.font.size="28px" subtitle.font.size="28px"?>
|
||||
<titleabbrev>OpenStack Object Storage Administration
|
||||
Guide</titleabbrev>
|
||||
<info>
|
||||
<author>
|
||||
<personname>
|
||||
<firstname/>
|
||||
<surname/>
|
||||
</personname>
|
||||
<affiliation>
|
||||
<orgname>OpenStack Foundation</orgname>
|
||||
</affiliation>
|
||||
</author>
|
||||
<copyright>
|
||||
<year>2010</year>
|
||||
<year>2011</year>
|
||||
<year>2012</year>
|
||||
<year>2013</year>
|
||||
<holder>OpenStack Foundation</holder>
|
||||
</copyright>
|
||||
<releaseinfo>Grizzly, 2013.1 with Object Storage
|
||||
1.8.0</releaseinfo>
|
||||
<productname>OpenStack Object Storage</productname>
|
||||
<pubdate/>
|
||||
<legalnotice role="apache2">
|
||||
<annotation>
|
||||
<remark>Copyright details are filled in by the
|
||||
template.</remark>
|
||||
</annotation>
|
||||
</legalnotice>
|
||||
<abstract>
|
||||
<para>OpenStack™ Object Storage offers open source
|
||||
software for cloud-based object storage for any
|
||||
organization. This manual provides guidance for
|
||||
installing, managing, and understanding the software
|
||||
that runs OpenStack Object Storage.</para>
|
||||
</abstract>
|
||||
<revhistory>
|
||||
<!-- ... continue adding more revisions here as you change this document using the markup shown below... -->
|
||||
<revision>
|
||||
<date>2013-05-09</date>
|
||||
<revdescription>
|
||||
<para>Updated the book title for
|
||||
consistency.</para>
|
||||
</revdescription>
|
||||
</revision>
|
||||
<revision>
|
||||
<date>2013-04-30</date>
|
||||
<revdescription>
|
||||
<para>Grizzly release.</para>
|
||||
</revdescription>
|
||||
</revision>
|
||||
<revision>
|
||||
<date>2012-12-06</date>
|
||||
<revdescription>
|
||||
<itemizedlist spacing="compact">
|
||||
<listitem>
|
||||
<para>Bug fixes and cleanup. Fix <link
|
||||
xlink:href="https://code.launchpad.net/bugs/1034144"
|
||||
>bug 1034144</link></para>
|
||||
<para>Add section about cluster
|
||||
health.</para>
|
||||
<para>Add a note about the apt-get install
|
||||
for swift3 from Ubuntu Cloud
|
||||
Archive.</para>
|
||||
</listitem>
|
||||
</itemizedlist>
|
||||
</revdescription>
|
||||
</revision>
|
||||
<revision>
|
||||
<date>2012-11-09</date>
|
||||
<revdescription>
|
||||
<itemizedlist spacing="compact">
|
||||
<listitem>
|
||||
<para>Folsom release of this
|
||||
document.</para>
|
||||
</listitem>
|
||||
</itemizedlist>
|
||||
</revdescription>
|
||||
</revision>
|
||||
<revision>
|
||||
<date>2012-08-21</date>
|
||||
<revdescription>
|
||||
<itemizedlist spacing="compact">
|
||||
<listitem>
|
||||
<para>Add note to the [filter:cache]
|
||||
section of the proxy-server.conf file
|
||||
reference indicating that the
|
||||
memcache_servers value is ignored if
|
||||
it is specified in
|
||||
/etc/swift/memcache.conf.</para>
|
||||
</listitem>
|
||||
<listitem>
|
||||
<para>Update Cyberduck tutorial.</para>
|
||||
</listitem>
|
||||
</itemizedlist>
|
||||
</revdescription>
|
||||
</revision>
|
||||
<revision>
|
||||
<date>2012-06-05</date>
|
||||
<revdescription>
|
||||
<itemizedlist spacing="compact">
|
||||
<listitem>
|
||||
<para>Fix <link
|
||||
xlink:href="https://bugs.launchpad.net/openstack-manuals/+bug/1009170"
|
||||
>bug 1009170</link>.</para>
|
||||
</listitem>
|
||||
</itemizedlist>
|
||||
</revdescription>
|
||||
</revision>
|
||||
<revision>
|
||||
<date>2012-05-03</date>
|
||||
<revdescription>
|
||||
<itemizedlist spacing="compact">
|
||||
<listitem>
|
||||
<para>Begin trunk designation.</para>
|
||||
</listitem>
|
||||
</itemizedlist>
|
||||
</revdescription>
|
||||
</revision>
|
||||
<revision>
|
||||
<date>2012-05-02</date>
|
||||
<revdescription>
|
||||
<itemizedlist spacing="compact">
|
||||
<listitem>
|
||||
<para>Essex release.</para>
|
||||
</listitem>
|
||||
</itemizedlist>
|
||||
</revdescription>
|
||||
</revision>
|
||||
<revision>
|
||||
<date>2012-04-27</date>
|
||||
<revdescription>
|
||||
<itemizedlist spacing="compact">
|
||||
<listitem>
|
||||
<para>Adds ring copy example.</para>
|
||||
</listitem>
|
||||
</itemizedlist>
|
||||
</revdescription>
|
||||
</revision>
|
||||
<revision>
|
||||
<date>2012-01-01</date>
|
||||
<revdescription>
|
||||
<itemizedlist spacing="compact">
|
||||
<listitem>
|
||||
<para>Formatting updates.</para>
|
||||
</listitem>
|
||||
</itemizedlist>
|
||||
</revdescription>
|
||||
</revision>
|
||||
</revhistory>
|
||||
</info>
|
||||
<!-- Chapters are referred from the book file through these include statements. You can add additional chapters using these types of statements. -->
|
||||
<xi:include href="../common/ch_getstart.xml"/>
|
||||
<xi:include href="aboutobjectstorage.xml"/>
|
||||
<xi:include
|
||||
href="../openstack-install/ch_installobjectstorage.xml"/>
|
||||
<xi:include href="objectstorageadmin.xml"/>
|
||||
<xi:include href="objectstoragetutorials.xml"/>
|
||||
<xi:include href="objectstoragemonitoring.xml"/>
|
||||
<xi:include href="../common/ch_support.xml"/>
|
||||
<xi:include href="../common/section_support-object-storage.xml"/>
|
||||
</book>
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 58 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 62 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 131 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 59 KiB |
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 621 KiB |
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@ -1,182 +0,0 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<chapter xmlns="http://docbook.org/ns/docbook"
|
||||
xmlns:xi="http://www.w3.org/2001/XInclude"
|
||||
xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0"
|
||||
xml:id="ch_openstack-object-storage-tutorials">
|
||||
|
||||
<title>OpenStack Object Storage Tutorials</title>
|
||||
<para>We want people to use OpenStack for practical problem solving, and the increasing size and
|
||||
density of web content makes for a great use-case for object storage. These tutorials show
|
||||
you how to use your OpenStack Object Storage installation for practical purposes, and it
|
||||
assumes Object Storage is already installed.</para>
|
||||
<section xml:id="storing-large-photos-or-videos-on-the-cloud">
|
||||
|
||||
<title>Storing Large Photos or Videos on the Cloud</title>
|
||||
|
||||
<para>In this OpenStack tutorial, we’ll walk through using an Object Storage installation to
|
||||
back up all your photos or videos. As the sensors on consumer-grade and pro-sumer grade
|
||||
cameras generate more and more megapixels, we all need a place to back our files to and
|
||||
know they are safe.</para>
|
||||
|
||||
<para>We'll go through this tutorial in parts:</para>
|
||||
<itemizedlist>
|
||||
<listitem>
|
||||
<para>Setting up secure access to Object Storage.</para></listitem>
|
||||
<listitem><para>Configuring Cyberduck for connecting to OpenStack Object Storage.</para></listitem>
|
||||
<listitem><para>Copying files to the cloud.</para></listitem>
|
||||
</itemizedlist>
|
||||
<section xml:id="part-i-setting-up-secure-access">
|
||||
<title>Part I: Setting Up Secure Access</title>
|
||||
<para>In this part, we'll get the proxy server running with SSL on the Object Storage
|
||||
installation. It's a requirement for using Cyberduck as a client interface to Object
|
||||
Storage.</para>
|
||||
<para>You will need a key and certificate to do this, which we can create as a
|
||||
self-signed for the tutorial since we can do the extra steps to have Cyberduck
|
||||
accept it. Creating a self-signed cert can usually be done with these commands on
|
||||
the proxy server:</para>
|
||||
<screen><prompt>$</prompt> <userinput>cd /etc/swift</userinput>
|
||||
<prompt>$</prompt> <userinput>openssl req -new -x509 -nodes -out cert.crt -keyout cert.key</userinput> </screen>
|
||||
|
||||
<para>Ensure these generated files are in /etc/swift/cert.crt and /etc/swift/cert.key.</para>
|
||||
<para>You also should configure your iptables to enable https traffic. Here's an example
|
||||
setup that works.</para>
|
||||
<programlisting>Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
|
||||
pkts bytes target prot opt in out source destination
|
||||
76774 1543M ACCEPT all -- lo any localhost anywhere
|
||||
416K 537M ACCEPT all -- any any anywhere anywhere state RELATED,ESTABLISHED
|
||||
106 6682 ACCEPT tcp -- any any anywhere anywhere tcp dpt:https
|
||||
13 760 ACCEPT tcp -- any any anywhere anywhere tcp dpt:ssh
|
||||
3 124 ACCEPT icmp -- any any anywhere anywhere icmp echo-request
|
||||
782 38880 DROP all -- any any anywhere anywhere
|
||||
|
||||
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
|
||||
pkts bytes target prot opt in out source destination
|
||||
0 0 DROP all -- any any anywhere anywhere
|
||||
|
||||
Chain OUTPUT (policy ACCEPT 397K packets, 1561M bytes)
|
||||
pkts bytes target prot opt in out source destination</programlisting>
|
||||
<para>If you don't have access to the Object Storage installation to configure these
|
||||
settings, ask your service provider to set up secure access for you.</para><para>Then, edit your <filename>proxy-server.conf</filename> file to
|
||||
include the following in the [DEFAULT] sections.</para>
|
||||
<programlisting>
|
||||
[DEFAULT]
|
||||
bind_port = 443
|
||||
cert_file = /etc/swift/cert.crt
|
||||
key_file = /etc/swift/cert.key
|
||||
</programlisting>
|
||||
<para>Also, make sure you use https: for all references to the URL for the server in the
|
||||
.conf files as needed.</para>
|
||||
<para>Verify that you can connect using the Public URL to Object Storage by using the
|
||||
"swift" tool:</para>
|
||||
<para>
|
||||
<screen><prompt>$</prompt> <userinput>swift -A https://yourswiftinstall.com:11000/v1.0 -U test:tester -K testing stat</userinput></screen>
|
||||
</para>
|
||||
|
||||
<para>Okay, you've created the access that Cyberduck expects for your Object Storage
|
||||
installation. Let's start configuring the Cyberduck side of things.</para>
|
||||
|
||||
</section>
|
||||
<section xml:id="part-ii-configuring-cyberduck">
|
||||
|
||||
<title>Part II: Configuring Cyberduck</title>
|
||||
<note>
|
||||
<para>
|
||||
See the <link xlink:href="http://trac.cyberduck.ch/wiki/help/en/howto/openstack">Cyberduck website</link>
|
||||
for further details.
|
||||
</para>
|
||||
</note>
|
||||
<para>
|
||||
After installing Cyberduck you'll need to change the path/context
|
||||
used for the authentication URL. The default value shipped with
|
||||
Cyberduck is incorrect.
|
||||
</para>
|
||||
<para>
|
||||
On OS X open a Terminal window and execute,
|
||||
<screen><prompt>$</prompt> <userinput>defaults write ch.sudo.cyberduck cf.authentication.context /auth/v1.0</userinput></screen>
|
||||
</para>
|
||||
<para>On Windows open the preferences file in a text
|
||||
editor. The exact location of this file is described
|
||||
<link
|
||||
xlink:href="http://trac.cyberduck.ch/wiki/help/en/faq#Windows1"
|
||||
>here</link>. If this path doesn't exist you may
|
||||
need to start and stop Cyberduck to have it generate
|
||||
the config file. Once you've opened the file add the
|
||||
setting,
|
||||
<programlisting><setting name="cf.authentication.context" value="/auth/v1.0" /></programlisting>
|
||||
</para>
|
||||
<para>
|
||||
To connect to Swift start Cyberduck and click the
|
||||
<emphasis>Open Connection</emphasis> toolbar button or choose
|
||||
<emphasis>File > Open Connection</emphasis>. Select
|
||||
<emphasis>Swift (OpenStack Object Storage)</emphasis> in the
|
||||
dropdown and enter your cluster details.
|
||||
<variablelist>
|
||||
<varlistentry>
|
||||
<term>Server</term>
|
||||
<listitem>
|
||||
<para>your proxy server's hostname or IP address</para>
|
||||
</listitem>
|
||||
</varlistentry>
|
||||
<varlistentry>
|
||||
<term>Port</term>
|
||||
<listitem><para>443</para></listitem>
|
||||
</varlistentry>
|
||||
<varlistentry>
|
||||
<term>Username</term>
|
||||
<listitem>
|
||||
<para>
|
||||
account name followed by a colon and then the user name,
|
||||
for example test:tester
|
||||
</para>
|
||||
</listitem>
|
||||
</varlistentry>
|
||||
<varlistentry>
|
||||
<term>Password</term>
|
||||
<listitem>
|
||||
<para>
|
||||
password for the account and user name entered above
|
||||
</para>
|
||||
</listitem>
|
||||
</varlistentry>
|
||||
</variablelist>
|
||||
<figure>
|
||||
<title>Example Cyberduck Swift Connection</title>
|
||||
<mediaobject>
|
||||
<imageobject>
|
||||
<imagedata fileref="figures/cyberduck_swift_connection.png"/>
|
||||
</imageobject>
|
||||
</mediaobject>
|
||||
</figure>
|
||||
</para>
|
||||
|
||||
<simplesect>
|
||||
<title>Connecting to a Unsecured Swift Cluster</title>
|
||||
<para>An Unsecured Swift Cluster does not use https
|
||||
connections. Download the <link
|
||||
xlink:href="http://trac.cyberduck.ch/raw-attachment/wiki/help/en/howto/openstack/Swift%20Unsecure.cyberduckprofile"
|
||||
>Unsecured Swift profile file</link> and
|
||||
double click it to import it into Cyberduck.</para>
|
||||
<para>
|
||||
When creating a new connection select <emphasis>Swift (HTTP)</emphasis>.
|
||||
Enter your connection details as described above. You'll need
|
||||
to change the port since presumably you won't want to use 443.
|
||||
</para>
|
||||
</simplesect>
|
||||
</section>
|
||||
<section xml:id="part-iii-copying-files">
|
||||
<title>Part III: Creating Containers (Folders) and Uploading Files</title>
|
||||
<para>Now you want to create containers to hold your files. Without containers, Object
|
||||
Storage doesn't know where to put the files. In the Action menu, choose New Folder
|
||||
and name the folder.</para>
|
||||
<para>Next you can drag and drop files into the created folder or select File > Upload
|
||||
to select files to upload to the OpenStack Object Storage service. <figure>
|
||||
<title>Example Cyberduck Swift Showing Uploads</title>
|
||||
<mediaobject>
|
||||
<imageobject>
|
||||
<imagedata fileref="figures/cyberduck_swift_uploads.png"/>
|
||||
</imageobject>
|
||||
</mediaobject>
|
||||
</figure></para><para>Et voila! You can back up terabytes of data if you just have the space and the data. That's a lot of pictures or video, so get snapping and rolling!</para>
|
||||
</section>
|
||||
</section>
|
||||
</chapter>
|
||||
@ -1,118 +0,0 @@
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
|
||||
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<groupId>org.openstack.docs</groupId>
|
||||
<artifactId>openstack-guide</artifactId>
|
||||
<version>1.0.0-SNAPSHOT</version>
|
||||
<packaging>jar</packaging>
|
||||
<name>OpenStack Administration Guides</name>
|
||||
|
||||
<properties>
|
||||
<!-- This is set by Jenkins according to the branch. -->
|
||||
<release.path.name>local</release.path.name>
|
||||
<comments.enabled>1</comments.enabled>
|
||||
</properties>
|
||||
<!-- ################################################ -->
|
||||
<!-- USE "mvn clean generate-sources" to run this POM -->
|
||||
<!-- ################################################ -->
|
||||
<build>
|
||||
<plugins>
|
||||
|
||||
<plugin>
|
||||
<groupId>com.rackspace.cloud.api</groupId>
|
||||
<artifactId>clouddocs-maven-plugin</artifactId>
|
||||
<version>1.9.2</version>
|
||||
<executions>
|
||||
<execution>
|
||||
<id>generate-webhelp</id>
|
||||
<goals>
|
||||
<goal>generate-webhelp</goal>
|
||||
</goals>
|
||||
<phase>generate-sources</phase>
|
||||
<configuration>
|
||||
<!-- These parameters only apply to webhelp -->
|
||||
<enableDisqus>${comments.enabled}</enableDisqus>
|
||||
<disqusShortname>os-object-admin</disqusShortname>
|
||||
<enableGoogleAnalytics>1</enableGoogleAnalytics>
|
||||
<googleAnalyticsId>UA-17511903-1</googleAnalyticsId>
|
||||
<generateToc> appendix toc,title
|
||||
article/appendix nop
|
||||
article toc,title
|
||||
book title,figure,table,example,equation
|
||||
chapter toc,title
|
||||
section toc part
|
||||
toc,title
|
||||
preface toc,title
|
||||
qandadiv toc
|
||||
qandaset toc
|
||||
reference toc,title
|
||||
set toc,title </generateToc>
|
||||
<!-- The following elements sets the autonumbering of sections in output for chapter numbers but no numbered sections-->
|
||||
<sectionAutolabel>0</sectionAutolabel>
|
||||
<tocDepthSection>1</tocDepthSection>
|
||||
<sectionLabelIncludesComponentLabel>0</sectionLabelIncludesComponentLabel>
|
||||
<webhelpDirname>admin</webhelpDirname>
|
||||
<targetDirectory>${basedir}/target/docbkx/webhelp/${release.path.name}/openstack-object-storage</targetDirectory>
|
||||
<includeDateInPdfFilename>0</includeDateInPdfFilename>
|
||||
<pdfFilenameBase>os-objectstorage-adminguide-${release.path.name}</pdfFilenameBase>
|
||||
</configuration>
|
||||
</execution>
|
||||
<execution>
|
||||
<id>cleanup</id>
|
||||
<goals>
|
||||
<goal>generate-webhelp</goal>
|
||||
</goals>
|
||||
<phase>generate-sources</phase>
|
||||
<configuration>
|
||||
<includes>dummy.xml</includes>
|
||||
<postProcess>
|
||||
<delete includeemptydirs="true"
|
||||
verbose="true">
|
||||
<fileset
|
||||
dir="${basedir}/target/docbkx/webhelp/${release.path.name}">
|
||||
<include name="**/*"/>
|
||||
<exclude name="openstack-object-storage/**"/>
|
||||
</fileset>
|
||||
</delete>
|
||||
</postProcess>
|
||||
</configuration>
|
||||
</execution>
|
||||
</executions>
|
||||
<configuration>
|
||||
<!-- These parameters apply to pdf and webhelp -->
|
||||
<xincludeSupported>true</xincludeSupported>
|
||||
<sourceDirectory>.</sourceDirectory>
|
||||
<includes> bk-objectstorage-adminguide.xml </includes>
|
||||
<canonicalUrlBase>http://docs.openstack.org/${release.path.name}/openstack-object-storage/admin/content/</canonicalUrlBase>
|
||||
<profileSecurity>reviewer</profileSecurity>
|
||||
<branding>openstack</branding>
|
||||
</configuration>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
<profiles>
|
||||
<profile>
|
||||
<id>Rackspace Research Repositories</id>
|
||||
<activation>
|
||||
<activeByDefault>true</activeByDefault>
|
||||
</activation>
|
||||
<repositories>
|
||||
<repository>
|
||||
<id>rackspace-research</id>
|
||||
<name>Rackspace Research Repository</name>
|
||||
<url>http://maven.research.rackspacecloud.com/content/groups/public/</url>
|
||||
</repository>
|
||||
</repositories>
|
||||
<pluginRepositories>
|
||||
<pluginRepository>
|
||||
<id>rackspace-research</id>
|
||||
<name>Rackspace Research Repository</name>
|
||||
<url>http://maven.research.rackspacecloud.com/content/groups/public/</url>
|
||||
</pluginRepository>
|
||||
</pluginRepositories>
|
||||
</profile>
|
||||
</profiles>
|
||||
</project>
|
||||
@ -31,4 +31,6 @@
|
||||
<xi:include href="section_nova_cli_manage_flavors.xml"/>
|
||||
<?hard-pagebreak?>
|
||||
<xi:include href="section_keystone_cli_set_quotas.xml"/>
|
||||
<?hard-pagebreak?>
|
||||
<xi:include href="section_swift_cli_analyze_log_files.xml"/>
|
||||
</chapter>
|
||||
|
||||
@ -0,0 +1,151 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<section
|
||||
xmlns="http://docbook.org/ns/docbook"
|
||||
xmlns:xi="http://www.w3.org/2001/XInclude"
|
||||
xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0"
|
||||
xml:id="analyzing-log-files-with-swift-cli">
|
||||
<title>Analyzing Log Files with Swift CLI</title>
|
||||
<para>When you want quick, command-line answers to
|
||||
questions about logs, you can use swift with the -o or
|
||||
-output option. The -o —output option can only be used
|
||||
with a single object download to redirect the data
|
||||
stream to either a different file name or to STDOUT
|
||||
(-). The ability to redirect the output to STDOUT
|
||||
allows you to pipe “|” data without saving it to disk
|
||||
first. One common use case is being able to do some
|
||||
quick log file analysis. First let’s use swift to
|
||||
setup some data for the examples. The “logtest”
|
||||
directory contains four log files with the following
|
||||
line format.</para>
|
||||
<para>
|
||||
<literallayout>Files:
|
||||
2010-11-16-21_access.log
|
||||
2010-11-16-22_access.log
|
||||
2010-11-15-21_access.log
|
||||
2010-11-15-22_access.log
|
||||
|
||||
Log lines:
|
||||
Nov 15 21:53:52 lucid64 proxy-server - 127.0.0.1 15/Nov/2010/22/53/52 DELETE /v1/AUTH_cd4f57824deb4248a533f2c28bf156d3/2eefc05599d44df38a7f18b0b42ffedd HTTP/1.0 204 - \
|
||||
- test%3Atester%2CAUTH_tkcdab3c6296e249d7b7e2454ee57266ff - - - txaba5984c-aac7-460e-b04b-afc43f0c6571 - 0.0432
|
||||
</literallayout>
|
||||
</para>
|
||||
<para>The swift tool can easily upload the four log files
|
||||
into a container named “logtest”:</para>
|
||||
<para><screen>
|
||||
<prompt>$</prompt> <userinput>cd logs</userinput>
|
||||
<prompt>$</prompt> <userinput>swift -A http://swift-auth.com:11000/v1.0 -U test:tester -K testing upload logtest *.log</userinput>
|
||||
</screen>
|
||||
<programlisting>
|
||||
2010-11-16-21_access.log
|
||||
2010-11-16-22_access.log
|
||||
2010-11-15-21_access.log
|
||||
2010-11-15-22_access.log
|
||||
</programlisting>
|
||||
Get statistics on the account: <screen><prompt>$</prompt> <userinput>swift -A http://swift-auth.com:11000/v1.0 -U test:tester -K testing -q stat</userinput></screen>
|
||||
<programlisting>
|
||||
Account: AUTH_cd4f57824deb4248a533f2c28bf156d3
|
||||
Containers: 1
|
||||
Objects: 4
|
||||
Bytes: 5888268
|
||||
</programlisting>
|
||||
Get statistics on the container:
|
||||
<screen><prompt>$</prompt> <userinput>swift -A http://swift-auth.com:11000/v1.0 -U test:tester -K testing stat logtest</userinput> </screen><programlisting>Account: AUTH_cd4f57824deb4248a533f2c28bf156d3
|
||||
Container: logtest
|
||||
Objects: 4
|
||||
Bytes: 5864468
|
||||
Read ACL:
|
||||
Write ACL:
|
||||
</programlisting>
|
||||
List all the objects in the container:
|
||||
<screen><prompt>$</prompt> <userinput>swift -A http:///swift-auth.com:11000/v1.0 -U test:tester -K testing list logtest</userinput></screen><programlisting>2010-11-15-21_access.log
|
||||
2010-11-15-22_access.log
|
||||
2010-11-16-21_access.log
|
||||
2010-11-16-22_access.log</programlisting>
|
||||
</para>
|
||||
<para>The following examples use the -o —output option
|
||||
with (-) to help answer questions about the uploaded
|
||||
log files. The swift command will download an object,
|
||||
stream it to awk to determine the breakdown of
|
||||
requests by return code for everything during 2200 on
|
||||
November 16th, 2010. Based on the log line format
|
||||
column 9 is the type of request and column 12 is the
|
||||
return code. After awk processes the data stream it is
|
||||
piped to sort and then uniq -c to sum up the number of
|
||||
occurrences for each combination of request type and
|
||||
return code.</para>
|
||||
<para><screen>
|
||||
<prompt>$</prompt> <userinput>swift -A http://swift-auth.com:11000/v1.0 -U test:tester -K testing download -o - logtest 2010-11-16-22_access.log | awk ‘{ print $9”-“$12}’ | sort | uniq -c</userinput>
|
||||
</screen>
|
||||
<programlisting>805 DELETE-204
|
||||
12 DELETE-404
|
||||
2 DELETE-409
|
||||
723 GET-200
|
||||
142 GET-204
|
||||
74 GET-206
|
||||
80 GET-304
|
||||
34 GET-401
|
||||
5 GET-403
|
||||
18 GET-404
|
||||
166 GET-412
|
||||
2 GET-416
|
||||
50 HEAD-200
|
||||
17 HEAD-204
|
||||
20 HEAD-401
|
||||
8 HEAD-404
|
||||
30 POST-202
|
||||
25 POST-204
|
||||
22 POST-400
|
||||
6 POST-404
|
||||
842 PUT-201
|
||||
2 PUT-202
|
||||
32 PUT-400
|
||||
4 PUT-403
|
||||
4 PUT-404
|
||||
2 PUT-411
|
||||
6 PUT-412
|
||||
6 PUT-413
|
||||
2 PUT-422
|
||||
8 PUT-499</programlisting>
|
||||
</para>
|
||||
<para>This example uses a bash for loop with awk, swift
|
||||
with its -o —output option with a hyphen (-) to find
|
||||
out how many PUT requests are in each log file. First
|
||||
create a list of objects by running swift with the
|
||||
list command on the “logtest” container; then for each
|
||||
item in the list run swift with download -o - then
|
||||
pipe the output into grep to filter the put requests
|
||||
and finally into wc -l to count the lines.</para>
|
||||
<para>
|
||||
<screen><prompt>$</prompt> <userinput>for f in `swift -A http://swift-auth.com:11000/v1.0 -U test:tester -K testing list logtest` ; \
|
||||
do echo -ne “PUTS - ” ; swift -A http://swift-auth.com:11000/v1.0 -U test:tester -K testing download -o - logtest $f | grep PUT | wc -l ; \
|
||||
done</userinput></screen>
|
||||
<programlisting>
|
||||
2010-11-15-21_access.log - PUTS - 402
|
||||
2010-11-15-22_access.log - PUTS - 1091
|
||||
2010-11-16-21_access.log - PUTS - 892
|
||||
2010-11-16-22_access.log - PUTS - 910
|
||||
</programlisting>
|
||||
</para>
|
||||
<para>By adding the -p —prefix option a prefix query is
|
||||
performed on the list to return only the object names
|
||||
that begin with a specific string. Let’s determine out
|
||||
how many PUT requests are in each object with a name
|
||||
beginning with “2010-11-15”. First create a list of
|
||||
objects by running swift with the list command on the
|
||||
“logtest” container with the prefix option -p
|
||||
2010-11-15. Then on each of item(s) returned run swift
|
||||
with the download -o - then pipe the output to grep
|
||||
and wc as in the previous example. The echo command is
|
||||
added to display the object name.</para>
|
||||
<para>
|
||||
<screen><prompt>$</prompt> <userinput>for f in `swift -A http://swift-auth.com:11000/v1.0 -U test:tester -K testing list -p 2010-11-15 logtest` ; \
|
||||
do echo -ne “$f - PUTS - ” ; swift -A http://127.0.0.1:11000/v1.0 -U test:tester -K testing download -o - logtest $f | grep PUT | wc -l ; \
|
||||
done</userinput></screen>
|
||||
<programlisting>2010-11-15-21_access.log - PUTS - 402
|
||||
2010-11-15-22_access.log - PUTS - 910</programlisting>
|
||||
</para>
|
||||
<para>The swift utility is simple, scalable, flexible and
|
||||
provides useful solutions all of which are core
|
||||
principles of cloud computing; with the -o output
|
||||
option being just one of its many features.</para>
|
||||
</section>
|
||||
@ -0,0 +1,52 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<section
|
||||
xmlns="http://docbook.org/ns/docbook"
|
||||
xmlns:xi="http://www.w3.org/2001/XInclude"
|
||||
xmlns:xlink="http://www.w3.org/1999/xlink"
|
||||
version="5.0"
|
||||
xml:id="managing-openstack-object-storage-with-swift-cli">
|
||||
<title>Managing OpenStack Object Storage with CLI
|
||||
Swift</title>
|
||||
<para>In the Object Store (swift) project there is a tool that
|
||||
can perform a variety of tasks on your storage cluster
|
||||
named swift. This client utility can be used for adhoc
|
||||
processing, to gather statistics, list items, update
|
||||
metadata, upload, download and delete files. It is based
|
||||
on the native swift client library client.py.
|
||||
Incorporating client.py into swift provides many benefits
|
||||
such as seamlessly re-authorizing if the current token
|
||||
expires in the middle of processing, retrying operations
|
||||
up to five times and a processing concurrency of 10. All
|
||||
of these things help make the swift tool robust and great
|
||||
for operational use.</para>
|
||||
<section xml:id="swift-acls">
|
||||
<title>Swift ACLs</title>
|
||||
<para>Swift ACLs work with users and accounts. Users have
|
||||
roles on accounts - such as '.admin', which allows
|
||||
full access to all containers and objects under the
|
||||
account. ACLs are set at the container level and
|
||||
support lists for read and write access, which are set
|
||||
with the X-Container-Read and X-Container-Write header
|
||||
respectively.</para>
|
||||
<para>The swift client can be used to set the acls, using
|
||||
the post subcommand with the option '-r' for the read
|
||||
ACL, and '-w' for the write ACL. This example allows
|
||||
the user 'testuser' to read objects in the container:
|
||||
<programlisting>
|
||||
$ swift post -r 'testuser'
|
||||
</programlisting>
|
||||
This could instead be a list of users.</para>
|
||||
<para>If you are using the StaticWeb middleware to allow
|
||||
OpenStack Object Storage to serve public web content,
|
||||
you should also be aware of the ACL syntax for
|
||||
managing allowed referrers. The syntax is '.r:'
|
||||
followed by a list of allowed referrers. For example,
|
||||
this command allows all referring domains access to
|
||||
the object:
|
||||
<programlisting>
|
||||
$ swift post -r '.r:*'
|
||||
</programlisting>
|
||||
</para>
|
||||
</section>
|
||||
|
||||
</section>
|
||||
Loading…
Reference in New Issue
Block a user