Browse Source

add documentation

Change-Id: I37a11f198d020067283ca5c154e610f2ff6770c1
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Doug Hellmann 2 years ago
parent
commit
34454593ff

+ 3
- 3
README.rst View File

@@ -1,6 +1,6 @@
1
-==================================================
2
- downpour -- OpenStack Tenant Data Migration Tool
3
-==================================================
1
+===================================================
2
+ downpour --- OpenStack Tenant Data Migration Tool
3
+===================================================
4 4
 
5 5
 downpour exports tenant data from an OpenStack cloud to create a set
6 6
 of Ansible playbooks for importing the data into another cloud.

+ 2
- 1
demo/tiny-resources.yml View File

@@ -1,7 +1,8 @@
1 1
 # Resource file for downpour using the instance created in tiny.yml.
2 2
 servers:
3 3
   - name: downpour-demo-tiny
4
-    # Create the server using a separate key than it was created with in tiny.yml.
4
+    # Create the server using a separate key than
5
+    # it was created with in tiny.yml.
5 6
     key_name: downpour-demo2
6 7
 keypairs:
7 8
   - name: downpour-demo

+ 57
- 0
doc/source/background.rst View File

@@ -0,0 +1,57 @@
1
+============
2
+ Background
3
+============
4
+
5
+Downpour is being created to solve the problem of moving workloads
6
+between clouds. It is only one of several possible approaches to the
7
+problem, and fits into a very specific niche at the hard end of the
8
+range of use cases.
9
+
10
+.. list-table::
11
+   :header-rows: 1
12
+
13
+   - * 
14
+     * Easy
15
+     * Moderate
16
+     * Hard
17
+   - * **Ownership**
18
+     * Operator
19
+     * Admin
20
+     * Tenant
21
+   - * **Backend**
22
+     * Shared storage
23
+     * Fast interconnect
24
+     * Shared nothing
25
+   - * **Applications**
26
+     * One per tenant
27
+     * Multi-app with naming conventions
28
+     * Rats nest
29
+
30
+Downpour does not assume the user has an special access to the cloud,
31
+either as an operator with access to backend systems or via admin
32
+APIs.
33
+
34
+Downpour does not assume that the source and destination clouds are
35
+connected in any way. Not only is it possible to move data between
36
+clouds that do not share backend services, it is possible to move data
37
+between clouds that cannot be accessed from the same system at the
38
+same time.
39
+
40
+Downpour does not make any assumptions about the mapping between
41
+applications and tenants. It is possible to extract only part of the
42
+resources owned by a tenant. The grouping is completely up to the
43
+user, and can represent an application or a single node in a multi-VM
44
+configuration.
45
+
46
+Downpour does not assume the source and destination clouds are build
47
+using the same architecture or configured in the same way. As long as
48
+the two clouds pass the standard OpenStack interoperability tests, it
49
+should be possible to use Downpour to move your workload.
50
+
51
+These requirements do come with trade-offs, the impact of which will
52
+depend on how "cloud native" an application really is. For example,
53
+the benefits of copy-on-write images may be lost during the migration
54
+if the entire image from each VM needs to be uploaded into the new
55
+cloud. The UUIDs associated with resources will also change, since
56
+there is no way to guarantee the assignment of a specific UUID for
57
+resources created in a separate cloud.

+ 2
- 2
doc/source/conf.py View File

@@ -23,7 +23,6 @@ sys.path.insert(0, os.path.abspath('../..'))
23 23
 extensions = [
24 24
     'sphinx.ext.autodoc',
25 25
     #'sphinx.ext.intersphinx',
26
-    'oslosphinx'
27 26
 ]
28 27
 
29 28
 # autodoc generation is a bit aggressive and a nuisance when doing heavy
@@ -38,7 +37,7 @@ master_doc = 'index'
38 37
 
39 38
 # General information about the project.
40 39
 project = u'downpour'
41
-copyright = u'2016, OpenStack Foundation'
40
+copyright = u'2017, Downpour contributors'
42 41
 
43 42
 # If true, '()' will be appended to :func: etc. cross-reference text.
44 43
 add_function_parentheses = True
@@ -57,6 +56,7 @@ pygments_style = 'sphinx'
57 56
 # html_theme_path = ["."]
58 57
 # html_theme = '_theme'
59 58
 # html_static_path = ['static']
59
+html_theme = 'nature'
60 60
 
61 61
 # Output file base name for HTML help builder.
62 62
 htmlhelp_basename = '%sdoc' % project

+ 4
- 3
doc/source/contributing.rst View File

@@ -1,4 +1,5 @@
1
-============
2
-Contributing
3
-============
1
+==============
2
+ Contributing
3
+==============
4
+
4 5
 .. include:: ../../CONTRIBUTING.rst

+ 10
- 0
doc/source/glossary.rst View File

@@ -0,0 +1,10 @@
1
+==========
2
+ Glossary
3
+==========
4
+
5
+.. glossary::
6
+
7
+   resource file
8
+     A YAML file containing explicitly identified resources to be
9
+     exported. See :doc:`resource_file` for more details.
10
+

+ 16
- 16
doc/source/index.rst View File

@@ -1,25 +1,25 @@
1
-.. downpour documentation master file, created by
2
-   sphinx-quickstart on Tue Jul  9 22:26:36 2013.
3
-   You can adapt this file completely to your liking, but it should at least
4
-   contain the root `toctree` directive.
1
+==================================================
2
+ downpour -- OpenStack Tenant Data Migration Tool
3
+==================================================
5 4
 
6
-Welcome to downpour's documentation!
7
-========================================================
5
+downpour exports tenant data from an OpenStack cloud to create a set
6
+of Ansible_ playbooks for importing the data into another cloud.
8 7
 
9
-Contents:
8
+.. note::
9
+
10
+  The project is in a very very early prototyping stage.
11
+
12
+.. _ansible: https://www.ansible.com
13
+
14
+Contents
15
+========
10 16
 
11 17
 .. toctree::
12 18
    :maxdepth: 2
13 19
 
14
-   readme
20
+   background
15 21
    installation
16 22
    usage
23
+   resource_file
17 24
    contributing
18
-
19
-Indices and tables
20
-==================
21
-
22
-* :ref:`genindex`
23
-* :ref:`modindex`
24
-* :ref:`search`
25
-
25
+   glossary

+ 22
- 7
doc/source/installation.rst View File

@@ -1,12 +1,27 @@
1
-============
2
-Installation
3
-============
1
+==============
2
+ Installation
3
+==============
4
+
5
+Prerequisites
6
+=============
7
+
8
+Downpour is written to take advantage of features of Python 3.5, so
9
+you will need a Python 3.5+ interpreter installed.
10
+
11
+Installing with pip
12
+===================
4 13
 
5 14
 At the command line::
6 15
 
7
-    $ pip install downpour
16
+    $ pip install os-downpour
17
+
18
+.. note:: The dist name for downpour is ``os-downpour``.
19
+
20
+Cloud Access Credentials
21
+========================
8 22
 
9
-Or, if you have virtualenvwrapper installed::
23
+downpour uses `os-client-config`_ for settings related to accessing
24
+the cloud. Fill in your ``clouds.yaml`` or use the environment
25
+variables or command line arguments provided.
10 26
 
11
-    $ mkvirtualenv downpour
12
-    $ pip install downpour
27
+.. _os-client-config: http://docs.openstack.org/developer/os-client-config/

+ 0
- 1
doc/source/readme.rst View File

@@ -1 +0,0 @@
1
-.. include:: ../../README.rst

+ 83
- 0
doc/source/resource_file.rst View File

@@ -0,0 +1,83 @@
1
+======================
2
+ Resource File Format
3
+======================
4
+
5
+A Downpour resource file is a YAML file containing explicitly
6
+identified resources to be exported, along with instructions for how
7
+to handle the export.
8
+
9
+``keypairs``
10
+============
11
+
12
+The keypairs section lists the names of the keypairs to be
13
+exported. Keys associated with servers are exported automatically, but
14
+if it is important to move keys not in use by any of the servers those
15
+keys can be listed separately.
16
+
17
+Each item in the keypairs list should be a mapping with a value for
18
+``name``.
19
+
20
+::
21
+
22
+  keypairs:
23
+    - name: downpour-demo
24
+
25
+``images``
26
+==========
27
+
28
+The images section lists the names of the images to be exported.
29
+
30
+Each item in the images list should be a mapping with a value for
31
+``name``.
32
+
33
+::
34
+
35
+  images:
36
+    - name: cirros-0.3.5-x86_64-disk
37
+
38
+``volumes``
39
+===========
40
+
41
+The volumes section lists the names and settings for the unattached
42
+volumes to be exported. This section should **not** include volumes
43
+attached to servers, because those are exported as part of exporting
44
+the server definition.
45
+
46
+Each item in the images list should be a mapping with a value for
47
+``name`` and an optional boolean value for ``save_state``, indicating
48
+whether the contents of the volume should be exported. If
49
+``save_state`` is false, a new volume with the same name and size will
50
+be created but it will be empty. The default is to save the contents
51
+of the volume.
52
+
53
+::
54
+
55
+  volumes:
56
+    - name: downpour-demo-unattached
57
+      save_state: false
58
+
59
+``servers``
60
+===========
61
+
62
+The servers section lists the names and settings for the virtual
63
+machines to be exported.
64
+
65
+Each item in the images list should be a mapping with a value for
66
+``name``. It can also contain an optional boolean value for
67
+``save_state``, indicating whether the contents of the VM should be
68
+exported. If ``save_state`` is false, a new VM with the same name and
69
+flavor will be created, but it will not contain any of the files from
70
+the current VM. The default is to save the contents of the volume.
71
+
72
+If an optional ``key_name`` setting is given, the new VM will be
73
+initialized using that ssh keypair instead of the one already
74
+associated with the server. The keypair does not need to exist on the
75
+source system.
76
+
77
+::
78
+
79
+   servers:
80
+     - name: downpour-demo-tiny
81
+       # Create the server using a separate key than
82
+       # it was created with in tiny.yml.
83
+       key_name: downpour-demo2

+ 114
- 5
doc/source/usage.rst View File

@@ -1,7 +1,116 @@
1
-========
2
-Usage
3
-========
1
+=======
2
+ Usage
3
+=======
4 4
 
5
-To use downpour in a project::
5
+Downpour uses a four step process. Between each step it is possible to
6
+stop and modify the data that has been prepared to pass to the next
7
+step.
6 8
 
7
-    import downpour
9
+1. Identify Resources to Export
10
+===============================
11
+
12
+The first phase of using Downpour is to identify exactly what
13
+resources will be exported from the cloud to build the :term:`resource
14
+file`. This step can be performed by hand by creating the required
15
+input file in a text editor, or the file can be build using the
16
+``query`` command.
17
+
18
+The resource file is a YAML file with sections for the principle
19
+resource types, ``keypairs``, ``images``, ``volumes``, and
20
+``servers``.  Resources are identified by name, and may also include
21
+extra parameters to control how the export and re-import operations
22
+are performed. For example, this resource file causes the
23
+``downpour-demo-tiny`` server to be exported but when it is recreated
24
+a different ssh key is used to provide access to log in.
25
+
26
+.. literalinclude:: ../../demo/tiny-resources.yml
27
+
28
+The ``downpour query`` command also can be used to find resources
29
+visible in the cloud, and add them to the resource file. It supports
30
+wildcard patterns in names and other criteria for filtering
31
+resources. For example, this command finds all servers with "``tiny``"
32
+in their name.
33
+
34
+::
35
+
36
+  $ downpour query --server-name '*tiny*' export.yml
37
+
38
+.. seealso::
39
+
40
+   :doc:`resource_file` includes more details about resource files.
41
+
42
+2. Exporting Resources
43
+======================
44
+
45
+The second phase of operation is to actually export the resources from
46
+the cloud using ``downpour export``, passing the resource file as
47
+input. Downpour starts by processing the resources listed in the file
48
+explicitly, and identifies any extra dependencies needed to recreate
49
+the configuration of those resources. For example, the networks,
50
+subnets, and security groups used by a server are exported
51
+automatically, as are the volumes attached to the server.
52
+
53
+::
54
+
55
+  $ downpour export export.yml ./export/
56
+
57
+The output for the export process is an Ansible_ playbook to recreate
58
+the resources, with all relationships intact. For images, volumes, and
59
+servers with the ``save-state`` flag set to true, the content of the
60
+resource will be downloaded and saved to the output directory where it
61
+can be used to recreate the resource.
62
+
63
+3. Importing Resources
64
+======================
65
+
66
+The import phase uses ``ansible-playbook`` to run the playbook created
67
+by the exporter.
68
+
69
+.. note::
70
+
71
+   Although Downpour currently requires Python 3.5 or greater, Ansible
72
+   is a Python 2.x application. If you are using ``pip`` and
73
+   ``virtualenv`` to install the tools, you will need to install them
74
+   in separate virtual environments.
75
+
76
+Ansible uses uses `os-client-config`_ for settings related to
77
+accessing the cloud. The simplest way to configure the cloud is via a
78
+``clouds.yaml`` file, but any mechanism supported by Ansible will
79
+work. The credentials used for the import phase do not need to be the
80
+same as the export phase. In fact, they're likely to be completely
81
+different because they will refer to a separate cloud's API endpoints.
82
+
83
+Downpour supports some customizations during export, such as changing
84
+the ssh key to be used for accessing a server. Other changes can be
85
+made by editing the playbook before running it.
86
+
87
+The playbook produced by Downpour creates each resource, then adds a
88
+line to a file ``uuids.csv`` to map the UUID in the source cloud to
89
+the UUID in the target cloud. This file may be useful for updating
90
+scripts or other configuration that rely on the UUID instead of a
91
+unique name for the resource.
92
+
93
+::
94
+
95
+   "Resource Type","Resource Name","Old","New"
96
+   "security group","downpour-demo","6deea469-54bd-4846-b12a-79fa6b482280","a4b80ffc-bc51-485c-915a-9ba9a7b4dcf0"
97
+   "volume","downpour-demo-tiny","256868c6-441f-4cd3-96fd-bda92c33822c","62e5616c-9a8c-44e2-bd14-4685b905ea94"
98
+   "security group","downpour-demo","3c7dcb77-d9ac-4af1-ba95-3f5d89a85227","a4b80ffc-bc51-485c-915a-9ba9a7b4dcf0"
99
+   "volume","downpour-demo-tiny","a6192546-c36e-4bee-ad00-8229e0b0efc5","62e5616c-9a8c-44e2-bd14-4685b905ea94"
100
+   "network","private","56a86bdb-13b2-4c9f-b8f5-a942d52602b5","f3027502-e4a2-4610-81fb-c6df99ead5c3"
101
+   "subnet","ipv6-private-subnet","8d736fe4-6b8f-4bf5-a38e-b511dce21f7f","01025e33-703b-4aa4-b6ec-80036bb3679b"
102
+   "subnet","private-subnet","e6baf9f4-09b5-4292-8236-3cca609ec2a3","2f9a1686-8125-4316-acd3-dbee51c44c1d"
103
+   "keypair","downpour-demo","downpour-demo","downpour-demo"
104
+   "image","cirros-0.3.5-x86_64-disk","570ec7bd-011b-4fbe-9968-626225654a7f","570ec7bd-011b-4fbe-9968-626225654a7f"
105
+
106
+.. _ansible: https://www.ansible.com
107
+.. _os-client-config: http://docs.openstack.org/developer/os-client-config/
108
+
109
+4. Decomissioning Resources
110
+===========================
111
+
112
+Downpour is not a live-migration tool, and it does not delete any
113
+resources from the source cloud. This allows you to perform
114
+application-specific migration (such as a final database sync) before
115
+updating any load balancers or DNS records and deleting old
116
+information.

+ 1
- 0
setup.cfg View File

@@ -28,6 +28,7 @@ console_scripts =
28 28
 source-dir = doc/source
29 29
 build-dir = doc/build
30 30
 all_files = 1
31
+warning-is-error = 1
31 32
 
32 33
 [upload_sphinx]
33 34
 upload-dir = doc/build/html

Loading…
Cancel
Save