doc: Update developer documentation
Change-Id: I531bfc11c93a0e0b44e421e443b2fc3697721195 Signed-off-by: Prashanth Pai <ppai@redhat.com>
This commit is contained in:
Prashanth Pai
parent
d8765aac78
commit
a765973600
160
HISTORY
160
HISTORY
@ -1,160 +0,0 @@
|
||||
This git repository is a child (of sorts) of the git.gluster.com/glusterfs.git
|
||||
repository, where origins of its contents come from the "swift" and "ufo"
|
||||
directories of its parent repository. The goal was to acknowledge that the
|
||||
gluster-swift integration work is in fact a separate and independent coding
|
||||
effort intersecting at the packaging level with GlusterFS upstream and its
|
||||
derivative products, for example, Red Hat's RHS.
|
||||
|
||||
To create this repository, we followed the steps below (in pseudo script
|
||||
form), which assumes that one is using an up-to-date glusterfs.git
|
||||
locally-cloned repository:
|
||||
|
||||
$ #
|
||||
$ # First list all of the changes to the "ufo" directory:
|
||||
$ #
|
||||
$ git checkout master
|
||||
$ git log --oneline ufo
|
||||
6d6205e object-storage: provide more common cfg defaults
|
||||
bf7b83d object-storage: cleanup err handling in Glusterfs.
|
||||
bbaa273 object-storage: turn off stat() for container list
|
||||
6a7d28c object-storage: use tox for unit tests; fix em too
|
||||
a56dca9 object-storage: Import missing sys and errno modules.
|
||||
93175bd Adds missing functions to ring.py, and more thorough tests.
|
||||
790cbb8 object-storage: Removed the redundant REMOTE_CLUSTER option.
|
||||
ee5df68 object-storage: Fixing the errors and warnings in unittest.
|
||||
598ca6b object-storage: Restoring multi volume support in UFO.
|
||||
311a5df object-storage: Use the wrapper functions provided by fs_utils.py to make system calls.
|
||||
6a6ec98 object-storage: Store the lock file in /var/run/swift.
|
||||
189eaec object-storage: yield during directory tree walks
|
||||
16d2801 object-storage: use temp file optimization
|
||||
8b87819 object-storage: add UFO unit tests
|
||||
cdcbc5b object-storage: only open file when requested
|
||||
a1f4a41 object-storage: don't sort object list, not req'd
|
||||
10ef8ef object-storage: do not list directory for delete
|
||||
1338ad1 object-storage: apply upstream DiskFile refactor
|
||||
2fd8af2 object-storage: a set of cleanups to DiskFile
|
||||
47e21af object-storage: Initial unittest of DiskFile class
|
||||
0c9f4d1 object-storage: unit tests to 100% cover utils
|
||||
29c23f2 object-storage: use temp file optimization for pkl
|
||||
c399ca2 object-storage: Use "-dev" in the version string
|
||||
7f9adbe object-storage: enforce RPM package dependencies
|
||||
a8d5907 object-storage: fix account mapping
|
||||
3d83e6c Replace "ssh foo gluster" with "gluster --remote-host=foo"
|
||||
f35d192 object-storage: Add a sorted list comparison to test
|
||||
a64a0d8 object-storage: del 1.4.8 ref from dir hierarchy
|
||||
$ #
|
||||
$ # Notice that it only goes back to the point where the "swift"
|
||||
$ # directory was renamed to "ufo", so we need to checkout the
|
||||
$ # previous revision by:
|
||||
$ #
|
||||
$ git log --oneline | grep -C 1 -F "del 1.4.8"
|
||||
1f64e5e gsyncd / geo-rep: include hostname in status cmd
|
||||
a64a0d8 object-storage: del 1.4.8 ref from dir hierarchy
|
||||
94ddaec Don't ignore all files that match (*o|*la|*lo)
|
||||
$ git checkout 94ddaec
|
||||
$ git log --oneline swift
|
||||
1129698 object-storage: final changes to remove swift diff
|
||||
e8d9565 object-storage: refactor to use swift devices
|
||||
05cc14c object-storage: make proper refs to module vars
|
||||
b696fc0 object-storage: reduce diffs, bury metadata update
|
||||
ffc7618 object-storage: swift-diff removal: hide obj only
|
||||
89aff74 object-storage: fix trailing whitespace
|
||||
d2040d1 object-storage: remove the device and part params
|
||||
6bd81f2 object-storage: only update metadata that changes
|
||||
7066b49 object-storage: DiskAccount subclassing of DiskDir
|
||||
92fed12 object-storage: fix bad refs to check_acct_exists
|
||||
f3daa0b object-storage: fix bad refs to REMOTE_CLUSTER
|
||||
ee28eae object-storage: rename self.name to self.container
|
||||
7fa7029 object-storage: unused parameter cleanup & handl'n
|
||||
a1fd8c8 object-storage: add missing docs for params
|
||||
a6b51d6 object-storage: minor code cleanups
|
||||
b0cb7aa object-storage: remove glusterfs filter requirement
|
||||
cae3216 object-storage: use constants for directory names
|
||||
6478b56 object-storeage: refactor to use one memcache key
|
||||
5c272f4 Minor refactorings to mark an internal routine
|
||||
8534cd5 Reduce the number of stat/fstat system calls made
|
||||
4f6aeb6 Further reduce extended attribute reads/writes
|
||||
3ab5404 Move _add_timestamp() closer to where it is used
|
||||
1a013e3 object-storage: reduce the number of getxattr system calls by one
|
||||
5110fc0 object-storage: Bump size of metadata stored per xattr key
|
||||
2b80439 object-storage: Don't catch generic exceptions around xattr calls
|
||||
dbe793b object-storage: Refactor code to reduce Swift diffs carried
|
||||
22050e5 object-storage: change logging of some errors to warnings
|
||||
ddb4652 Remove the test subdirectory diffs ahead of initial refactoring.
|
||||
49a3d88 Reduce the number of gratuitous differences in constraints.
|
||||
3c05477 Fix BZ 865858: remove unnecessary system calls around xattr ops
|
||||
49fad94 Refactor to use pyxattr's get/set/remove methods
|
||||
a9c613d Fix a small typ-o in the Swift README
|
||||
72301d4 swift: Donot display async_pending container.
|
||||
bf7634c swift: Fix for multiple mounts on the same mount point.
|
||||
93893b9 swift: Passing account name in container_update.
|
||||
530a44a swift: Gluster object storage plugin for Openstack Swift.
|
||||
$ #
|
||||
$ # Now we see that we have the entire history above.
|
||||
$ #
|
||||
$ # Next we need to generate patch files to apply to a clean tree.
|
||||
$ # To do that, we start by listing the patches for the swift tree
|
||||
$ # in reverse order and explicitly requesting a numbered format-
|
||||
$ # patch:
|
||||
$ let idx=1
|
||||
$ for i in $(git log --oneline --reverse swift | awk '{ print $1 }') ; do
|
||||
$ git checkout $i
|
||||
$ git format-patch --start-number $idx -1
|
||||
$ ((idx++))
|
||||
$ done
|
||||
$ #
|
||||
$ # Then we need to switch back to top-of-tree and do the same for
|
||||
$ # the "ufo" directory:
|
||||
$ #
|
||||
$ git checkout master
|
||||
$ for i in $(git log --oneline --reverse ufo | awk '{ print $1 }') ; do
|
||||
$ git checkout $i
|
||||
$ git format-patch --start-number $idx -1
|
||||
$ ((idx++))
|
||||
$ done
|
||||
$ #
|
||||
$ # At this point we have a set of sequentially numbered diffs that
|
||||
$ # can be applied to any git repo ... with two exceptions: patches
|
||||
$ # 0056-*.patch and 0061-*.patch have references to files that do not
|
||||
$ # exist ahead of time, so one has to edit out the cli* and .gitignore
|
||||
$ # diffs from those two patches, respectively, and then one can run
|
||||
$ # the following command to create a new repo:
|
||||
$ #
|
||||
$ cd /tmp # Or what ever directory you want to store the new repo in
|
||||
$ mkdir newdir
|
||||
$ cd newdir
|
||||
$ git init
|
||||
$ git am /path/to/*.patch # ... generated above
|
||||
$ git remote add -f origin ssh://<user>@git.gluster.com/gluster-swift.git
|
||||
$ git pull --no-commit origin master
|
||||
$ #
|
||||
$ # By default, a git pull will provide its own commit message, but
|
||||
$ # we list one below to be explicit in case some environments don't
|
||||
$ # have the proper "editor" set.
|
||||
$ #
|
||||
$ git commit -s -m "Merge up to master before committing initial changes"
|
||||
$ #
|
||||
$ # Once we have "Push" commit rights, the following updates the tree:
|
||||
$ #
|
||||
$ git push origin master
|
||||
Counting objects: 660, done.
|
||||
Delta compression using up to 4 threads.
|
||||
Compressing objects: 100% (584/584), done.
|
||||
Writing objects: 100% (659/659), 158.15 KiB, done.
|
||||
Total 659 (delta 316), reused 0 (delta 0)
|
||||
remote: Resolving deltas: 100% (316/316)
|
||||
remote: Processing changes: refs: 1, done
|
||||
remote: (W) ce1fd47: commit message lines >70 characters; manually wrap lines
|
||||
remote: (W) cfb17e8: commit subject >65 characters; use shorter first paragraph
|
||||
remote: (W) c48dcdc: commit message lines >70 characters; manually wrap lines
|
||||
remote: (W) dd826e2: commit message lines >70 characters; manually wrap lines
|
||||
remote: (W) b85ce6f: commit message lines >70 characters; manually wrap lines
|
||||
remote: (W) c565fc2: commit message lines >70 characters; manually wrap lines
|
||||
remote: (W) 6237313: commit message lines >70 characters; manually wrap lines
|
||||
remote: (W) 29d922d: commit message lines >70 characters; manually wrap lines
|
||||
remote: (W) ea1077a: commit message lines >70 characters; manually wrap lines
|
||||
remote: (W) c61aea1: commit message lines >70 characters; manually wrap lines
|
||||
remote: (W) a3a03e0: commit message lines >70 characters; manually wrap lines
|
||||
remote: (W) 3ad7809: commit message lines >70 characters; manually wrap lines
|
||||
To ssh://Portante@git.gluster.com/gluster-swift.git
|
||||
3303443..f534d66 master -> master
|
||||
@ -1,11 +1,14 @@
|
||||
# Developer Guide
|
||||
|
||||
## Development Environment Setup
|
||||
The workflow for SwiftOnFile is largely based upon the
|
||||
[GitHub work flow][] using the [fork and pull model][].
|
||||
The workflow for SwiftOnFile is largely based upon the [OpenStack Gerrit Workflow][].
|
||||
|
||||
### Account Setup
|
||||
You can create a free account on GitHub. It would be better to create keys and add your public key to github, else you can provide username/password each time you communicate with GitHub from any remote machine. Follow the the information given at [GitHub Generating SSH Keys][] if you need help creating your key. You have to create a fork of [swiftonfile repo][] for your development work. You can create your fork from the github Web UI.
|
||||
## Account Setup
|
||||
Please follow the guidelines at [OpenStack Gerrit Workflow][] to do the following:
|
||||
1. Create a Launchpad account.
|
||||
2. Create a Gerrit account.
|
||||
3. Upload SSH Keys to Gerrit.
|
||||
4. Install git-review tool.
|
||||
|
||||
### Package Requirements
|
||||
Type the following to install the required packages:
|
||||
@ -33,59 +36,51 @@ git config --global user.name "Firstname Lastname"
|
||||
git config --global user.email "your_email@youremail.com"
|
||||
~~~
|
||||
|
||||
### Clone your fork
|
||||
You can clone the fork you created using GitHub Web UI. By convention it will be called 'origin'.
|
||||
### Clone the source
|
||||
You can clone the swiftonfile repo from Gerrit:
|
||||
~~~
|
||||
git clone git@github.com:<your_username>/swiftonfile.git
|
||||
cd swiftonfile
|
||||
git clone ssh://<your-gerrit-username>@review.openstack.org:29418/stackforge/swiftonfile
|
||||
~~~
|
||||
|
||||
### Add upstream repo
|
||||
You can add swiftonfile project repo, to get the latest updates from the project. It will be called upstream by convention.
|
||||
Alternatively, if you just want to clone the source for trying things out,
|
||||
without setting up lp or Gerrit account or SSH keys, you can clone from the
|
||||
github mirror.
|
||||
~~~
|
||||
git remote add upstream git@github.com:swiftonfile/swiftonfile.git
|
||||
git clone git://github.com/stackforge/swiftonfile
|
||||
~~~
|
||||
|
||||
You can confirm these setting using 'git remote -v'. It should give you something like this:
|
||||
### Git Review
|
||||
Before installing git-review, make sure you have pip installed. Install the
|
||||
python `pip` tool by executing the following command:
|
||||
|
||||
~~~
|
||||
origin git@github.com:<your_username>/swiftonfile.git (fetch)
|
||||
origin git@github.com:<your_username>/swiftonfile.git (push)
|
||||
upstream git@github.com:swiftonfile/swiftonfile.git (fetch)
|
||||
upstream git@github.com:swiftonfile/swiftonfile.git (push)
|
||||
sudo easy_install pip
|
||||
~~~
|
||||
|
||||
### Code reviews
|
||||
These are the changes you need to make to the git configuration so you can download and verify someone's work.
|
||||
The tool `git review` is a simple tool to automate interaction with Gerrit.
|
||||
It is recommended to use this tool to upload, modify, and query changes in Gerrit.
|
||||
The tool can be installed by running the following command:
|
||||
|
||||
Open your .git/config file in your editor and locate the section for your GitHub remote. It should look something like this:
|
||||
~~~
|
||||
[remote "upstream"]
|
||||
url = git@github.com:<your_username>/swiftonfile.git
|
||||
fetch = +refs/heads/*:refs/remotes/upstream/*
|
||||
~~~
|
||||
We're going to add a new refspec to this section so that it now looks like this:
|
||||
~~~
|
||||
[remote "upstream"]
|
||||
url = git@github.com:<your_username>/swiftonfile.git
|
||||
fetch = +refs/heads/*:refs/remotes/upstream/*
|
||||
fetch = +refs/pull/*/head:refs/pull/upstream/*
|
||||
sudo pip install --upgrade git-review
|
||||
~~~
|
||||
|
||||
#### Download and Verify someone's pull request
|
||||
You can fetch all the pull requests using:
|
||||
While many Linux distributions offer a version of `git review`,
|
||||
they do not necessarily keep it up to date. Pip provides the latest version
|
||||
of the application which avoids problems with various versions of Gerrit.
|
||||
|
||||
The following command will setup an additional remote named 'gerrit' and
|
||||
also installs the gerrit Change-Id commit hook.
|
||||
~~~
|
||||
git fetch upstream
|
||||
# From github.com:swiftonfile/swiftonfile
|
||||
# * [new ref] refs/pull/1000/head -> refs/pull/upstream/1000
|
||||
# * [new ref] refs/pull/1002/head -> refs/pull/upstream/1002
|
||||
# * [new ref] refs/pull/1004/head -> refs/pull/upstream/1004
|
||||
# * [new ref] refs/pull/1009/head -> refs/pull/upstream/1009
|
||||
git review -s
|
||||
~~~
|
||||
|
||||
You should now be able to check out a pull request in your local repository as follows:
|
||||
If there is no output, then everything is setup correctly. If the output
|
||||
contains the string *We don't know where your gerrit is*, then you need to
|
||||
manually setup a remote repo with the name `gerrit`.
|
||||
~~~
|
||||
git checkout -b 999 pull/upstream/999
|
||||
# Switched to a new branch '999'
|
||||
git remote add gerrit ssh://<your-gerrit-username>@review.openstack.org:29418/stackforge/swiftonfile
|
||||
git remote -v
|
||||
~~~
|
||||
|
||||
### Tox and Nose
|
||||
@ -163,63 +158,54 @@ contains less than 70 characters.
|
||||
6. It may contain any external URL references like a launchpad blueprint.
|
||||
7. Blank line.
|
||||
|
||||
> Note: A bug or an enhancement both can be loged in github as an issue.
|
||||
|
||||
For more information on commit messages, please visit the
|
||||
[Git Commit Messages][] page in OpenStack.org.
|
||||
|
||||
### Uploading changes to Your Fork
|
||||
Once you have the changes ready for review, you can submit it to your github fork topic branch.
|
||||
### Uploading to Gerrit
|
||||
Once you have the changes ready for review, you can submit it to Gerrit
|
||||
by typing:
|
||||
|
||||
~~~
|
||||
git push origin TOPIC-BRANCH
|
||||
git review
|
||||
~~~
|
||||
|
||||
### Creating Pull request
|
||||
You pushed a commit to a topic branch in your fork, and now you would like it to be merged in the swiftonfile project.
|
||||
|
||||
Navigate to your forked repo, locate the branch you would like to be merged to swiftonfile and click on the Pull Request button.
|
||||
|
||||
Branch selection ==> Switch to your branch
|
||||
|
||||
Pull Request ==> Click the Compare & review button
|
||||
|
||||
Pull requests can be sent from any branch or commit but it's recommended that a topic branch be used so that follow-up commits can be pushed to update the pull request if necessary.
|
||||
|
||||
### Reviewing the pull request
|
||||
After starting the review, you're presented with a review page where you can get a high-level overview of what exactly has changed between your branch and the repository's master branch. You can review all comments made on commits, identify which files changed, and get a list of contributors to your branch.
|
||||
|
||||
After the change is reviewed, you might have to make some
|
||||
additional modifications to your change. You just need to do changes to your local topic branch, commit it, and push it to same branch on your github fork repo. If the branch is currently being used for a pull request, then the branch changes are automatically tracked by the pull request.
|
||||
additional modifications to your change. To continue the work for
|
||||
a specific change, you can query Gerrit for the change number by
|
||||
typing:
|
||||
|
||||
~~~
|
||||
git review -l
|
||||
~~~
|
||||
|
||||
Then download the change to make the new modifications by typing:
|
||||
|
||||
~~~
|
||||
git review -d CHANGE_NUMBER
|
||||
~~~
|
||||
|
||||
where CHANGE_NUMBER is the Gerrit change number.
|
||||
|
||||
If you need to create a new patch for a change and include your update(s)
|
||||
to your last commit type:
|
||||
|
||||
~~~
|
||||
git commit -as --amend
|
||||
~~~
|
||||
|
||||
Now that you have finished updating your change, you need to re-upload
|
||||
to Gerrit using the following command:
|
||||
|
||||
~~~
|
||||
git review
|
||||
~~~
|
||||
|
||||
If 'all goes well' your change will be merged to project swiftonfile. What 'all goes well' means, is this:
|
||||
|
||||
1. Travis-CI passes unit-tests.
|
||||
2. Jenkins passes functional-tests.
|
||||
3. It got +1 by at least 2 reviewers.
|
||||
4. A core-reviewer can give this pull request a +2 and merge it to the project repo.
|
||||
1. Jenkins passes unit tests and functional tests.
|
||||
2. Two core reviewers give +2.
|
||||
|
||||
|
||||
## Creating Distribution Packages
|
||||
|
||||
### Building RPMs for Fedora/RHEL/CentOS Systems
|
||||
Building RPMs. RPMs will be located in the *build* directory.
|
||||
|
||||
`$ bash makerpm.sh`
|
||||
|
||||
Building the RPM with a specific release value is useful for
|
||||
automatic Jenkin builds, or keeping track of different versions
|
||||
of the RPM:
|
||||
|
||||
`$ PKG_RELEASE=123 bash makerpm.sh`
|
||||
|
||||
[GitHub work flow]: https://guides.github.com/introduction/flow/index.html
|
||||
[fork and pull model]: https://help.github.com/articles/using-pull-requests
|
||||
[swiftonfile repo]: https://github.com/swiftonfile/swiftonfile
|
||||
[GitHub Generating SSH Keys]: https://help.github.com/articles/generating-ssh-keys
|
||||
[OpenStack Gerrit Workflow]: https://wiki.openstack.org/wiki/Gerrit_Workflow
|
||||
[SSH key]: http://review.openstack.org/#/settings/ssh-keys
|
||||
[PEP8]: http://www.python.org/dev/peps/pep-0008
|
||||
[Git Commit Messages]: https://wiki.openstack.org/wiki/GitCommitMessages
|
||||
[GlusterFS Compiling RPMS]: https://forge.gluster.org/glusterfs-core/pages/CompilingRPMS
|
||||
[README]: http://repos.fedorapeople.org/repos/openstack/openstack-trunk/README
|
||||
|
||||
|
||||
@ -64,7 +64,7 @@ Create */etc/httpd/conf.d/swift_wsgi.conf* configuration file that will define
|
||||
port and Virtual Host per each local service.
|
||||
|
||||
WSGISocketPrefix /var/run/wsgi
|
||||
|
||||
|
||||
#Proxy Service
|
||||
Listen 8080
|
||||
<VirtualHost *:8080>
|
||||
@ -78,7 +78,7 @@ port and Virtual Host per each local service.
|
||||
LogLevel debug
|
||||
CustomLog /var/log/httpd/proxy.log combined
|
||||
</VirtualHost>
|
||||
|
||||
|
||||
#Object Service
|
||||
Listen 6010
|
||||
<VirtualHost *:6010>
|
||||
@ -91,7 +91,7 @@ port and Virtual Host per each local service.
|
||||
LogLevel debug
|
||||
CustomLog /var/log/httpd/access.log combined
|
||||
</VirtualHost>
|
||||
|
||||
|
||||
#Container Service
|
||||
Listen 6011
|
||||
<VirtualHost *:6011>
|
||||
@ -104,7 +104,7 @@ port and Virtual Host per each local service.
|
||||
LogLevel debug
|
||||
CustomLog /var/log/httpd/access.log combined
|
||||
</VirtualHost>
|
||||
|
||||
|
||||
#Account Service
|
||||
Listen 6012
|
||||
<VirtualHost *:6012>
|
||||
Loading…
Reference in New Issue
Block a user