openstack/cookbook-openstack-object-storage
Code Issues Proposed changes

retire unmaintained project

Browse Source 
Change-Id: Icab1a4448c95a0dbeaaade9cb6572fe79c635d87
This commit is contained in:
Jan Klare
2016-12-14 16:05:40 +01:00
parent 7a9ba33ba6
commit 98fead1373
69 changed files with 5 additions and 5855 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
.gitignore vendored
View File

@@ -1,9 +0,0 @@
.bundle/
berks-cookbooks/
.kitchen
.vagrant
.coverage/
*.swp
Berksfile.lock
Vagrantfile
Gemfile.lock

4
.gitreview
View File

@@ -1,4 +0,0 @@
[gerrit]
host=review.openstack.org
port=29418
project=openstack/cookbook-openstack-object-storage.git

30
.rubocop.yml
View File

@@ -1,30 +0,0 @@
inherit_from: .rubocop_todo.yml
AllCops:
Include:
- metadata.rb
- Gemfile
- attributes/**
- libraries/**
- providers/**
- recipes/**
- resources/**
- spec/**
Exclude:
- .cookbooks/**/*
- berks-cookbooks/**/*
- .bundle/**/*
Encoding:
Exclude:
- metadata.rb
- Gemfile
NumericLiterals:
Enabled: false
LineLength:
Enabled: false
WordArray:
MinSize: 3

24
.rubocop_todo.yml
View File

@@ -1,24 +0,0 @@
# This configuration was generated by `rubocop --auto-gen-config`
# on 2015-05-29 11:53:01 -0500 using RuboCop version 0.29.1.
# The point is for the user to remove these configuration records
# one by one as the offenses are removed from the code base.
# Note that changes in the inspected code, or installation of new
# versions of RuboCop, may require this file to be generated again.
# Offense count: 4
Metrics/AbcSize:
Max: 169
# Offense count: 2
Metrics/PerceivedComplexity:
Max: 15
# Offense count: 8
# Configuration parameters: EnforcedStyle, SupportedStyles.
Style/ClassAndModuleChildren:
Enabled: false
# Offense count: 2
# Configuration parameters: EnforcedStyle, MinBodyLength, SupportedStyles.
Style/Next:
Enabled: false

10
Berksfile
View File

@@ -1,10 +0,0 @@
source "https://supermarket.chef.io"
metadata
cookbook "statsd",
github: "att-cloud/cookbook-statsd"
cookbook "openstack-common",
github: "openstack/cookbook-openstack-common"
cookbook "openstack-identity",
github: "openstack/cookbook-openstack-identity"

36
CONTRIBUTING.md
View File

@@ -1,36 +0,0 @@
Contributing
============
How To Get Started
------------------
If you would like to contribute to the development of OpenStack Chef Cookbooks,
you must follow the steps in this page:
http://docs.openstack.org/infra/manual/developers.html
Gerrit Workflow
---------------
Once those steps have been completed, changes to OpenStack
should be submitted for review via the Gerrit tool, following
the workflow documented at:
http://docs.openstack.org/infra/manual/developers.html#development-workflow
Pull requests submitted through GitHub will be ignored.
Bugs
----
Bugs should be filed on Launchpad, not GitHub:
https://bugs.launchpad.net/openstack-chef
Contacts
--------
Mailing list: groups.google.com/group/opscode-chef-openstack
IRC: #openstack-chef is our channel on irc.freenode.net
Wiki: https://wiki.openstack.org/wiki/Chef/GettingStarted and https://docs.getchef.com/openstack.html
Twitter: @chefopenstack

14
Gemfile
View File

@@ -1,14 +0,0 @@
## THIS GEMFILE IS DEPRECATED AND WILL BE REMOVED AFTER THE NEXT RELEASE
## THERE WON'T BE ANY UPDATES TO THIS FILE DURING THIS RELEASE CYCLE
## WE SWITCHED TO CHEFDK AS THE BUNDLE FOR THE NEEDED GEMS
source 'https://rubygems.org'
gem 'chef', '~> 11.18.6'
gem 'json', '<= 1.7.7' # chef 11 dependency
gem 'berkshelf', '~> 3.2.1'
gem 'hashie', '~> 2.0'
gem 'chefspec', '~> 4.0.0'
gem 'rspec', '~> 3.0.0'
gem 'foodcritic', '~> 4.0'
gem 'rubocop', '~> 0.29.1'

254
README.md
View File

@@ -1,250 +1,6 @@
Team and repository tags This project is no longer maintained.
========================
[![Team and repository tags](http://governance.openstack.org/badges/cookbook-openstack-object-storage.svg)](http://governance.openstack.org/reference/tags/index.html) The contents of this repository are still available in the Git
source code management system. To see the contents of this
<!-- Change things from this point on --> repository before it reached its end of life, please check out the
previous commit with "git checkout HEAD^1".
# THIS COOKBOOK IS CURRENTLY NOT MAINTAINED AND THERE WILL BE NO REALEASE FOR
STABLE/MITAKA. IF YOU ARE INTERESTED IN HELPING MAINTAINING IT, PLEASE JOIN OUR
WEEKLY MEETING ON IRC OR PING US IN THE #openstack-chef CHANNEL ON freenode.
Description
===========
Installs the OpenStack Object Storage service **Swift** as part of the OpenStack reference deployment Chef for OpenStack. The http://github.com/openstack/chef-openstack-repo contains documentation for using this cookbook in the context of a full OpenStack deployment. Swift is currently installed from packages.
https://wiki.openstack.org/wiki/Swift
Requirements
============
Clients
--------
* CentOS >= 7
* Ubuntu >= 14.04
Chef
---------
* 12
Dependent Cookbooks
-------------------
* openstack-common
* openstack-identity
* memcached
* statsd
Roles
=====
* swift-account-server - storage node for account data
* swift-container-server - storage node for container data
* swift-management-server - responsible for ring generation
* swift-object-server - storage node for object server
* swift-proxy-server - proxy for swift storage nodes
* swift-setup - server responsible for generating initial settings
The swift-management-server role performs the following functions:
* proxy node that knows super admin password
* ring repository and ring building workstation
* generally always has the swift-setup role too
* there can only be _one_ swift-management-server
There *must* be a node with the the swift-management-server role to act
as the ring repository.
In small environments, it is likely that all storage machines will
have all-in-one roles, with a load balancer ahead of it
In larger environments, where it is cost effective to split the proxy
and storage layer, storage nodes will carry
swift-{account,container,object}-server roles, and there will be
dedicated hosts with the swift-proxy-server role.
In really really huge environments, it's possible that the storage
node will be split into swift-{container,account}-server nodes and
swift-object-server nodes.
Recipes
=======
client
----
- Install the swift client packages
Attributes
==========
There are other attributes that must be set depending on authmode.
For "swauth", the following attributes are used:
* ```default[:swift][:authkey]``` - swauth super admin key if using swauth (defaults to test)
The following secrets can be defined in the databag defined in Common ['openstack']['secret']['secrets_data_bag']
```
{
"swift_hash": "1a7c0568fa84"
"swift_authkey": "keY4all"
"dispersion_auth_user": "ops:dispersion",
"dispersion_auth_key": "dispersionpass"
}
```
In addition, because swift is typically deployed as a cluster
there are some attributes used to find interfaces and ip addresses
on storage nodes:
* ```default[:swift][:git_builder_ip]``` - the IP address of the management server which other cluster members will use as their git pull target for ring updates (defaults to 127.0.0.1)
* ```default[:swift][:network][:proxy-bind-ip]``` - the IP address to bind to
on the proxy servers (defaults to 0.0.0.0 for all addresses)
* ```default[:swift][:network][:proxy-bind-port]``` - the port to bind to
on the proxy servers (defaults to 8080)
* ```default[:swift][:network][:account-bind-ip]``` - the IP address to bind to
on the account servers (defaults to 0.0.0.0 for all addresses)
* ```default[:swift][:network][:account-bind-port]``` - the port to bind to
on the account servers (defaults to 6002)
* ```default[:swift][:network][:container-bind-ip]``` - the IP address to bind to
on the container servers (defaults to 0.0.0.0 for all addresses)
* ```default[:swift][:network][:container-bind-port]``` - the port to bind to
on the container servers (defaults to 6002)
* ```default[:swift][:network][:object-bind-ip]``` - the IP address to bind to
on the object servers (defaults to 0.0.0.0 for all addresses)
* ```default[:swift][:network][:object-bind-port]``` - the port to bind to
on the container servers (defaults to 6002)
* ```default[:swift][:network][:object-cidr]``` - the CIDR network for your object
servers in order to build the ring (defaults to 10.0.0.0/24)
Proxy Plugins
=============
Formpost
-------
* ```default[:swift][:formpost][:enabled]``` - optionally enable the formpost proxy plugin (defaults to false)
TempURL
-------
* ```default[:swift][:tempurl][:enabled]``` - optionally enable the tempurl proxy plugin (defaults to false)
* ```default[:swift][:tempurl][:incoming_remove_headers]``` - The headers to remove from incoming requests (defaults to x-timestamp)
* ```default[:swift][:tempurl][:incoming_allow_headers]``` - The headers allowed as exceptions to incoming_remove_headers (defaults to empty string)
* ```default[:swift][:tempurl][:incoming_allow_headers]``` - The headers allowed as exceptions to incoming_remove_headers (defaults to empty string)
* ```default[:swift][:tempurl][:outgoing_remove_headers]``` - The headers to remove from outgoing responses (defaults to x-object-meta-*)
* ```default[:swift][:tempurl][:outgoing_allow_headers]``` - The headers allowed as exceptions to outgoing_remove_headers (defaults x-object-meta-public-*)
Domain Remap
------------
* ```default[:swift][:domain_remap][:enabled]``` - optionally enable the domain remap proxy plugin (defaults to false)
* ```default[:swift][:domain_remap][:storage_domain]``` - The domain remap reseller domain (defaults to example.com)
* ```default[:swift][:domain_remap][:root_path]``` - The domain remap root path (defaults to v1)
* ```default[:swift][:domain_remap][:reseller_prefixes]``` - The domain remap reseller prefixes (defaults to AUTH)
Staticweb
----------
* ```default[:swift][:staticweb][:enabled]``` - optionally enable the staticweb proxy plugin (defaults to false)
* ```default[:swift][:staticweb][:cache_timeout]``` - Seconds to cache container x-container-meta-web-* header values (defaults to 300)
Examples
========
Example environment
-------------------
```json
{
"default_attributes": {
"swift": {
"swift_hash": "107c0568ea84",
"authmode": "swauth",
"authkey": "test",
"auto_rebuild_rings": false,
"git_builder_ip": "10.0.0.10",
"swauth": {
"url": "http://10.0.0.10:8080/v1/"
}
}
},
"name": "swift",
"chef_type": "environment",
"json_class": "Chef::Environment"
}
```
This sets up defaults for a swauth-based cluster with the storage
network on 10.0.0.0/24.
Standalone Storage Server
-------------------------
```json
{
"name": "swift-object-server",
"json_class": "Chef::Role",
"run_list": [
"recipe[swift::object-server]"
],
"description": "A storage server role.",
"chef_type": "role"
}
```
Standalone Proxy Server
-----------------------
```json
"run_list": [
"role[swift-proxy-server]"
]
```
Testing
=======
Please refer to the [TESTING.md](TESTING.md) for instructions for testing the cookbook.
Berkshelf
=========
Berks will resolve version requirements and dependencies on first run and
store these in Berksfile.lock. If new cookbooks become available you can run
`berks update` to update the references in Berksfile.lock. Berksfile.lock will
be included in stable branches to provide a known good set of dependencies.
Berksfile.lock will not be included in development branches to encourage
development against the latest cookbooks.
License and Author
==================
| | |
|:---------------------|:---------------------------------------------------|
| **Author** | Alan Meadows (<alan.meadows@gmail.com>) |
| **Author** | Oisin Feeley (<of3434@att.com>) |
| **Author** | Ron Pedde (<ron.pedde@rackspace.com>) |
| **Author** | Will Kelly (<will.kelly@rackspace.com>) |
| **Author** | Chen Zhiwei (<zhiwchen@cn.ibm.com>) |
| **Author** | Mark Vanderwiel (<vanderwl@us.ibm.com>) |
| **Author** | Jan Klare (<j.klare@x-ion.de>) |
| | |
| **Copyright** | Copyright (c) 2013, AT&T, Inc. |
| **Copyright** | Copyright (c) 2012, Rackspace US, Inc. |
| **Copyright** | Copyright (c) 2013-2015 IBM, Corp. |
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

40
Rakefile
View File

@@ -1,40 +0,0 @@
task default: ["test"]
task :test #=> [:lint, :style, :unit]
desc "Vendor the cookbooks in the Berksfile"
task :berks_prep do
sh %{chef exec berks vendor}
end
desc "Run FoodCritic (lint) tests"
task :lint do
sh %{chef exec foodcritic --epic-fail any --tags ~FC003 --tags ~FC023 .}
end
desc "Run RuboCop (style) tests"
task :style do
sh %{chef exec rubocop}
end
desc "Run RSpec (unit) tests"
task :unit => :berks_prep do
sh %{chef exec rspec --format documentation}
end
desc "Remove the berks-cookbooks directory and the Berksfile.lock"
task :clean do
rm_rf [
'berks-cookbooks',
'Berksfile.lock'
]
end
desc "All-in-One Neutron build Infra using Common task"
task :integration do
# Use the common integration task
sh %(wget -nv -t 3 -O Rakefile-Common https://raw.githubusercontent.com/openstack/cookbook-openstack-common/master/Rakefile)
load './Rakefile-Common'
Rake::Task["common_integration"].invoke
end

30
TESTING.md
View File

@@ -1,30 +0,0 @@
# Testing the Cookbook #
This cookbook uses [chefdk](https://downloads.chef.io/chef-dk/) and [berkshelf](http://berkshelf.com/) to isolate dependencies. Make sure you have chefdk and the header files for `gecode` installed before continuing. Make sure that you're using gecode version 3. More info [here](https://github.com/opscode/dep-selector-libgecode/tree/0bad63fea305ede624c58506423ced697dd2545e#using-a-system-gecode-instead). For more detailed information on what needs to be installed, you can have a quick look into the bootstrap.sh file in this repository, which does install all the needed things to get going on ubuntu trusty. The tests defined in the Rakefile include lint, style and unit. For integration testing please refere to the [openstack-chef-repo](https://github.com/openstack/openstack-chef-repo).
We have three test suites which you can run either, individually (there are three rake tasks):
$ chef exec rake lint
$ chef exec rake style
$ chef exec rake unit
or altogether:
$ chef exec rake
The `rake` tasks will take care of installing the needed cookbooks with `berkshelf`.
## Rubocop ##
[Rubocop](https://github.com/bbatsov/rubocop) is a static Ruby code analyzer, based on the community [Ruby style guide](https://github.com/bbatsov/ruby-style-guide). We are attempting to adhere to this where applicable, slowly cleaning up the cookbooks until we can turn on Rubocop for gating the commits.
## Foodcritic ##
[Foodcritic](http://acrmp.github.io/foodcritic/) is a lint tool for Chef cookbooks. We ignore the following rules:
* [FC003](http://acrmp.github.io/foodcritic/#FC003) These cookbooks are not intended for Chef Solo.
* [FC023](http://acrmp.github.io/foodcritic/#FC023) Prefer conditional attributes.
## Chefspec
[ChefSpec](https://github.com/sethvargo/chefspec) is a unit testing framework for testing Chef cookbooks. ChefSpec makes it easy to write examples and get fast feedback on cookbook changes without the need for virtual machines or cloud servers.

515
attributes/default.rb
View File

@@ -1,515 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Attributes:: default
#
# Copyright 2014 IBM Corp.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
default['openstack']['object-storage']['service_tenant_name'] = 'service'
default['openstack']['object-storage']['service_user'] = 'swift'
default['openstack']['object-storage']['service_role'] = 'service'
# Default swift user
default['openstack']['object-storage']['user'] = 'swift'
# Default swift group
default['openstack']['object-storage']['group'] = 'swift'
# Default region
default['openstack']['object-storage']['region'] = node['openstack']['region']
# Set to some text value if you want templated config files
# to contain a custom banner at the top of the written file
default['openstack']['object-storage']['custom_template_banner'] = "
# This file autogenerated by Chef
# Do not edit, changes will be overwritten
"
#--------------------
# node/ring settings
#--------------------
default['openstack']['object-storage']['state'] = {}
# Hour to run swift_auditor on storage nodes
default['openstack']['object-storage']['audit_hour'] = '5'
# Eval-able expression that lists
# candidate disk nodes for disk probing. The result should be a hash
# with keys being the device name (without the leading "/dev/") and a
# hash block of any extra info associated with the device. For
# example: { "sdc" => { "model": "Hitachi 7K3000" }}. Largely,
# though, if you are going to make a list of valid devices, you
# probably know all the valid devices, and don't need to pass any
# metadata about them, so { "sdc" => {}} is probably enough. Example
# expression: Hash[('a'..'f').to_a.collect{|x| [ "sd{x}", {} ]}]
default['openstack']['object-storage']['disk_enum_expr'] = 'node[:block_device]'
# Flag to rebuild rings
default['openstack']['object-storage']['auto_rebuild_rings'] = false
# ip for git builder
default['openstack']['object-storage']['git_builder_ip'] = '127.0.0.1'
# How many server changes require before a rebalance is done
default['openstack']['object-storage']['wait_for'] = 1
# swift_hash_path_suffix and swift_hash_path_prefix are used as part of the
# the hashing algorithm when determining data placement in the cluster.
# These values should remain secret and MUST NOT change
# once a cluster has been deployed.
# Deprecated in juno, use swift_hash_path_prefix
default['openstack']['object-storage']['swift_hash'] = nil
# A value of nil will get the hash from the get_password library method in common
default['openstack']['object-storage']['swift_hash_path_prefix'] = node['openstack']['object-storage']['swift_hash']
default['openstack']['object-storage']['swift_hash_path_suffix'] = nil
# The swift-constraints section sets the basic constraints on data
# saved in the swift cluster. These constraints are automatically
# published by the proxy server in responses to /info requests.
# max_file_size is the largest "normal" object that can be saved in
# the cluster. This is also the limit on the size of each segment of
# a "large" object when using the large object manifest support.
# This value is set in bytes. Setting it to lower than 1MiB will cause
# some tests to fail. It is STRONGLY recommended to leave this value at
# the default (5 * 2**30 + 2).
default['openstack']['object-storage']['max_file_size'] = 5368709122
# max_meta_name_length is the max number of bytes in the utf8 encoding
# of the name portion of a metadata header.
default['openstack']['object-storage']['max_meta_name_length'] = 128
# max_meta_value_length is the max number of bytes in the utf8 encoding
# of a metadata value
default['openstack']['object-storage']['max_meta_value_length'] = 256
# max_meta_count is the max number of metadata keys that can be stored
# on a single account, container, or object
default['openstack']['object-storage']['max_meta_count'] = 90
# max_meta_overall_size is the max number of bytes in the utf8 encoding
# of the metadata (keys + values)
default['openstack']['object-storage']['max_meta_overall_size'] = 4096
# max_header_size is the max number of bytes in the utf8 encoding of each
# header. Using 8192 as default because eventlet use 8192 as max size of
# header line. This value may need to be increased when using identity
# v3 API tokens including more than 7 catalog entries.
# See also include_service_catalog in proxy-server.conf-sample
# (documented in overview_auth.rst)
default['openstack']['object-storage']['max_header_size'] = 8192
# max_object_name_length is the max number of bytes in the utf8 encoding
# of an object name
default['openstack']['object-storage']['max_object_name_length'] = 1024
# container_listing_limit is the default (and max) number of items
# returned for a container listing request
default['openstack']['object-storage']['container_listing_limit'] = 10000
# account_listing_limit is the default (and max) number of items returned
# for an account listing request
default['openstack']['object-storage']['account_listing_limit'] = 10000
# max_account_name_length is the max number of bytes in the utf8 encoding
# of an account name
default['openstack']['object-storage']['max_account_name_length'] = 256
# max_container_name_length is the max number of bytes in the utf8 encoding
# of a container name
default['openstack']['object-storage']['max_container_name_length'] = 256
# Use the openstack-common cookbook databags with the
# following keys:
# secret tokens
# "swift_hash_path_prefix"
# "swift_hash_path_suffix"
# "swift_authkey"
# "dispersion_auth_key"
# "dispersion_auth_user"
# The follow swift specific databag support is DEPRECATED.
# we support an optional secret databag where we will retrieve the
# following attributes overriding any default attributes here
#
# {
# "swift_hash": "107c0568ea84"
# "swift_authkey": "keW4all"
# "dispersion_auth_user": "test:test",
# "dispersion_auth_key": "test"
# }
default['openstack']['object-storage']['swift_secret_databag_name'] = nil
#--------------------
# roles
#--------------------
default['openstack']['object-storage']['setup_chef_role'] = 'os-object-storage-setup'
default['openstack']['object-storage']['management_server_chef_role'] = 'os-object-storage-management'
default['openstack']['object-storage']['proxy_server_chef_role'] = 'os-object-storage-proxy'
default['openstack']['object-storage']['object_server_chef_role'] = 'os-object-storage-object'
default['openstack']['object-storage']['account_server_chef_role'] = 'os-object-storage-account'
default['openstack']['object-storage']['container_server_chef_role'] = 'os-object-storage-container'
#--------------------
# authentication
#--------------------
# Authenitcation mode, either keystone or swauth
default['openstack']['object-storage']['authmode'] = 'keystone'
default['openstack']['object-storage']['authkey'] = nil
default['openstack']['object-storage']['swift_url'] = 'http://127.0.0.1:8080/v1/'
default['openstack']['object-storage']['swauth_url'] = 'http://127.0.0.1:8080/v1/'
default['openstack']['object-storage']['auth_url'] = 'http://127.0.0.1:8080/auth/v1.0'
# Keystone version
default['openstack']['object-storage']['api']['auth']['version'] = node['openstack']['api']['auth']['version']
# Keystone PKI signing directory
default['openstack']['object-storage']['api']['auth']['cache_dir'] = '/var/cache/swift/api'
#---------------------
# dispersion settings
#---------------------
default['openstack']['object-storage']['dispersion']['auth_user'] = nil
default['openstack']['object-storage']['dispersion']['auth_key'] = nil
# settings for the swift ring - these default settings are
# a safe setting for testing but part_power should be set to
# 26 in production to allow a swift cluster with 50,000 spindles
default['openstack']['object-storage']['ring']['part_power'] = 18
# The minimum number of hours before swift is allowed to migrate a partition
default['openstack']['object-storage']['ring']['min_part_hours'] = 1
# How many replicas swift should retain
default['openstack']['object-storage']['ring']['replicas'] = 3
# Zone, will look like "z1" on swift-ring-builder command line
default['openstack']['object-storage']['ring']['zone'] = '1'
# Region, will look like "r1" on swift-ring-builder command line
default['openstack']['object-storage']['ring']['region'] = '1'
#------------------
# statistics
#------------------
# Current statsd cookbook is not supported on rhel platforms
default['openstack']['object-storage']['statistics']['enabled'] = platform_family?('debian')
default['openstack']['object-storage']['statistics']['sample_rate'] = 1
# there are two ways to discover your graphite server ip for
# statsd to periodically publish to. You can directly set
# the ip below, or leave it set to nil and supply chef with
# the role name of your graphite server and the interface
# name to retrieve the appropriate internal ip address from
#
# if no servers with the role below can be found then
# 127.0.0.1 will be used
default['openstack']['object-storage']['statistics']['graphing_ip'] = nil
default['openstack']['object-storage']['statistics']['graphing_role'] = 'graphite-role'
default['openstack']['object-storage']['statistics']['graphing_interface'] = 'eth0'
# how frequently to run chef instantiated /usr/local/bin/swift_statsd_publish.py
# which publishes dispersion and recon statistics (in minutes)
default['openstack']['object-storage']['statistics']['report_frequency'] = 15
# enable or disable specific portions of generated report
default['openstack']['object-storage']['statistics']['enable_dispersion_report'] = true
default['openstack']['object-storage']['statistics']['enable_recon_report'] = true
default['openstack']['object-storage']['statistics']['enable_disk_report'] = true
# settings for statsd which should be configured to use the local
# statsd daemon that chef will install if statistics are enabled
default['openstack']['object-storage']['statistics']['statsd_host'] = '127.0.0.1'
default['openstack']['object-storage']['statistics']['statsd_port'] = '8125'
default['openstack']['object-storage']['statistics']['statsd_prefix'] = 'openstack.swift'
# paths to the recon cache files
default['openstack']['object-storage']['statistics']['recon_account_cache'] = '/var/cache/swift/account.recon'
default['openstack']['object-storage']['statistics']['recon_container_cache'] = '/var/cache/swift/container.recon'
default['openstack']['object-storage']['statistics']['recon_object_cache'] = '/var/cache/swift/object.recon'
#------------------
# network settings
#------------------
# the cidr configuration items are unimportant for a single server
# configuration, but in a multi-server setup, the cidr should match
# the interface appropriate to that service as they are used to
# resolve the appropriate addresses to use for internode
# communication
# Deprecated in Juno, use Common attributes instead
# proxy servers
default['openstack']['object-storage']['network']['proxy-bind-ip'] = nil
default['openstack']['object-storage']['network']['proxy-bind-port'] = nil
default['openstack']['object-storage']['network']['proxy-cidr'] = '127.0.0.0/8'
# account servers
default['openstack']['object-storage']['network']['account-bind-ip'] = '0.0.0.0'
default['openstack']['object-storage']['network']['account-bind-port'] = '6002'
# container servers
default['openstack']['object-storage']['network']['container-bind-ip'] = '0.0.0.0'
default['openstack']['object-storage']['network']['container-bind-port'] = '6001'
# object servers
default['openstack']['object-storage']['network']['object-bind-ip'] = '0.0.0.0'
default['openstack']['object-storage']['network']['object-bind-port'] = '6000'
default['openstack']['object-storage']['network']['object-cidr'] = '127.0.0.0/8'
#------------------
# sysctl
#------------------
# set sysctl properties for time waits
default['openstack']['sysctl']['net.ipv4.tcp_tw_recycle'] = 1
default['openstack']['sysctl']['net.ipv4.tcp_tw_reuse'] = 1
default['openstack']['sysctl']['net.ipv4.tcp_syncookies'] = 0
# N.B. conntrack_max may also need to be adjusted if
# server is running a stateful firewall
#------------------
# disk search
#------------------
# disk_test_filter is an array of predicates to test against disks to
# determine if a disk should be formatted and configured for swift.
# Each predicate is evaluated in turn, and a false from the predicate
# will result in the disk not being considered as a candidate for
# formatting.
# Each rule gets evaluated with "candidate" set to the device name
# without the leading "/dev/") and info set to the node hash value.
default['openstack']['object-storage']['disk_test_filter'] = [
'candidate =~ /(sd|hd|xvd|vd)(?!a$)[a-z]+/',
"File.exist?('/dev/' + candidate)",
"not system('/sbin/parted /dev/' + candidate + ' -s print | grep linux-swap')",
"not info.has_key?('removable') or info['removable'] == 0.to_s"]
#-------------------
# template overrides
#-------------------
# account-server
# Use an integer to override the number of pre-forked processes that will
# accept connections. Zero means no fork. Should default to the number of
# effective cpu cores in the system. It's worth noting that individual
# workers will # use many eventlet co-routines to service multiple concurrent
# requests.
default['openstack']['object-storage']['account-server']['workers'] = 'auto'
# Maximum number of clients one worker can process simultaneously (it will
# actually accept(2) N + 1). Setting this to one (1) will only handle one request
# at a time, without accepting another request concurrently. The default is 1024.
default['openstack']['object-storage']['account-server']['max_clients'] = 1024
# Parent directory or where devices are mounted. Default is /srv/node
default['openstack']['object-storage']['account-server']['devices'] = '/srv/node'
# Whether or not check if the devices are mounted to prevent accidentally writing to
# the root device. The default is set to true.
default['openstack']['object-storage']['account-server']['mount_check'] = true
# proxy-server
# Use an integer to override the number of pre-forked processes that will
# accept connections. Zero means no fork. Should default to the number of
# effective cpu cores in the system. It's worth noting that individual
# workers will # use many eventlet co-routines to service multiple concurrent
# requests.
default['openstack']['object-storage']['proxy-server']['workers'] = 'auto'
# Maximum number of clients one worker can process simultaneously (it will
# actually accept(2) N + 1). Setting this to one (1) will only handle one request
# at a time, without accepting another request concurrently. The default is 1024.
default['openstack']['object-storage']['proxy-server']['max_clients'] = 1024
# Request timeout to external services. The default is 10 seconds.
default['openstack']['object-storage']['proxy-server']['node_timeout'] = 10
# enable or disable formpost
default['openstack']['object-storage']['formpost']['enabled'] = false
# enable or disable tempurl
default['openstack']['object-storage']['tempurl']['enabled'] = false
# The headers to remove from incoming requests. Simply a whitespace delimited
# list of header names and names can optionally end with '*' to indicate a
# prefix match. incoming_allow_headers is a list of exceptions to these
# removals.
default['openstack']['object-storage']['tempurl']['incoming_remove_headers'] = 'x-timestamp'
# The headers allowed as exceptions to incoming_remove_headers. Simply a
# whitespace delimited list of header names and names can optionally end with
# '*' to indicate a prefix match.
default['openstack']['object-storage']['tempurl']['incoming_allow_headers'] = ''
# The headers to remove from outgoing responses. Simply a whitespace delimited
# list of header names and names can optionally end with '*' to indicate a
# prefix match. outgoing_allow_headers is a list of exceptions to these
# removals.
default['openstack']['object-storage']['tempurl']['outgoing_remove_headers'] = 'x-object-meta-*'
# The headers allowed as exceptions to outgoing_remove_headers. Simply a
# whitespace delimited list of header names and names can optionally end with
# '*' to indicate a prefix match.
default['openstack']['object-storage']['tempurl']['outgoing_allow_headers'] = 'x-object-meta-public-*'
# enable or disable domain_remap
default['openstack']['object-storage']['domain_remap']['enabled'] = false
# enable domain log name
default['openstack']['object-storage']['domain_remap']['log_name'] = 'domain_remap'
# domain remap log facilty
default['openstack']['object-storage']['domain_remap']['log_facility'] = 'LOG_LOCAL0'
# domain remap log level
default['openstack']['object-storage']['domain_remap']['log_level'] = 'INFO'
# domain remap log headers
default['openstack']['object-storage']['domain_remap']['log_headers'] = 'False'
# domain remap reseller domain
default['openstack']['object-storage']['domain_remap']['storage_domain'] = 'example.com'
# domain remap root path
default['openstack']['object-storage']['domain_remap']['path_root'] = 'v1'
# domain remap reseller prefixes
default['openstack']['object-storage']['domain_remap']['reseller_prefixes'] = 'AUTH'
# whether or not to enable staticweb in the swift proxy
default['openstack']['object-storage']['staticweb']['enabled'] = false
# Seconds to cache container x-container-meta-web-* header values.
default['openstack']['object-storage']['staticweb']['cache_timeout'] = 300
# staticweb logging options
default['openstack']['object-storage']['staticweb']['log_facility'] = 'LOG_LOCAL0'
default['openstack']['object-storage']['staticweb']['log_level'] = 'INFO'
default['openstack']['object-storage']['staticweb']['access_log_name'] = 'staticweb'
default['openstack']['object-storage']['staticweb']['access_log_facility'] = 'LOG_LOCAL0'
default['openstack']['object-storage']['staticweb']['access_log_level'] = 'INFO'
default['openstack']['object-storage']['staticweb']['log_headers'] = 'False'
# container-server
# Use an integer to override the number of pre-forked processes that will
# accept connections. Zero means no fork. Should default to the number of
# effective cpu cores in the system. It's worth noting that individual
# workers will # use many eventlet co-routines to service multiple concurrent
# requests.
default['openstack']['object-storage']['container-server']['workers'] = 'auto'
# Maximum number of clients one worker can process simultaneously (it will
# actually accept(2) N + 1). Setting this to one (1) will only handle one request
# at a time, without accepting another request concurrently. The default is 1024.
default['openstack']['object-storage']['container-server']['max_clients'] = 1024
# Parent directory or where devices are mounted. Default is /srv/node
default['openstack']['object-storage']['container-server']['devices'] = '/srv/node'
# Whether or not check if the devices are mounted to prevent accidentally writing to
# the root device. The default is set to true.
default['openstack']['object-storage']['container-server']['mount_check'] = true
# Override this with an allowed list of your various swift clusters if you wish
# to enable container sync for your end-users between clusters. This should
# be an array of fqdn hostnames for the cluster end-points that your end-users
# would access in the format of ['host1', 'host2', 'host3']
default['openstack']['object-storage']['container-server']['allowed_sync_hosts'] = []
# container-sync logging settings
default['openstack']['object-storage']['container-server']['container-sync']['log_name'] = 'container-sync'
default['openstack']['object-storage']['container-server']['container-sync']['log_facility'] = 'LOG_LOCAL0'
default['openstack']['object-storage']['container-server']['container-sync']['log_level'] = 'INFO'
# If you need to use an HTTP Proxy, set it here; defaults to no proxy.
default['openstack']['object-storage']['container-server']['container-sync']['sync_proxy'] = nil
# Will sync, at most, each container once per interval (in seconds)
default['openstack']['object-storage']['container-server']['container-sync']['interval'] = 300
# Maximum amount of time to spend syncing each container per pass (in seconds)
default['openstack']['object-storage']['container-server']['container-sync']['container_time'] = 60
# object-server
# Use an integer to override the number of pre-forked processes that will
# accept connections. Zero means no fork. Should default to the number of
# effective cpu cores in the system. It's worth noting that individual
# workers will # use many eventlet co-routines to service multiple concurrent
# requests.
default['openstack']['object-storage']['object-server']['workers'] = 'auto'
# Maximum number of clients one worker can process simultaneously (it will
# actually accept(2) N + 1). Setting this to one (1) will only handle one request
# at a time, without accepting another request concurrently. The default is 1024.
default['openstack']['object-storage']['object-server']['max_clients'] = 1024
# Parent directory or where devices are mounted. Default is /srv/node
default['openstack']['object-storage']['object-server']['devices'] = '/srv/node'
# Whether or not check if the devices are mounted to prevent accidentally writing to
# the root device. The default is set to true.
default['openstack']['object-storage']['object-server']['mount_check'] = true
# Time in seconds to wait between replication passes. The default is 30.
default['openstack']['object-storage']['object-server']['replicator']['run_pause'] = 30
# Time elapsed in seconds before an object can be reclaimed. The default is
# 604800 seconds.
default['openstack']['object-storage']['object-server']['replicator']['reclaim_age'] = 604800
#------------------
# swauth source
# -----------------
# Versions of swauth in Ubuntu Cloud Archive PPA can be outdated. This
# allows us to chose to install directly from a tagged branch of
# gholt's repository.
# values: package, git
default['openstack']['object-storage']['swauth_source'] = 'package'
default['openstack']['object-storage']['swauth_repository'] = 'https://github.com/gholt/swauth.git'
default['openstack']['object-storage']['swauth_version'] = '1.0.8'
#------------------
# packages
#------------------
# Leveling between distros
case platform_family
when 'rhel'
default['openstack']['object-storage']['platform'] = {
'disk_format' => 'ext4',
'proxy_packages' => %w(openstack-swift-proxy sudo cronie python-memcached),
'object_packages' => ['openstack-swift-object', 'sudo', 'cronie'],
'container_packages' => ['openstack-swift-container', 'sudo', 'cronie'],
'account_packages' => ['openstack-swift-account', 'sudo', 'cronie'],
'swift_packages' => ['openstack-swift', 'sudo', 'cronie'],
'swift_client_packages' => ['python-swiftclient'],
'swauth_packages' => ['openstack-swauth', 'sudo', 'cronie'],
'rsync_packages' => ['rsync'],
'git_packages' => ['xinetd', 'git', 'git-daemon'],
'service_prefix' => 'openstack-',
'service_suffix' => '',
'git_dir' => '/var/lib/git',
'git_service' => 'git',
'override_options' => '',
'swift_statsd_publish' => '/usr/bin/swift-statsd-publish.py'
}
when 'debian'
default['openstack']['object-storage']['platform'] = {
'disk_format' => 'xfs',
'proxy_packages' => ['swift-proxy', 'python-memcache'],
'object_packages' => ['swift-object'],
'container_packages' => ['swift-container'],
'account_packages' => ['swift-account', 'python-swiftclient'],
'swift_packages' => ['swift'],
'swift_client_packages' => ['python-swiftclient'],
'swauth_packages' => ['swauth'],
'rsync_packages' => ['rsync'],
'git_packages' => ['git-daemon-sysvinit'],
'service_prefix' => '',
'service_suffix' => '',
'git_dir' => '/var/cache/git',
'git_service' => 'git-daemon',
'override_options' => "-o Dpkg::Options:='--force-confold' -o Dpkg::Option:='--force-confdef'",
'swift_statsd_publish' => '/usr/local/bin/swift-statsd-publish.py'
}
end

9
bootstrap.sh
View File

@@ -1,9 +0,0 @@
#!/bin/bash -x
## This script is for installing all the needed packages on trusty to run the chef tests with 'chef exec rake'.
## It relies on the common bootstrap.sh from openstack/cookbook-openstack-common for installing common dependencies.
curl https://raw.githubusercontent.com/openstack/cookbook-openstack-common/master/bootstrap.sh \
--retry 3 \
--silent \
--show-error \
| /bin/bash -x

53
files/default/5EDB1B62EC4926EA
View File

@@ -1,53 +0,0 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1.4.11 (GNU/Linux)
mQINBFAqSlgBEADPKwXUwqbgoDYgR20zFypxSZlSbrttOKVPEMb0HSUx9Wj8VvNC
r+mT4E9wAyq7NTIs5ad2cUhXoyenrjcfGqK6k9R6yRHDbvAxCSWTnJjw7mzsajDN
ocXC6THKVW8BSjrh0aOBLpht6d5QCO2vyWxw65FKM65GOsbX03ZngUPMuOuiOEHQ
Zo97VSH2pSB+L+B3d9B0nw3QnU8qZMne+nVWYLYRXhCIxSv1/h39SXzHRgJoRUFH
vL2aiiVrn88NjqfDW15HFhVJcGOFuACZnRA0/EqTq0qNo3GziQO4mxuZi3bTVL5s
GABiYW9uIlokPqcS7Fa0FRVIU9R+bBdHZompcYnKAeGag+uRvuTqC3MMRcLUS9Oi
/P9I8fPARXUPwzYN3fagCGB8ffYVqMunnFs0L6td08BgvWwer+Buu4fPGsQ5OzMc
lgZ0TJmXyOlIW49lc1UXnORp4sm7HS6okA7P6URbqyGbaplSsNUVTgVbi+vc8/jY
dfExt/3HxVqgrPlq9htqYgwhYvGIbBAxmeFQD8Ak/ShSiWb1FdQ+f7Lty+4mZLfN
8x4zPZ//7fD5d/PETPh9P0msF+lLFlP564+1j75wx+skFO4v1gGlBcDaeipkFzeo
zndAgpegydKSNTF4QK9iTYobTIwsYfGuS8rV21zE2saLM0CE3T90aHYB/wARAQAB
tD1DYW5vbmljYWwgQ2xvdWQgQXJjaGl2ZSBTaWduaW5nIEtleSA8ZnRwbWFzdGVy
QGNhbm9uaWNhbC5jb20+iQI3BBMBCAAhBQJQKkpYAhsDBQsJCAcDBRUKCQgLBRYC
AwEAAh4BAheAAAoJEF7bG2LsSSbqKxkQAIKtgImrk02YCDldg6tLt3b69ZK0kIVI
3Xso/zCBZbrYFmgGQEFHAa58mIgpv5GcgHHxWjpX3n4tu2RM9EneKvFjFBstTTgo
yuCgFr7iblvs/aMW4jFJAiIbmjjXWVc0CVB/JlLqzBJ/MlHdR9OWmojN9ZzoIA+i
+tWlypgUot8iIxkR6JENxit5v9dN8i6anmnWybQ6PXFMuNi6GzQ0JgZIVs37n0ks
2wh0N8hBjAKuUgqu4MPMwvNtz8FxEzyKwLNSMnjLAhzml/oje/Nj1GBB8roj5dmw
7PSul5pAqQ5KTaXzl6gJN5vMEZzO4tEoGtRpA0/GTSXIlcx/SGkUK5+lqdQIMdyS
n8bImU6V6rDSoOaI9YWHZtpv5WeUsNTdf68jZsFCRD+2+NEmIqBVm11yhmUoasC6
dYw5l9P/PBdwmFm6NBUSEwxb+ROfpL1ICaZk9Jy++6akxhY//+cYEPLin02r43Z3
o5Piqujrs1R2Hs7kX84gL5SlBzTM4Ed+ob7KVtQHTefpbO35bQllkPNqfBsC8AIC
8xvTP2S8FicYOPATEuiRWs7Kn31TWC2iwswRKEKVRmN0fdpu/UPdMikyoNu9szBZ
RxvkRAezh3WheJ6MW6Fmg9d+uTFJohZt5qHdpxYa4beuN4me8LF0TYzgfEbFT6b9
D6IyTFoT0LequQINBFAqSlgBEADmL3TEq5ejBYrA+64zo8FYvCF4gziPa5rCIJGZ
/gZXQ7pm5zek/lOe9C80mhxNWeLmrWMkMOWKCeaDMFpMBOQhZZmRdakOnH/xxO5x
+fRdOOhy+5GTRJiwkuGOV6rB9eYJ3UN9caP2hfipCMpJjlg3j/GwktjhuqcBHXhA
HMhzxEOIDE5hmpDqZ051f8LGXld9aSL8RctoYFM8sgafPVmICTCq0Wh03dr5c2JA
gEXy3ushYm/8i2WFmyldo7vbtTfx3DpmJc/EMpGKV+GxcI3/ERqSkde0kWlmfPZb
o/5+hRqSryqfQtRKnFEQgAqAhPIwXwOkjCpPnDNfrkvzVEtl2/BWP/1/SOqzXjk9
TIb1Q7MHANeFMrTCprzPLX6IdC4zLp+LpV91W2zygQJzPgWqH/Z/WFH4gXcBBqmI
8bFpMPONYc9/67AWUABo2VOCojgtQmjxuFn+uGNw9PvxJAF3yjl781PVLUw3n66d
wHRmYj4hqxNDLywhhnL/CC7KUDtBnUU/CKn/0Xgm9oz3thuxG6i3F3pQgpp7MeMn
tKhLFWRXo9Bie8z/c0NV4K5HcpbGa8QPqoDseB5WaO4yGIBOt+nizM4DLrI+v07y
Xe3Jm7zBSpYSrGarZGK68qamS3XPzMshPdoXXz33bkQrTPpivGYQVRZuzd/R6b+6
IurV+QARAQABiQIfBBgBCAAJBQJQKkpYAhsMAAoJEF7bG2LsSSbq59EP/1U3815/
yHV3cf/JeHgh6WS/Oy2kRHp/kJt3ev/l/qIxfMIpyM3u/D6siORPTUXHPm3AaZrb
w0EDWByA3jHQEzlLIbsDGZgrnl+mxFuHwC1yEuW3xrzgjtGZCJureZ/BD6xfRuRc
mvnetAZv/z98VN/oj3rvYhUi71NApqSvMExpNBGrdO6gQlI5azhOu8xGNy4OSke8
J6pAsMUXIcEwjVEIvewJuqBW/3rj3Hh14tmWjQ7shNnYBuSJwbLeUW2e8bURnfXE
TxrCmXzDmQldD5GQWCcD5WDosk/HVHBmHlqrqy0VO2nE3c73dQlNcI4jVWeC4b4Q
SpYVsFz/6Iqy5ZQkCOpQ57MCf0B6P5nF92c5f3TYPMxHf0x3DrjDbUVZytxDiZZa
XsbZzsejbbc1bSNp4hb+IWhmWoFnq/hNHXzKPHBTapObnQju+9zUlQngV0BlPT62
hOHOw3Pv7suOuzzfuOO7qpz0uAy8cFKe7kBtLSFVjBwaG5JX89mgttYW+lw9Rmsb
p9Iw4KKFHIBLOwk7s+u0LUhP3d8neBI6NfkOYKZZCm3CuvkiOeQP9/2okFjtj+29
jEL+9KQwrGNFEVNe85Un5MJfYIjgyqX3nJcwypYxidntnhMhr2VD3HL2R/4CiswB
Oa4g9309p/+af/HU1smBrOfIeRoxb8jQoHu3
=xg4S
-----END PGP PUBLIC KEY BLOCK-----

88
files/default/cluster_stats.py
View File

@@ -1,88 +0,0 @@
#!/usr/bin/env python
import os
import re
import errno
import subprocess
USING_COLLECTD=0
try:
import collectd
USING_COLLECTD=1
except:
pass
# ===============================================================================
# [2012-06-19 21:37:04] Checking ring md5sum's on 3 hosts...
# 3/3 hosts matched, 0 error[s] while checking hosts.
# ===============================================================================
def get_md5sums():
retval = 0
output = subprocess.Popen(['swift-recon', '--objmd5'], stdout=subprocess.PIPE).communicate()[0]
for line in output.split("\n"):
result = re.search("([0-9]+) error", line)
if result:
retval = result.group(1)
return retval
# ===============================================================================
# [2012-06-19 21:36:27] Checking replication times on 3 hosts...
# [Replication Times] shortest: 0.00546943346659, longest: 0.00739345153173, avg: 0.00669538444943
# ===============================================================================
def get_replication_times():
retval = {}
output = subprocess.Popen(['swift-recon', '-r'], stdout=subprocess.PIPE).communicate()[0]
for line in output.split("\n"):
result = re.search("shortest: ([0-9\.]+), longest: ([0-9\.]+), avg: ([0-9\.]+)", line)
if result:
retval['shortest'] = float(result.group(1))
retval['longest'] = float(result.group(2))
retval['average'] = float(result.group(3))
return retval
def get_all():
stats = {}
stats['md5sums'] = get_md5sums()
stats['replication_times'] = get_replication_times()
return stats
def config_callback(conf):
pass
def read_callback():
stats = get_all()
if not stats:
return
# blarg, this should be fixed
for key in stats.keys():
path = '%s' % key
value = stats[key]
if type(value) != type({}):
# must be an int
val = collectd.Values(plugin=path)
val.type = 'gauge'
val.values = [int(value)]
val.dispatch()
else:
# must be a hash
for subvalue in value.keys():
path = '%s.%s' % (key, subvalue)
val = collectd.Values(plugin=path)
val.type = 'gauge'
if type(value[subvalue]) == type("string"):
val.values = [int(value[subvalue])]
else:
val.values = value[subvalue]
val.dispatch()
if not USING_COLLECTD:
stats = get_all()
print stats
else:
collectd.register_config(config_callback)
collectd.register_read(read_callback)

16
files/default/git-daemon.default
View File

@@ -1,16 +0,0 @@
# Defaults for git-daemon initscript
# sourced by /etc/init.d/git-daemon
# installed at /etc/default/git-daemon by the maintainer scripts
#
# This is a POSIX shell fragment
#
GIT_DAEMON_ENABLE=true
GIT_DAEMON_USER=gitdaemon
GIT_DAEMON_DIRECTORY=/var/cache/git
GIT_DAEMON_BASE_PATH=/var/cache/git
# Additional options that are passed to the Daemon.
GIT_DAEMON_OPTIONS=""

44
files/default/rsync.init
View File

@@ -1,44 +0,0 @@
#! /bin/sh
#
# chkconfig: 2345 50 50
# description: rsync service
# source function library
. /etc/rc.d/init.d/functions
PROG='/usr/bin/rsync'
BASE=${0##*/}
# Adapt the --config parameter to point to your rsync daemon configuration
# The config file must contain following line:
# pid file = /var/run/<filename>.pid
# Where <filename> is the filename of the init script (= this file)
OPTIONS="--daemon --config=/etc/rsyncd.conf"
case "$1" in
start)
echo -n $"Starting $BASE: "
daemon --check $BASE $PROG $OPTIONS
RETVAL=$?
[ $RETVAL -eq 0 ] && touch /var/lock/subsys/$BASE
echo
;;
stop)
echo -n $"Shutting down $BASE: "
killproc $BASE
RETVAL=$?
[ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/$BASE
echo
;;
restart|force-reload)
$0 stop
sleep 1
$0 start
;;
*)
echo "Usage: $0 {start|stop|restart|force-reload}" >&2
exit 1
;;
esac
exit 0

98
files/default/swift_stats.py
View File

@@ -1,98 +0,0 @@
#!/usr/bin/env python
import os
import errno
from resource import getpagesize
USING_COLLECTD=0
try:
import collectd
USING_COLLECTD=1
except:
pass
def get_unmounts(mountpath="/srv/node/"):
try:
candidates = [ x for x in os.listdir(mountpath) if os.path.isdir(mountpath + x) ]
except OSError as e:
if e.errno != errno.ENOENT:
raise
return 0
mounts = []
with open('/proc/mounts', 'r') as procmounts:
for line in procmounts:
_, mounted_path, _, _, _, _ = line.rstrip().split()
if mounted_path.startswith(mountpath):
mounts.append(mounted_path.split('/')[-1])
return len(set(candidates) - set(mounts))
def get_sockstats():
sockstat = {}
try:
with open('/proc/net/sockstat') as proc_sockstat:
for entry in proc_sockstat:
if entry.startswith("TCP: inuse"):
tcpstats = entry.split()
sockstat['tcp_in_use'] = int(tcpstats[2])
sockstat['orphan'] = int(tcpstats[4])
sockstat['time_wait'] = int(tcpstats[6])
sockstat['tcp_mem_allocated_bytes'] = \
int(tcpstats[10]) * getpagesize()
except OSError as e:
if e.errno != errno.ENOENT:
raise
try:
with open('/proc/net/sockstat6') as proc_sockstat6:
for entry in proc_sockstat6:
if entry.startswith("TCP6: inuse"):
sockstat['tcp6_in_use'] = int(entry.split()[2])
except IOError as e:
if e.errno != errno.ENOENT:
raise
return sockstat
def get_all():
stats = {}
stats['socket'] = get_sockstats()
stats['unmounts'] = get_unmounts()
return stats
def config_callback(conf):
pass
def read_callback():
stats = get_all()
if not stats:
return
# blarg, this should be fixed
for key in stats.keys():
path = "%s" % key
value = stats[key]
if type(value) != type({}):
# must be an int
val = collectd.Values(plugin=path)
val.type = 'gauge'
val.values = [int(value)]
val.dispatch()
else:
# must be a hash
for subvalue in value.keys():
path = '%s.%s' % (key, subvalue)
val = collectd.Values(plugin=path)
val.type = 'gauge'
val.values = [int(value[subvalue])]
val.dispatch()
if not USING_COLLECTD:
stats = get_all()
print stats
else:
collectd.register_config(config_callback)
collectd.register_read(read_callback)

38
libraries/drive_utils.rb
View File

@@ -1,38 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Library:: drive_utils
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Author: Ron Pedde <ron.pedde@rackspace.com>
#
# Drive Inspection Related Utilities
# rubocop:disable Eval, UselessAssignment
# TODO(chrislaco) This is a tragedy, and needs refactored
module DriveUtils
def locate_disks(enum_expression, filter_expressions)
candidate_disks = eval(enum_expression)
candidate_expression = 'candidate_disks.select{|candidate,info| (' +
filter_expressions.map { |x| "(#{x})" }.join(' and ') +
')}'
# TODO(mancdaz): fix this properly so the above works in the first place
candidate_expression.gsub!(/\[\'removable\'\] = 0/, "['removable'].to_i == 0")
drives = Hash[eval(candidate_expression)]
Chef::Log.info("Using candidate drives: #{drives.keys.join(', ')}")
drives.keys
end
end

46
libraries/ip_utils.rb
View File

@@ -1,46 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Library:: ip_utils
#
# Copyright 2013, ATT Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Author: Alan Meadows <alan.meadows@gmail.com>
#
require 'ipaddr'
# IPAddress Related Utilities
module IPUtils
# TODO(chrislaco) This needs yanked/refactored into common/libraries/network
def locate_ip_in_cidr(network, node) # rubocop:disable MethodLength
Chef::Log.debug("Searching for ip within #{network} on node #{node.name}")
net = IPAddr.new(network)
node['network']['interfaces'].each do |interface|
if interface[1].key?('addresses')
interface[1]['addresses'].each do |k, v|
if v['family'] == 'inet6' || (v['family'] == 'inet' && v['prefixlen'] != '32')
addr = IPAddr.new(k)
return k if net.include?(addr)
end
end
end
end
error = "Can't find address within network #{network} for node #{node.name}"
Chef::Log.error(error)
fail error
end
end

26
libraries/service_utils.rb
View File

@@ -1,26 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Library:: service_utils
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Service Related Utilities
module ServiceUtils
# Build platform specific service name
def svc_name(service_name)
platform_options = node['openstack']['object-storage']['platform']
platform_options['service_prefix'] + service_name + platform_options['service_suffix']
end
end

24
metadata.rb
View File

@@ -1,24 +0,0 @@
name 'openstack-object-storage'
maintainer 'openstack-chef'
maintainer_email 'openstack-dev@lists.openstack.org'
license 'Apache 2.0'
description 'Installs and configures Openstack Swift'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '12.0.0'
recipe 'openstack-object-storage::account-server', 'Installs the swift account server'
recipe 'openstack-object-storage::client', 'Install the swift client'
recipe 'openstack-object-storage::container-server', 'Installs the swift container server'
recipe 'openstack-object-storage::management-server', 'Installs the swift management server'
recipe 'openstack-object-storage::object-server', 'Installs the swift object server'
recipe 'openstack-object-storage::proxy-server', 'Installs the swift proxy server'
recipe 'openstack-object-storage::setup', 'Does initial setup of a swift cluster'
%w(centos ubuntu redhat).each do |os|
supports os
end
depends 'memcached', '>= 1.7.2'
depends 'statsd', '>= 0.1.5'
depends 'openstack-common', '>= 12.0.0'
depends 'openstack-identity', '>= 12.0.0'

254
providers/disk.rb
View File

@@ -1,254 +0,0 @@
# encoding: UTF-8
#
# Copyright 2011, Dell
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Author: andi abes
#
require 'chef/mixin/shell_out'
include Chef::Mixin::ShellOut
# rubocop:disable MethodLength
def load_current_resource
dev_name = @new_resource.name
@current = Chef::Resource::OpenstackObjectStorageDisk.new(dev_name)
parted_partition_parse dev_name
parts = @current.part
unless @current.blocks
# parted didn't return anything -- empty disk.
# get size from sfdisk
sfdisk_get_size(dev_name)
end
Chef::Log.info("About to print partition table for #{dev_name}")
s = <<EOF
current state for dev #{dev_name}
Size in 1K blocks: #{@current.blocks}
EOF
Chef::Log.info("Printing partition table for #{dev_name}")
num = 0
parts.each do |p|
s << 'partition ' << num
s << " start/end/size (1k): #{p[:start]}/#{p[:end]}/#{p[:size]}"
s << " type: #{p[:type]}"
s << "\n"
num += 1
end unless parts.nil?
Chef::Log.info(s)
end
# sample output
# sfdisk /dev/sdb -g
# /dev/sdb: 261 cylinders, 255 heads, 63 sectors/track
def sfdisk_get_size(dev_name)
out = Mixlib::ShellOut.new("sfdisk #{dev_name} -s").run_command.stdout
Chef::Log.info("size 1k blocks: #{out.to_i} for #{dev_name}")
# sfdisk sees the world as 1k blocks
@current.blocks(out.to_i)
end
def parted_partition_parse(dev_name)
Chef::Log.info("reading partition table for #{dev_name}")
# Run parted to get basic info about the disk
# sample output:
# # parted -m -s /dev/sda unit b print
# BYT;
# /dev/vda:8589934592B:virtblk:512:512:msdos:Virtio Block Device;
# 1:1048576B:8589934591B:8588886016B:ext3::;
pipe = IO.popen("parted -m -s #{dev_name} unit b print") # this can return 1, but it's ok (if no partition table present, we'll create it)
result = pipe.readlines
parted_parse_results result
end
def parted_parse_results(input)
Chef::Log.debug('partition table: ' + input.inspect)
input = input.to_a
part_tab = []
catch :parse_error do
line = input.shift # Error or BYT;
throw :parse_error if line =~ /^Error:/
line = input.shift
throw :parse_error unless line =~ %r{/dev/([^/]+):([0-9]+)B:(.*):.*$}
blocks = Regexp.last_match(2).to_i / 1024
if @current.blocks && @current.blocks != blocks
throw "Our disk size changed. Expecting: #{@current.blocks}, got #{blocks}"
end
@current.blocks(blocks)
input.each do |input_line|
# 1:1048576B:8589934591B:8588886016B:ext3::;
throw :parse_error unless input_line =~ /([0-9]):([0-9]+)B:([0-9]+)B:([0-9]+)B:(.*):(.*):(.*);$/
part_num = Regexp.last_match(1).to_i
part_info = {
num: part_num,
start: Regexp.last_match(2).to_i / 1024,
end: Regexp.last_match(3).to_i / 1024,
size: Regexp.last_match(4).to_i / 1024,
type: Regexp.last_match(5),
system: Regexp.last_match(6),
flags: Regexp.last_match(7)
}
part_tab << part_info
end
end
@current.part(part_tab)
part_tab
end
action :list do
Chef::Log.info("at some point there'll be a list")
new_resource.updated_by_last_action(update)
end
####
# compare the requested partition table parameters to what exists
# if differences found - remove all current partitions, and create new ones.
# An existing partition is considered a match if:
# - it has the same serial # (1,2,3)
# - it has the same size
#
# We also want to start to partition at 1M to be correctly aligned
# even due to 4K sector size and controller stripe sizes.
#
# Plus, then parted doesn't bitch every time you run it.
action :ensure_exists do
Chef::Log.info('Entering :ensure_exists')
req = @new_resource.part
cur = @current.part
dev_name = @new_resource.name
update = false
recreate = false
disk_blocks = @current.blocks # 1k blocks
if cur.nil?
recreate = true
else
idx = 0
current_block = 0
Chef::Log.info("Checking partition #{idx} for #{dev_name}")
req.each do |params|
if cur[idx].nil?
recreate = true
Chef::Log.info("no current #{idx}")
next
end
req_size = params[:size] # size in Mb - convert to blocks
if req_size == :remaining
req_size = disk_blocks - current_block
else
req_size *= 1024
end
cur_size = cur[idx][:size]
cur_min = req_size * 0.9
cur_max = req_size * 1.1
recreate = true unless (cur_size > cur_min) && (cur_size < cur_max)
current_block += cur[idx][:size]
Chef::Log.info("partition #{idx} #{(recreate ? 'differs' : 'is same')}: #{cur_size}/#{req_size} for #{dev_name}")
idx += 1
end
end
if !recreate
Chef::Log.info('partition table matches for #{dev_name} - not recreating')
else
### make sure to ensure that there are no mounted
### filesystems on the device
re = /^(#{Regexp.escape(dev_name)}[0-9]+)/
mounted = []
shell_out!('mount').stdout.each_line do |line|
md = re.match(line)
next unless md
mounted << md[1]
end
mounted.each do |m|
Chef::Log.info("unmounting #{m} for #{dev_name}")
shell_out!("umount #{m}")
end
# Nuke current partition table.
Chef::Log.info("Creating partition table for #{dev_name}")
cmd = Mixlib::ShellOut.new("parted -s -m #{dev_name} mktable gpt").run_command
Chef::Log.info("Created partition table for #{dev_name} out:#{cmd.stdout.strip} err:#{cmd.stderr.strip}")
# create new partitions
idx = 0
req.each do |params|
start_block = 0
start_block = '1M' if idx == 0
if params[:size] == :remaining
requested_size = '100%'
else
requested_size = "#{params[:size]}M"
end
Chef::Log.info("Creating partition #{idx + 1} for #{dev_name}")
cmd = Mixlib::ShellOut.new("parted -m -s #{dev_name} mkpart #{idx} #{start_block} #{requested_size}").run_command
Chef::Log.info("Created partition #{idx + 1} for #{dev_name} out:#{cmd.stdout.strip} err:#{cmd.stderr.strip}")
idx += 1
end
update = true
end
# walk through the partitions and enforce disk format
idx = 1
req.each do |params|
device = "#{dev_name}#{idx}"
Chef::Log.info("Testing file system on #{device} for type #{params[:type]}")
case params[:type]
when 'xfs'
if Mixlib::ShellOut.new("xfs_admin -l #{device}").run_command.error?
Chef::Log.info("Creating file system on #{device} for type #{params[:type]}")
cmd = Mixlib::ShellOut.new("mkfs.xfs -L swift -f -i size=512 #{device}").run_command
Chef::Log.info("Created file system on #{device} for type #{params[:type]} out:#{cmd.stdout.strip} err:#{cmd.stderr.strip}")
update = true
end
when 'ext4'
unless Mixlib::ShellOut.new("tune2fs -l #{device} | awk \'/Filesystem volume name:/{print $4}\' | grep -v \"<none>\"").run_command.exitstatus
Chef::Log.info("Creating file system on #{device} for type #{params[:type]}")
cmd = Mixlib::ShellOut.new("mkfs.ext4 -L swift #{device}").run_command
Chef::Log.info("Created file system on #{device} for type #{params[:type]} out:#{cmd.stdout.strip} err:#{cmd.stderr.strip}")
update = true
end
end
end
new_resource.updated_by_last_action(update)
end

173
providers/mounts.rb
View File

@@ -1,173 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Provider:: mounts
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Author: Ron Pedde <ron.pedde@rackspace.com>
#
require 'chef/util/file_edit'
require 'pp'
action :ensure_exists do
proposed_devices = @new_resource.devices
fsck = @new_resource.pass
path = @new_resource.name
dev_info = {}
Chef::Log.info('IN MOUNTS')
new_resource.updated_by_last_action(false)
# walk through the devices, gathering information
proposed_devices.each do |device|
next unless ::File.exist?("/dev/#{device}")
info = {}
info['device'] = device
info['ip'] = @new_resource.ip
info['format'] = @new_resource.format
info['uuid'] = Mixlib::ShellOut.new("blkid /dev/#{device} -s UUID -o value").run_command.stdout.strip
info['mountpoint'] = info['uuid'].split('-').join('')
info['mounted'] = Mixlib::ShellOut.new("mount | grep '#{path}/#{info['mountpoint']}\'").run_command.status
info['size'] = Mixlib::ShellOut.new("sfdisk -s /dev/#{device}").run_command.stdout.to_i / 1024
next if info['uuid'] == ''
dev_info[info['uuid']] = info
end
Chef::Log.info('Physical Inventory:')
dev_info.each do |_, v|
Chef::Log.info("Device: #{v['device']}, UUID: #{v['uuid']}, Mounted: #{v['mounted']}, Format: #{v['format']}")
end
# make sure we have a "path"
Directory(path) do
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
recursive true
end.run_action(:create)
# find what should be mounted, and what IS mounted
mounts = node['filesystem'].reduce({}) { |hsh, (k, v)| hsh.merge(v['mount'] => k) }
valid_mounts = dev_info.reduce([]) { |ary, (_, v)| ary << "#{path}/#{v['mountpoint']}" }
mountpoints = Dir.new(path).reject { |x| x[/^\./] }.map { |d| "#{path}/#{d}" }
inverted_mounts = dev_info.reduce({}) { |hsh, (k, v)| hsh.merge(v['mountpoint'] => v.merge('uuid' => k)) }
fstabs = ::File.readlines('/etc/fstab').reduce({}) do |hash, line|
line = line.split('#')[0].split
Chef::Log.debug("#{line[0]} ... #{line[1]}")
hash.merge(line[1] => line[0])
end
fstabs.reject! { |k, v| !k || !v || !k.length || !v.length }
Chef::Log.info("Mounts: #{PP.pp(mounts, '')}")
Chef::Log.info("Valid Mounts: #{PP.pp(valid_mounts, '')}")
Chef::Log.info("Mountpoints: #{PP.pp(mountpoints, '')}")
Chef::Log.info("Fstabs: #{PP.pp(fstabs, '')}")
# mounts in /srv/node that shouldn't be there
(mounts.keys.select { |x| x && x[/^#{path}/] } - valid_mounts).each do |dev|
Chef::Log.info("Unmounting #{dev}")
Mixlib::ShellOut.new("umount #{dev}").run_command if Mixlib::ShellOut.new("mount | grep '#{dev}'").run_command.status
new_resource.updated_by_last_action(true)
end
# fstab entries that don't need to be there anymore
(fstabs.keys.select { |k| k.start_with?(path) } - valid_mounts).each do |dev|
fe = Chef::Util::FileEdit.new('/etc/fstab')
fe.search_file_delete_line(Regexp.new(dev.gsub('/', '\/')))
fe.write_file
new_resource.updated_by_last_action(true)
end
# directories/mountpoints in /srv/node that are now useless
(mountpoints - valid_mounts).each do |mountpoint|
Chef::Log.info("rmdiring #{mountpoint}")
begin
Dir.rmdir(mountpoint)
rescue SystemCallError
Chef::Log.info("Directory #{mountpoint} appears non-empty")
end
new_resource.updated_by_last_action(true)
end
# new, unmounted devices
(valid_mounts - mounts.keys).each do |mountpoint|
info = inverted_mounts[mountpoint.gsub("#{path}/", '')]
Chef::Log.info("mounting #{mountpoint} (#{info['device']})")
mount_path = "#{path}/#{info['mountpoint']}"
Directory(mount_path) do
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
recursive true
end.run_action(:create)
case info['format']
when 'ext4'
mount_options = 'noatime,nodiratime,nobarrier,user_xattr'
when 'xfs'
case node['platform_family']
when 'debian'
mount_options = 'noatime,nodiratime,nobarrier,logbufs=8,nobootwait'
else
mount_options = 'noatime,nodiratime,nobarrier,logbufs=8'
end
end
mt = Mount(mount_path) do
device info['uuid']
device_type :uuid
options mount_options
dump 0
fstype info['format']
action :nothing
pass fsck
end
unless fstabs.key?(mount_path)
# then its a brand-new drive, freshly formatted
Chef::Log.info("Mounting new device #{info['mountpoint']}")
mt.run_action(:enable)
mt.run_action(:mount)
end
new_resource.updated_by_last_action(true)
end
dev_info.reject { |_k, v| v['mounted'] }.keys.each do |uuid|
dev_info[uuid]['mounted'] = Mixlib::ShellOut.new("mount | grep '#{path}/#{dev_info[uuid]['mountpoint']}\'").run_command.status
end
if @new_resource.publish_attributes && dev_info != {}
dev_info.each do |k, v|
node.set['openstack']['object-storage']['state']['devs'][k] = {
device: v['device'],
size: v['size'],
uuid: v['uuid'],
mounted: v['mounted'],
format: v['format'],
mountpoint: v['mountpoint'],
ip: v['ip']
}
end
Chef::Log.info("State: #{PP.pp(node['openstack']['object-storage']['state']['devs'], '')}")
end
end

283
providers/ring_script.rb
View File

@@ -1,283 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Resource:: ring_script
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Author: Ron Pedde <ron.pedde@rackspace.com>
#
require 'pp'
# rubocop:disable PerlBackrefs, CyclomaticComplexity, MethodLength
def generate_script # rubocop:disable Metrics/AbcSize
# need to load and parse the existing rings.
ports = { 'object' => node['openstack']['object-storage']['network']['object-bind-port'],
'container' => node['openstack']['object-storage']['network']['container-bind-port'],
'account' => node['openstack']['object-storage']['network']['account-bind-port'] }
must_rebalance = false
ring_path = @new_resource.ring_path
ring_data = { raw: {}, parsed: {}, in_use: {} }
disk_data = {}
dirty_cluster_reasons = []
['account', 'container', 'object'].each do |which|
ring_data[:raw][which] = nil
if ::File.exist?("#{ring_path}/#{which}.builder")
IO.popen("su swift -c 'swift-ring-builder #{ring_path}/#{which}.builder'") do |pipe|
ring_data[:raw][which] = pipe.readlines
Chef::Log.debug("#{which} Ring data raw:\n#{PP.pp(ring_data[:raw][which], '')}")
ring_data[:parsed][which] = parse_ring_output(ring_data[:raw][which])
Chef::Log.info("#{which} Ring data parsed:\n#{PP.pp(ring_data[:parsed][which], '')}")
node.set['openstack']['object-storage']['state']['ring'][which] = ring_data[:parsed][which]
end
else
Chef::Log.info("#{which} ring builder files do not exist!")
end
# collect all the ring data, and note what disks are in use. All I really
# need is a hash of device and id
ring_data[:in_use][which] ||= {}
if ring_data[:parsed][which][:hosts]
ring_data[:parsed][which][:hosts].each do |_ip, dev|
dev.each do |_dev_id, devhash|
ring_data[:in_use][which].store(devhash[:device], devhash[:id])
end
end
end
Chef::Log.info("#{which} Ring - In use: #{PP.pp(ring_data[:in_use][which], '')}")
# figure out what's present in the cluster
disk_data[which] = {}
role = node['openstack']['object-storage']["#{which}_server_chef_role"]
disk_state = Chef::Search::Query.new.search(:node, "chef_environment:#{node.chef_environment} AND roles:#{role}")
Chef::Log.info("#{which} node count: #{disk_state.count} for role: #{role}")
# for a running track of available disks
disk_data[:available] ||= {}
disk_data[:available][which] ||= {}
disk_state.each do |swiftnode|
Chef::Log.info("#{which} node: #{swiftnode[:hostname]} state:\n#{PP.pp(swiftnode['openstack']['object-storage']['state'], '')}")
if swiftnode['openstack']['object-storage']['state']['devs']
swiftnode['openstack']['object-storage']['state']['devs'].each do |k, v|
disk_data[which][v[:ip]] = disk_data[which][v[:ip]] || {}
disk_data[which][v[:ip]][k] = {}
v.keys.each { |x| disk_data[which][v[:ip]][k].store(x, v[x]) }
disk_data[which][v[:ip]][k]['region'] = swiftnode['openstack']['object-storage']['ring']['region']
disk_data[which][v[:ip]][k]['zone'] = swiftnode['openstack']['object-storage']['ring']['zone']
disk_data[:available][which][v[:mountpoint]] = v[:ip]
unless v[:mounted]
dirty_cluster_reasons << "Disk #{v[:name]} (#{v[:uuid]}) is not mounted on host #{v[:ip]} (#{swiftnode[:hostname]})"
end
end
end
end
Chef::Log.info("#{which} Ring - Avail:\n#{PP.pp(disk_data[:available][which], '')}")
end
# Have the raw data, now bump it together and drop the script
s = "#!/bin/bash\n\n# This script is automatically generated.\n"
s << "# Running it will likely blow up your system if you don't review it carefully.\n"
s << "# You have been warned.\n\n"
s << "set -x\n"
unless node['openstack']['object-storage']['auto_rebuild_rings']
s << "if [ \"$1\" != \"--force\" ]; then\n"
s << " echo \"Auto rebuild rings is disabled, so you must use --force to generate rings\"\n"
s << " exit 0\n"
s << "fi\n\n"
end
Chef::Log.info("Disk data: #{PP.pp(disk_data, '')}")
new_disks = {}
missing_disks = {}
new_servers = []
['account', 'container', 'object'].each do |which|
# remove available disks that are already in the ring
new_disks[which] = disk_data[:available][which].reject { |k, _v| ring_data[:in_use][which].key?(k) }
# find all in-ring disks that are not in the cluster
missing_disks[which] = ring_data[:in_use][which].reject { |k, _v| disk_data[:available][which].key?(k) }
Chef::Log.info("#{which} Ring - Missing:\n#{PP.pp(missing_disks[which], '')}")
Chef::Log.info("#{which} Ring - New:\n#{PP.pp(new_disks[which], '')}")
s << "\n# -- #{which.capitalize} Servers --\n\n"
disk_data[which].keys.sort.each do |ip|
s << "# #{ip}\n"
disk_data[which][ip].keys.sort.each do |k|
v = disk_data[which][ip][k]
s << '# ' + v.keys.sort.select { |x| ['ip', 'device', 'uuid'].include?(x) }.map { |x| v[x] }.join(', ')
if new_disks[which].key?(v['mountpoint'])
s << ' (NEW!)'
new_servers << ip unless new_servers.include?(ip)
end
s << "\n"
end
end
# for all those servers, check if they are already in the ring. If not,
# then we need to add them to the ring. For those that *were* in the
# ring, and are no longer in the ring, we need to delete those.
s << "\n"
# add the new disks
disk_data[which].keys.sort.each do |ip|
disk_data[which][ip].keys.sort.each do |uuid|
v = disk_data[which][ip][uuid]
if new_disks[which].key?(v['mountpoint'])
s << "swift-ring-builder #{ring_path}/#{which}.builder add r#{v['region']}z#{v['zone']}-#{v['ip']}:#{ports[which]}/#{v['device']} #{v['size']}\n"
must_rebalance = true
end
end
end
# remove the disks -- sort to ensure consistent order
missing_disks[which].keys.sort.each do |mountpoint|
diskinfo = ring_data[:parsed][which][:hosts].select { |_k, v| v.key?(mountpoint) }.map { |_, v| v[mountpoint] }[0]
Chef::Log.info("#{which} Missing diskinfo:\n#{PP.pp(diskinfo, '')}")
description = Hash[diskinfo.select { |k, _v| [:zone, :ip, :device].include?(k) }].map { |k, v| "#{k}: #{v}" }.join(', ')
s << "# #{description}\n"
s << "swift-ring-builder #{ring_path}/#{which}.builder remove d#{missing_disks[which][mountpoint]}\n"
must_rebalance = true
end
s << "\n"
if must_rebalance
# we'll only rebalance if we meet the minimums for new adds
if node['openstack']['object-storage']['wait_for'] > new_servers.count
Chef::Log.info("#{which} New servers, but not enough to force a rebalance")
must_rebalance = false
else
s << "swift-ring-builder #{ring_path}/#{which}.builder rebalance\n\n\n"
end
else
s << "# #{which.capitalize} ring has no outstanding changes!\n\n"
end
end
[s, must_rebalance]
end
# Parse the raw output of swift-ring-builder
def parse_ring_output(ring_data)
output = { state: {} }
ring_data.each do |line|
if line =~ /build version ([0-9]+)/
output[:state][:build_version] = $1
elsif line =~ /^Devices:\s+id\s+region\s+zone\s+/
next
elsif line =~ /^Devices:\s+id\s+zone\s+/
next
elsif line =~ /\soverload factor\s/
next
elsif line =~ /^\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+\.\d+\.\d+\.\d+)\s+(\d+)\s+(\d+\.\d+\.\d+\.\d+)\s+(\d+)\s+(\S+)\s+([0-9.]+)\s+(\d+)\s+([-0-9.]+)\s*$/
output[:hosts] ||= {}
output[:hosts][$4] ||= {}
output[:hosts][$4][$8] ||= {}
output[:hosts][$4][$8][:id] = $1
output[:hosts][$4][$8][:region] = $2
output[:hosts][$4][$8][:zone] = $3
output[:hosts][$4][$8][:ip] = $4
output[:hosts][$4][$8][:port] = $5
output[:hosts][$4][$8][:replication_ip] = $6
output[:hosts][$4][$8][:replication_port] = $7
output[:hosts][$4][$8][:device] = $8
output[:hosts][$4][$8][:weight] = $9
output[:hosts][$4][$8][:partitions] = $10
output[:hosts][$4][$8][:balance] = $11
elsif line =~ /^\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+\.\d+\.\d+\.\d+)\s+(\d+)\s+(\S+)\s+([0-9.]+)\s+(\d+)\s+([-0-9.]+)\s*$/
output[:hosts] ||= {}
output[:hosts][$4] ||= {}
output[:hosts][$4][$6] ||= {}
output[:hosts][$4][$6][:id] = $1
output[:hosts][$4][$6][:region] = $2
output[:hosts][$4][$6][:zone] = $3
output[:hosts][$4][$6][:ip] = $4
output[:hosts][$4][$6][:port] = $5
output[:hosts][$4][$6][:device] = $6
output[:hosts][$4][$6][:weight] = $7
output[:hosts][$4][$6][:partitions] = $8
output[:hosts][$4][$6][:balance] = $9
elsif line =~ /^\s+(\d+)\s+(\d+)\s+(\d+\.\d+\.\d+\.\d+)\s+(\d+)\s+(\S+)\s+([0-9.]+)\s+(\d+)\s+([-0-9.]+)\s*$/
output[:hosts] ||= {}
output[:hosts][$3] ||= {}
output[:hosts][$3][$5] ||= {}
output[:hosts][$3][$5][:id] = $1
output[:hosts][$3][$5][:zone] = $2
output[:hosts][$3][$5][:ip] = $3
output[:hosts][$3][$5][:port] = $4
output[:hosts][$3][$5][:device] = $5
output[:hosts][$3][$5][:weight] = $6
output[:hosts][$3][$5][:partitions] = $7
output[:hosts][$3][$5][:balance] = $8
elsif line =~ /(\d+) partitions, (\d+\.\d+) replicas, (\d+) regions, (\d+) zones, (\d+) devices, (\d+\.\d+) balance/
output[:state][:partitions] = $1
output[:state][:replicas] = $2
output[:state][:regions] = $3
output[:state][:zones] = $4
output[:state][:devices] = $5
output[:state][:balance] = $6
elsif line =~ /(\d+) partitions, (\d+) replicas, (\d+) zones, (\d+) devices, (\d+\.\d+) balance$/
output[:state][:partitions] = $1
output[:state][:replicas] = $2
output[:state][:zones] = $3
output[:state][:devices] = $4
output[:state][:balance] = $5
elsif line =~ /^The minimum number of hours before a partition can be reassigned is (\d+)$/
output[:state][:min_part_hours] = $1
else
fail "Cannot parse ring builder output for #{line}"
end
end
output
end
action :ensure_exists do
Chef::Log.info("Ensuring #{new_resource.name}")
new_resource.updated_by_last_action(false)
s, must_update = generate_script
script_file = File new_resource.name do
owner new_resource.owner
group new_resource.group
mode new_resource.mode
content s
end
script_file.run_action(:create)
new_resource.updated_by_last_action(must_update)
end

66
recipes/account-server.rb
View File

@@ -1,66 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Recipe:: account-server
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
include_recipe 'openstack-object-storage::common'
include_recipe 'openstack-object-storage::storage-common'
include_recipe 'openstack-object-storage::disks'
class Chef::Recipe # rubocop:disable Documentation
include ServiceUtils
end
platform_options = node['openstack']['object-storage']['platform']
platform_options['account_packages'].each.each do |pkg|
package pkg do
action :upgrade
options platform_options['override_options'] # retain configs
end
end
svc_names = {}
%w(swift-account swift-account-auditor swift-account-reaper swift-account-replicator).each do |svc|
svc_names[svc] = svc_name(svc)
end
svc_names.values.each do |svc|
service svc do
supports status: true, restart: true
action [:enable, :start]
only_if '[ -e /etc/swift/account-server.conf ] && [ -e /etc/swift/account.ring.gz ]'
end
end
# create account server template
template '/etc/swift/account-server.conf' do
source 'account-server.conf.erb'
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
mode 00600
variables(
'bind_ip' => node['openstack']['object-storage']['network']['account-bind-ip'],
'bind_port' => node['openstack']['object-storage']['network']['account-bind-port']
)
notifies :restart, "service[#{svc_names['swift-account']}]", :immediately
notifies :restart, "service[#{svc_names['swift-account-auditor']}]", :immediately
notifies :restart, "service[#{svc_names['swift-account-reaper']}]", :immediately
notifies :restart, "service[#{svc_names['swift-account-replicator']}]", :immediately
end

28
recipes/client.rb
View File

@@ -1,28 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Recipe:: client
#
# Copyright 2014, IBM Corp.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
platform_options = node['openstack']['object-storage']['platform']
platform_options['swift_client_packages'].each do |pkg|
package pkg do
options platform_options['package_overrides']
action :upgrade
end
end

130
recipes/common.rb
View File

@@ -1,130 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Recipe:: swift-common
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
class ::Chef::Recipe # rubocop:disable Documentation
include ::Openstack
end
class Chef::Recipe # rubocop:disable Documentation
include DriveUtils
end
include_recipe 'openstack-common::sysctl'
#-------------
# stats
#-------------
# optionally statsd daemon for stats collection
if node['openstack']['object-storage']['statistics']['enabled']
node.set['statsd']['relay_server'] = true
include_recipe 'statsd::server'
end
# find graphing server address
if Chef::Config[:solo] && !node['recipes'].include?('chef-solo-search')
Chef::Log.warn('This recipe uses search. Chef Solo does not support search.')
graphite_servers = []
else
graphite_servers = search(:node, "roles:#{node['openstack']['object-storage']['statistics']['graphing_role']} AND chef_environment:#{node.chef_environment}")
end
graphite_host = '127.0.0.1'
unless graphite_servers.empty?
graphite_host = graphite_servers[0]['network']["ipaddress_#{node['openstack']['object-storage']['statistics']['graphing_interface']}"]
end
if node['openstack']['object-storage']['statistics']['graphing_ip'].nil?
node.set['statsd']['graphite_host'] = graphite_host
else
node.set['statsd']['graphite_host'] = node['openstack']['object-storage']['statistics']['graphing_ip']
end
#--------------
# swift common
#--------------
platform_options = node['openstack']['object-storage']['platform']
platform_options['swift_packages'].each do |pkg|
package pkg do
options platform_options['package_overrides']
action :upgrade
end
end
directory '/etc/swift' do
action :create
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
mode 00700
end
# determine hash
if node['openstack']['object-storage']['swift_secret_databag_name'].nil?
swift_hash_path_prefix = node['openstack']['object-storage']['swift_hash_path_prefix']
swift_hash_path_prefix = get_password 'token', 'swift_hash_path_prefix' if swift_hash_path_prefix.nil?
swift_hash_path_suffix = node['openstack']['object-storage']['swift_hash_path_suffix']
swift_hash_path_suffix = get_password 'token', 'swift_hash_path_suffix' if swift_hash_path_suffix.nil?
else
# Deprecated, else case to be removed.
swift_secrets = Chef::EncryptedDataBagItem.load 'secrets', node['openstack']['object-storage']['swift_secret_databag_name']
swift_hash_path_prefix = swift_secrets['swift_hash']
end
template '/etc/swift/swift.conf' do
source 'swift.conf.erb'
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
mode 00600
variables(
swift_hash_path_prefix: swift_hash_path_prefix,
swift_hash_path_suffix: swift_hash_path_suffix
)
end
# need a swift user
user node['openstack']['object-storage']['user'] do
shell '/bin/bash'
action :modify
end
package 'git' do
options platform_options['package_overrides']
action :upgrade
end
# drop a ring puller script
# TODO: make this smarter
git_builder_ip = node['openstack']['object-storage']['git_builder_ip']
template '/etc/swift/pull-rings.sh' do
source 'pull-rings.sh.erb'
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
mode 00700
variables(
builder_ip: git_builder_ip,
service_prefix: platform_options['service_prefix']
)
end
execute '/etc/swift/pull-rings.sh' do
cwd '/etc/swift'
only_if '[ -x /etc/swift/pull-rings.sh ]'
end

95
recipes/container-server.rb
View File

@@ -1,95 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Recipe:: swift-container-server
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
include_recipe 'openstack-object-storage::common'
include_recipe 'openstack-object-storage::storage-common'
include_recipe 'openstack-object-storage::disks'
class Chef::Recipe # rubocop:disable Documentation
include ServiceUtils
end
platform_options = node['openstack']['object-storage']['platform']
platform_options['container_packages'].each do |pkg|
package pkg do
action :upgrade
options platform_options['override_options']
end
end
svc_names = {}
%w(swift-container swift-container-auditor swift-container-replicator swift-container-updater).each do |svc|
svc_names[svc] = svc_name(svc)
end
svc_names.values.each do |svc|
service svc do
supports status: true, restart: true
action [:enable, :start]
only_if '[ -e /etc/swift/container-server.conf ] && [ -e /etc/swift/container.ring.gz ]'
end
end
memcache_servers = memcached_servers.join ','
template '/etc/swift/container-reconciler.conf' do
source 'container-reconciler.conf.erb'
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
mode 00600
variables(
'memcache_servers' => memcache_servers
)
end
template '/etc/swift/container-server.conf' do
source 'container-server.conf.erb'
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
mode 00600
variables(
'bind_ip' => node['openstack']['object-storage']['network']['container-bind-ip'],
'bind_port' => node['openstack']['object-storage']['network']['container-bind-port']
)
notifies :restart, "service[#{svc_names['swift-container']}]", :immediately
notifies :restart, "service[#{svc_names['swift-container-replicator']}]", :immediately
notifies :restart, "service[#{svc_names['swift-container-updater']}]", :immediately
notifies :restart, "service[#{svc_names['swift-container-auditor']}]", :immediately
end
unless node['openstack']['object-storage']['container-server']['allowed_sync_hosts'] == []
service_name = svc_name('swift-container-sync')
template '/etc/swift/container-sync-realms.conf' do
source 'container-sync-realms.conf.erb'
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
mode 00600
notifies :restart, "service[#{service_name}]", :immediately
end
service service_name do
supports status: false, restart: true
action [:enable, :start]
only_if '[ -e /etc/swift/container-server.conf ] && [ -e /etc/swift/container.ring.gz ]'
end
end

68
recipes/disks.rb
View File

@@ -1,68 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Recipe:: disks
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Author: Ron Pedde <ron.pedde@rackspace.com>
# Inspired by: Andi Abes @ Dell
class Chef::Recipe # rubocop:disable Documentation
include IPUtils
include DriveUtils
end
platform_options = node['openstack']['object-storage']['platform']
package 'xfsprogs' do
options platform_options['package_overrides']
action :upgrade
end
%w(parted util-linux).each do |pkg|
package pkg do
options platform_options['package_overrides']
action :upgrade
end
end
disk_enum_expr = node['openstack']['object-storage']['disk_enum_expr']
disk_test_filter = node['openstack']['object-storage']['disk_test_filter']
disks = locate_disks(disk_enum_expr, disk_test_filter)
Chef::Log.info("Located disks: #{disks}")
disks.each do |disk|
openstack_object_storage_disk "/dev/#{disk}" do
part [{ type: platform_options['disk_format'], size: :remaining }]
action :ensure_exists
end
end
# FIXME: "#{x}1" is only really valid for {v,s,h}dx. Doesn't
# work for loop or probably for hp-style /dev/cciss/c0d0p1x0t0g0m1whatever
#
# additionally, there is an implicit assumption that bind ports
# for all object/container/account services are on the same net
disk_ip = locate_ip_in_cidr(node['openstack']['object-storage']['network']['object-cidr'], node)
openstack_object_storage_mounts '/srv/node' do
action :ensure_exists
publish_attributes 'swift/state/devs'
devices disks.map { |x| "#{x}1" }
ip disk_ip
format platform_options['disk_format']
end

99
recipes/identity_registration.rb
View File

@@ -1,99 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Recipe:: identity_registration
#
# Copyright 2013, AT&T Services, Inc.
# Copyright 2013, Craig Tracey <craigtracey@gmail.com>
# Copyright 2013, Opscode, Inc.
# Copyright 2015, IBM Corp.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
require 'uri'
class ::Chef::Recipe # rubocop:disable Documentation
include ::Openstack
end
identity_admin_endpoint = admin_endpoint 'identity-admin'
token = get_password 'token', 'openstack_identity_bootstrap_token'
auth_url = ::URI.decode identity_admin_endpoint.to_s
admin_api_endpoint = admin_endpoint 'object-storage-api'
internal_api_endpoint = internal_endpoint 'object-storage-api'
public_api_endpoint = public_endpoint 'object-storage-api'
service_pass = get_password 'service', 'openstack-object-storage'
service_tenant_name = node['openstack']['object-storage']['service_tenant_name']
service_user = node['openstack']['object-storage']['service_user']
service_role = node['openstack']['object-storage']['service_role']
region = node['openstack']['object-storage']['region']
# Register Object Storage Service
openstack_identity_register 'Register Object Storage Service' do
auth_uri auth_url
bootstrap_token token
service_name 'swift'
service_type 'object-store'
service_description 'Swift Service'
action :create_service
end
# Register Object Storage Endpoint
openstack_identity_register 'Register Object Storage Endpoint' do
auth_uri auth_url
bootstrap_token token
service_type 'object-store'
endpoint_region region
endpoint_adminurl admin_api_endpoint.to_s
endpoint_internalurl internal_api_endpoint.to_s
endpoint_publicurl public_api_endpoint.to_s
action :create_endpoint
end
# Register Service Tenant
openstack_identity_register 'Register Service Tenant' do
auth_uri auth_url
bootstrap_token token
tenant_name service_tenant_name
tenant_description 'Service Tenant'
action :create_tenant
end
# Register Service User
openstack_identity_register "Register #{service_user} User" do
auth_uri auth_url
bootstrap_token token
tenant_name service_tenant_name
user_name service_user
user_pass service_pass
action :create_user
end
## Grant Service role to Service User for Service Tenant ##
openstack_identity_register "Grant '#{service_role}' Role to #{service_user} User for #{service_tenant_name} Tenant" do
auth_uri auth_url
bootstrap_token token
tenant_name service_tenant_name
user_name service_user
role_name service_role
action :grant_role
end

94
recipes/management-server.rb
View File

@@ -1,94 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Recipe:: management-server
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
include_recipe 'openstack-object-storage::common'
# FIXME: This should probably be a role (ring-builder?), so you don't end up
# with multiple repos!
include_recipe 'openstack-object-storage::ring-repo'
platform_options = node['openstack']['object-storage']['platform']
if node['openstack']['object-storage']['authmode'] == 'swauth'
case node['openstack']['object-storage']['swauth_source']
when 'package'
platform_options['swauth_packages'].each do |pkg|
package pkg do
action :upgrade
options platform_options['override_options']
end
end
when 'git'
git "#{Chef::Config[:file_cache_path]}/swauth" do
repository node['openstack']['object-storage']['swauth_repository']
revision node['openstack']['object-storage']['swauth_version']
action :sync
end
bash 'install_swauth' do
cwd "#{Chef::Config[:file_cache_path]}/swauth"
user 'root'
group 'root'
code <<-EOH
python setup.py install
EOH
environment 'PREFIX' => '/usr/local'
end
end
end
# determine where to find dispersion login information
if node['openstack']['object-storage']['swift_secret_databag_name'].nil?
auth_user = node['openstack']['object-storage']['dispersion']['auth_user']
auth_user = get_password 'token', 'dispersion_auth_user' if auth_user.nil?
auth_key = node['openstack']['object-storage']['dispersion']['auth_key']
auth_key = get_password 'token', 'dispersion_auth_key' if auth_key.nil?
else
# Deprecated, else case to be removed.
swift_secrets = Chef::EncryptedDataBagItem.load 'secrets', node['openstack']['object-storage']['swift_secret_databag_name']
auth_user = swift_secrets['dispersion_auth_user']
auth_key = swift_secrets['dispersion_auth_key']
end
if node['openstack']['object-storage']['statistics']['enabled']
template platform_options['swift_statsd_publish'] do
source 'swift-statsd-publish.py.erb'
owner 'root'
group 'root'
mode 00755
end
cron 'cron_swift_statsd_publish' do
command "#{platform_options['swift_statsd_publish']} > /dev/null 2>&1"
minute "*/#{node['openstack']['object-storage']['statistics']['report_frequency']}"
end
end
template '/etc/swift/dispersion.conf' do
source 'dispersion.conf.erb'
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
mode 00600
variables(
'auth_url' => node['openstack']['object-storage']['auth_url'],
'auth_user' => auth_user,
'auth_key' => auth_key
)
end

21
recipes/memcached.rb
View File

@@ -1,21 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Recipe:: memcached
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
include_recipe 'memcached'

83
recipes/object-server.rb
View File

@@ -1,83 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Recipe:: swift-object-server
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
include_recipe 'openstack-object-storage::common'
include_recipe 'openstack-object-storage::storage-common'
include_recipe 'openstack-object-storage::disks'
class Chef::Recipe # rubocop:disable Documentation
include ServiceUtils
end
platform_options = node['openstack']['object-storage']['platform']
platform_options['object_packages'].each do |pkg|
package pkg do
action :upgrade
options platform_options['override_options']
end
end
svc_names = {}
%w(swift-object swift-object-replicator swift-object-auditor swift-object-updater).each do |svc|
svc_names[svc] = svc_name(svc)
end
svc_names.values.each do |svc|
service svc do
supports status: false, restart: true
action [:enable, :start]
only_if '[ -e /etc/swift/object-server.conf ] && [ -e /etc/swift/object.ring.gz ]'
end
end
memcache_servers = memcached_servers.join ','
template '/etc/swift/object-expirer.conf' do
source 'object-expirer.conf.erb'
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
mode 00600
variables(
'memcache_servers' => memcache_servers
)
end
template '/etc/swift/object-server.conf' do
source 'object-server.conf.erb'
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
mode 00600
variables(
'bind_ip' => node['openstack']['object-storage']['network']['object-bind-ip'],
'bind_port' => node['openstack']['object-storage']['network']['object-bind-port']
)
notifies :restart, "service[#{svc_names['swift-object']}]", :immediately
notifies :restart, "service[#{svc_names['swift-object-replicator']}]", :immediately
notifies :restart, "service[#{svc_names['swift-object-updater']}]", :immediately
notifies :restart, "service[#{svc_names['swift-object-auditor']}]", :immediately
end
cron 'swift-recon' do
minute '*/5'
command 'swift-recon-cron /etc/swift/object-server.conf'
user node['openstack']['object-storage']['user']
end

146
recipes/proxy-server.rb
View File

@@ -1,146 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Recipe:: proxy-server
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
include_recipe 'openstack-object-storage::common'
include_recipe 'openstack-object-storage::memcached'
class Chef::Recipe # rubocop:disable Documentation
include IPUtils
include ServiceUtils
end
if node.run_list.expand(node.chef_environment).recipes.include?('openstack-object-storage::setup')
Chef::Log.info('I ran the openstack-object-storage::setup so I will use my own swift passwords')
else
setup_role = node['openstack']['object-storage']['setup_chef_role']
setup = search(:node, "chef_environment:#{node.chef_environment} AND roles:#{setup_role}")
if setup.length == 0
Chef::Application.fatal! 'You must have run the openstack-object-storage::setup recipe (on this or another node) before running the swift::proxy recipe on this node'
elsif setup.length == 1
Chef::Log.info "Found openstack-object-storage::setup node: #{setup[0].name}"
node.set['openstack']['object-storage']['service_pass'] = setup[0]['swift']['service_pass']
elsif setup.length > 1
Chef::Application.fatal! 'You have multiple nodes in your environment that have run swift-setup, and that is not allowed'
end
end
platform_options = node['openstack']['object-storage']['platform']
# upgrade platform-specific packages
platform_options['proxy_packages'].each do |pkg|
package pkg do
action :upgrade
options platform_options['override_options']
end
end
case node['openstack']['object-storage']['authmode']
when 'swauth'
case node['openstack']['object-storage']['swauth_source']
when 'package'
platform_options['swauth_packages'].each do |pkg|
package pkg do
action :upgrade
options platform_options['override_options']
end
end
when 'git'
git "#{Chef::Config[:file_cache_path]}/swauth" do
repository node['openstack']['object-storage']['swauth_repository']
revision node['openstack']['object-storage']['swauth_version']
action :sync
end
bash 'install_swauth' do
cwd "#{Chef::Config[:file_cache_path]}/swauth"
user 'root'
group 'root'
code <<-EOH
python setup.py install
EOH
environment 'PREFIX' => '/usr/local'
end
end
when 'keystone'
package 'python-keystoneclient' do
action :upgrade
end
identity_endpoint = internal_endpoint 'identity-internal'
identity_admin_endpoint = admin_endpoint 'identity-admin'
service_pass = get_password 'service', 'openstack-object-storage'
auth_uri = auth_uri_transform identity_endpoint.to_s, node['openstack']['object-storage']['api']['auth']['version']
identity_uri = identity_uri_transform(identity_admin_endpoint)
end
package 'python-swift-informant' do
action :upgrade
only_if { node['openstack']['object-storage']['use_informant'] }
end
directory '/var/cache/swift' do
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
mode 00700
end
proxy_service_name = svc_name('swift-proxy')
service proxy_service_name do
supports status: true, restart: true
action [:enable, :start]
only_if '[ -e /etc/swift/proxy-server.conf ] && [ -e /etc/swift/object.ring.gz ]'
end
# determine authkey to use
if node['openstack']['object-storage']['swift_secret_databag_name'].nil?
authkey = node['openstack']['object-storage']['authkey']
authkey = get_password 'token', 'swift_authkey' if authkey.nil?
else
# Deprecated, else case to be removed.
swift_secrets = Chef::EncryptedDataBagItem.load 'secrets', node['openstack']['object-storage']['swift_secret_databag_name']
authkey = swift_secrets['swift_authkey']
end
proxy_api_bind = endpoint 'object-storage-api-bind'
proxy_api_bind_port = node['openstack']['object-storage']['network']['proxy-bind-port']
proxy_api_bind_port = proxy_api_bind.port if proxy_api_bind_port.nil?
proxy_api_bind_host = node['openstack']['object-storage']['network']['proxy-bind-ip']
proxy_api_bind_host = proxy_api_bind.host if proxy_api_bind_host.nil?
memcache_servers = memcached_servers.join ','
# create proxy config file
template '/etc/swift/proxy-server.conf' do
source 'proxy-server.conf.erb'
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
mode 00600
variables(
'authmode' => node['openstack']['object-storage']['authmode'],
'bind_host' => proxy_api_bind_host,
'bind_port' => proxy_api_bind_port,
'authkey' => authkey,
'memcache_servers' => memcache_servers,
'auth_uri' => auth_uri,
'identity_uri' => identity_uri,
'service_pass' => service_pass
)
notifies :restart, "service[#{proxy_service_name}]", :immediately
end

169
recipes/ring-repo.rb
View File

@@ -1,169 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Recipe:: ring-repo
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# This recipe creates a git ring repository on the management node
# for purposes of ring synchronization
#
platform_options = node['openstack']['object-storage']['platform']
ring_options = node['openstack']['object-storage']['ring']
git_config_email = "git config user.email 'chef@openstack.org'"
git_config_name = "git config user.name 'Chef'"
platform_options['git_packages'].each do |pkg|
package pkg do
options platform_options['package_overrides']
action :upgrade
end
end
service 'xinetd' do
supports status: false, restart: true
action [:enable, :start]
only_if { platform_family?('rhel') }
end
execute 'create empty git repo' do
cwd '/tmp'
umask 022
command "mkdir $$; cd $$; git init; echo \"backups\" \> .gitignore; #{git_config_email} ; #{git_config_name} ; git add .gitignore; git commit -m 'initial commit' --author='chef <chef@openstack>'; git push file:///#{platform_options['git_dir']}/rings master"
user node['openstack']['object-storage']['user']
action :nothing
end
directory 'git-directory' do
path "#{platform_options['git_dir']}/rings"
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
mode 00755
recursive true
action :create
end
execute 'initialize git repo' do
cwd "#{platform_options['git_dir']}/rings"
umask 022
user node['openstack']['object-storage']['user']
command 'git init --bare && touch git-daemon-export-ok'
creates "#{platform_options['git_dir']}/rings/config"
action :run
notifies :run, 'execute[create empty git repo]', :immediately
end
case node['platform_family']
when 'rhel'
service 'git-daemon' do
service_name platform_options['git_service']
action [:enable]
end
when 'debian'
service 'git-daemon' do
service_name platform_options['git_service']
action [:enable, :start]
end
end
cookbook_file '/etc/default/git-daemon' do
owner 'root'
group 'root'
mode 00644
source 'git-daemon.default'
action :create
notifies :restart, 'service[git-daemon]', :immediately
not_if { platform_family?('rhel') }
end
directory '/etc/swift/ring-workspace' do
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
mode 00755
action :create
end
execute 'checkout-rings' do # ~FC040
cwd '/etc/swift/ring-workspace'
command "git clone file://#{platform_options['git_dir']}/rings"
user node['openstack']['object-storage']['user']
creates '/etc/swift/ring-workspace/rings'
end
['account', 'container', 'object'].each do |ring_type|
part_power = ring_options['part_power']
min_part_hours = ring_options['min_part_hours']
replicas = ring_options['replicas']
Chef::Log.info("Building initial ring #{ring_type} using part_power=#{part_power}, "\
"min_part_hours=#{min_part_hours}, replicas=#{replicas}")
execute "add #{ring_type}.builder" do
cwd '/etc/swift/ring-workspace/rings'
command "git add #{ring_type}.builder && #{git_config_email} ; #{git_config_name} && git commit -m 'initial ring builders' --author='chef <chef@openstack>'"
user node['openstack']['object-storage']['user']
action :nothing
end
execute "create #{ring_type} builder" do
cwd '/etc/swift/ring-workspace/rings'
command "swift-ring-builder #{ring_type}.builder create #{part_power} #{replicas} #{min_part_hours}"
user node['openstack']['object-storage']['user']
creates "/etc/swift/ring-workspace/rings/#{ring_type}.builder"
notifies :run, "execute[add #{ring_type}.builder]", :immediate
end
end
bash 'rebuild-rings' do
action :nothing
cwd '/etc/swift/ring-workspace/rings'
user node['openstack']['object-storage']['user']
code <<-EOF
set -x
# Should this be done?
git reset --hard
git clean -df
../generate-rings.sh
for d in object account container; do swift-ring-builder ${d}.builder; done
add=0
if test -n "$(find . -maxdepth 1 -name '*gz' -print -quit)"
then
git add *builder *gz
add=1
else
git add *builder
add=1
fi
if [ $add -ne 0 ]
then
git commit -m "Autobuild of rings on $(date +%Y%m%d) by Chef" --author="chef <chef@openstack>"
git push
fi
EOF
end
openstack_object_storage_ring_script '/etc/swift/ring-workspace/generate-rings.sh' do
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
mode 00700
ring_path '/etc/swift/ring-workspace/rings'
action :ensure_exists
notifies :run, 'bash[rebuild-rings]', :immediate
end

76
recipes/rsync.rb
View File

@@ -1,76 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Recipe:: rsync
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
platform_options = node['openstack']['object-storage']['platform']
platform_options['rsync_packages'].each do |pkg|
package pkg do
action :upgrade
options platform_options['override_options']
end
end
# rhel based systems install rsync and run it with rsync. We don't want to do that
cookbook_file '/etc/init.d/rsyncd' do
owner 'root'
group 'root'
mode 00755
source 'rsync.init'
action :create
only_if { platform_family?('rhel') }
end
# FIXME: chicken and egg
case node['platform_family']
when 'rhel'
# enable rsyncd
rsync_servicename = 'rsyncd'
service 'rsyncd' do
supports status: false, restart: true, start: true, stop: true
action [:enable, :start]
only_if '[ -f /etc/rsyncd.conf ]'
end
# disable rsync (the one via xinetd)
service 'rsync' do
supports status: false, restart: false, start: false, stop: false
action [:disable]
end
when 'debian'
rsync_servicename = 'rsync'
service 'rsync' do
supports status: false, restart: true
action [:enable, :start]
only_if '[ -f /etc/rsyncd.conf ]'
end
end
template '/etc/rsyncd.conf' do
source 'rsyncd.conf.erb'
mode 00644
notifies :restart, "service[#{rsync_servicename}]", :immediately
end
execute 'enable rsync' do
command "sed -i 's/RSYNC_ENABLE=false/RSYNC_ENABLE=true/' /etc/default/rsync"
only_if "grep -q 'RSYNC_ENABLE=false' /etc/default/rsync"
notifies :restart, 'service[rsync]', :immediately
action :run
not_if { platform_family?('rhel') }
end

88
recipes/setup.rb
View File

@@ -1,88 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Recipe:: setup
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
include_recipe 'openstack-object-storage::common'
# make sure we die if there are multiple swift-setups
if Chef::Config[:solo]
Chef::Application.fatal! 'This recipe uses search. Chef Solo does not support search.'
else
setup_role = node['openstack']['object-storage']['setup_chef_role']
setup_role_count = search(:node, "chef_environment:#{node.chef_environment} AND roles:#{setup_role}").length
if setup_role_count > 1
Chef::Application.fatal! 'You can only have one node with the swift-setup role'
end
end
unless node['openstack']['object-storage']['service_pass']
Chef::Log.info('Running swift setup - setting swift passwords')
end
platform_options = node['openstack']['object-storage']['platform']
# install platform-specific packages
platform_options['proxy_packages'].each do |pkg|
package pkg do
action :upgrade
options platform_options['override_options']
end
end
case node['openstack']['object-storage']['authmode']
when 'swauth'
case node['openstack']['object-storage']['swauth_source']
when 'package'
platform_options['swauth_packages'].each do |pkg|
package pkg do
action :upgrade
options platform_options['override_options']
end
end
when 'git'
git "#{Chef::Config[:file_cache_path]}/swauth" do
repository node['openstack']['object-storage']['swauth_repository']
revision node['openstack']['object-storage']['swauth_version']
action :sync
end
bash 'install_swauth' do
cwd "#{Chef::Config[:file_cache_path]}/swauth"
user 'root'
group 'root'
code <<-EOH
python setup.py install
EOH
environment 'PREFIX' => '/usr/local'
end
else
Chef::Log.fatal("Object storage swauth source #{node['openstack']['object-storage']['swauth_source']} is not supported")
end
when 'keystone'
package 'python-keystoneclient' do
action :upgrade
end
include_recipe 'openstack-object-storage::identity_registration'
else
Chef::Log.fatal("Object storage authmode #{node['openstack']['object-storage']['authmode']} is not supported")
end
package 'python-swift-informant' do
action :upgrade
only_if { node['openstack']['object-storage']['use_informant'] }
end

42
recipes/storage-common.rb
View File

@@ -1,42 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Recipe:: storage-common
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
include_recipe 'openstack-object-storage::rsync'
template '/etc/swift/drive-audit.conf' do
source 'drive-audit.conf.erb'
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
mode 00600
end
cron 'drive-audit' do
hour node['openstack']['object-storage']['audit_hour']
minute '10'
command 'swift-drive-audit /etc/swift/drive-audit.conf'
end
directory '/var/cache/swift' do
owner node['openstack']['object-storage']['user']
group node['openstack']['object-storage']['group']
recursive true
action :create
mode 00700
end

43
resources/disk.rb
View File

@@ -1,43 +0,0 @@
# encoding: UTF-8
#
# Copyright 2011, Dell
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Author: andi abes
#
# Ensure that a disk's partition table matches expectations.
# Sample use:
#
# openstack_object_storage_disk '/dev/sdb' do
# part([
# { type: "xfs", size: swift_disk::ONE_GIG*4 },
# { type: "xfs", size: swift_disk::remaining }
# ])
# action :ensure_exists
# end
actions :ensure_exists
def initialize(*args)
super
@action = :ensure_exists
end
attribute :name, kind_of: String
attribute :size, kind_of: Integer
attribute :blocks, kind_of: Integer
attribute :device, kind_of: String
attribute :part, kind_of: Array
attribute :status, kind_of: Symbol

70
resources/mounts.rb
View File

@@ -1,70 +0,0 @@
# encoding: UTF-8
#
# Cookbook Name:: openstack-object-storage
# Resource:: mounts
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Author: Ron Pedde <ron.pedde@rackspace.com>
#
# Ensure that swift mounts are strongly enforced. This
# will ensure specified drives are mounted, and unspecified
# drives are not mounted. In addition, if there is a stale
# mountpoint (from disk failure, maybe?), then that mountpoint
# will try to be unmounted
#
# Sample use:
#
# openstack_object_storage_mounts '/srv/node' do
# devices [ 'sdb1', 'sdc1' ]
# action :ensure_exists
# ip '10.1.1.1'
# end
# It will force mounts based on fs uuid (mangled to remove
# dashes) and return a structure that describes the disks
# mounted.
# As this is expected to be consumed for the purposes of
# swift, the ip address should be the address that gets
# embedded into the ring (i.e. the listen port of the storage server)
# Example return structure:
# { '2a9452c5-d929-43d9-9631-4340ace45279': {
# 'device': 'sdb1',
# 'ip': '10.1.1.1',
# 'mounted': 'true',
# 'mountpoint': '2a9452c5d92943d996314340ace45279',
# 'size': 1022 (in 1k increments)
# 'uuid': '2a9452c5-d929-43d9-9631-4340ace45279'
# },
# ...
# }
actions :ensure_exists
def initialize(*args)
super
@action = :ensure_exists
end
attribute :name, kind_of: String
attribute :devices, kind_of: Array
attribute :ip, kind_of: String, default: '127.0.0.1'
attribute :publish_attributes, kind_of: String, default: nil
attribute :format, kind_of: String, default: 'xfs'
attribute :pass, kind_of: Integer, default: 2

42
resources/ring_script.rb
View File

@@ -1,42 +0,0 @@
# encoding: UTF-8
#
# Copyright 2012, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Author: Ron Pedde <ron.pedde@rackspace.com>
#
# Build a proposed ring-building script
# Sample use:
#
# openstack_object_storage_ring_script '/tmp/build-rings.sh' do
# owner 'root'
# group 'swift'
# mode '0700'
# ring_path '/etc/swift/ring-workspace'
# action :ensure_exists
# end
actions :ensure_exists
def initialize(*args)
super
@action = :ensure_exists
end
attribute :name, kind_of: String
attribute :owner, kind_of: String, default: 'root'
attribute :group, kind_of: String, default: 'root'
attribute :mode, kind_of: Integer, default: 00600
attribute :ring_path, kind_of: String, default: '/etc/swift'

61
spec/account_spec.rb
View File

@@ -1,61 +0,0 @@
# encoding: UTF-8
require_relative 'spec_helper'
describe 'openstack-object-storage::account-server' do
describe 'ubuntu' do
let(:runner) { ChefSpec::SoloRunner.new(UBUNTU_OPTS) }
let(:node) { runner.node }
let(:chef_run) do
node.set['openstack']['object-storage']['disk_enum_expr'] = "[{ 'sda' => {}}]"
node.set['openstack']['object-storage']['disk_test_filter'] = [
'candidate =~ /sd[^a]/ or candidate =~ /hd[^a]/ or candidate =~ /vd[^a]/ or candidate =~ /xvd[^a]/',
"File.exist?('/dev/' + candidate)",
"not system('/sbin/parted /dev/' + candidate + ' -s print | grep linux-swap')",
"not info.has_key?('removable') or info['removable'] == 0.to_s"]
# mock out an interface on the storage node
node.set['network'] = MOCK_NODE_NETWORK_DATA['network']
runner.converge(described_recipe)
end
include_context 'swift-stubs'
it 'upgrades swift account packages' do
expect(chef_run).to upgrade_package('swift-account')
end
it 'upgrades swiftclient package' do
expect(chef_run).to upgrade_package('python-swiftclient')
end
it 'starts swift account services on boot' do
%w(swift-account swift-account-auditor swift-account-reaper swift-account-replicator).each do |svc|
expect(chef_run).to enable_service(svc)
end
end
describe '/etc/swift/account-server.conf' do
let(:file) { chef_run.template('/etc/swift/account-server.conf') }
it_behaves_like 'custom template banner displayer' do
let(:file_name) { file.name }
end
it 'creates account-server.conf' do
expect(chef_run).to create_template(file.name).with(
user: 'swift',
group: 'swift',
mode: 00600
)
end
describe 'default attribute values' do
it_behaves_like 'a common swift server default attribute values checker', 'account', '0.0.0.0', '6002'
end
it_behaves_like 'a common swift server configurator', 'account', '0.0.0.0', '6002'
it_behaves_like 'some common swift server values'
end
end
end

16
spec/client_spec.rb
View File

@@ -1,16 +0,0 @@
# encoding: UTF-8
require_relative 'spec_helper'
describe 'openstack-object-storage::client' do
describe 'ubuntu' do
let(:runner) { ChefSpec::SoloRunner.new(UBUNTU_OPTS) }
let(:node) { runner.node }
let(:chef_run) do
runner.converge(described_recipe)
end
it 'installs packages' do
expect(chef_run).to upgrade_package('python-swiftclient')
end
end
end

162
spec/common_spec.rb
View File

@@ -1,162 +0,0 @@
# encoding: UTF-8
require_relative 'spec_helper'
describe 'openstack-object-storage::common' do
describe 'ubuntu' do
let(:runner) { ChefSpec::SoloRunner.new(UBUNTU_OPTS) }
let(:node) { runner.node }
let(:chef_run) do
runner.converge(described_recipe)
end
include_context 'swift-stubs'
it 'includes openstack-common::sysctl' do
expect(chef_run).to include_recipe('openstack-common::sysctl')
end
it 'upgrades swift packages' do
expect(chef_run).to upgrade_package('swift')
end
it 'does not create swift user' do
expect(chef_run).not_to create_user('swift').with(
shell: '/bin/bash'
)
end
it 'executes pull rings' do
expect(chef_run).to run_execute('/etc/swift/pull-rings.sh').with(
cwd: '/etc/swift'
)
end
describe '60-openstack.conf' do
let(:file) { chef_run.template('/etc/sysctl.d/60-openstack.conf') }
it 'creates /etc/sysctl.d/60-openstack.conf' do
expect(chef_run).to create_template(file.name).with(
user: 'root',
group: 'root',
mode: 00644
)
end
it 'sets the net.ipv4.tcp_tw_recycle' do
match = 'net.ipv4.tcp_tw_recycle = 1'
expect(chef_run).to render_file(file.name).with_content(match)
end
it 'sets the net.ipv4.tcp_tw_reuse' do
match = 'net.ipv4.tcp_tw_reuse = 1'
expect(chef_run).to render_file(file.name).with_content(match)
end
it 'sets the net.ipv4.tcp_syncookies' do
match = 'net.ipv4.tcp_syncookies = 0'
expect(chef_run).to render_file(file.name).with_content(match)
end
end
it 'upgrades git package for ring management' do
expect(chef_run).to upgrade_package('git')
end
describe '/etc/swift' do
let(:dir) { chef_run.directory('/etc/swift') }
it 'creates /etc/swift' do
expect(chef_run).to create_directory(dir.name).with(
user: 'swift',
group: 'swift',
mode: 00700
)
end
end
describe '/etc/swift/swift.conf' do
let(:file) { chef_run.template('/etc/swift/swift.conf') }
it_behaves_like 'custom template banner displayer' do
let(:file_name) { file.name }
end
it 'creates swift.conf' do
expect(chef_run).to create_template(file.name).with(
user: 'swift',
group: 'swift',
mode: 00600
)
end
it 'template contents' do
[
/^swift_hash_path_prefix = swift_hash_path_prefix-secret$/,
/^swift_hash_path_suffix = swift_hash_path_suffix-secret$/,
/^max_file_size = 5368709122$/,
/^max_meta_name_length = 128$/,
/^max_meta_value_length = 256$/,
/^max_meta_count = 90$/,
/^max_meta_overall_size = 4096$/,
/^max_header_size = 8192$/,
/^max_object_name_length = 1024$/,
/^container_listing_limit = 10000$/,
/^account_listing_limit = 10000$/,
/^max_account_name_length = 256$/,
/^max_container_name_length = 256$/
].each do |content|
expect(chef_run).to render_file(file.name).with_content(content)
end
end
it 'template contents with hash overrides' do
node.set['openstack']['object-storage']['swift_hash_path_prefix'] = '1234'
node.set['openstack']['object-storage']['swift_hash_path_suffix'] = '4321'
[
/^swift_hash_path_prefix = 1234$/,
/^swift_hash_path_suffix = 4321$/
].each do |content|
expect(chef_run).to render_file(file.name).with_content(content)
end
end
end
describe '/etc/swift/pull-rings.sh' do
let(:file) { chef_run.template('/etc/swift/pull-rings.sh') }
it_behaves_like 'custom template banner displayer' do
let(:file_name) { file.name }
end
it 'creates pull-rings.sh' do
expect(chef_run).to create_template(file.name).with(
user: 'swift',
group: 'swift',
mode: 00700
)
end
describe 'default attribute values' do
it 'uses default attribute value for platform service_prefix' do
expect(chef_run.node['openstack']['object-storage']['platform']['service_prefix']).to eq('')
end
it 'uses default attribute value for git_builder_ip' do
expect(chef_run.node['openstack']['object-storage']['git_builder_ip']).to eq('127.0.0.1')
end
end
describe 'template contents' do
it 'uses the builder_ip variable' do
node.set['openstack']['object-storage']['git_builder_ip'] = 'git_builder_ip_value'
expect(chef_run).to render_file(file.name).with_content(%r{git clone git://git_builder_ip_value/rings /etc/swift/rings})
end
it 'uses the service_prefix variable' do
node.set['openstack']['object-storage']['platform']['service_prefix'] = 'service_prefix_'
expect(chef_run).to render_file(file.name).with_content(/service service_prefix_swift-\$\{d\}-replicator restart$/)
end
end
end
end
end

158
spec/container_spec.rb
View File

@@ -1,158 +0,0 @@
# encoding: UTF-8
require_relative 'spec_helper'
describe 'openstack-object-storage::container-server' do
describe 'ubuntu' do
let(:runner) { ChefSpec::SoloRunner.new(UBUNTU_OPTS) }
let(:node) { runner.node }
let(:chef_run) do
node.set['openstack']['object-storage']['disk_enum_expr'] = "[{ 'sda' => {}}]"
node.set['openstack']['object-storage']['disk_test_filter'] = [
'candidate =~ /sd[^a]/ or candidate =~ /hd[^a]/ or candidate =~ /vd[^a]/ or candidate =~ /xvd[^a]/',
"File.exist?('/dev/' + candidate)",
"not system('/sbin/parted /dev/' + candidate + ' -s print | grep linux-swap')",
"not info.has_key?('removable') or info['removable'] == 0.to_s"]
# mock out an interface on the storage node
node.set['network'] = MOCK_NODE_NETWORK_DATA['network']
runner.converge(described_recipe)
end
include_context 'swift-stubs'
it 'upgrades swift container packages' do
expect(chef_run).to upgrade_package('swift-container')
end
it 'starts swift container services on boot' do
node.set['openstack']['object-storage']['container-server']['allowed_sync_hosts'] = %w(host1 host2)
%w(swift-container swift-container-auditor swift-container-replicator swift-container-updater swift-container-sync).each do |svc|
expect(chef_run).to enable_service(svc)
end
end
describe '/etc/swift/container-server.conf' do
let(:file) { chef_run.template('/etc/swift/container-server.conf') }
it_behaves_like 'custom template banner displayer' do
let(:file_name) { file.name }
end
it 'creates account-server.conf' do
expect(chef_run).to create_template(file.name).with(
user: 'swift',
group: 'swift',
mode: 00600
)
end
it 'has allowed sync hosts' do
node.set['openstack']['object-storage']['container-server']['allowed_sync_hosts'] = %w(host1 host2)
expect(chef_run).to render_file(file.name).with_content('allowed_sync_hosts = host1,host2')
end
{ 'bind_ip' => '0.0.0.0',
'bind_port' => '6001',
'log_statsd_default_sample_rate' => '1',
'log_statsd_metric_prefix' => 'openstack.swift.Fauxhai' }.each do |k, v|
it "sets the #{k}" do
expect(chef_run).to render_file(file.name).with_content(/^#{Regexp.quote("#{k} = #{v}")}$/)
end
end
end
describe '/etc/swift/container-server.conf' do
let(:file) { chef_run.template('/etc/swift/container-server.conf') }
it 'creates account-server.conf' do
expect(chef_run).to create_template(file.name).with(
user: 'swift',
group: 'swift',
mode: 00600
)
end
describe 'default attribute values' do
it_behaves_like 'a common swift server default attribute values checker', 'container', '0.0.0.0', '6001'
it 'for allowed_sync_hosts' do
expect(chef_run.node['openstack']['object-storage']['container-server']['allowed_sync_hosts']).to eq([])
end
end
describe 'template contents' do
it_behaves_like 'a common swift server configurator', 'container', '0.0.0.0', '6001'
it_behaves_like 'some common swift server values'
it 'sets allowed_sync_hosts when present' do
node.set['openstack']['object-storage']['container-server']['allowed_sync_hosts'] = %w(host1 host2)
expect(chef_run).to render_file(file.name).with_content(/^allowed_sync_hosts = host1,host2$/)
end
it 'does not set allowed_sync_hosts when not present' do
node.set['openstack']['object-storage']['container-server']['allowed_sync_hosts'] = false
expect(chef_run).not_to render_file(file.name).with_content(/^allowed_sync_hosts = $/)
end
context 'container-sync' do
it 'sets sync_proxy when present' do
node.set['openstack']['object-storage']['container-server']['container-sync']['sync_proxy'] = 'sync_proxy_value'
expect(chef_run).to render_file(file.name).with_content(/^sync_proxy = sync_proxy_value$/)
end
it 'does not set allowed_sync_hosts when not present' do
node.set['openstack']['object-storage']['container-server']['container-sync']['sync_proxy'] = false
expect(chef_run).not_to render_file(file.name).with_content(/^sync_proxy = $/)
end
%w(log_name log_facility log_level interval container_time).each do |attr|
it "sets the container-sync #{attr} attribute" do
node.set['openstack']['object-storage']['container-server']['container-sync'][attr] = "#{attr}_value"
expect(chef_run).to render_file(file.name).with_content(/^#{attr} = #{attr}_value$/)
end
end
end
end
end
describe '/etc/swift/object-reconciler.conf' do
let(:file) { chef_run.template('/etc/swift/container-reconciler.conf') }
it_behaves_like 'custom template banner displayer' do
let(:file_name) { file.name }
end
it 'creates object-reconciler.conf' do
expect(chef_run).to create_template(file.name).with(
user: 'swift',
group: 'swift',
mode: 00600
)
end
it 'sets the memcache_servers attribute' do
expect(chef_run).to render_file(file.name).with_content(/^memcache_servers = host1:111,host2:222$/)
end
end
describe '/etc/swift/container-sync-realms.conf' do
before do
node.set['openstack']['object-storage']['container-server']['allowed_sync_hosts'] = ['host1', 'host2', 'host3']
end
let(:file) { chef_run.template('/etc/swift/container-sync-realms.conf') }
it_behaves_like 'custom template banner displayer' do
let(:file_name) { file.name }
end
it 'creates container-sync-realms.conf' do
expect(chef_run).to create_template(file.name).with(
user: 'swift',
group: 'swift',
mode: 00600
)
end
end
end
end

31
spec/disks_spec.rb
View File

@@ -1,31 +0,0 @@
# encoding: UTF-8
require_relative 'spec_helper'
describe 'openstack-object-storage::disks' do
describe 'ubuntu' do
let(:runner) { ChefSpec::SoloRunner.new(UBUNTU_OPTS) }
let(:node) { runner.node }
let(:chef_run) do
node.set['openstack']['object-storage']['disk_enum_expr'] = "[{ 'sda' => {}}]"
node.set['openstack']['object-storage']['disk_test_filter'] = [
'candidate =~ /sd[^a]/ or candidate =~ /hd[^a]/ or candidate =~ /vd[^a]/ or candidate =~ /xvd[^a]/',
"File.exist?('/dev/' + candidate)",
"not system('/sbin/parted /dev/' + candidate + ' -s print | grep linux-swap')",
"not info.has_key?('removable') or info['removable'] == 0.to_s"]
# mock out an interface on the storage node
node.set['network'] = MOCK_NODE_NETWORK_DATA['network']
runner.converge(described_recipe)
end
include_context 'swift-stubs'
it 'upgrades xfs progs package' do
expect(chef_run).to upgrade_package('xfsprogs')
end
it 'upgrades parted package' do
expect(chef_run).to upgrade_package('parted')
end
end
end

186
spec/identity_registration_spec.rb
View File

@@ -1,186 +0,0 @@
# encoding: UTF-8
require_relative 'spec_helper'
describe 'openstack-object-storage::identity_registration' do
let(:runner) { ChefSpec::SoloRunner.new(UBUNTU_OPTS) }
let(:node) { runner.node }
let(:chef_run) do
runner.converge(described_recipe)
end
include_context 'swift-stubs'
it 'registers object storage service' do
expect(chef_run).to create_service_openstack_identity_register(
'Register Object Storage Service'
).with(
auth_uri: 'http://127.0.0.1:35357/v2.0',
bootstrap_token: 'bootstrap-token',
service_type: 'object-store',
service_description: 'Swift Service',
action: [:create_service]
)
end
context 'registers object storage endpoint' do
it 'with default values' do
expect(chef_run).to create_endpoint_openstack_identity_register(
'Register Object Storage Endpoint'
).with(
auth_uri: 'http://127.0.0.1:35357/v2.0',
bootstrap_token: 'bootstrap-token',
service_type: 'object-store',
endpoint_region: 'RegionOne',
endpoint_adminurl: 'http://127.0.0.1:8080/v1/AUTH_%(tenant_id)s',
endpoint_internalurl: 'http://127.0.0.1:8080/v1/AUTH_%(tenant_id)s',
endpoint_publicurl: 'http://127.0.0.1:8080/v1/AUTH_%(tenant_id)s',
action: [:create_endpoint]
)
end
it 'with different admin URL ' do
general_url = 'http://general.host:456/general_path'
admin_url = 'https://admin.host:123/admin_path'
# Set general endpoint
node.set['openstack']['endpoints']['object-storage-api']['uri'] = general_url
# Set the admin endpoint override
node.set['openstack']['endpoints']['admin']['object-storage-api']['uri'] = admin_url
expect(chef_run).to create_endpoint_openstack_identity_register(
'Register Object Storage Endpoint'
).with(
auth_uri: 'http://127.0.0.1:35357/v2.0',
bootstrap_token: 'bootstrap-token',
service_type: 'object-store',
endpoint_region: 'RegionOne',
endpoint_adminurl: admin_url,
endpoint_internalurl: general_url,
endpoint_publicurl: general_url,
action: [:create_endpoint]
)
end
it 'with different internal URL ' do
general_url = 'http://general.host:456/general_path'
internal_url = 'http://internal.host:456/internal_path'
# Set general endpoint
node.set['openstack']['endpoints']['object-storage-api']['uri'] = general_url
# Set the internal endpoint override
node.set['openstack']['endpoints']['internal']['object-storage-api']['uri'] = internal_url
expect(chef_run).to create_endpoint_openstack_identity_register(
'Register Object Storage Endpoint'
).with(
auth_uri: 'http://127.0.0.1:35357/v2.0',
bootstrap_token: 'bootstrap-token',
service_type: 'object-store',
endpoint_region: 'RegionOne',
endpoint_adminurl: general_url,
endpoint_internalurl: internal_url,
endpoint_publicurl: general_url,
action: [:create_endpoint]
)
end
it 'with different public URL ' do
general_url = 'http://general.host:456/general_path'
public_url = 'https://public.host:789/public_path'
# Set general endpoint
node.set['openstack']['endpoints']['object-storage-api']['uri'] = general_url
# Set the public endpoint override
node.set['openstack']['endpoints']['public']['object-storage-api']['uri'] = public_url
expect(chef_run).to create_endpoint_openstack_identity_register(
'Register Object Storage Endpoint'
).with(
auth_uri: 'http://127.0.0.1:35357/v2.0',
bootstrap_token: 'bootstrap-token',
service_type: 'object-store',
endpoint_region: 'RegionOne',
endpoint_adminurl: general_url,
endpoint_internalurl: general_url,
endpoint_publicurl: public_url,
action: [:create_endpoint]
)
end
it 'with all different URLs ' do
admin_url = 'https://admin.host:123/admin_path'
internal_url = 'http://internal.host:456/internal_path'
public_url = 'https://public.host:789/public_path'
node.set['openstack']['endpoints']['admin']['object-storage-api']['uri'] = admin_url
node.set['openstack']['endpoints']['internal']['object-storage-api']['uri'] = internal_url
node.set['openstack']['endpoints']['public']['object-storage-api']['uri'] = public_url
expect(chef_run).to create_endpoint_openstack_identity_register(
'Register Object Storage Endpoint'
).with(
auth_uri: 'http://127.0.0.1:35357/v2.0',
bootstrap_token: 'bootstrap-token',
service_type: 'object-store',
endpoint_region: 'RegionOne',
endpoint_adminurl: admin_url,
endpoint_internalurl: internal_url,
endpoint_publicurl: public_url,
action: [:create_endpoint]
)
end
it 'with custom region override' do
node.set['openstack']['object-storage']['region'] = 'swiftRegion'
expect(chef_run).to create_endpoint_openstack_identity_register(
'Register Object Storage Endpoint'
).with(
auth_uri: 'http://127.0.0.1:35357/v2.0',
bootstrap_token: 'bootstrap-token',
service_type: 'object-store',
endpoint_region: 'swiftRegion',
endpoint_adminurl: 'http://127.0.0.1:8080/v1/AUTH_%(tenant_id)s',
endpoint_internalurl: 'http://127.0.0.1:8080/v1/AUTH_%(tenant_id)s',
endpoint_publicurl: 'http://127.0.0.1:8080/v1/AUTH_%(tenant_id)s',
action: [:create_endpoint]
)
end
end
it 'registers service tenant' do
expect(chef_run).to create_tenant_openstack_identity_register(
'Register Service Tenant'
).with(
auth_uri: 'http://127.0.0.1:35357/v2.0',
bootstrap_token: 'bootstrap-token',
tenant_name: 'service',
tenant_description: 'Service Tenant',
action: [:create_tenant]
)
end
it 'registers service user' do
expect(chef_run).to create_user_openstack_identity_register(
'Register swift User'
).with(
auth_uri: 'http://127.0.0.1:35357/v2.0',
bootstrap_token: 'bootstrap-token',
tenant_name: 'service',
user_name: 'swift',
user_pass: 'swift-pass',
action: [:create_user]
)
end
it 'grants service role to service user for service tenant' do
expect(chef_run).to grant_role_openstack_identity_register(
"Grant 'service' Role to swift User for service Tenant"
).with(
auth_uri: 'http://127.0.0.1:35357/v2.0',
bootstrap_token: 'bootstrap-token',
tenant_name: 'service',
role_name: 'service',
user_name: 'swift',
action: [:grant_role]
)
end
end

190
spec/management_spec.rb
View File

@@ -1,190 +0,0 @@
# encoding: UTF-8
require_relative 'spec_helper'
describe 'openstack-object-storage::management-server' do
describe 'ubuntu' do
let(:runner) { ChefSpec::SoloRunner.new(UBUNTU_OPTS) }
let(:node) { runner.node }
let(:chef_run) do
runner.converge(described_recipe)
end
include_context 'swift-stubs'
it 'does not upgrade swauth package' do
expect(chef_run).not_to upgrade_package 'swauth'
end
describe '/etc/swift/dispersion.conf' do
let(:file) { chef_run.template('/etc/swift/dispersion.conf') }
it_behaves_like 'custom template banner displayer' do
let(:file_name) { file.name }
end
it 'creates dispersion.conf' do
expect(chef_run).to create_template(file.name).with(
user: 'swift',
group: 'swift',
mode: 00600
)
end
describe 'default attribute values' do
it 'uses default attribute value for auth_url' do
expect(chef_run.node['openstack']['object-storage']['auth_url']).to eq('http://127.0.0.1:8080/auth/v1.0')
end
it 'uses default attribute value for swift_secret_databag_name' do
expect(chef_run.node['openstack']['object-storage']['swift_secret_databag_name']).to be_nil
end
it 'uses default attribute value for dispersion auth_user' do
expect(chef_run.node['openstack']['object-storage']['dispersion']['auth_user']).to eq(nil)
end
it 'uses default attribute value for dispersion auth_key' do
expect(chef_run.node['openstack']['object-storage']['dispersion']['auth_key']).to eq(nil)
end
end
describe 'template contents' do
it 'sets the auth_url' do
node.set['openstack']['object-storage']['auth_url'] = 'auth_url_value'
expect(chef_run).to render_file(file.name).with_content(/^auth_url = auth_url_value$/)
end
context 'with user databag' do
%w(user key).each do |attr|
it "sets the auth_#{attr}" do
expect(chef_run).to render_file(file.name).with_content(/^auth_#{attr} = dispersion_auth_#{attr}-secret$/)
end
end
end
context 'with swift databag' do
let(:swift_secrets) do
{ 'dispersion_auth_user' => 'dispersion_auth_user_value',
'dispersion_auth_key' => 'dispersion_auth_key_value' }
end
before do
node.set['openstack']['object-storage']['swift_secret_databag_name'] = 'swift_secret_databag_name_value'
allow(Chef::EncryptedDataBagItem).to receive(:load)
.with('secrets', 'swift_secret_databag_name_value')
.and_return(swift_secrets)
end
%w(user key).each do |attr|
it "sets the auth_#{attr}" do
expect(chef_run).to render_file(file.name).with_content(/^auth_#{attr} = dispersion_auth_#{attr}_value$/)
end
end
end
context 'without swift databag' do
before do
node.set['openstack']['object-storage']['swift_secret_databag_name'] = nil
end
%w(user key).each do |attr|
it "sets the auth_#{attr}" do
node.set['openstack']['object-storage']['dispersion']["auth_#{attr}"] = "auth_#{attr}_value"
expect(chef_run).to render_file(file.name).with_content(/^auth_#{attr} = auth_#{attr}_value$/)
end
end
end
it 'template default contents' do
[
%r{^auth_url = http://127.0.0.1:8080/auth/v1.0$},
/^auth_user = dispersion_auth_user-secret$/,
/^auth_key = dispersion_auth_key-secret$/
].each do |content|
expect(chef_run).to render_file(file.name).with_content(content)
end
end
it 'has template overrides' do
node.set['openstack']['object-storage']['auth_url'] = 'url'
node.set['openstack']['object-storage']['dispersion']['auth_user'] = 'user'
node.set['openstack']['object-storage']['dispersion']['auth_key'] = 'key'
[
/^auth_url = url$/,
/^auth_user = user$/,
/^auth_key = key$/
].each do |content|
expect(chef_run).to render_file(file.name).with_content(content)
end
end
end
end
describe '/usr/local/bin/swift-statsd-publish.py' do
let(:file) { chef_run.template('/usr/local/bin/swift-statsd-publish.py') }
it 'creates /usr/local/bin/swift-statsd-publish.py' do
expect(chef_run).to create_template(file.name).with(
user: 'root',
group: 'root',
mode: 00755
)
end
describe 'default attribute values' do
it 'uses default attribute value for statsd_host' do
expect(chef_run.node['openstack']['object-storage']['statistics']['statsd_host']).to eq('127.0.0.1')
end
it 'uses default attribute value for statsd_port' do
expect(chef_run.node['openstack']['object-storage']['statistics']['statsd_port']).to eq('8125')
end
it 'uses default attribute value for statsd_prefix' do
expect(chef_run.node['openstack']['object-storage']['statistics']['statsd_prefix']).to eq('openstack.swift')
end
it 'uses default attribute value for enable_dispersion_report' do
expect(chef_run.node['openstack']['object-storage']['statistics']['enable_dispersion_report']).to eq(true)
end
it 'uses default attribute value for enable_recon_report' do
expect(chef_run.node['openstack']['object-storage']['statistics']['enable_recon_report']).to eq(true)
end
it 'uses default attribute value for enable_disk_report' do
expect(chef_run.node['openstack']['object-storage']['statistics']['enable_disk_report']).to eq(true)
end
%w(account container object).each do |server_type|
it "uses default attribute value for recon_#{server_type}_cache" do
expect(chef_run.node['openstack']['object-storage']['statistics']["recon_#{server_type}_cache"]).to eq("/var/cache/swift/#{server_type}.recon")
end
end
end
describe 'template contents' do
%w(statsd_host statsd_prefix recon_account_cache recon_container_cache recon_object_cache).each do |attr|
it "sets the #{attr} attribute" do
node.set['openstack']['object-storage']['statistics'][attr] = "#{attr}_value"
expect(chef_run).to render_file(file.name).with_content(/self\.#{attr}\s*= '#{attr}_value'$/)
end
end
it 'sets the statsd_port attribute' do
node.set['openstack']['object-storage']['statistics']['statsd_port'] = 'statsd_port_value'
expect(chef_run).to render_file(file.name).with_content(/self\.statsd_port\s*= statsd_port_value$/)
end
%w(enable_dispersion_report enable_recon_report enable_disk_report).each do |attr|
[true, false].each do |value|
it "sets the #{attr} attribute to #{value}" do
node.set['openstack']['object-storage']['statistics'][attr] = value
expect(chef_run).to render_file(file.name).with_content(/self\.#{attr}\s*= #{value.to_s.capitalize}$/)
end
end
end
end
end
end
end

92
spec/object_spec.rb
View File

@@ -1,92 +0,0 @@
# encoding: UTF-8
require_relative 'spec_helper'
describe 'openstack-object-storage::object-server' do
describe 'ubuntu' do
let(:runner) { ChefSpec::SoloRunner.new(UBUNTU_OPTS) }
let(:node) { runner.node }
let(:chef_run) do
node.set['openstack']['object-storage']['disk_enum_expr'] = "[{ 'sda' => {}}]"
node.set['openstack']['object-storage']['disk_test_filter'] = [
'candidate =~ /sd[^a]/ or candidate =~ /hd[^a]/ or candidate =~ /vd[^a]/ or candidate =~ /xvd[^a]/',
"File.exist?('/dev/' + candidate)",
"not system('/sbin/parted /dev/' + candidate + ' -s print | grep linux-swap')",
"not info.has_key?('removable') or info['removable'] == 0.to_s"]
# mock out an interface on the storage node
node.set['network'] = MOCK_NODE_NETWORK_DATA['network']
runner.converge(described_recipe)
end
include_context 'swift-stubs'
it 'upgrades swift packages' do
expect(chef_run).to upgrade_package('swift-object')
end
it 'starts swift object services on boot' do
%w(swift-object swift-object-replicator swift-object-auditor swift-object-updater).each do |svc|
expect(chef_run).to enable_service(svc)
end
end
describe '/var/spool/crontab/root' do
it 'template contents' do
skip 'TODO: check for recon script'
end
end
describe '/etc/swift/object-server.conf' do
let(:file) { chef_run.template('/etc/swift/object-server.conf') }
it_behaves_like 'custom template banner displayer' do
let(:file_name) { file.name }
end
it 'creates object-server.conf' do
expect(chef_run).to create_template(file.name).with(
user: 'swift',
group: 'swift',
mode: 00600
)
end
describe 'default attribute values' do
it_behaves_like 'a common swift server default attribute values checker', 'object', '0.0.0.0', '6000'
end
it_behaves_like 'a common swift server configurator', 'object', '0.0.0.0', '6000'
it_behaves_like 'some common swift server values'
describe 'default replicator values' do
{ 'run_pause' => '30',
'reclaim_age' => '604800' }.each do |k, v|
it "sets the default for #{k}" do
expect(chef_run).to render_config_file(file.name).with_section_content('object-replicator', /^#{Regexp.quote("#{k} = #{v}")}$/)
end
end
end
end
describe '/etc/swift/object-expirer.conf' do
let(:file) { chef_run.template('/etc/swift/object-expirer.conf') }
it_behaves_like 'custom template banner displayer' do
let(:file_name) { file.name }
end
it 'creates object-expirerr.conf' do
expect(chef_run).to create_template(file.name).with(
user: 'swift',
group: 'swift',
mode: 00600
)
end
it 'sets the memcache_servers attribute' do
expect(chef_run).to render_file(file.name).with_content(/^memcache_servers = host1:111,host2:222$/)
end
end
end
end

351
spec/proxy_spec.rb
View File

@@ -1,351 +0,0 @@
# encoding: UTF-8
require_relative 'spec_helper'
describe 'openstack-object-storage::proxy-server' do
describe 'ubuntu' do
let(:runner) { ChefSpec::SoloRunner.new(UBUNTU_OPTS) }
let(:node) { runner.node }
let(:chef_run) do
node.automatic['cpu']['total'] = 6
runner.converge(described_recipe)
end
include_context 'swift-stubs'
include_examples 'keystone-authmode'
it 'upgrades memcache python packages' do
expect(chef_run).to upgrade_package('python-memcache')
end
it 'upgrades swift packages' do
expect(chef_run).to upgrade_package('swift-proxy')
end
it 'does not upgrade swauth package' do
expect(chef_run).not_to upgrade_package('swauth')
end
it 'starts swift-proxy on boot' do
expect(chef_run).to enable_service('swift-proxy')
end
describe '/etc/swift/proxy-server.conf' do
let(:file) { chef_run.template('/etc/swift/proxy-server.conf') }
it_behaves_like 'custom template banner displayer' do
let(:file_name) { file.name }
end
it 'creates proxy-server.conf' do
expect(chef_run).to create_template(file.name).with(
user: 'swift',
group: 'swift',
mode: 00600
)
end
describe 'default attribute values' do
it_behaves_like 'a common swift server default attribute values checker', 'proxy', nil, nil
it 'has default contents' do
array = [
/^pipeline = catch_errors healthcheck cache ratelimit authtoken keystoneauth proxy-logging proxy-server$/,
/^workers = auto$/
]
array.each do |content|
expect(chef_run).to render_file(file.name).with_content(content)
end
end
it 'uses default attribute value for authmode' do
expect(chef_run.node['openstack']['object-storage']['authmode']).to eq('keystone')
end
%w(tempurl formpost domain_remap staticweb).each do |attr|
it "uses default attribute value for #{attr} enabled" do
expect(chef_run.node['openstack']['object-storage'][attr]['enabled']).to eq(false)
end
end
%w(swift_url swauth_url).each do |attr|
it "uses default attribute value for #{attr}" do
expect(chef_run.node['openstack']['object-storage'][attr]).to eq('http://127.0.0.1:8080/v1/')
end
end
it 'uses default attribute value for container-server allowed_sync_hosts' do
expect(chef_run.node['openstack']['object-storage']['container-server']['allowed_sync_hosts']).to eq([])
end
it 'uses default attribute value for domain_remap log_name' do
expect(chef_run.node['openstack']['object-storage']['domain_remap']['log_name']).to eq('domain_remap')
end
it 'uses default attribute value for domain_remap log_facility' do
expect(chef_run.node['openstack']['object-storage']['domain_remap']['log_facility']).to eq('LOG_LOCAL0')
end
it 'uses default attribute value for domain_remap log_level' do
expect(chef_run.node['openstack']['object-storage']['domain_remap']['log_level']).to eq('INFO')
end
it 'uses default attribute value for domain_remap log_headers' do
expect(chef_run.node['openstack']['object-storage']['domain_remap']['log_headers']).to eq('False')
end
it 'uses default attribute value for domain_remap storage_admin' do
expect(chef_run.node['openstack']['object-storage']['domain_remap']['storage_domain']).to eq('example.com')
end
it 'uses default attribute value for domain_remap path_root' do
expect(chef_run.node['openstack']['object-storage']['domain_remap']['path_root']).to eq('v1')
end
it 'uses default attribute value for domain_remap reseller_prefixes' do
expect(chef_run.node['openstack']['object-storage']['domain_remap']['reseller_prefixes']).to eq('AUTH')
end
it 'uses default attribute value for staticweb cache_timeout' do
expect(chef_run.node['openstack']['object-storage']['staticweb']['cache_timeout']).to eq(300)
end
it 'uses default attribute value for staticweb log_name' do
expect(chef_run.node['openstack']['object-storage']['staticweb']['log_name']).to be_nil
end
it 'uses default attribute value for staticweb log_facility' do
expect(chef_run.node['openstack']['object-storage']['staticweb']['log_facility']).to eq('LOG_LOCAL0')
end
it 'uses default attribute value for staticweb log_level' do
expect(chef_run.node['openstack']['object-storage']['staticweb']['log_level']).to eq('INFO')
end
it 'uses default attribute value for staticweb access_log_name' do
expect(chef_run.node['openstack']['object-storage']['staticweb']['access_log_name']).to eq('staticweb')
end
it 'uses default attribute value for staticweb access_log_facility' do
expect(chef_run.node['openstack']['object-storage']['staticweb']['access_log_facility']).to eq('LOG_LOCAL0')
end
it 'uses default attribute value for staticweb access_log_level' do
expect(chef_run.node['openstack']['object-storage']['staticweb']['access_log_level']).to eq('INFO')
end
it 'uses default attribute value for staticweb log_headers' do
expect(chef_run.node['openstack']['object-storage']['staticweb']['log_headers']).to eq('False')
end
it 'uses default attribute value for tempurl incoming_remove_headers' do
expect(chef_run.node['openstack']['object-storage']['tempurl']['incoming_remove_headers']).to eq('x-timestamp')
end
it 'uses default attribute value for tempurl incoming_allow_headers' do
expect(chef_run.node['openstack']['object-storage']['tempurl']['incoming_allow_headers']).to eq('')
end
it 'uses default attribute value for tempurl outgoing_remove_headers' do
expect(chef_run.node['openstack']['object-storage']['tempurl']['outgoing_remove_headers']).to eq('x-object-meta-*')
end
it 'uses default attribute value for tempurl outgoing_allow_headers' do
expect(chef_run.node['openstack']['object-storage']['tempurl']['outgoing_allow_headers']).to eq('x-object-meta-public-*')
end
end
describe 'template contents' do
it_behaves_like 'a common swift server configurator', 'proxy', '127.0.0.1', '8080'
context 'workers' do
it 'sets the number of workers' do
expect(chef_run).to render_file(file.name).with_content(/^workers = auto$/)
end
end
context 'pipeline' do
%w(domain_remap formpost staticweb).each do |pipeline_item|
it "includes #{pipeline_item} in the pipeline when present" do
node.set['openstack']['object-storage'][pipeline_item]['enabled'] = true
expect(chef_run).to render_file(file.name).with_content(/^pipeline = .*#{pipeline_item}.*$/)
end
it "does not include #{pipeline_item} in the pipeline when not present" do
node.set['openstack']['object-storage'][pipeline_item]['enabled'] = false
expect(chef_run).not_to render_file(file.name).with_content(/^pipeline = .*#{pipeline_item}.*$/)
end
end
it 'includes the tempurl element when it is enabled and authmode is swauth' do
node.set['openstack']['object-storage']['authmode'] = 'swauth'
node.set['openstack']['object-storage']['tempurl']['enabled'] = true
expect(chef_run).to render_file(file.name).with_content(/^pipeline = .*tempurl.*$/)
end
it 'does not include the tempurl element when it is disabled' do
node.set['openstack']['object-storage']['tempurl']['enabled'] = false
expect(chef_run).not_to render_file(file.name).with_content(/^pipeline = .*tempurl.*$/)
end
it 'does not includes the tempurl element when authmode is not swauth' do
node.set['openstack']['object-storage']['authmode'] = 'not_swauth'
expect(chef_run).not_to render_file(file.name).with_content(/^pipeline = .*tempurl.*$/)
end
it 'includes keystone related items when authmode is keystone' do
expect(chef_run).to render_file(file.name).with_content(/^pipeline = .*authtoken keystoneauth.*$/)
end
it 'does not include keystone related items when authmode is not keystone' do
node.set['openstack']['object-storage']['authmode'] = 'not_keystone'
expect(chef_run).not_to render_file(file.name).with_content(/^pipeline = .*authtoken keystoneauth.*$/)
end
it 'includes swauth item when authmode is swauth' do
node.set['openstack']['object-storage']['authmode'] = 'swauth'
expect(chef_run).to render_file(file.name).with_content(/^pipeline = .*swauth.*$/)
end
it 'does not include swauth item when authmode is not swauth' do
node.set['openstack']['object-storage']['authmode'] = 'not_swauth'
expect(chef_run).not_to render_file(file.name).with_content(/^pipeline = .*swauth.*$/)
end
end
it 'sets account_autocreate when authmode is keystone' do
expect(chef_run).to render_file(file.name).with_content(/^account_autocreate = true$/)
end
it 'does not set account_autocreate when authmode is not keystone' do
node.set['openstack']['object-storage']['authmode'] = 'not_keystone'
expect(chef_run).not_to render_file(file.name).with_content(/^account_autocreate = true$/)
end
context 'swauth enabled' do
before do
node.set['openstack']['object-storage']['authmode'] = 'swauth'
end
it 'has auth key override' do
node.set['openstack']['object-storage']['authkey'] = '1234'
expect(chef_run).to render_file(file.name).with_content(/^super_admin_key = 1234$/)
end
it 'sets allow_account_management attribute when authmode is swauth' do
expect(chef_run).to render_file(file.name).with_content(/^allow_account_management = true$/)
end
it 'sets the default_swift_cluster attribute' do
node.set['openstack']['object-storage']['swift_url'] = 'swift_url_value'
node.set['openstack']['object-storage']['swauth_url'] = 'swauth_url_value'
expect(chef_run).to render_file(file.name).with_content(/^default_swift_cluster = local#swift_url_value#swauth_url_value$/)
end
it 'sets allow_overrides when tempurl is enabled' do
node.set['openstack']['object-storage']['tempurl']['enabled'] = true
expect(chef_run).to render_file(file.name).with_content(/^allow_overrides = true$/)
end
it 'does not set allow_overrides when tempurl is disabled' do
node.set['openstack']['object-storage']['tempurl']['enabled'] = false
expect(chef_run).not_to render_file(file.name).with_content(/^allow_overrides = true$/)
end
it 'sets allowed_sync_hosts when present' do
node.set['openstack']['object-storage']['container-server']['allowed_sync_hosts'] = %w(host1 host2)
expect(chef_run).to render_file(file.name).with_content(/^allowed_sync_hosts = host1,host2$/)
end
it 'does not set allowed_sync_hosts when not present' do
node.set['openstack']['object-storage']['container-server']['allowed_sync_hosts'] = nil
expect(chef_run).not_to render_file(file.name).with_content(/^allowed_sync_hosts = $/)
end
end
context 'swauth disabled' do
before do
node.set['openstack']['object-storage']['authmode'] = 'not_swauth'
end
it 'sets allow_account_management attribute when authmode is not swauth' do
expect(chef_run).to render_file(file.name).with_content(/^allow_account_management = false$/)
end
it 'does not set the default_swift_cluster attribute' do
expect(chef_run).not_to render_file(file.name).with_content(/^default_swift_cluster = local.*$/)
end
it 'does not set allow_overrides when tempurl is enabled' do
node.set['openstack']['object-storage']['tempurl']['enabled'] = true
expect(chef_run).not_to render_file(file.name).with_content(/^allow_overrides = true$/)
end
it 'does not set allowed_sync_hosts when present' do
node.set['openstack']['object-storage']['container-server']['allowed_sync_hosts'] = %w(host1 host2)
expect(chef_run).not_to render_file(file.name).with_content(/^allowed_sync_hosts = host1,host2$/)
end
end
context 'authtoken enabled' do
{ 'paste.filter_factory' => 'keystoneclient.middleware.auth_token:filter_factory',
'auth_uri' => 'http://127.0.0.1:5000/v2.0',
'identity_uri' => 'http://127.0.0.1:35357/',
'auth_version' => 'v2.0',
'admin_tenant_name' => 'service',
'admin_user' => 'swift',
'admin_password' => 'swift-pass',
'signing_dir' => '/var/cache/swift/api'
}.each do |k, v|
it "sets the default for #{k}" do
expect(chef_run).to render_config_file(file.name).with_section_content('filter:authtoken', /^#{Regexp.quote("#{k} = #{v}")}$/)
end
end
end
it 'sets the memcache_servers attribute' do
expect(chef_run).to render_file(file.name).with_content(/^memcache_servers = host1:111,host2:222$/)
end
context 'domain_remap' do
%w(log_name log_facility log_level log_headers).each do |attr|
it "sets the #{attr} attribute" do
node.set['openstack']['object-storage']['domain_remap'][attr] = "#{attr}_value"
expect(chef_run).to render_file(file.name).with_content(/^set #{attr} = #{attr}_value$/)
end
end
%w(storage_domain path_root reseller_prefixes).each do |attr|
it "sets the #{attr} attribute" do
node.set['openstack']['object-storage']['domain_remap'][attr] = "#{attr}_value"
expect(chef_run).to render_file(file.name).with_content(/^#{attr} = #{attr}_value$/)
end
end
end
context 'staticweb' do
it 'sets the cache_timeout attribute' do
node.set['openstack']['object-storage']['staticweb']['cache_timeout'] = 'cache_timeout_value'
expect(chef_run).to render_file(file.name).with_content(/^cache_timeout = cache_timeout_value$/)
end
%w(log_name log_facility log_level access_log_name access_log_facility access_log_level log_headers).each do |attr|
it "sets the #{attr} attribute" do
node.set['openstack']['object-storage']['staticweb'][attr] = "#{attr}_value"
expect(chef_run).to render_file(file.name).with_content(/^set #{attr} = #{attr}_value$/)
end
end
end
context 'tempurl' do
%w(incoming_remove_headers incoming_allow_headers outgoing_remove_headers outgoing_allow_headers).each do |attr|
it "sets the #{attr} attribute" do
node.set['openstack']['object-storage']['tempurl'][attr] = "#{attr}_value"
expect(chef_run).to render_file(file.name).with_content(/^#{attr} = #{attr}_value$/)
end
end
end
end
end
end
end

31
spec/ring-repo_spec.rb
View File

@@ -1,31 +0,0 @@
# encoding: UTF-8
require_relative 'spec_helper'
describe 'openstack-object-storage::ring-repo' do
describe 'ubuntu' do
let(:runner) { ChefSpec::SoloRunner.new(UBUNTU_OPTS) }
let(:node) { runner.node }
let(:chef_run) do
runner.converge(described_recipe)
end
include_context 'swift-stubs'
it 'upgrades git package for ring management' do
expect(chef_run).to upgrade_package('git-daemon-sysvinit')
end
it 'should not start xinetd services on boot' do
expect(chef_run).not_to enable_service('xinetd')
end
# FIXME(galstrom21): This spec file should just check that the LWRP
# is called with the appropriate paramaters. It should not be checking
# the file contents.
describe '/etc/swift/ring-workspace/generate-rings.sh' do
it 'gets installed' do
skip 'TODO: determine some way to ensure this LWRP script gets created'
end
end
end
end

55
spec/ring-script_spec.rb
View File

@@ -1,55 +0,0 @@
# encoding: UTF-8
require_relative 'spec_helper'
describe 'openstack-object-storage::ring-repo' do
describe 'ubuntu' do
let(:runner) { ChefSpec::SoloRunner.new(UBUNTU_OPTS) }
let(:node) { runner.node }
let(:chef_run) { runner.converge(described_recipe) }
let(:events) { Chef::EventDispatch::Dispatcher.new }
let(:cookbook_collection) { Chef::CookbookCollection.new([]) }
let(:run_context) { Chef::RunContext.new(node, cookbook_collection, events) }
describe 'ring_script' do
let(:resource) do
Chef::Resource::OpenstackObjectStorageRingScript.new('script1', run_context)
end
let(:provider) do
Chef::Provider::OpenstackObjectStorageRingScript.new(resource, run_context)
end
it 'parse_ring_output parses ring data for normal ring data' do
ring_data = ['/etc/swift/ring-workspace/rings/account.builder, build version 0',
'262144 partitions, 3.000000 replicas, 0 regions, 0 zones, 0 devices, 0.00 balance, 0.00 dispersion',
'The minimum number of hours before a partition can be reassigned is 1',
'The overload factor is 0.00% (0.000000)']
expect(
provider.send(:parse_ring_output, ring_data)
).to eq(
state:
{ build_version: '0',
partitions: '262144',
replicas: '3.000000',
regions: '0',
zones: '0',
devices: '0',
balance: '0.00',
min_part_hours: '1'
}
)
end
it 'parse_ring_output parsing fails for bad data' do
ring_data = ['ugly data']
exception = false
begin
provider.send(:parse_ring_output, ring_data)
rescue
exception = true
end
expect(exception).to be true
end
end
end
end

40
spec/rsync_spec.rb
View File

@@ -1,40 +0,0 @@
# encoding: UTF-8
require_relative 'spec_helper'
describe 'openstack-object-storage::rsync' do
describe 'ubuntu' do
let(:runner) { ChefSpec::SoloRunner.new(UBUNTU_OPTS) }
let(:node) { runner.node }
let(:chef_run) do
runner.converge(described_recipe)
end
include_context 'swift-stubs'
it 'upgrades git package for ring management' do
expect(chef_run).to upgrade_package('rsync')
end
it 'starts rsync service on boot' do
expect(chef_run).to enable_service('rsync')
end
describe '/etc/rsyncd.conf' do
let(:file) { chef_run.template('/etc/rsyncd.conf') }
it_behaves_like 'custom template banner displayer' do
let(:file_name) { file.name }
end
it 'creates /etc/rsyncd.conf' do
expect(chef_run).to create_template(file.name).with(
mode: 00644
)
end
it 'template contents' do
skip 'TODO: implement'
end
end
end
end

26
spec/setup_spec.rb
View File

@@ -1,26 +0,0 @@
# encoding: UTF-8
require_relative 'spec_helper'
describe 'openstack-object-storage::setup' do
describe 'ubuntu' do
let(:runner) { ChefSpec::SoloRunner.new(UBUNTU_OPTS) }
let(:node) { runner.node }
let(:chef_run) do
runner.converge(described_recipe)
end
include_context 'swift-stubs'
include_examples 'keystone-authmode'
it 'does not include the identity registration recipe' do
node.set['openstack']['object-storage']['authmode'] = 'swauth'
expect(chef_run).not_to include_recipe('openstack-object-storage::identity_registration')
end
it 'includes the identity registration recipe' do
expect(chef_run).to include_recipe('openstack-object-storage::identity_registration')
end
# TODO: flush out rest of this spec
end
end

192
spec/spec_helper.rb
View File

@@ -1,192 +0,0 @@
# encoding: UTF-8
require 'chefspec'
require 'chefspec/berkshelf'
require 'chef/application'
ChefSpec::Coverage.start! { add_filter 'openstack-compute' }
LOG_LEVEL = :fatal
REDHAT_OPTS = {
platform: 'redhat',
version: '7.1',
log_level: LOG_LEVEL
}
UBUNTU_OPTS = {
platform: 'ubuntu',
version: '14.04',
log_level: LOG_LEVEL
}
SUSE_OPTS = {
platform: 'suse',
version: '11.3',
log_level: LOG_LEVEL
}
MOCK_NODE_NETWORK_DATA =
{
'ipaddress' => '10.0.0.2',
'fqdn' => 'localhost.localdomain',
'hostname' => 'localhost',
'network' => {
'default_interface' => 'eth0',
'interfaces' => {
'eth0' => {
'addresses' => {
'fe80::a00:27ff:feca:ab08' => { 'scope' => 'Link', 'prefixlen' => '64', 'family' => 'inet6' },
'10.0.0.2' => { 'netmask' => '255.255.255.0', 'broadcast' => '10.0.0.255', 'family' => 'inet' },
'08:00:27:CA:AB:08' => { 'family' => 'lladdr' }
}
},
'lo' => {
'addresses' => {
'::1' => { 'scope' => 'Node', 'prefixlen' => '128', 'family' => 'inet6' },
'127.0.0.1' => { 'netmask' => '255.0.0.0', 'family' => 'inet' }
}
}
}
}
}
shared_context 'swift-stubs' do
before do
stub_command('dpkg -s memcached').and_return(true)
stub_command('getent passwd memcache').and_return(true)
stub_command('/usr/bin/id swift').and_return(true)
stub_command('[ -x /etc/swift/pull-rings.sh ]').and_return(true)
stub_command('[ -f /etc/rsyncd.conf ]').and_return(true)
stub_command("grep -q 'RSYNC_ENABLE=false' /etc/default/rsync").and_return(true)
stub_command('[ -e /etc/swift/account-server.conf ] && [ -e /etc/swift/account.ring.gz ]').and_return(true)
stub_command('[ -e /etc/swift/container-server.conf ] && [ -e /etc/swift/container.ring.gz ]').and_return(true)
stub_command('[ -e /etc/init/swift-container-sync.conf ]').and_return(false)
stub_command('[ -e /etc/init.d/swift-container-sync ]').and_return(false)
stub_command('[ -e /etc/swift/object-server.conf ] && [ -e /etc/swift/object.ring.gz ]').and_return(true)
stub_command('[ -e /etc/swift/proxy-server.conf ] && [ -e /etc/swift/object.ring.gz ]').and_return(true)
# create mock cluster
n = Chef::Node.new
n.name('manager')
n.default_attrs = {
'swift' => {
'service_pass' => 'foobar'
}
}
allow_any_instance_of(Chef::Recipe).to receive(:search).with(:node, 'chef_environment:_default AND roles:os-object-storage-setup').and_return([n])
allow(Chef::Application).to receive(:fatal!)
allow_any_instance_of(Chef::Recipe).to receive(:get_password)
.with('token', 'openstack_identity_bootstrap_token')
.and_return('bootstrap-token')
allow_any_instance_of(Chef::Recipe).to receive(:get_password)
.with('service', 'openstack-object-storage')
.and_return('swift-pass')
allow_any_instance_of(Chef::Recipe).to receive(:get_password)
.with('token', 'swift_hash_path_prefix')
.and_return('swift_hash_path_prefix-secret')
allow_any_instance_of(Chef::Recipe).to receive(:get_password)
.with('token', 'swift_hash_path_suffix')
.and_return('swift_hash_path_suffix-secret')
allow_any_instance_of(Chef::Recipe).to receive(:get_password)
.with('token', 'swift_authkey')
.and_return('swift_authkey-secret')
allow_any_instance_of(Chef::Recipe).to receive(:get_password)
.with('token', 'dispersion_auth_user')
.and_return('dispersion_auth_user-secret')
allow_any_instance_of(Chef::Recipe).to receive(:get_password)
.with('token', 'dispersion_auth_key')
.and_return('dispersion_auth_key-secret')
allow_any_instance_of(Chef::Recipe).to receive(:memcached_servers)
.and_return(['host1:111', 'host2:222'])
end
end
shared_examples 'keystone-authmode' do
describe 'authorization mode' do
it 'does upgrade keystoneclient package' do
expect(chef_run).to upgrade_package('python-keystoneclient')
end
end
end
shared_examples 'a common swift server configurator' do |server_type, bind_ip, bind_port|
{ 'bind_ip' => "#{bind_ip}",
'bind_port' => "#{bind_port}",
'log_statsd_default_sample_rate' => '1',
'log_statsd_metric_prefix' => 'openstack.swift.Fauxhai',
'max_clients' => '1024',
'workers' => 'auto' }.each do |k, v|
it "sets the default for #{k}" do
expect(chef_run).to render_config_file(file.name).with_section_content('DEFAULT', /^#{Regexp.quote("#{k} = #{v}")}$/)
end
end
%w(ip port).each do |attr|
it "sets the bind_#{attr} attr" do
node.set['openstack']['object-storage']['network']["#{server_type}-bind-#{attr}"] = "#{attr}_value"
expect(chef_run).to render_config_file(file.name).with_section_content('DEFAULT', /^bind_#{attr} = #{attr}_value$/)
end
end
context 'statistics enabled' do
before do
node.set['openstack']['object-storage']['statistics']['enabled'] = true
end
it 'sets the log_statsd_default_sample_rate attribute' do
node.set['openstack']['object-storage']['statistics']['sample_rate'] = 'sample_rate_value'
expect(chef_run).to render_config_file(file.name).with_section_content('DEFAULT', /^log_statsd_default_sample_rate = sample_rate_value$/)
end
it 'sets the log_statsd_metric_prefix attribute' do
node.set['openstack']['object-storage']['statistics']['statsd_prefix'] = 'statsd_prefix_value'
chef_run.node.automatic['hostname'] = 'myhostname'
expect(chef_run).to render_config_file(file.name).with_section_content('DEFAULT', /^log_statsd_metric_prefix = statsd_prefix_value\.myhostname$/)
end
end
it 'does not show statistic related attributed when disabled' do
node.set['openstack']['object-storage']['statistics']['enabled'] = false
expect(chef_run).not_to render_config_file(file.name).with_section_content('DEFAULT', /^log_statsd_host = localhost$/)
end
end
shared_examples 'a common swift server default attribute values checker' do |server_type, bind_ip, bind_port|
it 'bind_ip' do
expect(chef_run.node['openstack']['object-storage']['network']["#{server_type}-bind-ip"]).to eq(bind_ip)
end
it 'bind_port' do
expect(chef_run.node['openstack']['object-storage']['network']["#{server_type}-bind-port"]).to eq(bind_port)
end
it 'log_statsd_default_sample_rate' do
expect(chef_run.node['openstack']['object-storage']['statistics']['sample_rate']).to eq(1)
end
it 'statsd_prefix' do
expect(chef_run.node['openstack']['object-storage']['statistics']['statsd_prefix']).to eq('openstack.swift')
end
it 'hostname' do
expect(chef_run.node['hostname']).to eq('Fauxhai')
end
it 'workers' do
expect(chef_run.node['openstack']['object-storage']["#{server_type}-server"]['workers']).to eq('auto')
end
end
shared_examples 'custom template banner displayer' do
it 'shows the custom banner' do
node.set['openstack']['object-storage']['custom_template_banner'] = 'custom_template_banner_value'
expect(chef_run).to render_file(file_name).with_content(/^custom_template_banner_value$/)
end
end
shared_examples 'some common swift server values' do
{ 'devices' => '/srv/node',
'mount_check' => 'true' }.each do |k, v|
it "sets the default for #{k}" do
expect(chef_run).to render_config_file(file.name).with_section_content('DEFAULT', /^#{Regexp.quote("#{k} = #{v}")}$/)
end
end
end

46
spec/storage-common_spec.rb
View File

@@ -1,46 +0,0 @@
# encoding: UTF-8
require_relative 'spec_helper'
describe 'openstack-object-storage::storage-common' do
describe 'ubuntu' do
let(:runner) { ChefSpec::SoloRunner.new(UBUNTU_OPTS) }
let(:node) { runner.node }
let(:chef_run) do
runner.converge(described_recipe)
end
include_context 'swift-stubs'
describe '/var/cache/swift' do
let(:dir) { chef_run.directory('/var/cache/swift') }
it 'creates /var/cache/swift' do
expect(chef_run).to create_directory(dir.name).with(
user: 'swift',
group: 'swift',
mode: 00700
)
end
end
describe '/etc/swift/drive-audit.conf' do
let(:file) { chef_run.template('/etc/swift/drive-audit.conf') }
it_behaves_like 'custom template banner displayer' do
let(:file_name) { file.name }
end
it 'creates drive-audit.conf' do
expect(chef_run).to create_template(file.name).with(
user: 'swift',
group: 'swift',
mode: 00600
)
end
it 'template contents' do
skip 'TODO: implement'
end
end
end
end

27
templates/default/account-server.conf.erb
View File

@@ -1,27 +0,0 @@
<%= node['openstack']['object-storage']['custom_template_banner'] %>
[DEFAULT]
bind_ip = <%= @bind_ip %>
bind_port = <%= @bind_port %>
workers = <%= node['openstack']['object-storage']['account-server']['workers'] %>
max_clients = <%= node['openstack']['object-storage']['account-server']['max_clients'] %>
devices = <%= node['openstack']['object-storage']['account-server']['devices'] %>
mount_check = <%= node['openstack']['object-storage']['account-server']['mount_check'] %>
<% if node['openstack']['object-storage']['statistics']['enabled'] -%>
log_statsd_host = localhost
log_statsd_port = 8125
log_statsd_default_sample_rate = <%= node['openstack']['object-storage']['statistics']['sample_rate'] %>
log_statsd_metric_prefix = <%= node['openstack']['object-storage']['statistics']['statsd_prefix'] %>.<%= node['hostname'] %>
<% end %>
[pipeline:main]
pipeline = account-server
[app:account-server]
use = egg:swift#account
[account-replicator]
[account-auditor]
[account-reaper]

54
templates/default/container-reconciler.conf.erb
View File

@@ -1,54 +0,0 @@
<%= node['openstack']['object-storage']['custom_template_banner'] %>
[DEFAULT]
# swift_dir = /etc/swift
# user = swift
# You can specify default log routing here if you want:
# log_name = swift
# log_facility = LOG_LOCAL0
# log_level = INFO
# log_address = /dev/log
#
# comma separated list of functions to call to setup custom log handlers.
# functions get passed: conf, name, log_to_console, log_route, fmt, logger,
# adapted_logger
# log_custom_handlers =
#
# If set, log_udp_host will override log_address
# log_udp_host =
# log_udp_port = 514
#
# You can enable StatsD logging here:
# log_statsd_host = localhost
# log_statsd_port = 8125
# log_statsd_default_sample_rate = 1.0
# log_statsd_sample_rate_factor = 1.0
# log_statsd_metric_prefix =
[container-reconciler]
# The reconciler will re-attempt reconciliation if the source object is not
# available up to reclaim_age seconds before it gives up and deletes the entry
# in the queue.
# reclaim_age = 604800
# The cycle time of the daemon
# interval = 30
# Server errors from requests will be retried by default
# request_tries = 3
[pipeline:main]
pipeline = catch_errors proxy-logging cache proxy-server
[app:proxy-server]
use = egg:swift#proxy
# See proxy-server.conf-sample for options
[filter:cache]
use = egg:swift#memcache
memcache_servers = <%= @memcache_servers %>
[filter:proxy-logging]
use = egg:swift#proxy_logging
[filter:catch_errors]
use = egg:swift#catch_errors
# See proxy-server.conf-sample for options

45
templates/default/container-server.conf.erb
View File

@@ -1,45 +0,0 @@
<%= node['openstack']['object-storage']['custom_template_banner'] %>
[DEFAULT]
bind_ip = <%= @bind_ip %>
bind_port = <%= @bind_port %>
workers = <%= node['openstack']['object-storage']['container-server']['workers'] %>
max_clients = <%= node['openstack']['object-storage']['container-server']['max_clients'] %>
devices = <%= node['openstack']['object-storage']['container-server']['devices'] %>
mount_check = <%= node['openstack']['object-storage']['container-server']['mount_check'] %>
<% if node['openstack']['object-storage']['statistics']['enabled'] -%>
log_statsd_host = localhost
log_statsd_port = 8125
log_statsd_default_sample_rate = <%= node['openstack']['object-storage']['statistics']['sample_rate'] %>
log_statsd_metric_prefix = <%= node['openstack']['object-storage']['statistics']['statsd_prefix'] %>.<%= node['hostname'] %>
<% end -%>
<% if node['openstack']['object-storage']["container-server"]["allowed_sync_hosts"] -%>
allowed_sync_hosts = <%= node['openstack']['object-storage']["container-server"]["allowed_sync_hosts"].join(",") %>
<% end -%>
[pipeline:main]
pipeline = container-server
[app:container-server]
use = egg:swift#container
[container-replicator]
[container-updater]
[container-auditor]
[container-sync]
# You can override the default log routing for this app here (don't use set!):
log_name = <%= node['openstack']['object-storage']["container-server"]["container-sync"]["log_name"] %>
log_facility = <%= node['openstack']['object-storage']["container-server"]["container-sync"]["log_facility"] %>
log_level = <%= node['openstack']['object-storage']["container-server"]["container-sync"]["log_level"] %>
# If you need to use an HTTP Proxy, set it here; defaults to no proxy.
<% if node['openstack']['object-storage']["container-server"]["container-sync"]["sync_proxy"] -%>
sync_proxy = <%= node['openstack']['object-storage']["container-server"]["container-sync"]["sync_proxy"] %>
<% end -%>
# Will sync, at most, each container once per interval
interval = <%= node['openstack']['object-storage']["container-server"]["container-sync"]["interval"] %>
# Maximum amount of time to spend syncing each container per pass
container_time = <%= node['openstack']['object-storage']["container-server"]["container-sync"]["container_time"] %>

49
templates/default/container-sync-realms.conf.erb
View File

@@ -1,49 +0,0 @@
<%= node['openstack']['object-storage']['custom_template_banner'] %>
# [DEFAULT]
# The number of seconds between checking the modified time of this config file
# for changes and therefore reloading it.
# mtime_check_interval = 300
# [realm1]
# key = realm1key
# key2 = realm1key2
# cluster_name1 = https://host1/v1/
# cluster_name2 = https://host2/v1/
#
# [realm2]
# key = realm2key
# key2 = realm2key2
# cluster_name3 = https://host3/v1/
# cluster_name4 = https://host4/v1/
# Each section name is the name of a sync realm. A sync realm is a set of
# clusters that have agreed to allow container syncing with each other. Realm
# names will be considered case insensitive.
#
# The key is the overall cluster-to-cluster key used in combination with the
# external users' key that they set on their containers' X-Container-Sync-Key
# metadata header values. These keys will be used to sign each request the
# container sync daemon makes and used to validate each incoming container sync
# request.
#
# The key2 is optional and is an additional key incoming requests will be
# checked against. This is so you can rotate keys if you wish; you move the
# existing key to key2 and make a new key value.
#
# Any values in the realm section whose names begin with cluster_ will indicate
# the name and endpoint of a cluster and will be used by external users in
# their containers' X-Container-Sync-To metadata header values with the format
# "realm_name/cluster_name/container_name". Realm and cluster names are
# considered case insensitive.
#
# The endpoint is what the container sync daemon will use when sending out
# requests to that cluster. Keep in mind this endpoint must be reachable by all
# container servers, since that is where the container sync daemon runs. Note
# the the endpoint ends with /v1/ and that the container sync daemon will then
# add the account/container/obj name after that.
#
# Distribute this container-sync-realms.conf file to all your proxy servers
# and container servers.

15
templates/default/dispersion.conf.erb
View File

@@ -1,15 +0,0 @@
<%= node['openstack']['object-storage']['custom_template_banner'] %>
[dispersion]
auth_url = <%= @auth_url %>
auth_user = <%= @auth_user %>
auth_key = <%= @auth_key %>
auth_version = 1.0
endpoint_type = publicURL
swift_dir = /etc/swift
dispersion_coverage = 5
retries = 5
concurrency = 25
container_report = yes
object_report = yes
dump_json = no

7
templates/default/drive-audit.conf.erb
View File

@@ -1,7 +0,0 @@
<%= node['openstack']['object-storage']['custom_template_banner'] %>
[drive-audit]
log_facility = LOG_LOCAL0
log_level = INFO
device_dir = /srv/node
minutes = 60

111
templates/default/object-expirer.conf.erb
View File

@@ -1,111 +0,0 @@
<%= node['openstack']['object-storage']['custom_template_banner'] %>
[DEFAULT]
# swift_dir = /etc/swift
# user = swift
# You can specify default log routing here if you want:
# log_name = swift
# log_facility = LOG_LOCAL0
# log_level = INFO
# log_address = /dev/log
# The following caps the length of log lines to the value given; no limit if
# set to 0, the default.
# log_max_line_length = 0
#
# comma separated list of functions to call to setup custom log handlers.
# functions get passed: conf, name, log_to_console, log_route, fmt, logger,
# adapted_logger
# log_custom_handlers =
#
# If set, log_udp_host will override log_address
# log_udp_host =
# log_udp_port = 514
#
# You can enable StatsD logging here:
# log_statsd_host = localhost
# log_statsd_port = 8125
# log_statsd_default_sample_rate = 1.0
# log_statsd_sample_rate_factor = 1.0
# log_statsd_metric_prefix =
[object-expirer]
# interval = 300
# auto_create_account_prefix = .
# expiring_objects_account_name = expiring_objects
# report_interval = 300
# concurrency is the level of concurrency o use to do the work, this value
# must be set to at least 1
# concurrency = 1
# processes is how many parts to divide the work into, one part per process
# that will be doing the work
# processes set 0 means that a single process will be doing all the work
# processes can also be specified on the command line and will override the
# config value
# processes = 0
# process is which of the parts a particular process will work on
# process can also be specified on the command line and will overide the config
# value
# process is "zero based", if you want to use 3 processes, you should run
# processes with process set to 0, 1, and 2
# process = 0
# The expirer will re-attempt expiring if the source object is not available
# up to reclaim_age seconds before it gives up and deletes the entry in the
# queue.
# reclaim_age = 604800
# recon_cache_path = /var/cache/swift
[pipeline:main]
pipeline = catch_errors proxy-logging cache proxy-server
[app:proxy-server]
use = egg:swift#proxy
# See proxy-server.conf-sample for options
[filter:cache]
use = egg:swift#memcache
memcache_servers = <%= @memcache_servers %>
[filter:catch_errors]
use = egg:swift#catch_errors
# See proxy-server.conf-sample for options
[filter:proxy-logging]
use = egg:swift#proxy_logging
# If not set, logging directives from [DEFAULT] without "access_" will be used
# access_log_name = swift
# access_log_facility = LOG_LOCAL0
# access_log_level = INFO
# access_log_address = /dev/log
#
# If set, access_log_udp_host will override access_log_address
# access_log_udp_host =
# access_log_udp_port = 514
#
# You can use log_statsd_* from [DEFAULT] or override them here:
# access_log_statsd_host = localhost
# access_log_statsd_port = 8125
# access_log_statsd_default_sample_rate = 1.0
# access_log_statsd_sample_rate_factor = 1.0
# access_log_statsd_metric_prefix =
# access_log_headers = false
#
# If access_log_headers is True and access_log_headers_only is set only
# these headers are logged. Multiple headers can be defined as comma separated
# list like this: access_log_headers_only = Host, X-Object-Meta-Mtime
# access_log_headers_only =
#
# By default, the X-Auth-Token is logged. To obscure the value,
# set reveal_sensitive_prefix to the number of characters to log.
# For example, if set to 12, only the first 12 characters of the
# token appear in the log. An unauthorized access of the log file
# won't allow unauthorized usage of the token. However, the first
# 12 or so characters is unique enough that you can trace/debug
# token usage. Set to 0 to suppress the token completely (replaced
# by '...' in the log).
# Note: reveal_sensitive_prefix will not affect the value
# logged with access_log_headers=True.
# reveal_sensitive_prefix = 16
#
# What HTTP methods are allowed for StatsD logging (comma-sep); request methods
# not in this list will have "BAD_METHOD" for the <verb> portion of the metric.
# log_statsd_valid_http_methods = GET,HEAD,POST,PUT,DELETE,COPY,OPTIONS

33
templates/default/object-server.conf.erb
View File

@@ -1,33 +0,0 @@
<%= node['openstack']['object-storage']['custom_template_banner'] %>
[DEFAULT]
bind_ip = <%= @bind_ip %>
bind_port = <%= @bind_port %>
workers = <%= node['openstack']['object-storage']['object-server']['workers'] %>
max_clients = <%= node['openstack']['object-storage']['object-server']['max_clients'] %>
devices = <%= node['openstack']['object-storage']['object-server']['devices'] %>
mount_check = <%= node['openstack']['object-storage']['object-server']['mount_check'] %>
<% if node['openstack']['object-storage']['statistics']['enabled'] -%>
log_statsd_host = localhost
log_statsd_port = 8125
log_statsd_default_sample_rate = <%= node['openstack']['object-storage']['statistics']['sample_rate'] %>
log_statsd_metric_prefix = <%= node['openstack']['object-storage']['statistics']['statsd_prefix'] %>.<%= node['hostname'] %>
<% end %>
[pipeline:main]
pipeline = recon object-server
[app:object-server]
use = egg:swift#object
[filter:recon]
use = egg:swift#recon
[object-replicator]
run_pause = <%= node['openstack']['object-storage']['object-server']['replicator']['run_pause'] %>
reclaim_age = <%= node['openstack']['object-storage']['object-server']['replicator']['reclaim_age'] %>
[object-updater]
[object-auditor]

141
templates/default/proxy-server.conf.erb
View File

@@ -1,141 +0,0 @@
<%= node['openstack']['object-storage']['custom_template_banner'] %>
<%
pipeline = []
# the pipeline ordering is important and certain inclusions
# must either preceed or follow the selected auth module
if (node['openstack']['object-storage']['authmode'] == 'swauth' and node['openstack']['object-storage']['tempurl']['enabled'] == true)
pipeline << 'tempurl'
end
if node['openstack']['object-storage']['domain_remap']['enabled']
pipeline << 'domain_remap'
end
if node['openstack']['object-storage']['formpost']['enabled']
pipeline << 'formpost'
end
case @authmode
when 'keystone'
pipeline << 'authtoken'
pipeline << 'keystoneauth'
when 'swauth'
pipeline << 'swauth'
end
if node['openstack']['object-storage']['staticweb']['enabled']
pipeline << 'staticweb'
end
if pipeline.include?('swauth')
account_management = true
else
account_management = false
end
-%>
[DEFAULT]
bind_ip = <%= @bind_host %>
bind_port = <%= @bind_port %>
workers = <%= node['openstack']['object-storage']['proxy-server']['workers'] %>
max_clients = <%= node['openstack']['object-storage']['proxy-server']['max_clients'] %>
<% if node['openstack']['object-storage']['statistics']['enabled'] -%>
log_statsd_host = localhost
log_statsd_port = 8125
log_statsd_default_sample_rate = <%= node['openstack']['object-storage']['statistics']['sample_rate'] %>
log_statsd_metric_prefix = <%= node['openstack']['object-storage']['statistics']['statsd_prefix'] %>.<%= node['hostname'] %>
<% end %>
[pipeline:main]
pipeline = catch_errors healthcheck cache ratelimit <%= pipeline.join(' ') %> proxy-logging proxy-server
[app:proxy-server]
use = egg:swift#proxy
allow_account_management = <%= account_management %>
node_timeout = <%= node['openstack']['object-storage']['proxy-server']['node_timeout'] %>
<% if pipeline.include?('keystoneauth') -%>
account_autocreate = true
<% end %>
<% if pipeline.include?('swauth') -%>
[filter:swauth]
use = egg:swauth#swauth
super_admin_key = <%= @authkey %>
default_swift_cluster = local#<%= node['openstack']['object-storage']['swift_url'] %>#<%= node['openstack']['object-storage']['swauth_url'] %>
<% if pipeline.include?('tempurl') -%>
allow_overrides = true
<% end %>
<% if node['openstack']['object-storage']['container-server']['allowed_sync_hosts'].any? -%>
allowed_sync_hosts = <%= node['openstack']['object-storage']['container-server']['allowed_sync_hosts'].join(',') %>
<% end %>
<% end %>
[filter:healthcheck]
use = egg:swift#healthcheck
[filter:cache]
use = egg:swift#memcache
memcache_servers = <%= @memcache_servers %>
[filter:ratelimit]
use = egg:swift#ratelimit
[filter:domain_remap]
use = egg:swift#domain_remap
set log_name = <%= node['openstack']['object-storage']['domain_remap']['log_name'] %>
set log_facility = <%= node['openstack']['object-storage']['domain_remap']['log_facility'] %>
set log_level = <%= node['openstack']['object-storage']['domain_remap']['log_level'] %>
set log_headers = <%= node['openstack']['object-storage']['domain_remap']['log_headers'] %>
storage_domain = <%= node['openstack']['object-storage']['domain_remap']['storage_domain'] %>
path_root = <%= node['openstack']['object-storage']['domain_remap']['path_root'] %>
reseller_prefixes = <%= node['openstack']['object-storage']['domain_remap']['reseller_prefixes'] %>
[filter:catch_errors]
use = egg:swift#catch_errors
[filter:cname_lookup]
use = egg:swift#cname_lookup
[filter:staticweb]
use = egg:swift#staticweb
cache_timeout = <%= node['openstack']['object-storage']['staticweb']['cache_timeout'] %>
set log_name = <%= node['openstack']['object-storage']['staticweb']['log_name'] %>
set log_facility = <%= node['openstack']['object-storage']['staticweb']['log_facility'] %>
set log_level = <%= node['openstack']['object-storage']['staticweb']['log_level'] %>
set access_log_name = <%= node['openstack']['object-storage']['staticweb']['access_log_name'] %>
set access_log_facility = <%= node['openstack']['object-storage']['staticweb']['access_log_facility'] %>
set access_log_level = <%= node['openstack']['object-storage']['staticweb']['access_log_level'] %>
set log_headers = <%= node['openstack']['object-storage']['staticweb']['log_headers'] %>
[filter:tempurl]
use = egg:swift#tempurl
incoming_remove_headers = <%= node['openstack']['object-storage']['tempurl']['incoming_remove_headers'] %>
incoming_allow_headers = <%= node['openstack']['object-storage']['tempurl']['incoming_allow_headers'] %>
outgoing_remove_headers = <%= node['openstack']['object-storage']['tempurl']['outgoing_remove_headers'] %>
outgoing_allow_headers = <%= node['openstack']['object-storage']['tempurl']['outgoing_allow_headers'] %>
[filter:formpost]
use = egg:swift#formpost
<% if pipeline.include?('authtoken') -%>
[filter:authtoken]
paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory
auth_uri = <%= @auth_uri %>
identity_uri = <%= @identity_uri %>
auth_version = <%= node['openstack']['object-storage']['api']['auth']['version'] %>
admin_tenant_name = <%= node['openstack']['object-storage']['service_tenant_name'] %>
admin_user = <%= node['openstack']['object-storage']['service_user'] %>
admin_password = <%= @service_pass %>
signing_dir = <%= node['openstack']['object-storage']['api']['auth']['cache_dir'] %>
[filter:keystoneauth]
use = egg:swift#keystoneauth
operator_roles = admin, swiftoperator
<% end %>
[filter:proxy-logging]
use = egg:swift#proxy_logging

30
templates/default/pull-rings.sh.erb
View File

@@ -1,30 +0,0 @@
#!/bin/bash
<%= node['openstack']['object-storage']['custom_template_banner'] %>
# this has to be run as root to restart the services...
if [ ! -d /etc/swift/rings ] || [ ! -e /etc/swift/rings/.git/config ]; then
rm -rf /etc/swift/rings
git clone git://<%= @builder_ip %>/rings /etc/swift/rings
fi
cd /etc/swift/rings
git reset --hard
git clean -df
git pull
[ -e /etc/swift/rings ] && chown -R swift: /etc/swift/rings
for d in object account container; do
if [ -e /etc/swift/rings/${d}.ring.gz ]; then
if [ ! -e ../${d}.ring.gz ] || [ "$(md5sum ${d}.ring.gz | cut -f1 -d' ')" != "$(md5sum ../${d}.ring.gz | cut -f1 -d' ')" ]; then
cp ${d}.ring.gz ../${d}.ring.new
chown swift: ../${d}.ring.new
mv ../${d}.ring.new ../${d}.ring.gz
if [ -e /etc/swift/${d}-server.conf ]; then
service <%= @service_prefix %>swift-${d}-replicator restart
fi
fi
fi
done

26
templates/default/rsyncd.conf.erb
View File

@@ -1,26 +0,0 @@
<%= node['openstack']['object-storage']['custom_template_banner'] %>
uid = swift
gid = swift
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
address = 0.0.0.0
[account]
max connections = 10
path = /srv/node/
read only = false
lock file = /var/lock/account.lock
[container]
max connections = 10
path = /srv/node/
read only = false
lock file = /var/lock/container.lock
[object]
max connections = 10
path = /srv/node/
read only = false
lock file = /var/lock/object.lock

157
templates/default/swift-statsd-publish.py.erb
View File

@@ -1,157 +0,0 @@
#!/usr/bin/env python
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# coding=utf-8
#
# Author: Alan Meadows <alan.meadows@gmail.com>
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
<%= node['openstack']['object-storage']['custom_template_banner'] %>
"""
Openstack swift collector for recon and dispersion reports. Will send
back dispersion reporting metrics as well as swift recon statistics
to a statsd server for graphite consumption
"""
from subprocess import Popen, PIPE, check_call
from socket import socket, AF_INET, SOCK_DGRAM
import re
import os
try:
import json
json # workaround for pyflakes issue #13
except ImportError:
import simplejson as json
class OpenStackSwiftStatisticsCollector(object):
def __init__(self):
'''Setup some initial values defined by chef'''
self.statsd_host = '<%= node['openstack']['object-storage']['statistics']['statsd_host'] %>'
self.statsd_port = <%= node['openstack']['object-storage']['statistics']['statsd_port'] %>
self.statsd_prefix = '<%= node['openstack']['object-storage']['statistics']['statsd_prefix'] %>'
<% if node['openstack']['object-storage']['statistics']['enable_dispersion_report'] -%>
self.enable_dispersion_report = True
<% else %>
self.enable_dispersion_report = False
<% end %>
<% if node['openstack']['object-storage']['statistics']['enable_recon_report'] -%>
self.enable_recon_report = True
<% else %>
self.enable_recon_report = False
<% end %>
<% if node['openstack']['object-storage']['statistics']['enable_disk_report'] -%>
self.enable_disk_report = True
<% else %>
self.enable_disk_report = False
<% end %>
self.recon_account_cache = '<%= node['openstack']['object-storage']['statistics']['recon_account_cache'] %>'
self.recon_container_cache = '<%= node['openstack']['object-storage']['statistics']['recon_container_cache'] %>'
self.recon_object_cache = '<%= node['openstack']['object-storage']['statistics']['recon_object_cache'] %>'
def _dispersion_report(self):
"""
Swift Dispersion Report Collection
"""
p = Popen(['/usr/bin/swift-dispersion-report', '-j'],
stdout=PIPE, stderr=PIPE)
stdout, stderr = p.communicate()
self.publish('%s.dispersion.errors' % self.statsd_prefix, len(stderr.split('\n')) - 1)
data = json.loads(stdout)
for t in ('object', 'container'):
for (k, v) in data[t].items():
self.publish('%s.dispersion.%s.%s' % (self.statsd_prefix, t, k), v)
def _recon_report(self):
"""
Swift Recon Collection
"""
recon_cache = {'account': self.recon_account_cache,
'container': self.recon_container_cache,
'object': self.recon_object_cache}
for recon_type in recon_cache:
if not os.access(recon_cache[recon_type], os.R_OK):
continue
try:
f = open(recon_cache[recon_type])
try:
rmetrics = json.loads(f.readlines()[0].strip())
metrics = self._process_cache(rmetrics)
for k, v in metrics:
metric_name = '%s.%s.%s' % (self.statsd_prefix, recon_type, ".".join(k))
if isinstance(v, (int, float)):
self.publish(metric_name, v)
except (ValueError, IndexError):
continue
finally:
f.close()
def _disk_report(self):
"""
Swift Disk Capacity Report
"""
p = Popen(['/usr/bin/swift-recon', '-d'],
stdout=PIPE, stderr=PIPE)
stdout, stderr = p.communicate()
used, total = 0, 0
match = re.search(r'.* space used: ([0-9]*\.?[0-9]+) of ([0-9]*\.?[0-9]+)', stdout, re.M|re.I)
if match:
used, total = [int(i) for i in match.groups()]
highest, avg = 0, 0
match = re.search(r'.* lowest:.+highest: ([0-9]*\.?[0-9]+)%, avg: ([0-9]*\.?[0-9]+)%', stdout, re.M|re.I)
if match:
highest, avg = match.groups()
self.publish('%s.capacity.bytes_used' % self.statsd_prefix, used)
self.publish('%s.capacity.bytes_free' % self.statsd_prefix, total-used)
self.publish('%s.capacity.bytes_utilization' % self.statsd_prefix, int((used/total)*100))
self.publish('%s.capacity.single_disk_utilization_highest' % self.statsd_prefix, highest)
self.publish('%s.capacity.single_disk_utilization_average' % self.statsd_prefix, avg)
def collect(self):
if (self.enable_dispersion_report):
self._dispersion_report()
if (self.enable_recon_report):
self._recon_report()
if (self.enable_disk_report):
self._disk_report()
def publish(self, metric_name, value):
"""Publish a metric to statsd server"""
# TODO: IPv6 support
print '%s:%s|g' % (metric_name.encode('utf-8'), value), (self.statsd_host, self.statsd_port)
udp_sock = socket(AF_INET, SOCK_DGRAM)
udp_sock.sendto('%s:%s|g' % (metric_name.encode('utf-8'), value), (self.statsd_host, self.statsd_port))
def _process_cache(self, d, path=()):
"""Recusively walk a nested recon cache dict to obtain path/values"""
metrics = []
for k, v in d.iteritems():
if not isinstance(v, dict):
metrics.append((path + (k,), v))
else:
self._process_cache(v, path + (k,))
return metrics
if __name__ == '__main__':
collector = OpenStackSwiftStatisticsCollector()
collector.collect()

50
templates/default/swift.conf.erb
View File

@@ -1,50 +0,0 @@
<%= node['openstack']['object-storage']['custom_template_banner'] %>
[swift-hash]
swift_hash_path_prefix = <%= @swift_hash_path_prefix %>
swift_hash_path_suffix = <%= @swift_hash_path_suffix %>
# storage policies are defined here and determine various characteristics
# about how objects are stored and treated. Policies are specified by name on
# a per container basis. Names are case-insensitive. The policy index is
# specified in the section header and is used internally. The policy with
# index 0 is always used for legacy containers and can be given a name for use
# in metadata however the ring file name will always be 'object.ring.gz' for
# backwards compatibility. If no policies are defined a policy with index 0
# will be automatically created for backwards compatibility and given the name
# Policy-0. A default policy is used when creating new containers when no
# policy is specified in the request. If no other policies are defined the
# policy with index 0 will be declared the default. If multiple policies are
# defined you must define a policy with index 0 and you must specify a
# default. It is recommended you always define a section for
# storage-policy:0.
[storage-policy:0]
name = Policy-0
default = yes
# the following section would declare a policy called 'silver', the number of
# replicas will be determined by how the ring is built. In this example the
# 'silver' policy could have a lower or higher # of replicas than the
# 'Policy-0' policy above. The ring filename will be 'object-1.ring.gz'. You
# may only specify one storage policy section as the default. If you changed
# this section to specify 'silver' as the default, when a client created a new
# container w/o a policy specified, it will get the 'silver' policy because
# this config has specified it as the default. However if a legacy container
# (one created with a pre-policy version of swift) is accessed, it is known
# implicitly to be assigned to the policy with index 0 as opposed to the
# current default.
#[storage-policy:1]
#name = silver
[swift-constraints]
max_file_size = <%= node['openstack']['object-storage']['max_file_size'] %>
max_meta_name_length = <%= node['openstack']['object-storage']['max_meta_name_length'] %>
max_meta_value_length = <%= node['openstack']['object-storage']['max_meta_value_length'] %>
max_meta_count = <%= node['openstack']['object-storage']['max_meta_count'] %>
max_meta_overall_size = <%= node['openstack']['object-storage']['max_meta_overall_size'] %>
max_header_size = <%= node['openstack']['object-storage']['max_header_size'] %>
max_object_name_length = <%= node['openstack']['object-storage']['max_object_name_length'] %>
container_listing_limit = <%= node['openstack']['object-storage']['container_listing_limit'] %>
account_listing_limit = <%= node['openstack']['object-storage']['account_listing_limit'] %>
max_account_name_length = <%= node['openstack']['object-storage']['max_account_name_length'] %>
max_container_name_length = <%= node['openstack']['object-storage']['max_container_name_length'] %>