set_value method of Puppet::Util::IniFile uses '' for separator when
called without the separator argument[1]. Set the argument explicitly
to avoid broken rendering. Note that the argument was added in
puppetlabs-inifile 2.0.0 so we don't need to bump the min version.
[1] https://github.com/puppetlabs/puppetlabs-inifile/issues/544
Closes-Bug: #2081303
Change-Id: I6112c6d12e860bd4b4030e5f784952a7358f617f
(cherry picked from commit 93f15aa78c378dd853e000156ad14dfff15fa394)
(cherry picked from commit 5769e2a9bd787966d444b7964921c2e8e373dca2)
(cherry picked from commit a0dbacabb44350c816e56924ff50fe74ee7da81c)
swift 2.10.0 was released in 2016. We no longer need to support even
older versions. The output format depends on the version currently
installed, thus the latest logic should work as long as the same
release (or even a bit older release) is installed.
This also updates the logic to skip the header lines. The new logic
is considered to be more robust, because it works even when additional
lines are added.
Change-Id: I694ca4c046f490e4ffecf2101f3f147a69045162
This change ensures the shortened address representations are always
used for ring devices to avoid broken idempotency caused by different
representations (shortened[1] vs canonical[2]).
[1] ::1
[2] 0000:0000:0000:0000:0000:0000:0000:0001
Closes-Bug: #1997313
Change-Id: I762f0780ba25536aa616482a49d59c7db69924d6
This change introduces capability to customize internal-client.conf,
which is used by several daemon processes like container-sharder.
Closes-Bug: #1986599
Related-Bug: #1921319
Change-Id: I200e3a4ff323a07778ec6c4d7dd96824c72a12cf
Regex validation for aliases in swift_storage_policy provider does not
allow single alias value, like for exemple:
swift::storage::policy { '1':
policy_name => '3-Replica-Policy',
policy_aliases => 'green',
This fixes the accepted pattenr to allow single-value for policy_aliases
parameter.
Also, the whole expression is updated to more strictly validate
the pattern, especially when more than 2 aliases are given.
Change-Id: I62c86097c717d2a74c100386cf20e07976571805
The change 63688a14e57282ae97072b406c81d0da2f61965c introduced support
of storage policies and since the change was merged builder_file_path
takes one argument which is policy_index .
This change fixes the function call without argument to avoid failure
caused by missing argument.
Closes-Bug: #1949334
Change-Id: I987a37cb0606b6c217f906fbae17a765e40f4580
This patch is adding the configuration of the number of workers,
and the size of the listen queue in Debian, which uses uwsgi to run
Swift. Therefore, this patch adds new swift_proxy_uwsgi_config,
swift_account_uwsgi_config, and swift_container_uwsgi_config, providers
as well as e new swift::wsgi::uwsgi_proxy, swift::wsgi::uwsgi_container,
and swift::wsgi::uwsgi_account classes.
Change-Id: Ib18f3479bc6a4513a7b3d823cda480c4ace2f2c0
Updating region/zone of a device is actually implemented and supported
in swift, while puppet-swift doesn't support that operation yet.
This patch updates error messages when an operator tries to update
region/zone of a device, so that they notice more clearly that the
cause exists in puppet-swift, not in swift.
Change-Id: I7c7f633630ac52578a0cf6beb2033465a16bd251
Adding barbican configuration to Swift will
allow enabling data-at-rest encryption in Swift.
Change-Id: I78c6003f5f599a422193dc47422ee607ce05c715
Signed-off-by: Thiago da Silva <thiago@redhat.com>
This commit adds support for the memcache_servers option in
object-expirer and container-reconciler. Without this, both
currently default to 127.0.0.1:11211.
Closes-Bug: #1627927
Change-Id: Ie139f018c4a742b014dd4d682970e154d66a8c6d
This change implements storage policies as defined by swift:
http://docs.openstack.org/developer/swift/overview_policies.html
There are two primary areas of change in this review
- Add storage policy support to the ringbuilder and ring devices, along with
associated spec tests.
- Adding storage policy support and enforcing rules in swift.conf using the
swift_storage_policy type and provider. Also: updated spec tests and an
update to swift acceptance test to use storage policies to configure an
additional 3 replica based ring.
See release notes and the README update for more details/instructions.
Change-Id: I2b8db751790704df3f1027a14f61e231591537f3
This patch: https://review.openstack.org/#/c/319387
modified the output of rings, so we need to add the case where this
regex is found and how to use it.
Change-Id: Id9f1a478670e7b618ce9d222c5bc9739e3dd6286
Do not use the puppet file resource from within
the swiftinit service provider. In puppet versions
4.4.0+ it has unexpected results. Instead use ruby
File to write the init/service files directly.
Change-Id: If0dc56e0742e591e3db3ed18154edf3489d5ef1a
Add possibility to configure container_sync middleware. Also
allow to configure swift container-sync-realms, which specifying
the allowable clusters and their information, which is required
for container_sync.
Change-Id: I183841cca3daf72a30db160b88c9e3a9335ad7dc
If a ring is balanced or does not need an update, swift-ring-builder
will print an additional line:
"Ring file /etc/swift/container.ring.gz is up-to-date"
Skip this line in parsing the output to avoid "Warning: Unexpected line:"
Change-Id: Ibc021f62f699c94f36c76b11acefe1000f6a2288
Added function to check if ubuntu is < 16.04.
If ubuntu < 16.04 then default provider was upstart.
If ubuntu = 16.04 then use systemd support.
Closes-Bug: #1578408
Change-Id: Icb2d6ff5152e782066b957d5e6e74357496c6b28
Compare contents of the init/systemd file on disk against
swiftinit provider template contents. If they don't match
then the distro provided init/systemd file has not been replaced.
swiftinit provider will call service enable and write out correct
init/systemd file.
Change-Id: I722f6cfa8c537d62a607880cc37863f2d1139636
Starting in swift 2.2.2 an extra line for overload is printed
by swift-ring-builder.
Check for this and skip that line to avoid avoid warning messages.
Change-Id: I2d81db0a29249101116c1899565f024aaac69000
Closes-Bug: #1524936
The 'swiftinit' provider is a custom provider of the service type
that manages swift services using swift-init.
Use of the swiftinit service provider is optional, the default is
to use service providers specified in params file. This provider
also manages swift services starting at boot by adding or removing
a templated init or services file and making systemctl calls.
See README for more detail.
A wrapper defined type 'swift::service' has been created to simplify
use of the swiftinit provider without adding logic to every class.
this wrapper also aids in input validation and testing of the
swiftinit provider.
Two extra runs of apply_manifest have been added to the
basic_swift_spec acceptance test. The service_provider
is set to "swiftinit". The first run catches any errors upgrading
to the swiftinit service provider and the second run tests
idempotency.
This patch is an initial step towards using swift-init to manage
multiple swift services out of different configuration files such
as is needed to run a separate replication network.
Change-Id: I2f71c82c7a6c463f8c76a193409c0a17daa15bda
This commit moves conditionals based on values obtained from Facter
inside code blocks that are only executed on agents and replaces
the use of Facter['osfamily'].value with Facter.value:(:osfamily).
Without this change we are returning values based off the master's
facts, so if a Debian agent checks into a RedHat master then the type
is currently making a decision based off of RedHat and not the actual
OS of the agent running the catalog. Code ran outside blocks is
evaluated on masters and inside a block is evaluated by the agent when
the catalog is executed. We do not notice this because all our
testing uses "puppet apply" and autorequire only matters when they
find a matching resource name in the catalog. The latter results
in no error because the relationship is simply ignored on the Debian
agent if a package resource with the RedHat name is not found.
This issue was found by running facter 3, the C++11 re-implementation
under the jruby and clojure implemented puppetserver. The clojure
jni shim written so that facter can be ran inside of puppetserver does
not implement the [] method for the Facter module but the ruby shim
does. So I noticed that the code had an issue when
Facter['osfamily'].value was executed on the master, which returned a
not method error. Once again, something we didn't notice becuase we
do not test against a master. I decided to migrate to
Facter.value(:osfamily) to simply keep to an API that is consistent
across both puppet and puppetserver but fixing where the code runs
does actually solve the not method found error.
Change-Id: I24b0b20b2839c7bc33a76ac14849783f2285579f
The swift-object-expirer service was not supported by
puppet-swift. Adding support by creating a new class
(swift::objectexpirer), and its associated custom type/provider.
Change-Id: I498ffe525a7316c0091e4c9d8b7d9658234231f6
With the creation of the new openstack_config provider, some processing
that was done in swift_config has been centralized in
openstack_config.
Impacted methods are :
* section
* setting
* separator
Also, this commit adds the fact that, when passing a specific string
(ensure_absent_val) the provider will behave as if ensure => absent was
specified. '<SERVICE DEFAULT>' is the default value for
ensure_absent_val.
The use case is the following :
swift_config { 'DEFAULT/foo' : value => 'bar' } # will work as usual
swift_config { 'DEFAULT/foo' : value => '<SERVICE DEFAULT>' } # will mean absent
That means that all the current :
if $myvar {
swift_config { 'DEFAULT/foo' : value => $myvar }
} else {
swift_config { 'DEFAULT/foo' : ensure => absent }
}
can be removed in favor of :
swift_config { 'DEFAULT/foo' : value => $myvar }
If for any reason '<SERVICE DEFAULT>' turns out to be a valid value for
a specific parameter. One could by pass that doing the following :
swift_config { 'DEFAULT/foo' : value => '<SERVICE DEFAULT>',
ensure_absent_val => 'foo' }
Change-Id: I9281d2cae81f9799327f7f6e04498d6bc723f233
Depends-On: I0eeebde3aac2662cc7e69bfad7f8d2481463a218
Currently we specify the ordering of config resources wherever it is
necessary based on the presence of the file it will write to, or the
presence of the package in charge of providing the file it will write
to.
Those kind of ordering can be specified directly at the resource level
using the autorequire mechanism. With this patch, any config resource
will make sure the package in charge of providing the file will be
installed first.
Change-Id: Icb3464cc0a747d40326d610d38806d059c9a0fc3
One shold never parse uri with split(':') as it was in types
ring_account_device, ring_container_device and ring_object_device.
This is fixed by using uri and just setting variables from that.
Also swift_ring_builder didn't add brackets to ipv6 address before
adding port information to it.
Change-Id: I55b7032143c8446abeed4f6c266c0fff0cb6b3b3
Since Swift 1.8.0, there is the region layer of topology above zones.
swift-ring-builder supports this natively and this commit allows a user to
specify a region when creating devices.
We maintain backwards compatibility by defaulting to region '1'.
The coverage of the ring builder spec tests were also improved as part of
this commit.
Change-Id: I67cbe6b87c84778f71df59cf00f2c5175342bc1b
This change also adds an autorequire in the swift_config
type to ensure the /etc/swift/swift.conf exists.
Change-Id: Ic591a3ceb040c4599339ed7f750a5273d2052ac1
Implements: blueprint puppet-swift-ini-settings
Previous configuration file management method used templates.
This method had limited flexibility and was cumbersome
to use and maintain between upstream versions.
This change introduces new custom types based on ini_setting:
* swift_account_config: /etc/swift/account-server.conf
* swift_bench_config: /etc/swift/swift-bench.conf
* swift_config: /etc/swift/swift.conf
* swift_container_config: /etc/swift/container-server.conf
* swift_dispersion_config: /etc/swift/dispersion.conf
* swift_object_config: /etc/swift/object-server.conf
* swift_proxy_config: /etc/swift/proxy-server.conf
The use of ini_setting offers multiple advantages:
* Ability to individually manage configuration settings without
the need to modify templates. This allows the user to manage or
override settings that aren't handled by the module.
* Use upstream default values if none is provided.
* Remove the need to maintain and synchronize templates
to match upstream versions. Such maintenance could become
problematic and complicated if multiple upstream versions
are supported but have different base configuration files.
Implements: blueprint puppet-swift-ini-settings
Change-Id: Iba7be64581b81b8ab4f33cf7096df37e64ccc4b4
Updates swift_ring_builder rebalancing parsing to support the
latest upstream replicator_id and replicator_port changes.
This correspondes with swift 1.8+ upstream development.
Fixes LP Bug #1182197.
Change-Id: Ibbaf75ca22cad95b22f9b5f03b547db8c47a9ea4
In some case, there are no spaces before - of the balance in swift 1.8.
The change fix this.
/etc/swift/object.builder, build version 9
32768 partitions, 3.000000 replicas, 1 regions, 3 zones, 9 devices, 200.00 balance
The minimum number of hours before a partition can be reassigned is 24
Devices: id region zone ip address port name weight partitions balance meta
0 1 1 10.68.0.45 6000 sdd 100.00 32768 200.00
1 1 2 10.68.0.46 6000 sdd 100.00 0-100.00
...
Change-Id: Ic7757a22ce94e4d51ae11780d4626c90a9ed430f
Updates the swift_ring_builder so that it parses ring builder output
from the most recent Swift releases correctly (1.7.6 or greated).
Backward compat for the previous parsing is still supported.
Change-Id: I4263b349310e16f6825960805eb27e4bfe3ff002
Previously, the Puppet modules only supported a single device per
endpoint.
This commits allows multiple devices per endpoint by making the device
name a part of the namevar for the swift ring devices.
it also updates the relevent configs.
Previously, the swift caches were being built on the class
instances themselves.
This could result in stale values when puppet runs multiple times
using the same process (like in the case of puppet agent)
This commit uses prefetch to ensure that the retrieved swift ring
values are updated per puppet run.
The swift ring db converts integers into floats with 2 decimal
points percisions. This was causing the ring resources to constantly
resync.
This commit munges weight in the type to the proper format.
The code was not robust when no
rows were returned from
swift_ringbuilder(builder_file_path).
The calls to retrieve an array range from the
array of this commands output was returning nil
which caused the failure: undef method each on nil
This change explicitly checks for the case where
there are no rows returned by swift-ring-builder.
This commit adds a class, defined resources and
native types that can be used together to manage
the creation, configuration, and rebalancing of
rings.
Creates the following native types:
ring_account_device
ring_container_device
ring_object_device
The implementation of these types is provided
by swift-ring-builder and most of the functionality
is provided by the parent provider (SwiftRingBuilder)
Each of the providers is hard-coded to find the
relevent ring database in /etc/swift. This directory
location can currently not be configured.
Each provider implements self.instances so the
current state can of the ring can be queried from
puppet.
These providers do not handle either the creation
or the rebalancing of the rings, they are intended
to be used in combination with defined resource types
that perform that functionality:
swift::ringbuilder::create
swift::ringbuilder::rebalance
The swift::ringbuilder class is also created as
a part of this commit. This class does the following:
- creates object, account, and container rings
(it uses the same parameters to create these rings)
- creates the rebalance defines
- sets up the following relationships:
for each ring type, create should be applied before
the native types which should refresh the rebalance
