Commit Graph

803 Commits (053614be148a5b085b186c82ba630bf6ce2aa067)

Author SHA1 Message Date
Alfredo Moralejo 053614be14 Do not include ETag when puting manifest in chunked uploads
While testing glance with Ceph Rados Gateway using latest Ceph release
(Nautilus), i've found that glance fails to upload the manifest using
dynamic large objects mode because of the value used in ETag request.

This issue has been reported to Ceph as it seems related to some recent
change in radosgw code [1].

However, checking at the upload workflow used by glance and comparing
to Swift documentation [2], I wonder if adding the etag is actually
providing any value. In the Swift the ETag header is used to validate
integrity when uploading chunks, not the manifest while glance is doing
exactly the oposite, not sending the etag in the chunks (I guess to
avoid checksuming big images, which makes sense to me) and sending it
when puting the manifest.

This patch is removing the etag header when sending the PUT request for
the manifest in chunked uploads.


Closes-bug: #1824533
Change-Id: I0b563dfcdc30026669fb089c82db8c3df7edc808
(cherry picked from commit ed356fc6b4)
2019-04-17 16:29:26 +00:00
wangxiyuan 9c8364bacf Prevent unicode object error from zero-byte read
During large file uploads under py3, we are occasionally seeing a
"unicode objects must be encoded before hashing" error even though
we are reading from a byte stream.  From what I can tell, it looks
like it's happening when a zero-byte read is requested, so we handle
that case explicitly.  This is a band-aid fix; we still need to track
down the source.

Co-authored-by: wangxiyuan <>
Co-authored-by: Brian Rosmaita <>

Related-bug: #1805332
Change-Id: Ia7653f9fcbe902abc203c10c80ab44a641a4d8f9
(cherry picked from commit 1d25a2b7a2)
2019-03-20 14:09:09 +00:00
OpenStack Release Bot 58229944f2 Update UPPER_CONSTRAINTS_FILE for stable/stein
Update the URL to the upper-constraints file to point to the redirect
rule on so that anyone working on this branch
will switch to the correct upper-constraints list automatically when
the requirements repository branches.

Until the requirements repository has as stable/stein branch, tests will
continue to use the upper-constraints list on master.

Change-Id: I08bf9c65c59f808a0b2d813f08ae49886aa81c4a
2019-03-18 14:39:15 +00:00
OpenStack Release Bot 8591aafbe3 Update .gitreview for stable/stein
Change-Id: I27f3f92be35608e0d8ac9ec0280f30ac1a95a79f
2019-03-18 14:39:13 +00:00
Erno Kuvaja 5b68597400 Stein cycle Release Notes
Change-Id: I1b234fb4ca9c8895e53d8063f72e3eb8e3b94efc
2019-02-28 17:29:56 +00:00
Zuul 83a8b2cc06 Merge "Filesystem driver: add chunk size config option" 2019-02-28 08:15:52 +00:00
Zuul bb46e542f7 Merge "add python 3.7 unit test job" 2019-02-25 15:29:30 +00:00
Lucian Petrut ea1cad2bed Filesystem driver: add chunk size config option
At the moment, the filesystem driver uses a hardcoded 64 KB chunk
size, used when reading/writing image files.

This can be extremely inefficient, especially when file shares are

For this reason, this change will make it configurable.


Change-Id: I1559c03308d36ecf9305c7a72ad658ecd1f2dc76
Closes-Bug: #1792869
2019-02-25 14:47:07 +02:00
Zuul f503633da0 Merge "Fix python3 compatibility of rbd get_fsid" 2019-02-22 07:51:30 +00:00
Michal Arbet 9c73370358 Fix python3 compatibility of rbd get_fsid
In python3 conn.get_fsid() is represented as binary.
Because of this, direct_url is corrupted in DB.
This patch is fixing this. More informations in
closing bug.

Closes-Bug: #1816721

Change-Id: I6a89042fae63bf5c9096a6bee6774aebf3f5864b
2019-02-20 12:37:48 +00:00
Cyril Roelandt d54ee8437f Do not raise StopIteration
The current code will not work in Python3.7 because of PEP479. This
commit fixes it so that it works with both 3.6 and 3.7.


Closes-bug: #1815335
Change-Id: I6adf23e0569db8655806a07d53c5000f5fd0918f
2019-02-15 13:10:59 +00:00
Corey Bryant b5f4d46136 add python 3.7 unit test job
This is a mechanically generated patch to add a unit test job running
under Python 3.7.

See ML discussion here [1] for context.


Change-Id: Iae860f4db0da99cf8fc564c085cba7b234606751
Story: #2004073
Task: #27415
2019-02-14 22:40:49 -05:00
Zuul db75918208 Merge "Fix some types in the FS and VMware drivers" 2019-01-28 06:14:25 +00:00
Zuul 7cdda1be12 Merge "Remove moxstubout usage" 2019-01-24 05:23:15 +00:00
Cyril Roelandt ac5abddb6f Fix some types in the FS and VMware drivers
The _get_datadir_path_and_priority and _parse_datastore_info_and_weight methods
may return the priority (or weight) as an integer or a string. This causes a few

- the caller must cast the result to an integer
- the code is harder to read
- static checkers such as mypy have a hard time processing these methods

We make sure that both methods now return the priority as an integer.

Closes-Bug: #1813092
Change-Id: I9435c88560b01737f9310fe2cba1ca4c84e0f3fa
2019-01-24 01:14:26 +01:00
Zuul 8814fe8273 Merge "Fix defaults for ConfigParser" 2019-01-15 12:57:15 +00:00
Zuul 5f983942e0 Merge "Imported Translations from Zanata" 2019-01-15 12:57:14 +00:00
Zuul ebca8475a7 Merge "Use template for lower-constraints" 2019-01-15 12:57:13 +00:00
OpenStack Proposal Bot f5843e4dee Imported Translations from Zanata
For more information about this automatic import see:

Change-Id: I1072be24590ceb10911042ba1bcef7bec5be1f19
2019-01-13 08:16:01 +00:00
Zuul de27161e78 Merge "Update deprecation notices" 2019-01-08 16:21:43 +00:00
Zuul 8f3668d846 Merge "Catch rbd NoSpace exception" 2018-12-26 23:24:49 +00:00
yfzhao 4c328e7630 Update mailinglist from dev to discuss
openstack-dev was decomissioned this night in
Update openstack-dev to openstack-discuss

Co-authored-by: LeopardMa <>

Change-Id: I06acef3eafe1b9f7842636e9bcb5a637f2d78cc3
Closes-Bug: #1808929
2018-12-21 11:11:48 +08:00
Andreas Jaeger 86927ea13c Use template for lower-constraints
Small cleanups:

* Use openstack-lower-constraints-jobs template, remove individual
* Sort list of templates

Change-Id: Iebc92d703af81053547c7b09c863e580e803381c
2018-12-20 20:59:06 +01:00
Brian Rosmaita 429402b541 Update deprecation notices
Removal of 'stores' and 'default_store' has been postponed until
Train; update the deprecation text and add a release note.

Change-Id: Ib01be1746f9a1b414d6a9436032c34034a2e6f45
2018-12-20 11:24:44 -05:00
Liang Fang 7377cf2638 Catch rbd NoSpace exception
Unhelpful error (HTTP 500, Internal Server Error) received if image
creation failed due to insufficient free space. This patch catches rbd
NoSpace exception and re-raise the exact exception
glance_store.StorageFull, and the http client will receive "HTTP 413
Request Entity Too Large".

Closes-Bug: #1808456
Change-Id: Id787d3ae6f761aaa31d2fc7dd26d2f7d327367d4
Co-authored-by: Stefan Dinescu <>
Signed-off-by: Liang Fang <>
2018-12-14 10:45:47 +08:00
Chuck Short aefad17de8 Remove moxstubout usage
As of version 3.5.0 moxstub will be deprecated, so remove it where it has
been used.

Change-Id: I2622c457871815311241d2eea562d7a3c70b0795
Signed-off-by: Chuck Short <>
2018-10-29 10:22:05 -04:00
Zuul b8b873433e Merge "Provision to add new config options in sample config file" 2018-10-25 17:19:11 +00:00
Brian Rosmaita ad46c752ef Add statement explaining "tips" job configuration
The configuration of the periodic "tips" jobs in .zuul.yaml may
be confusing, particularly after a stable branch is cut.   Add a
comment explaining what we're doing and why to reduce confusion.

Change-Id: I7bf676ac972af835c7a6b8d04815629752685765
2018-10-11 12:03:44 -04:00
Zuul fb5bd5db32 Merge "Imported Translations from Zanata" 2018-09-18 13:27:29 +00:00
Abhishek Kekane da494d4abb Provision to add new config options in sample config file
In Rocky we have added support for multiple backend as a EXPERIMENTAL
feature. However configuration options related to multiple backend
are not generated in sample config file due to some issue. We have
added below 2 new config options for multiple backend.

1. enabled_backneds (added in glance)
2. default_backend (added in glance_store)

Made provision to add option 2 from above to sample config

Change-Id: I63571e4a8f85003e304f16653d60cbd38e6b6bde
Partial-Bug: #1793057
2018-09-18 10:18:54 +00:00
OpenStack Proposal Bot bb965c7037 Imported Translations from Zanata
For more information about this automatic import see:

Change-Id: I28a2c529cd63db05e7a08ab433ff3aba25bd4146
2018-09-18 06:07:16 +00:00
Nguyen Hai 63c730e53b add lib-forward-testing-python3 test job
This is a mechanically generated patch to add a functional test job
running under Python 3 as part of the python3-first goal.

See the python3-first goal document for details:

Change-Id: I3a9fd029aa958761990aea6d10b20c7904b3ea05
Story: #2002586
Task: #24297
2018-09-18 04:15:07 +00:00
Nguyen Hai 969f81ef48 add python 3.6 unit test job
This is a mechanically generated patch to add a unit test job running
under Python 3.6 as part of the python3-first goal.

See the python3-first goal document for details:

Change-Id: Ie77088a8539ad1f6a83db0a6e57a023977e15a52
Story: #2002586
Task: #24297
2018-09-18 04:15:01 +00:00
Nguyen Hai ce9d742ad1 switch documentation job to new PTI
This is a mechanically generated patch to switch the documentation
jobs to use the new PTI versions of the jobs as part of the
python3-first goal.

See the python3-first goal document for details:

Change-Id: I92412e449b0c42099698870d2c3000aa70d320da
Story: #2002586
Task: #24297
2018-09-18 04:14:54 +00:00
Cyril Roelandt 6ce857397d Fix defaults for ConfigParser
As of 3.7, the configparser module will not allow defaults values to be None.
This patch replaces such values with "default".

Change-Id: Id5a414412cd66d479fb8f8784cba5deddc628dfd
Closes-Bug: #1785641
2018-09-04 18:47:31 +02:00
Nguyen Hai c7f78ef6b3 import zuul job settings from project-config
This is a mechanically generated patch to complete step 1 of moving
the zuul job settings out of project-config and into each project

Because there will be a separate patch on each branch, the branch
specifiers for branch-specific jobs have been removed.

Because this patch is generated by a script, there may be some
cosmetic changes to the layout of the YAML file(s) as the contents are

See the python3-first goal document for details:

Change-Id: Ic16227b9c5b30dcb8c400e392536774ff40aded1
Story: #2002586
Task: #24297
2018-08-23 14:34:07 +00:00
Brian Rosmaita 11db636c34 Refactor periodic "tips" jobs
Two changes:
1 - declare the abstract base tips jobs 'abstract'
2 - add a branch matcher to where the periodic tips jobs
    are invoked in the project definition so that they only
    apply to the master branch

Change-Id: I1093de2583da862f58372e6c03b2d9b604dbd630
2018-08-15 15:40:07 -04:00
Zuul 5b09ab7472 Merge "Move doc8 to test requirements" 2018-08-15 14:58:15 +00:00
OpenStack Proposal Bot 045dc83718 Imported Translations from Zanata
For more information about this automatic import see:

Change-Id: I3271db43146fa028e9f1507d47366b4785a8ed09
2018-08-15 06:21:39 +00:00
James Page ef44579c35 Move doc8 to test requirements
doc8 is not required to use this module, only for style
checking of RST documentation.

Move from requirements.txt -> test-requirements.txt to avoid
generating a run-time dependency on doc8.

Change-Id: I3ae3909401eb53d4442e6d369228e235e1d5d60e
2018-08-14 14:23:44 +01:00
zhangbailin f49c211fbd Remove team diversity tags note in README
In this commit( removed
diversity tags, it should also be updated here.

Change-Id: I461e724d5880dc4fa4784d50749f25c4b54f474a
2018-08-10 10:01:52 +08:00
Brian Rosmaita 4d4cf1559c Wrap interface function for multihash correctly
The store_add_to_backend_with_multihash() function was not wrapped
correctly so that Glance could consume the secure hash value.  This
patch adds the correct wrapping.

Closes-bug: #1784420

Change-Id: I7a94dfffb61fc212abb58c1e1aec5884474b6aa8
2018-07-31 12:50:18 +00:00
Tomoki Sekiyama c369ba013f cinder: Support os-brick privsep filters
Currently os-brick that glance_store depends on uses oslo-privsep
to execute commands with root privileges, so we do not need to
maintain each command for os-brick in the rootwrap filters, but
only need to add the privsep-helper command.

This replaces the filters for os-brick with the privsep helper.

Related changes in Nova:
  # nova: Add os-brick rootwrap filter for privsep
  # nova: Initialise oslo.privsep early in main
Related changes in Cinder:
  # cinder: Add os-brick rootwrap filter for privsep
  # cinder: Initialise oslo.privsep early in main

Change-Id: Idbebaf796eaf89189f64f64167371b81e56b366e
2018-07-22 10:10:49 -04:00
OpenStack Release Bot df0780bd0e Update reno for stable/rocky
Change-Id: I992873b9e1f74cf57d9a749395a1928e28586b97
2018-07-19 20:41:01 +00:00
Zuul 7141c77f18 Merge "Consider Cinder back-end as production ready" 2018-07-19 18:04:46 +00:00
Zuul 87c35999cd Merge "Remove config option help translation" 2018-07-19 17:40:04 +00:00
Zuul f84d3044df Merge "Address multi store nits" 2018-07-19 17:25:15 +00:00
Erno Kuvaja 2c35f16ff1 Consider Cinder back-end as production ready
Closes-Bug: #1780764
Change-Id: I02153190d72bd861c509f61caad92c20a787f848
2018-07-19 13:56:40 +00:00
Sean McGinnis 1bbe9b3f8a Remove config option help translation
Config option help is used to generate sample config files.  This
is done statically and only in English. Translation is done
dynamically at runtime by loading the translation catalogs. So
putting the _() translation around config option help creates
unnecessary work for the translators as this will never be loaded
and translated.

This removes translation markers from config option definitions.

Change-Id: I0597a5b69ef4e179324885cb35bbb72fceb2b830
2018-07-19 13:53:32 +00:00
Brian Rosmaita 90f4b82aec Deprecate store_add_to_backend()
A replacement, store_add_to_backend_with_multihash(), has been
introduced in Rocky.  The old function is kept in Rocky for
backward compatability.  This deprecation notice registers our
intent to remove it in Stein.

Change-Id: If09ae10ee160e3fbfafce94e4f3607994af63ec5
2018-07-18 12:02:43 -04:00