1002 Commits

Author SHA1 Message Date
Takashi Kajinami
1315cf9c36 Pick up jdk version from environment
... instead of hard-coding the version in ant build file.

Change-Id: I382803ee7c557e02fc02bda53f140b46aa0f2146
2024-02-10 14:22:55 +09:00
40031d0420 reno: Update master for unmaintained/yoga
Update the yoga release notes configuration to build from
unmaintained/yoga.

Change-Id: I70df86fafd317d924dd635f4f1634ef2b0880d3e
2024-02-05 16:55:19 +00:00
Zuul
3e1f097e11 Merge "Fix redundant default posargs for unit tests" 2024-02-02 01:28:26 +00:00
Zuul
aa192c23f5 Merge "Update repository structure" 2024-02-01 19:21:43 +00:00
Zuul
773b2817aa Merge "Remove unused .unittests" 2024-02-01 19:21:42 +00:00
Zuul
b9499dfb06 Merge "Skip functional tests for change in .gitignore" 2024-02-01 19:03:02 +00:00
Takashi Kajinami
cafb36b1af Fix redundant default posargs for unit tests
The same is already defined by .stestr.conf .

Change-Id: I9afd30ef02570cc8eee43dd48f2b9b6a52f05d2d
2024-02-01 19:01:55 +00:00
Zuul
bc48403c79 Merge "Skip functional tests when README is updated" 2024-02-01 18:40:03 +00:00
Zuul
a1d4e8a20a Merge "Update links in README" 2024-02-01 18:28:50 +00:00
Takashi Kajinami
0ccfe557e5 Remove unused .unittests
We generally use tox to run unit tests.

Change-Id: I94409abebb7670a0174557db95bb385cf7f1eec7
2024-02-02 03:14:18 +09:00
Takashi Kajinami
c9e35be319 Skip functional tests for change in .gitignore
Change-Id: I7756b3c6f7d343f7246d1e368799c32c47b9c16c
2024-02-02 03:12:09 +09:00
Takashi Kajinami
77509add21 Skip functional tests when README is updated
Change-Id: I058531f9aade756720085555939ca659f85590b2
2024-02-02 03:07:20 +09:00
Zuul
b84c9a634e Merge "Remove unused STORLETS_DOCKER_SWIFT_*_ID" 2024-02-01 17:55:10 +00:00
Takashi Kajinami
7c8a63a274 Update repository structure
The structure currently described is very old and does not match
the current one after refactoring.

Change-Id: I2b39a6f14175f0478c7fc14698f4e477ce05351b
2024-02-02 02:47:49 +09:00
Takashi Kajinami
1b5dd530d9 Update links in README
... to use docs.openstack.org . Also the deployment guide is no longer
maintained so remove the link to it.

Change-Id: I51378ae474e1edc515858ae629aa88a8b4d2c6a4
2024-02-02 02:36:22 +09:00
Takashi Kajinami
9d80bd25a1 Remove unused STORLETS_DOCKER_SWIFT_*_ID
Change-Id: I80b47c126957cea7908dbb8fdbb059a2648c17d4
2024-02-01 21:41:21 +09:00
Takashi Kajinami
7e3b21de8d Ensure '-' in storlet object metadata
... to avoid detecting malformed headers like;
  X-Object-Meta-StorletFoo

Change-Id: I16b6488c48f75cf2519d8dd387449f4fde69f3b1
2024-01-31 12:17:36 +09:00
Takashi Kajinami
88369fee13 Run codespell to fix typos
This fixes the spell fix by;

$ tox -e codespell -- -q 2 -S '*.csv,*.po,*.yaml' -w

Change-Id: I11a3142d6eef40bec2324b98bfe4b0d871779f64
2024-01-30 19:09:27 +09:00
Zuul
f565b62fa5 Merge "Skip storlet execution for symlink at object-server" 2024-01-29 09:54:56 +00:00
Takashi Kajinami
c227b51739 Refactor functional tests with broken python storlet
... and also add a test case to cover proxy execution scenario.

Change-Id: Ic2c508ca57d37878b44f97425f326d7d7e46f506
2024-01-29 18:16:56 +09:00
Takashi Kajinami
b43dca80a2 Skip storlet execution for symlink at object-server
... because the actual object content is not available in symlink.

Change-Id: I5e4ba77eb5c689b83a7503682ccabddea8aa1608
2024-01-29 18:10:03 +09:00
Zuul
3eb72a4bda Merge "Refactor SLO tests" 2024-01-29 06:16:20 +00:00
Takashi Kajinami
50a4d4f73a Refactor SLO tests
* Use context manager to open files
* Response codes are already checked within swiftclient methods
* etag is now returned by put_object method

Change-Id: I446118a314929c7d646c66bc2d8a0f1fa41e4fed
2024-01-29 14:29:22 +09:00
Zuul
dc90c16808 Merge "Generalize container related options" 2024-01-29 02:23:38 +00:00
Zuul
6b80385cda Merge "Add option to limit pids within a container" 2024-01-29 01:51:36 +00:00
Zuul
c17b6ef0e6 Merge "Remove unused bin/ files" 2024-01-29 01:25:28 +00:00
Takashi Kajinami
5818bc046d Add option to limit pids within a container
Because we run container side applications using the swift user, we
need a mechanism to limit number of processes launched inside storlet
containers so that user workloads do not consume all allowed pids in
host.

This introduces a new option to set pids limit in each container.

Change-Id: Idc07807ab7dba12c795d19d6405fc998e7b09893
2024-01-29 10:01:01 +09:00
Takashi Kajinami
f7caa971ef Remove unnecessary client config file generation
This is no longer needed, since we split out file manager logic.

Change-Id: I2072825ef09e01cfeade3aa98472b3716b4126a7
2024-01-28 17:52:58 +00:00
Zuul
156f9f4cb3 Merge "Refactor config file generation/modification" 2024-01-28 17:24:02 +00:00
Takashi Kajinami
4aaa9d351a Remove unused bin/ files
These are now generated by setuptools.

Change-Id: I31751308f54ec384967dc491a55f39edcdc8f54d
2024-01-29 01:58:12 +09:00
Takashi Kajinami
513f6f777d Refactor config file generation/modification
This replaces the python script to update swift config files by set
of raw commands or functions in bash.

This also removes the hard-coded storlet-proxy-server.conf and
replaces it by the common internal-client.conf .

Change-Id: I45e92e9323d144230d5850274692c5cbf6aa0014
2024-01-28 16:38:46 +00:00
Takashi Kajinami
3ed0275b2b Generalize container related options
This renames a few options of docker gateway to generalize name of
these options in case we support any different runtimes such as podman.
  - docker_repo to container_image_namespace
  - default_docker_image to default_container_image

Change-Id: Id256e85f79912b5f5130e259475e9c9db28d7912
2024-01-29 01:16:25 +09:00
Takashi Kajinami
ede1fe4b33 Allow binding storlet applications to specific CPUs or memory nodes
Docker supports binding a container to specific cpu cores memory nodes.
This introduces a few options to allow operators to limit resources
used by storlet containers.

Change-Id: Ifa734850f4284724a5ee06a172b81e04e5fb7015
2024-01-28 12:56:55 +00:00
Takashi Kajinami
ddf7d26259 Limit cpu and memory used by each storlet container
... to avoid overload caused by applications executed by users.

Change-Id: I1157dabe476119af39604fdc30a30b63503ee46a
2024-01-28 21:56:35 +09:00
Zuul
1a096cdad6 Merge "coveragerc: Remove non-existent path" 2024-01-25 15:47:05 +00:00
Takashi Kajinami
4bc6c2c86a coveragerc: Remove non-existent path
The storlets/openstack directory does not exist.

Change-Id: I9d6414e170dc3ffaec0d8684e1b6c04bc46b71a0
2024-01-25 23:57:26 +09:00
Takashi Kajinami
60e75cff17 Drop redundant try-except block
We can omit try-except in case we just re-raise all exceptions.

Also, we should fail in case fd is requested but the log file is not
open.

Change-Id: I2d8588cc8356e32f5bb199416243b52edad30ef6
2024-01-22 23:13:37 +09:00
Zuul
0b6628f972 Merge "Remove redundant cast" 2024-01-21 11:01:21 +00:00
Zuul
e2e25a1f1d Merge "Add Python 3.11 to supported runtime" 2024-01-21 10:18:09 +00:00
Zuul
12664baecc Merge "Restrict access to storlet log file" 2024-01-21 08:38:22 +00:00
Zuul
3a29d3760f Merge "Restrict access to local storlet/dependency files" 2024-01-21 08:38:20 +00:00
Takashi Kajinami
70d354028d Add Python 3.11 to supported runtime
Now Python 3.11 is part of the tested runtimes.

Change-Id: I0479ad0f2ee491327a6a32dde15e495299ea3175
2024-01-21 16:48:23 +09:00
Takashi Kajinami
dd612dd4d4 Remove redundant cast
The st_mtime property of stat result is float, so we don't need
the casting to float.

Change-Id: Ib56e82a623d6d8e9dabe242046645ff205591827
2024-01-21 16:41:08 +09:00
Takashi Kajinami
dc81cefc1b Restrict access to storlet log file
Change-Id: I47f936e821e254867f847c947abd30388c126830
2024-01-21 03:11:17 +09:00
Takashi Kajinami
5ad58804af Restrict access to local storlet/dependency files
Now swift and storlet daemon inside containers run with consistent uid
so we don't need group/other permissions.

Also chown should be executed before actual file content is written,
so that the content is not read by a different user.

Closes-Bug: #2047723
Change-Id: I7790e51556875be1fc6438d1e2c599b693ca3b5b
2024-01-20 18:07:05 +00:00
Zuul
43362ddfc5 Merge "Remove note about old pip's behavior" 2024-01-08 16:28:05 +00:00
Takashi Kajinami
8f17a119ee Remove note about old pip's behavior
... because the behavior is different in recent versions. The same note
was already removed from requirements.txt.

Change-Id: Icd256666bf50d8c43535079399576d6626f51b0c
2024-01-08 23:50:43 +09:00
Zuul
f382843bf2 Merge "Use devstack functions to manage swift/keystone services" 2024-01-08 07:11:34 +00:00
Zuul
cf717b73af Merge "Drop explicit install of uwsgi-plugin-python3" 2024-01-08 05:02:23 +00:00
Takashi Kajinami
07ca6fcf06 Use devstack functions to manage swift/keystone services
... instead of implementing our own logics. Also, the fstab entry is
already managed by the core devstack.

Change-Id: I6a8fbcfebf1e501f41ac82117688572533ae624d
2024-01-08 13:42:35 +09:00