SeaweedFS for StarlingX

This repository provides the SeaweedFS file server integration for StarlingX, including:

• Helm charts to deploy SeaweedFS and its operator
• A StarlingX application plugin (python3-k8sapp-seaweedfs) to register and manage the charts
• A StarlingX application Helm package (stx-seaweedfs-helm) with FluxCD manifests
• A Keystone-auth component (stx-seaweedfs-auth) used by the file server proxy to enforce X-Auth-Token

---

Components

SeaweedFS file server (Filer) and proxy — seaweedfs-extras-helm

helm-charts/custom/seaweedfs-extras-helm/seaweedfs-extras-helm/seaweedfs-extras-helm/

This chart deploys the SeaweedFS resources needed to provide the file server functionality:

• SeaweedFS masters (cluster coordination)
• SeaweedFS filer (the file server API and metadata)
• SeaweedFS volumes (data storage)
• A Filer Proxy service that front-ends the filer

The Filer Proxy pattern:

• NGINX receives client requests
• stx-seaweedfs-auth validates X-Auth-Token against Keystone
• If allowed, the request is proxied to the SeaweedFS filer service

Client-facing service:

• http://seaweed-filer-proxy.seaweedfs.svc.cluster.local:8888

Direct filer service (internal use/debug):

• seaweed-filer (ClusterIP)

Operator — seaweedfs-operator-helm

helm-charts/upstream/seaweedfs-operator-helm/

Installs the SeaweedFS operator and its webhook components so that SeaweedFS custom resources can be reconciled.

Typical runtime outcome:

• seaweedfs-operator deployment running
• webhook service available
• webhook cert patch job/serviceaccount present

StarlingX application plugin — python3-k8sapp-seaweedfs

python3-k8sapp-seaweedfs/

Registers and manages the SeaweedFS charts within the StarlingX application framework:

• charts registration (operator + extras)
• lifecycle operations (install/upgrade/remove)
• overrides handling
• kustomize integration as needed by the platform

Application Helm package + FluxCD manifests — stx-seaweedfs-helm

stx-seaweedfs-helm/

Delivers the application payload used by the platform deployment tooling:

• FluxCD base objects (namespace, HelmRepository, Kustomization)
• HelmRelease definitions for:
  • seaweedfs-operator
  • seaweedfs-extras
• Static and system overrides YAMLs

Keystone auth image — stx-seaweedfs-auth (image seaweedfs-filer-auth)

stx-seaweedfs-helm/docker/stx-seaweedfs-auth/

This is the Keystone-aware component used by the Filer Proxy to authenticate requests.

High-level behavior:

• Receives auth checks from NGINX (sidecar/auth-request pattern)
• Validates X-Auth-Token against Keystone
• Returns allow/deny to gate access to filer endpoints

Image reference used in-cluster:

• registry.local:9001/starlingx/seaweedfs-filer-auth:1.0

---

Build and publish stx-seaweedfs-auth

Build the image and packages

This app is not part of the build, since its installation in /usr/local/share/applications/helm is commented out. Uncomment the relevant lines in stx-seaweedfs-helm/debian/all/deb_folder/rules and stx-seaweedfs-helm/debian/all/deb_folder/stx-seaweedfs-helm.install to install the tarball in the applications directory.

docker pull starlingx/stx-debian:master-stable-latest

./build-stx-images.sh \
  --base starlingx/stx-debian:master-stable-latest \
  --no-pull-base \
  --only stx-seaweedfs-auth \
  --cache

Export, load, and push to the local registry

docker save -o swfs-auth.tar registry.local:9001/starlingx/seaweedfs-filer-auth:1.0

On the target controller:

sudo docker load -i swfs-auth.tar
sudo docker push registry.local:9001/starlingx/seaweedfs-filer-auth:1.0

Mirror required images into registry.local:9001

sudo docker login registry.local:9001 -u admin

# TODO (ecandotti): remove when the images are added to bootstrap
imgs=(
  docker.io/chrislusf/seaweedfs-operator:1.0.5
  docker.io/chrislusf/seaweedfs:3.97
  docker.io/nginx:1.27-alpine
)

for i in "${imgs[@]}"; do
  sudo docker pull "$i"
  sudo docker tag  "$i" "registry.local:9001/$i"
  sudo docker push "registry.local:9001/$i"
done

Validation

Check workload status

kubectl -n seaweedfs get pods,pv,pvc,sc,svc,endpoints

Example expected resources:

NAME                                       READY   STATUS    RESTARTS       AGE
pod/seaweed-filer-0                        1/1     Running   2 (6h7m ago)   6h7m
pod/seaweed-filer-proxy-767fd9b65d-tlfn4   2/2     Running   4 (6h7m ago)   6h7m
pod/seaweed-master-0                       1/1     Running   2 (6h7m ago)   6h7m
pod/seaweed-master-1                       1/1     Running   1 (6h7m ago)   6h7m
pod/seaweed-master-2                       1/1     Running   1 (6h7m ago)   6h7m
pod/seaweed-volume-0                       1/1     Running   2 (6h6m ago)   6h7m
pod/seaweedfs-operator-75f849f89f-rdx9m    1/1     Running   0              6h18m

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                               STORAGECLASS   VOLUMEATTRIBUTESCLASS   REASON   AGE
persistentvolume/pvc-326b3cb9-104c-40c5-b677-8f1fc8e2bff4   10Gi       RWO            Delete           Bound    seaweedfs/mount0-seaweed-volume-0   general        <unset>                          6h7m

NAME                                            STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
persistentvolumeclaim/mount0-seaweed-volume-0   Bound    pvc-326b3cb9-104c-40c5-b677-8f1fc8e2bff4   10Gi       RWO            general        <unset>                 6h7m

NAME                                            PROVISIONER           RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
storageclass.storage.k8s.io/cephfs              cephfs.csi.ceph.com   Delete          Immediate           true                   21d
storageclass.storage.k8s.io/general (default)   rbd.csi.ceph.com      Delete          Immediate           true                   21d

NAME                                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                       AGE
service/seaweed-filer                ClusterIP   aefd:207::710    <none>        8888/TCP,18888/TCP,8333/TCP   6h7m
service/seaweed-filer-peer           ClusterIP   None             <none>        8888/TCP,18888/TCP,8333/TCP   6h7m
service/seaweed-filer-proxy          ClusterIP   aefd:207::d7e0   <none>        8888/TCP                      6h7m
service/seaweed-master               ClusterIP   aefd:207::125e   <none>        9333/TCP,19333/TCP            6h7m
service/seaweed-master-peer          ClusterIP   None             <none>        9333/TCP,19333/TCP            6h7m
service/seaweed-volume-0             ClusterIP   aefd:207::6225   <none>        8444/TCP,18444/TCP            6h7m
service/seaweed-volume-peer          ClusterIP   None             <none>        8444/TCP,18444/TCP            6h7m
service/seaweedfs-operator           ClusterIP   aefd:207::447e   <none>        8080/TCP                      6h20m
service/seaweedfs-operator-webhook   ClusterIP   aefd:207::cf0d   <none>        443/TCP                       6h20m

NAME                                   ENDPOINTS
endpoints/seaweed-filer                [aefd:206::a4ce:fec1:5423:e316]:8888,[aefd:206::a4ce:fec1:5423:e316]:8333,[aefd:206::a4ce:fec1:5423:e316]:18888
endpoints/seaweed-filer-peer           [aefd:206::a4ce:fec1:5423:e316]:8888,[aefd:206::a4ce:fec1:5423:e316]:8333,[aefd:206::a4ce:fec1:5423:e316]:18888
endpoints/seaweed-filer-proxy          [aefd:206::8e22:765f:6121:eb74]:8080
endpoints/seaweed-master               [aefd:206::8e22:765f:6121:eb75]:19333,[aefd:206::a4ce:fec1:5423:e315]:19333,[aefd:206::a4ce:fec1:5423:e317]:19333
endpoints/seaweed-master-peer          [aefd:206::8e22:765f:6121:eb75]:19333,[aefd:206::a4ce:fec1:5423:e315]:19333,[aefd:206::a4ce:fec1:5423:e317]:19333
endpoints/seaweed-volume-0             [aefd:206::a4ce:fec1:5423:e318]:8444,[aefd:206::a4ce:fec1:5423:e318]:18444
endpoints/seaweed-volume-peer          [aefd:206::a4ce:fec1:5423:e318]:8444,[aefd:206::a4ce:fec1:5423:e318]:18444
endpoints/seaweedfs-operator           [aefd:206::8e22:765f:6121:eb70]:8080
endpoints/seaweedfs-operator-webhook   [aefd:206::8e22:765f:6121:eb70]:9443

Authenticated upload through the Filer Proxy

source /etc/platform/openrc
TOKEN=$(openstack token issue -f value -c id)

curl -v -fSs \
  -H "X-Auth-Token: $TOKEN" \
  -X PUT --upload-file test-file \
  "http://seaweed-filer-proxy.seaweedfs.svc.cluster.local:8888/backups/fake-subcloud99999/26.03/test-file"

Authenticated download and integrity check

source /etc/platform/openrc
TOKEN=$(openstack token issue -f value -c id)
OUT=/tmp/test-file.bin

curl -v -fSs \
  -H "X-Auth-Token: $TOKEN" \
  "http://seaweed-filer-proxy.seaweedfs.svc.cluster.local:8888/backups/fake-subcloud99999/26.03/test-file" \
  -o "$OUT"

md5sum /tmp/test-file.bin
md5sum test-file

Expected result: both MD5 hashes match.

Troubleshooting

• ImagePullBackOff

  • Confirm the image exists in registry.local:9001/...
  • Confirm secret default-registry-key exists in seaweedfs
  • Confirm serviceaccounts used by deployments/jobs reference imagePullSecrets

• Operator webhook/cert patch job failures

  • Confirm the job’s serviceaccount can pull images
  • Confirm required RBAC and namespace objects exist