The stx-init-env script sets up a minikube profile
and immediately starts building the container images.
Occasionally, the minikube network has not yet finished setup
when the build image step starts, which causes some URLs
to fail to resolve, most often deb.debian.org
While the root cause investigation is pending,
this commit adds a short pause in between these steps
to prevent the issue.
In addition, some extra logging is added for clarity.
Test Plan:
pass - ./stx-init-env --rebuild
Partial-Bug: 2093371
Change-Id: I4f5ce7849799b3fdd3a0bb4e64bab244b95cca94
Signed-off-by: Leonardo Fagundes Luz Serrano <Leonardo.FagundesLuzSerrano@windriver.com>
LAT docker file downloads the installer from a hard-coded URL that
points to https://mirror.starlingx.windriver.com . Allow users to
override this location by defining STX_MIRROR_URL in the host
environment. By default, guess mirror location from stx.conf.
TESTS
==========================
* Rebuild LAT container and make sure it uses the mirror URL from
stx.conf
* Set STX_MIRROR_URL prior to calling stx-init-env and make sure it
gets picked up by the docker file
Story: 2010055
Task: 49883
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: Id8ea88407f74003db934337efd574451658633d8
Avoid "minikube profile list" when checking whether the profile exists.
The list command attempts to connect to each profile and is quite slow.
Use "minikube status -p $MINIKUBENAME" instead.
Story: 2011038
Task: 49570
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: If799840d749de00af907de7867ec68fb9908afa3
* stx script:
- New command "stx control is-started" to complement start/stop
- New option "stx control {start,stop} --wait"
* stx-init-env:
- new option --reset: delete chroots + restart pods
- new option --reset-hard: stop pods, delete local workspaces,
chroots, aptly, docker & minikube profile
- rename option "--nuke" to "--delete-minikube-profile"; old spelling
is still accepted with a warning
- renamed & refactored some functions
* import-stx:
- new env var STX_RM_METHOD: may be optionally set to "docker" for
deleting root-owned files via "docker run", rather than "sudo"
TESTS
=========================
* Misc sanity checks using minikube & k8s
* Manually tested blacklist checks in safe_rm()
* rm via "sudo" vs "docker run"
* Using minikube:
- stx-init-env
- stx-init-env --rebuild
- stx start, build all packages, --reset, build all packages
- stx start, build all packages, --reset-hard, stx-init-env,
build all packages
Story: 2011038
Task: 49549
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: Ife4172ae9fa7b58332ac7ad65beb99525bc2a1a3
New option to skip (re-)starting the pods. This will allow us to better
control build stages in Jenkins.
TESTS
=====================
Run with and without the new option
Story: 2010226
Task: 49211
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: I367a6db9503a13d376966376a8b25cbedccd1f35
I fixed the issue with minikube_started() and helm_started() where the
grep command was not grepping the correct docker image and helm chart
due to matching substring. It was solved using helm ls command and the
option --kube-context when minikube eixist to pass minikube profile
and helm ls with the option --namespace when minikube is not exist.
TESTS
========================================
* setup a use profile names "MINIKUBENAME" Eg "stx" and "stx1"
* setup a similar environment to reproduce the issue
Closes-Bug: 1995400
Change-Id: Ic9b2224240e0e02be30957a96e7deb24da122136
Signed-off-by: Willy Njomen <willy.njomen@windriver.com>
Previous commit added a new option "--dockerhub-login/-L". Remove the
"-L" alias since we want to reserve it for future use.
TESTS
=======================
Run script with the new option and with/without --rebuild. Make sure
"docker login" is executed against the host docker daemon and minikube's
docker daemon.
Story: 2010055
Task: 48435
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: I06703f58e1598e35e53b12160743516ae7b20c14
New option -L/--dockerhub-login: authenticate local docker daemon and
minikube's docker daemon before building or downloading docker images.
This makes sure DockerHub's download rate limit is counted towards the
current user's DockerHub account.
TESTS
=======================
Run script with the new option and with/without --rebuild. Make sure
"docker login" is executed against the host docker daemon and minikube's
docker daemon.
Story: 2010055
Task: 48435
Change-Id: Ia87d54a383f164c36b2380b1683a56407b1f5974
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Stx-tool always uses "stx-init-env --rebuild" from the docker hub
master branch. This causes random problems with people trying to
set up an environment
I used "stx-init-env --rebuild" to make sure the images are built
locally, it does not make sense to always use the latest tag for a
release.
TESTS
========================================
* Start stx-tools containers
* Pull an image inside "stx shell"
* Stop stx-tools container
* Run "stx-init-env" without the "--rebuild"
* to make sure the images are
built locally
Story: 2010055
Task: 47970
Signed-off-by: Willy Njomen <willy.njomen@windriver.com>
Change-Id: Id466fbd243df1d77330473850fa4ba7d649941b7
New environment variables to set docker image tags for the images used
by the helm chart, as well as the images we download from Docker Hub.
When these new vars are missing, the scripts behave as before this
patch.
TESTS
============================
* Re-create the environment with/without the new env vars, with/without
"--rebuild"
* Make sure "stx control start/stop" works
* Make sure images created by stx-init-env use tags matching new env
vars
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: I66e061aef52fdb26c99303ecfbcd1fa9b43bc6bc
Add the checking for minikube_exists to avoid the KeyError error,
when the command './stx-init-env --nuke' is executed and the minikube
cluster doesn't exist.
Story: 2009897
Task: 45824
Test Plan:
Pass: export MINIKUBENAME=abc ......
'./stx-init-env --nuke' works without the KeyError exception.
Signed-off-by: Zhixiong Chi <zhixiong.chi@windriver.com>
Change-Id: I2cf46e01df312604ca2a0afa38a498e7397747b8
Changed the "--cache" option to skip pulling images that already exist
locally. Otherwise a locally-built image would be overwritten by a
downloaded one.
TESTS
===============================
Call the script with various permutations of --cache & --rebuild, with &
without deleting cached images prior.
Story: 2009897
Task: 45225
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: I37435d3d7a6d1a20b87794530cc937d759674685
This patch adds any missing keys to stx.conf while restarting pods via
stx-init-env. This should minimize disruption to developers when new
keys are added to the reference file, stx.conf.sample.
TESTS
========================
- Remove some required keys from stx.conf, then run stx-init-env and
make sure they get re-added
Story: 2009897
Task: 45179
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: Ifbed10e01dc2e5653b827274a556f6dad166f1d8
This patch allows stx-init-env to rebuild specific images, rather than
all.
TESTS
===============
Run the script with various --rebuild options:
* without --rebuild
* with --rebuild
* with --rebuild=builder,lat-sdk
* with --rebuild=builder,INVALID_IMAGE_NAME
Story: 2009897
Task: 45177
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: I96801d7f44842303d3b9e4374abcae677873d8a7
By default we use the "--no-cache" option when building containers. Add
a command-line option to avoid this, to make testing this script easier.
TESTS
=====================
Rebuild containers with and without the "--cache" option
Story: 2008846
Task: 45124
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: I79d61cc9a7606d6de65478ced5bab325f7264278
Extends the build tools to add support to Kubernetes. For kubernetes,
it supports single node cluster and host path for the volumes.
New environment variables:
- PROJECT: build project name
- STX_PLATFORM: minikube or kubernetes
- STX_BUILD_CPUS: replaces MINIKUBECPUS
- STX_K8S_NAMESPACE: kubernetes namespace name
* Default values are set to minikube, more details added in
import-stx.README.
To deploy stx on k8s you need to follow the below steps:
- create k8s namespace
- export PROJECT, included to support multiproject environments it is
used by the new default build home and also for docker tagging.
- export STX_BUILD_HOME, e.g: /localdisk/user/$PROJECT
- export STX_K8S_NAMESPACE="namespace_name"
- export STX_PLATFORM="kubernetes"
- export KUBECONFIG to your kubernetes config file
- STX_BUILD_CPUS replaces MINIKUBECPUS, this variable is used
by build-pkgs parallel jobs ($MAX_CPUS)
- Create your build home $STX_BUILD_HOME
- Init repo & repo sync
- source import-stx, check the env variables
- stx-init-env
- stx control status/start/stop/enter
Test Plan:
Pass: Create env on minikube
Pass: Create env on Kubernetes
Pass: Apply patch on current minikube env and continue to work on the
environment without issues
Pass: build package on Debian
Pass: build Debian image
Story: 2009812
Task: 44391
Signed-off-by: Luis Sampaio <luis.sampaio@windriver.com>
Change-Id: I7b760fbf1454f6aa90dd93dd9ff3a61d5fbd1b5c
Add the support to allow to define the cpu and memory resources
when we start the minikube.
Story: 2008862
Task: 44079
Signed-off-by: Zhixiong Chi <zhixiong.chi@windriver.com>
Change-Id: I4ca4f7b5bb6d6eb59ceafbb8114b2fd4a856cb0d
Add the '--clean' option for stx-init-env to support to refresh
the config file stx.conf, and to drop the configmap data and the
intermediate files.
Story: 2008862
Task: 44080
Signed-off-by: Zhixiong Chi <zhixiong.chi@windriver.com>
Change-Id: I53cdb50e7f3244940ec4ab26cd68e62aa15ba64f
Create the mirror directory before the minikube cluster is created
otherwise it will be automatically created with root permission.
Log the minikube varibales MINIKUBE_HOME, MINIKUBENAME and
STX_BUILD_HOME into minikube_history.log.
Story: 2008862
Task: 43862
Signed-off-by: Zhixiong Chi <zhixiong.chi@windriver.com>
Change-Id: I72c877cc171f49633d8b1802834cf7da48a9e7b5
- import-stx: allow users to override MINIKUBENAME
- stx-init-env: push VolumePath via the environment. This script appears
to have been designed to be sourced (it uses "return" statements
rather than "exit"), yet it attempts to process a command-line
argument ($1), which doesn't work in sourced files. Solution: define
docker mount point in import-stx as an environment variable.
Story: 2008846
Task: 43753
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: I489be2276bc2f0798affaf6c334614648f5922c2
Start the minikube env for stx tool, so that we can implement
the four containers then begin to execute the subtasks.
Story: 2008862
Task: 43583
Change-Id: I9941395ac2853f5edcbd306941d14f690477e9e7
Signed-off-by: Zhixiong Chi <zhixiong.chi@windriver.com>
Implement the stx control module so that the developer could
manage the four containers with the 'stx control xxx' command.
Containers: (stx-builder|stx-pkgbuilder|stx-lat-tool|stx-repomgr)
Now we assume that we have a minikube env to be ready for this
module, we can use it directly. For the minikube env installation
we can build it with the stx-init-env script later.
Now support the action: [status|start|stop|enter|upgrade]
stx control status: Check status of the containers
stx control start: Boot the containers up
stx control stop: Shutdown the containers
stx control enter: Login the container(default builder)
stx control upgrade: Upgrade the containers
This module will be used after the developer have a minikube env,
Please refer to the more help information with the command
'stx control --help'
Story: 2008862
Task: 42514
Signed-off-by: Zhixiong Chi <zhixiong.chi@windriver.com>
Change-Id: I0c25e0f784671e90c56d5d367b12a27c3f3fa7cf
Implement the stx config module so that the developer
could define the customized configuration before the
starlingx building.
Now support the action: show|add|get|unset
The usage of stx config module is like the git command.
Please refer to the more help information with the command
'stx config --help'
Story: 2008862
Task: 42515
Signed-off-by: Zhixiong Chi <zhixiong.chi@windriver.com>
Change-Id: I0310d1b591415f111e30efdf5bd76b555f725416