Readme for airship-utils
Along with the readme change the following small changes were made: - make it possible to install arbitrary packages with install_packages.sh script - add rsh and nis to list of default blacklisted packages to satisfy initial requirements. Change-Id: I84d81422e11f5de2d99fa30b1974513f62313386
This commit is contained in:
parent
1baf1269b0
commit
26f94f8a81
190
README.md
190
README.md
@ -1,65 +1,185 @@
|
|||||||
# docker-aptly
|
# Airship-utils
|
||||||
|
|
||||||
## Features
|
Airship-utils is a collection of tools that can accompany other airship-* projects.
|
||||||
|
Currently, airship-utils contains the following components:
|
||||||
|
|
||||||
- Packages are downloaded during the docker image build
|
- miniMirror
|
||||||
- GPG keys for signature may be generated during the docker image build or existing ones are used
|
|
||||||
- Nginx blacklist support at runtime
|
|
||||||
|
|
||||||
## Quickstart
|
## miniMirror
|
||||||
|
|
||||||
The main difference with the upstream repo is packages saved inside a docker image.
|
miniMirror is a combination of package mirroring tool (Aptly) and a web server (Nginx)
|
||||||
During the image building /opt/update_mirror_ubuntu.sh is called to create mirrors, update them,
|
packed into a container and a helm chart for k8s deployment. See [1] for additional info.
|
||||||
merge all in one snapshot and publish it. By default, a new GPG key is generated for making a signature for repo.
|
|
||||||
|
|
||||||
There are two modes: filtered build that fetches only packages specified in assets/packages and
|
### Features
|
||||||
unfiltered build that fetches all packages. The filtered build is used by default.
|
|
||||||
|
|
||||||
To fetch all packages the following command can be used:
|
- Packages are downloaded during the docker image build.
|
||||||
|
A list of packages can be specified with particular versions or
|
||||||
|
without them to get the current snapshots. Currently miniMirror
|
||||||
|
focuses on Ubuntu repositories only.
|
||||||
|
- GPG key for signature can be generated during the docker image build
|
||||||
|
or existing one can be used. To make a signature for packages Aptly
|
||||||
|
requires a private key, it is not possible to use a signature from
|
||||||
|
original mirror.
|
||||||
|
- Packages blacklist support at runtime. An additional Nginx
|
||||||
|
configuration can be provided to block specific package
|
||||||
|
installation. By default packages contains the following regexp in name are blocked:
|
||||||
|
- telnet
|
||||||
|
- ftp
|
||||||
|
- \brsh\b
|
||||||
|
- \bnis\b
|
||||||
|
|
||||||
|
### How to build miniMirror image?
|
||||||
|
|
||||||
|
#### General desription
|
||||||
|
|
||||||
|
As Debian packages are downloaded and saved inside a docker image, it
|
||||||
|
may take some time to build the image. The process of building the image
|
||||||
|
includes the following steps:
|
||||||
|
|
||||||
|
- Prepare GPG environment (see assets/startup.sh for details).
|
||||||
|
- Put into right places or generate GPG key depending on the build
|
||||||
|
configuration. See "Step by step guide" step 2 for details.
|
||||||
|
- Update GPG keyring.
|
||||||
|
- Create packages infrastructure (see assets/update_mirror_ubuntu.sh).
|
||||||
|
- Create Aptly mirrors.
|
||||||
|
- Fetch packages from upstream repositories according to the mirror
|
||||||
|
configurations.
|
||||||
|
- Merge repositories. For example, by default xenial, xenial-updates,
|
||||||
|
and xenial-security are used. Packages from each repository are
|
||||||
|
merged into one with latest wins strategy.
|
||||||
|
- Publish repository to directory Nginx will serve static files
|
||||||
|
from.
|
||||||
|
|
||||||
|
#### Configuration
|
||||||
|
|
||||||
|
The following build args are available:
|
||||||
|
|
||||||
|
Repository configuration:
|
||||||
|
|
||||||
|
- UPSTREAM_URL - a URL packages are downloaded from (mirror URL).
|
||||||
|
- UPSTREAM_KEY_URL - a URL for public GPG key if it is not on a default location.
|
||||||
|
- UBUNTU_RELEASE - a release name for a Ubuntu distribution.
|
||||||
|
- COMPONENTS - a list of repository components separated by space.
|
||||||
|
For example, values can be main, universe, restricted, multiverse [2].
|
||||||
|
- REPOS - a list of repository types separated by space.
|
||||||
|
For example xenial, xenial-updates, xenial-security, xenial-backports.
|
||||||
|
|
||||||
|
Packages configuration:
|
||||||
|
- MODE - a string determining if all packages should be downloaded or
|
||||||
|
specific only. Possible values: packages or all.
|
||||||
|
- PACKAGE_FILE - a file name where a list of packages is saved. If
|
||||||
|
MODE=packages the file must be available in assets/packages
|
||||||
|
directory.
|
||||||
|
|
||||||
|
GPG key configuration:
|
||||||
|
By default GPG key for making package signature is generated during
|
||||||
|
the build. If you have a GPG key already you can put private and
|
||||||
|
public keys in assets/gpg dir. Keys must have special names: aptly.sec
|
||||||
|
and aptly.pub. You may configure GPG key params via the following arguments:
|
||||||
|
- FULL_NAME - a full name for a GPG key.
|
||||||
|
- EMAIL_ADDRESS - an email for a GPG key.
|
||||||
|
- GPG_PASSWORD - a passphrase for a GPG key. This can be used both for
|
||||||
|
GPG key generation and GPG key usage.
|
||||||
|
|
||||||
|
Nginx configuration:
|
||||||
|
- HOSTNAME - server_name configuration for Nginx.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/urpylka/docker-aptly.git
|
git clone https://git.openstack.org/openstack/airship-utils
|
||||||
docker build docker-aptly --build-arg MODE=all
|
docker build airship-utils \
|
||||||
|
--UBUNTU_RELEASE=bionic \
|
||||||
|
--build-arg FULL_NAME="John Smith" \
|
||||||
|
--build-arg EMAIL_ADDRESS="john.smith@example.com" \
|
||||||
|
--build-arg GPG_PASSWORD="PickAPassword" \
|
||||||
|
--build-arg HOSTNAME=_
|
||||||
```
|
```
|
||||||
|
|
||||||
By default GPG key for making package signature are generated during the build.
|
### Step by step guide
|
||||||
You may configure GPG key params via build arguments: FULL_NAME, EMAIL_ADDRESS, and GPG_PASSWORD, like:
|
|
||||||
|
This is an example of how miniMirror can be used.
|
||||||
|
|
||||||
|
1) Prepare a list of packages needed for a miniMirror image.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker build docker-aptly \
|
cd airship-utils
|
||||||
--build-arg FULL_NAME="First Last" \
|
cat << 'EOF' > assets/packages/my_packages
|
||||||
--build-arg EMAIL_ADDRESS="youremail@example.com" \
|
mysql-client-5.7 (= 5.7.24-0ubuntu0.16.04.1)
|
||||||
--build-arg GPG_PASSWORD="PickAPassword"
|
mysql-client-core-5.7
|
||||||
|
postgresql-client-9.5 (= 9.5.14-0ubuntu0.16.04)
|
||||||
|
postgresql-client-common
|
||||||
|
EOF
|
||||||
```
|
```
|
||||||
|
|
||||||
If you have a GPG key already you can put private and public key in assets/gpg dir.
|
2) Prepare a GPG key for making package signature.
|
||||||
Keys must have special names: aptly.sec and aptly.pub
|
|
||||||
For example:
|
GPG public and private keys should be named as assets/gpg/aptly.pub and assets/gpg/aptly.key.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cp <my private key> docker-aptly/assets/gpg/aptly.sec
|
mkdir -p /opt/aptly
|
||||||
cp <my public key> docker-aptly/assets/gpg/aptly.pub
|
export FULL_NAME='John Smith'
|
||||||
|
export EMAIL_ADDRESS='john.smith@example.com'
|
||||||
docker build docker-aptly \
|
export GPG_PASSWORD='my_passphrase'
|
||||||
--build-arg GPG_PASSWORD="GPG passphrase for my private key"
|
bash assets/gpg_batch.sh
|
||||||
|
gpg -v --batch --gen-key /opt/gpg_batch
|
||||||
|
mv /opt/aptly/* assets/gpg/
|
||||||
|
rm /opt/gpg_batch
|
||||||
```
|
```
|
||||||
|
|
||||||
To use the Nginx blacklist feature a volume with Nginx config has to be mounted at runtime.
|
3) Build docker image.
|
||||||
By default, the following keywords are blocked: telnet, ftp.
|
|
||||||
If no volume is mounted then no blacklist will be used.
|
```bash
|
||||||
|
docker build . -t mini-mirror \
|
||||||
|
--build-arg PACKAGE_FILE=my_packages \
|
||||||
|
--build-arg GPG_PASSWORD="$GPG_PASSWORD"
|
||||||
|
```
|
||||||
|
|
||||||
|
4) Test miniMirror container.
|
||||||
|
|
||||||
|
Start miniMirror container.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run -d \
|
||||||
|
--publish 8080:80 \
|
||||||
|
--volume $(pwd)/assets/nginx:/opt/nginx \
|
||||||
|
--name mini-mirror \
|
||||||
|
mini-mirror
|
||||||
|
```
|
||||||
|
|
||||||
|
Run another container and install packages there.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run --network host \
|
||||||
|
--env PACKAGES='mysql-client-5.7 postgresql-client-9.5' \
|
||||||
|
--name target \
|
||||||
|
--volume $(pwd)/tools:/opt \
|
||||||
|
ubuntu:16.04 /opt/install_packages.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
### How to blacklist miniMirror packages
|
||||||
|
|
||||||
|
To use the Nginx blacklist feature a volume with Nginx config has to
|
||||||
|
be mounted at runtime. If no volume is mounted then no blacklist will
|
||||||
|
be used.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker run \
|
docker run \
|
||||||
--name aptly \
|
--name mini-mirror \
|
||||||
--detach \
|
--detach \
|
||||||
--publish 8080:80 \
|
--publish 8080:80 \
|
||||||
--volume $(pwd)/assets/nginx:/opt/nginx \
|
--volume $(pwd)/assets/nginx:/opt/nginx \
|
||||||
aptly:test
|
mini-mirror
|
||||||
```
|
```
|
||||||
___
|
|
||||||
|
|
||||||
For additional docs see https://github.com/amadev/docker-aptly
|
## References
|
||||||
|
|
||||||
|
* [1] https://review.openstack.org/#/c/611376/
|
||||||
|
* [2] https://help.ubuntu.com/community/Repositories
|
||||||
|
|
||||||
|
## Copyright
|
||||||
|
|
||||||
|
* Copyright 2018 AT&T Intellectual Property
|
||||||
* Copyright 2018 Artem B. Smirnov
|
* Copyright 2018 Artem B. Smirnov
|
||||||
* Copyright 2016 Bryan J. Hong
|
* Copyright 2016 Bryan J. Hong
|
||||||
* Licensed under the Apache License, Version 2.0
|
* Licensed under the Apache License, Version 2.0
|
||||||
|
@ -1,3 +1,3 @@
|
|||||||
location ~ (telnet|ftp) {
|
location ~ (telnet|ftp|\brsh\b|\bnis\b) {
|
||||||
return 404;
|
return 404;
|
||||||
}
|
}
|
||||||
|
@ -9,5 +9,8 @@ apt install -y curl
|
|||||||
curl -s localhost:8080/aptly_repo_signing.key | apt-key add -
|
curl -s localhost:8080/aptly_repo_signing.key | apt-key add -
|
||||||
echo 'deb http://localhost:8889 xenial main' > /etc/apt/sources.list
|
echo 'deb http://localhost:8889 xenial main' > /etc/apt/sources.list
|
||||||
apt-get update
|
apt-get update
|
||||||
apt-cache policy accountsservice
|
PACKAGES=${PACKAGES:-accountsservice}
|
||||||
apt-get install -y accountsservice
|
for package in $PACKAGES; do
|
||||||
|
apt-cache policy "$package"
|
||||||
|
apt-get install -y "$package"
|
||||||
|
done
|
||||||
|
Loading…
Reference in New Issue
Block a user