Commit Graph

50 Commits (214f0e8d5ba24996193739a8ad1ce7268884a4b2)

Author SHA1 Message Date
Pete Vander Giessen c123af21d3 Strict confinement related fixes
(Not complete strict confinement, but these don't break anything
devmode related, and get us closer to having strict confinement

Added more needed interfaces to snapcraft.yaml.

Created a wrapper around dnsmasq so that we can run as the snap_daemon
user. Added snap_daemon user to snapcraft.yaml.

Added a utility script for connecting interfaces that don't auto
connect (tools/ Not useful for production, but saves a lot
of time when testing.

libvirt no longer uses unix sock group "sudo" (can't run setguid in
strict confinement).

Got rid of "find_missing_plugins" in init script. By the time we
release strict confinement to production, all those plugins will auto

Change-Id: I8324ac7bd0332c41cac17703eb15d7301e7babf3
2020-04-02 17:31:54 +00:00
Pete Vander Giessen 5611bc9759 Strict confinement (devmode)
Make MicroStack strictly confined, albeit in devmode for now.

Addresses unpredictable breakages with apt package upgrades in eoan
and focal, and sets the stage for a better isolated, less fragile snap
going forward.

We now use layouts to handle libvirt and qemu setting paths at compile
time. This is cleaner than the organize hack.

Moved away from calls to systemctl in init, as a strictly confined
snap cannot call systemctl on a non snappy system.

Disabled call to sysctl to set ipv4_fowarding, as we don't have access
to sysctl in a strictly confined snap. This may break some users, and
we need to figure out a way to address the breakage.

Got rid of routine, moving rabbitmq setup into a
bash script instead (it's just cleaner).

Moved keypair creation into launch script, as it's difficult to do
sensible things with keypair creation in the init script, which is
running using sudo, and therefore doesn't have access to

Added (but commented out) code that will check to verify that plugs
are connected before running microstack.init or ovs-vsctl. This code
may go away entirely, as we plan on auto connecting all of our
interfaces, and don't technically need to guard against not having
them connected.

Added temporary local upper-constraints file, to fix an issue where
upstream upper-constraints was breaking pip install by setting a
neutron version. This needs a better long term fix, but works for now.

Closes-bug: 1860660
Change-Id: Iaf1f1482609f05285ed9061317b32e90bffd2da0
2020-03-05 09:31:15 +00:00
Pete Vander Giessen d262cbed49 Revert "Initial DPDK support"
This reverts commit ce5e82e319.

MicroStack cannot currently install due to a missing ovs-related
library. This is possibly due to recent changes in spacraft, or
possibly due to the workarounds for those changes. Regardless, it
appears that backing out the DPDK changes gets us back to a state
where we can install.

Partial-Bug: 1862911
Change-Id: I060c1a0095470639f9158cb9e9ebe8281a65a678
2020-02-12 20:10:59 +00:00
Nikolay Vinogradov 9019ad3529 Added Filebeat, NRPE and Telegraf to Microstack
- Snapped binary packages of Filebeat, NRPE and Telegraf (disabled by default)
- Added W/A of Telegraf segfault after ELF patching by snapcraft
- Implemented IPMI input tuning for Telegraf
- Allowed to run NRPE as root:root (from custom PPA)
- Implemented Filebeat, NRPE and Telegraf control scripts and config on top of snap-overlay
- Added support for checking Microstack systemd services by NRPE
- Added few generic and Microstack-specific NRPE checks
- Added possibility to override default config paths for the daemons
- Added support for in-band IPMI input to Telegraf
- Stick LMA wrappers and services naming to Microstack conventions
- Increase build timeout in .zuul conf by 30min

Change-Id: I68dbdb11248cf0c1e22e9333af3cf0f88954f557
2020-02-11 14:33:26 +00:00
Zuul be4e35ab29 Merge "Initial DPDK support" 2020-02-11 01:43:32 +00:00
Pete Vander Giessen a89f5574c3 Added microstack.remove command
Running microstack.remove will remove the br-ex virtual bridge device,
then uninstall MicroStack.

We do this because we can't use ovs-ctl to remove the bridge as part
of a remove hook, as the Open vSwitch daemons are not running at that
point. The microstack.remove command gives operators a way to cleanly
uninstall the snap, without needing to reboot to get rid of br-ex.

Added test exercising the code to

Rerranged entry points a bit (moved some things into to make
code sharing easier, and to prevent a proliferation of entry point
scripts in our root dir.

Change-Id: I9ff25864cd96ada3a9b3da8992c2b33955eff0b4
Closes-Bug: #1852147
2019-12-17 17:11:32 +00:00
Gabor Meszaros ce5e82e319 Initial DPDK support
Adding initial support for OVS DPDK.

Signed-off-by: Gabor Meszaros <>
Change-Id: I1f87188bd8b4b928108674a35e9a9c309e9f6d17
2019-12-09 16:45:11 +01:00
Pete Vander Giessen 1a25e50a17 Made horizon, rabbit and mysql ports configurable
Addresses requests to make it easier to avoid conflicts between the
Horizon dashboard and http services that might already be running on
the machine.

Configurable via snap config. Exposing via arguments to .init and
testing post init configuration is left for a separate PR.

Eventually, these may move to non standard ports by default. This PR
sets the stage for that, but further discussion is needed before we
decide whether to implement.

(This commit also contains a sneaky fix for the username display at the
end of the launch script.)

Closes-Bug: 1814829
Change-Id: If728d6ec8024bca4d3e809637fbdcc03ed4e6934
2019-11-27 15:27:54 +00:00
Pete Vander Giessen 2915caf98e Fixed dashboard_allowed_hosts config setting.
Now happens in a template, just like all the other values, which fixes
an issue where it doesn't get overridden during an upgrade.

Change-Id: Ied84ddc0282c77de6797f90efc8923ae66a9d59e
2019-11-21 08:47:04 +01:00
Pete Vander Giessen 590a7bc3d5 Refresh now sets missing config values
Broke default config settings into a separate script, which the
install hook always called, and post-refresh calls if it can't find
the new config hierarchy.

This fixes issues w/ upgrading from beta to edge.

Change-Id: I6b11109c2a2f6aca142a18c9fa274b332891d5c8
2019-11-21 08:46:41 +01:00
Pete Vander Giessen f7d2064561 set-br-ex now idempotent
When refreshing a snap, set-br-ex could fail because the iptables
rules that it sets were already setup. We now exit zero if this is
true, which prevents us from breaking on upgrades.

Change-Id: Ibfee98cabfa3e35bf53dbd191de2cf46f3709a51
2019-11-21 08:46:15 +01:00
Pete Vander Giessen 5404a261aa Clustering prototype
This enables basic clustering functionality. We add:

tools/cluster/cluster/ A server that handles validation of
cluster passwords.

tools/cluster/cluster/ A client for this server.

Important Note: This prototype does not support TLS, and the
functionality in the client and server is basic. Before we roll
clustering out to production, we need to have those two chat over TLS,
and be much more careful about verifying credentials.

Also included ...

Various fixes and changes to the init script and config templates to
support cluster configuration, and allow for the fact that we may have
endpoint references for two network ips.

Updates to snapcraft.yaml, adding the new tooling.

A more formalized config infrastructure. It's still a TODO to move the
specification out of the implicit definition in the install hook, and
into a nice, explicit, well documented yaml file.

Added nesting to the Question classes in the init script, as well as
strings pointing at config keys, rather than having the config be
implicitly indicated by the Question subclass' name. (This allows us
to put together a config spec that doesn't require the person reading
the spec to understand what Questions are, and how they are

Renamed and unified the "unit" and "lint" tox environments, to allow
for the multiple Python tools that we want to lint and test.

Added hooks in the init script to make it possible to do automated
testing, and added an automated test for a cluster. Run with "tox -e

Added cirros image to snap, to work around sporadic issues downloading
it from

Removed ping logic from snap, to workaround failures in gate. Need to
add it back in once we fix them.

Change-Id: I44ccd16168a7ed41486464df8c9e22a14d71ccfd
2019-11-04 13:03:41 +00:00
Pete Vander Giessen 0399955cf1 Ported and updated launch script
Moved security rules and keypair creation into init first.

Launch script now takes image name as positional argument, and name of
instance as a named argument. This makes it work more like launch in
other Canonical tools.

Written in Python, for ease of maintenance.

--retry and --wait args allow it to behave like tests expect it to,
while humans will get a much more intuitive (and much less noisy)

Also increased time we wait for a ping on the host, to allow for
slower, pure qemu, emulation times, and bring it in line with what
Tempest does in similar situations.

Change-Id: I11dcc098012468e9c88dcc7af78cde6920f31ecd
2019-10-23 01:09:42 +00:00
Pete Vander Giessen dfd1d5ec68 Added testing for control nodes
Ported to, and folded in

Made a testing framework for shared components.


Got rid of default .stestr.conf, as we're going to have multiple tests
running, and one conf is confusing.

Manually ordering functional tests for now, as stestr noms too much
output, and runs things in parallel, which doesn't work for our
functional tests.

Skipping compute node test for now, as it won't work until we can
connect to a control node with databases and such.

Moved to tools/ It's really
more of a tool for manual testing than an automated test.

Added test-requirements and updated gitignore.

Moved auto-detection of kvm extensions to init, rather than test, as
it makes more sense there.

Change-Id: Iba7f7fe07cbb066790f802cf2a7c87c68994062c
2019-10-16 15:44:38 +00:00
Pete Vander Giessen 7525ebcded Enable loading and saving of question answers.
This lays the groundwork for interactive init, as well as being able
to specify control and compute nodes.

Added preliminary config lists for control and compute nodes. Added
appropriate default snapctl config settings in install script.

Also changed "binary" questions to "boolean" questions, as that's
better wording, and it means that my docstrings are not a confusing
mix of "boolean" and "binary" when I forget which term I used.

Snuck in a fix for the "basic" testing environment -- it was missing
the Python requirements, and was therefore failing!

Change-Id: I7f95ab68f924fa4d4280703c372b807cc7c77758
2019-10-10 15:14:00 +00:00
Pete Vander Giessen b4f90c6eca Make logging better
Move logging output for most services to systemd.

Add a hook in snap.openstack to tell OpenStack services to wait
until we set database.ready in the snap config before starting. This
prevents spamming systemd with error messages before we run
microstack.init (See matching PR against snap.openstack, coming soon.)

Incidentally fix issue w/ the way that was running
CalledProcessError and parsing output.

Order patches part after uca-sources, to work around an issue we
discovered with apt update while those two parts are running in
parallel. (python-apt segfaults, and no fun is had by anyone.)

Remaining gaps in our logging: systemd still displays some errors
during init, which might be fixable with further ordering of snapctl
start invocations. We're also relying on MySQL and RabbitMQ log output
to know when those services are started, so we haven't moved their
output to systemd just yet.

Dropped in a fix to work w/ updated version of snap.openstack.

Change-Id: I130ed730c14ab35b8b677b9c9f573fa6fe1e8f13
2019-10-08 13:13:23 +00:00
Pete Vander Giessen 73d39dc8ce Make Horizon Dashboard settings operator overrideable.
We want to allow operators to override Horizon's default
settings. This involves moving local_settings.d out of the read only
snap filesystem, and into $SNAP_COMMON. This is a little bit tricky.

First, we patch and as we're building the
snap, to include a LOCAL_PATH in $SNAP_COMMON.

Then, we add a template with the rest of our default overrides,
and write it out to $SNAP_COMMON/horizon/local_settings.d

Finally we tweak our tests so that we can give our overrides a
spin. As a bonus, this makes a lot easier to run
in our multipass testing scenario!

`tox -e basic` now also runs selenium tests, as well.

Change-Id: Ic0ce18cfa1b97a93191da749095d8aa2270d5aeb
2019-09-24 18:27:39 +00:00
Pete Vander Giessen 0b4c7a22a3 Fix horizon dashboard
Port the python2.7 local settings overrides to the python3.6 directory

Move all overrides into as part
of troubleshooting some remaining problems. Everything is more
organized and functional now :-)

Added selenium tests.

Change-Id: I54923e1dc9c7ffa47c2ef6fb90ea9d224b0d2eee
2019-09-16 13:25:39 +00:00
Pete Vander Giessen 93f412fc93 Better init script
Written in Python. Easier to maintain. Easier to make interactive.

Change-Id: Ib579b43c1564b55165de5c2f3d20387122448b19
2019-08-23 16:30:31 +00:00
Pete Vander Giessen 92a6ea8dc7 Broke configuration out into a manual step.
Renamed the old and outdated "configure-openstack" script to ""

Updated and folded most of the configure hook into it.

Removed database installation step from install hook.

We can now install microstack without a database dump, which helps
immensely in updating. And we have a logical place to put additional
configuraiton, including some of the manual steps in, which
could be scripted if we gave users a chance to skip the system changes
that they wanted to skip.

Also updated README and DEMO file to match new flow. Updated test

Future cleanup and features documented in Trello, but not included in
this PR, which is big enough already :-)

Change-Id: I8d926a8b463124494ddb7a4696adbe86f89db7d5
2019-08-01 21:12:05 +00:00
Pete Vander Giessen 95efd789b4 Added snap configuration.
Moved openstack env variables out of snapcraft.yaml, and into a

Made openstack-wrapper, which auto sources microstack.rc.

Key off of revised snap.openstack, which supports snapctl config
values in templates. Moved a lot of the overlay into
snap-overlay/templates, and added config values to them. Also writes
templates just once, when you run snap-openstack setup.

Change-Id: Ib0f2e3fc97f491d9ed9dfbafc61dc8e27a8a2b48
2019-07-25 16:37:51 +00:00
Pete Vander Giessen 8ea5dc8679 Added automated testing via tox and zuul.
Tweaked tests/ functional test so that it no longer
requires multpass, and can run without kvm cpu extensions being
enabled (not all machines in the game have cpu extensions.)

Added tox.ini, wrapper script for building and installation, etc.

Change-Id: I968116dd7bec412a55813c896d60cfc86c7070db
2019-07-24 08:04:38 +01:00
Pete Vander Giessen 7b4e61201b
Cruft cleanup. (#75)
* Cruft cleanup.

Organized snapcraft.yaml better, and got rid of unecessary bits.

* Cleaned up scripts.

Moved into snap-overlay, to be more consistent with
everything else.

Got rid of outdated and confusing configure-the-things script.

* Clarified a comment.

* Got rid of erroneous config file.

* Undeleted config file -- wasn't erroneous!
2019-06-25 13:45:00 -04:00
Pete Vander Giessen f45bfd913d
Refactor snap to work with core18. (#73)
Refactor snap to work with core18.

Giving the snapcraft.yaml a base property helps tremendously with the
efficiency of the build process, and I believe that it puts us in a
better position to reliably support non Ubuntu distros going forward.

This also bases us on long supported bionic libraries, and gives us a
nice place to work from as we add Python 3 and Stein support, as well
as general polish and fixes.
2019-06-25 11:31:42 -04:00
Ryan Beisner dfeb664207
Merge pull request #63 from CanonicalLtd/bug/1823016-default-dns
Added default dns server for microstack guests.
2019-04-11 15:08:25 -05:00
Pete Vander Giessen dfdd85ecb4 Added newline at end of dhcp_agent.ini 2019-04-10 14:57:44 -04:00
Pete Vander Giessen d8c34fa1dc Added default dns server for microstack guests. 2019-04-10 14:56:30 -04:00
Pete Vander Giessen dcf6dd98d8 Switched back to kvm and host-passthrough vm defaults. This will break Virtualbox, but it will make machines on bare metal and kvm setups run a lot, lot faster. 2019-04-10 14:21:05 -04:00
Pete Vander Giessen 56cd24f5d0 Moved from database dump to hack.
Dropped a command to change the endpoints from localhost to
in the configure hook.

This is a temporary solution, pending automation of the database
update. (I was burning too much time getting a manual dump to work for
2019-04-10 14:12:01 -04:00
Pete Vander Giessen 369abc1f6f Updated mysql tarball. 2019-04-08 15:10:18 -04:00
Pete Vander Giessen 34f962dd97 Added updated mysql tarball, and updated tests.
Updated the mysql tarball, which has the localhost ->

Test for pinging Internet was failing due to not waiting long enough,
so added a proper retry strategy.
2019-04-05 17:10:41 -04:00
Pete Vander Giessen 6eaf44bbfa Udpated microstack api endpoints from localhost to
Addresses the issue where services such as a juju controller cannot
access endpoints from within an instance.

Updated all references to localhost to, and added the
address to ALLOWED_HOSTS.

Also updated version of qemu lib.
2019-04-04 10:52:18 -04:00
Pete Vander Giessen fdd341824c
Added iptables rules and sysctl vars for internet access (#57)
Fixes lp#1812415

Prior to this fix, instances spun up by microstack could not
successfully route to and from the Internet. Setting a rule for ipv4
forwarding and iptables rules for the nat fix the issue.
2019-03-27 12:38:52 -04:00
Nicolas Pochet 4e4d802b43 Add ubuntu theme (#55) 2019-02-05 10:07:27 -05:00
Pete Vander Giessen 8bcda8bdd3
Disabled themes. (#52)
This is a bandaid for, where switching
to the material theme can get your GUI stuck in an errored
state. We'll re-enable everything once that bug is fixed.
2019-01-09 16:26:33 -05:00
Pete Vander Giessen c563717f6e Switched to qemu and host-model libvirt settings.
These settings are less performant, but more universally compatible.

A seperate task would be to expose these settings via the snap config,
in some sensible way. For now, this means that microstack will "just
work" in more places.
2018-12-12 22:08:59 +00:00
Pete Vander Giessen 597ffc99d8 Perist external bridge over reboots.
This fixes an issue where we weren't creating a netplan config and/or
and entry in /etc/network/interfaces for br-ex, and thus losing
external access to our virtual networks after reboot.

Since we don't actually want to touch the host system's networking
config, we just drop a oneshot daemon into place that sets br-ex up
each time the snap services are started.
2018-12-05 21:25:11 +00:00
James Page 0c955d6e2c Mysql quick init (#26)
* Preseed mysql data dir with something useful

* Configure fernet keys on first install

* Update overlay

* Ensure image is synced
2018-11-11 16:29:27 -05:00
James Page 6e422029a2 Switch region name to microstack 2018-11-08 09:41:49 +00:00
Pete Vander Giessen 320a522d73 Revert speedup merge (couldn't launch instances) 2018-11-08 03:14:37 +00:00
Pete Vander Giessen fb04856e71
Mysql quick init (#24)
* Preseed mysql data dir with something useful

* Configure fernet keys on first install

* Update missing instances directory

* Updated mysql.tar.xz

* Folded configure command into configure hook.
2018-11-07 21:02:56 -05:00
Pete Vander Giessen 8c1d0d3f78 Fixed pathing issue w/ horizon dashboard. 2018-11-07 22:58:37 +00:00
Pete Vander Giessen c686b13d8f
Fix horizon (#23)
Fixed dashboard errors.

Added a proper uwsgi script with matching .ini templates.

Added django settings files that fix issues with Django attempting to write files to the read only snap filesystem, and generally make things work well in the snap.

Fixed snap-openstack.yaml entries so that our file and directory setup is accurate.
2018-11-07 16:30:20 -05:00
James Page 65b3cc6531 Fixup instances path creation 2018-11-06 18:00:16 +00:00
James Page 5f2d12c306 Add router and external network configuration 2018-11-06 12:03:32 +00:00
James Page 58b4650cf4 Split service configuration from configure hook 2018-11-06 10:01:54 +00:00
James Page 2e63e927b1 Drop use of deprecated nova.neutron options 2018-11-05 18:01:11 +00:00
James Page e1091643c8 Configure for metadata proxy access 2018-11-05 18:00:10 +00:00
James Page de7792812e Drop use of configure hook for now 2018-11-05 12:59:27 +00:00
James Page 0f590885e4 Major restructure 2018-11-02 10:55:42 +00:00