Strict confinement (devmode)

Make MicroStack strictly confined, albeit in devmode for now. Addresses unpredictable breakages with apt package upgrades in eoan and focal, and sets the stage for a better isolated, less fragile snap going forward. We now use layouts to handle libvirt and qemu setting paths at compile time. This is cleaner than the organize hack. Moved away from calls to systemctl in init, as a strictly confined snap cannot call systemctl on a non snappy system. Disabled call to sysctl to set ipv4_fowarding, as we don't have access to sysctl in a strictly confined snap. This may break some users, and we need to figure out a way to address the breakage. Got rid of questions.shell.shell routine, moving rabbitmq setup into a bash script instead (it's just cleaner). Moved keypair creation into launch script, as it's difficult to do sensible things with keypair creation in the init script, which is running using sudo, and therefore doesn't have access to /home/<someuser>/snap Added (but commented out) code that will check to verify that plugs are connected before running microstack.init or ovs-vsctl. This code may go away entirely, as we plan on auto connecting all of our interfaces, and don't technically need to guard against not having them connected. Added temporary local upper-constraints file, to fix an issue where upstream upper-constraints was breaking pip install by setting a neutron version. This needs a better long term fix, but works for now. Closes-bug: 1860660 Change-Id: Iaf1f1482609f05285ed9061317b32e90bffd2da0
2020-02-06 02:29:20 +00:00 · 2020-02-06 02:29:20 +00:00 · 5611bc9759
commit 5611bc9759
parent d262cbed49
15 changed files with 1031 additions and 309 deletions
--- a/.gitignore
+++ b/.gitignore
@ -10,6 +10,7 @@ prime/
 snap/.snapcraft
 stage/
 dump.tar.gz
+squashfs-root

 # Emacs
 *~
--- a/README.md
+++ b/README.md
@ -29,6 +29,11 @@ At this time you can install from the `--beta` or `--edge` snap channels:

    sudo snap install microstack --classic --beta

+The edge channel is moving toward a strictly confined snap. At this time, it
+must be installed in devmode:
+
+    sudo snap install microstack --devmode --edge
+
 ## Initialisation

 Initialisation will set up databases, networks, flavors, an SSH keypair, a
--- a/patches/upper-constraints.txt
+++ b/patches/upper-constraints.txt
@ -0,0 +1,573 @@
+ntlm-auth===1.2.0
+voluptuous===0.11.5
+chardet===3.0.4
+rsa===4.0
+restructuredtext-lint===1.2.2
+netmiko===2.3.0
+instack-undercloud===9.4.0
+PasteDeploy===2.0.1
+typing===3.6.6
+python-saharaclient===2.2.1
+python-hnvclient===0.1.0
+Routes===2.4.1
+rtslib-fb===2.1.66
+XStatic-Angular-Bootstrap===2.2.0.0
+paunch===4.5.2
+WebOb===1.8.5
+sphinxcontrib-actdiag===0.8.5
+docopt===0.6.2
+pecan===1.3.2
+ryu===4.30
+os-api-ref===1.6.0
+python-ldap===3.1.0
+oslo.concurrency===3.29.1
+websocket-client===0.55.0
+osprofiler===2.6.1
+os-resource-classes===0.3.0
+python-ironic-inspector-client===3.5.0
+lxml===4.3.2
+python-kingbirdclient===0.2.1
+setproctitle===1.1.10
+pytest===4.3.0
+python-etcd===0.4.5
+raven===6.10.0
+cursive===0.2.2
+oslo.service===1.38.1
+django-appconf===1.0.3
+pykerberos===1.2.1
+certifi===2018.11.29
+sphinxcontrib-nwdiag===0.9.5
+requests-aws===0.1.8
+alabaster===0.7.12
+pbr===5.1.3
+munch===2.3.2
+attrs===19.1.0
+microversion-parse===0.2.1
+Pint===0.9
+oslo.i18n===3.23.1
+jsonpath-rw-ext===1.2.0
+python-mistralclient===3.8.1
+oslo.context===2.22.1
+python-senlinclient===1.10.1
+rcssmin===1.0.6
+pycadf===2.9.0
+grpcio===1.15.0
+skydive-client===0.5.0
+pysendfile===2.0.1
+fixtures===3.0.0
+neutron-lib===1.25.1
+XStatic-FileSaver===1.3.2.0
+pystache===0.5.4
+XStatic-Font-Awesome===4.7.0.0
+nose===1.3.7
+nosehtmloutput===0.0.5
+waitress===1.2.1
+os-refresh-config===10.2.2
+pysnmp===4.4.9
+sphinxcontrib-websupport===1.1.0
+Mako===1.0.7
+XStatic-angular-ui-router===0.3.1.2
+pyScss===1.3.4
+XStatic-jQuery===1.12.4.1
+jsonmodels===2.4
+ddt===1.2.1
+pyserial===3.4
+ipaddress===1.0.22;python_version=='2.7'
+python-freezerclient===2.1.0
+os-xenapi===0.3.4
+python-vitrageclient===2.7.0
+nosexcover===1.0.11
+krest===1.3.1
+psycopg2===2.7.7
+networkx===2.2
+bashate===0.6.0
+XStatic-Angular===1.5.8.0
+pyngus===2.3.0
+Pillow===5.4.1
+zuul-sphinx===0.3.0
+python-mimeparse===1.6.0
+tripleo-common===10.8.2
+Tempita===0.5.2
+ply===3.11
+requests-toolbelt===0.9.1
+simplejson===3.16.0
+suds-jurko===0.6
+python-swiftclient===3.7.1
+pyOpenSSL===19.0.0
+monasca-common===2.13.0
+scipy===1.2.1
+mypy-extensions===0.4.1;python_version=='3.4'
+mypy-extensions===0.4.1;python_version=='3.5'
+mypy-extensions===0.4.1;python_version=='3.6'
+rsd-lib===0.4.0
+XStatic-Jasmine===2.4.1.2
+python-glanceclient===2.16.0
+pyinotify===0.9.6
+debtcollector===1.21.0
+requests-unixsocket===0.1.5
+asn1crypto===0.24.0
+croniter===0.3.29
+octavia-lib===1.1.1
+python-watcherclient===2.2.0
+MarkupSafe===1.1.1
+pypowervm===1.1.20
+doc8===0.8.0
+pymongo===3.7.2
+soupsieve===1.8
+sqlparse===0.2.4
+oslotest===3.7.1
+jsonpointer===2.0
+defusedxml===0.5.0
+relativetimebuilder===0.2.0
+netaddr===0.7.19
+pyghmi===1.2.16
+sphinxcontrib-blockdiag===1.5.5
+thrift===0.11.0
+gnocchiclient===7.0.5
+wcwidth===0.1.7
+sphinxcontrib.datatemplates===0.1.0
+jsonpath-rw===1.4.0
+prettytable===0.7.2
+vine===1.2.0
+taskflow===3.5.0
+traceback2===1.4.0
+semantic-version===2.6.0
+virtualbmc===1.4.0
+deprecation===2.0.6
+SQLAlchemy===1.2.18
+pyroute2===0.5.4
+google-auth===1.6.3
+kazoo===2.6.1
+XStatic-roboto-fontface===0.5.0.0
+pyudev===0.21.0
+eventlet===0.24.1
+openstack-doc-tools===1.8.0
+frozendict===1.2
+oslo.messaging===9.5.1
+jira===2.0.0
+extras===1.0.0
+PyJWT===1.7.1
+zVMCloudConnector===1.4.0
+paramiko===2.4.2
+reno===2.11.2
+unicodecsv===0.14.1;python_version=='2.7'
+imagesize===1.1.0
+pydot===1.4.1
+pathlib===1.0.1;python_version=='2.7'
+urllib3===1.24.1
+graphviz===0.10.1
+PyKMIP===0.8.0
+whereto===0.4.0
+python-subunit===1.3.0
+tornado===4.5.3;python_version=='3.4'
+tornado===4.5.3;python_version=='3.5'
+tornado===4.5.3;python_version=='3.6'
+tornado===4.5.3;python_version=='2.7'
+pycparser===2.19
+mock===2.0.0
+PyYAML===3.13
+beautifulsoup4===4.7.1
+os-net-config===10.4.2
+ovs===2.10.0
+cryptography===2.6.1
+adal===1.2.1
+backports.ssl-match-hostname===3.7.0.1;python_version=='2.7'
+openstack-release-test===1.4.2
+pylxd===2.2.9
+ruamel.ordereddict===0.4.13;python_version=='2.7'
+pycryptodomex===3.7.3
+anyjson===0.3.3
+requests-mock===1.5.2
+os-apply-config===10.3.0
+prometheus-client===0.6.0
+oslosphinx===4.18.0
+mox3===0.27.0
+gunicorn===19.9.0
+textfsm===0.4.1
+unittest2===1.1.0
+django-compressor===2.2
+libvirt-python===5.1.0
+python-zunclient===3.3.0
+asyncio===3.4.3;python_version=='3.4'
+asyncio===3.4.3;python_version=='3.5'
+asyncio===3.4.3;python_version=='3.6'
+tzlocal===1.5.1
+python-novaclient===13.0.2
+bcrypt===3.1.6
+fixtures-git===0.1.0
+os-client-config===1.32.0
+XStatic-Angular-Gettext===2.3.8.0
+XStatic-Hogan===2.0.0.2
+XStatic-objectpath===1.2.1.0
+python-manilaclient===1.27.0
+requests===2.21.0
+snowballstemmer===1.2.1
+Jinja2===2.10
+XStatic-Bootstrap-SCSS===3.3.7.1
+pyzabbix===0.7.5
+ptyprocess===0.6.0
+threadloop===1.0.2
+amqp===2.4.2
+ruamel.yaml===0.15.89
+websockify===0.8.0
+XStatic-JQuery.quicksearch===2.0.3.1
+mpmath===1.1.0
+django-debreach===1.5.2
+sphinx-feature-classification===0.3.2
+XStatic-JQuery-Migrate===1.2.1.1
+appdirs===1.4.3
+tinyrpc===0.9.4
+google-auth-httplib2===0.0.3
+Flask-SQLAlchemy===2.3.2
+daiquiri===1.5.0
+influxdb===5.1.0
+funcparserlib===0.3.6
+passlib===1.7.1
+dib-utils===0.0.11
+cliff===2.14.1
+os-brick===2.8.3
+ansible-runner===1.2.0
+trollius===2.2;python_version=='2.7'
+scp===0.13.0
+python-zaqarclient===1.11.0
+funcsigs===1.0.2;python_version=='2.7'
+zhmcclient===0.22.0
+lockfile===0.12.2
+dnspython3===1.15.0;python_version=='3.4'
+dnspython3===1.15.0;python_version=='3.5'
+dnspython3===1.15.0;python_version=='3.6'
+ldappool===2.4.1
+termcolor===1.1.0
+hiredis===1.0.0
+google-api-python-client===1.7.8
+castellan===1.2.2
+oslo.versionedobjects===1.35.1
+webcolors===1.8.1
+aodhclient===1.2.0
+autobahn===19.2.1
+SQLAlchemy-Utils===0.33.11
+pluggy===0.9.0
+coverage===4.5.2
+freezegun===0.3.11
+python-pytun===2.3.0
+pyperclip===1.7.0
+cassandra-driver===3.17.0
+mox===0.5.3
+XStatic-Angular-Schema-Form===0.8.13.0
+gabbi===1.45.0
+nwdiag===1.0.4
+XStatic-bootswatch===3.3.7.0
+XStatic-JS-Yaml===3.8.1.0
+XStatic-term.js===0.0.7.0
+oslo.log===3.42.3
+nodeenv===1.3.3
+pylev===1.3.0
+python-searchlightclient===1.5.1
+oslo.middleware===3.37.1
+XStatic-mdi===1.6.50.2
+django-pyscss===2.0.2
+uritemplate===3.0.0
+django-babel===0.6.2
+docutils===0.14
+notifier===1.0.3
+os-ken===0.3.1
+pycrypto===2.6.1
+ujson===1.35
+selenium===3.141.0
+python-glareclient===0.5.3
+mypy===0.670;python_version=='3.4'
+mypy===0.670;python_version=='3.5'
+mypy===0.670;python_version=='3.6'
+mistral-lib===1.1.1
+Click===7.0
+dogtag-pki===10.3.5.1
+XStatic-Angular-UUID===0.0.4.0
+sphinxcontrib-seqdiag===0.8.5
+os-win===4.2.1
+dictdiffer===0.7.2
+retrying===1.3.3
+shade===1.31.0
+pathlib2===2.3.3
+pydotplus===2.0.2
+flask-oslolog===0.1
+jeepney===0.4;python_version=='3.4'
+jeepney===0.4;python_version=='3.5'
+jeepney===0.4;python_version=='3.6'
+stestr===2.2.0
+singledispatch===3.4.0.3;python_version=='2.7'
+oslo.serialization===2.28.2
+warlock===1.3.0
+exabgp===4.0.10
+sphinxcontrib-httpdomain===1.7.0
+metalsmith===0.11.1
+thriftpy===0.3.9;python_version=='2.7'
+text-unidecode===1.2
+murano-pkg-check===0.3.0
+oslo.vmware===2.32.2
+sqlalchemy-migrate===0.12.0
+python-monascaclient===1.15.0
+ldap3===2.5.2
+requests-ntlm===1.1.0
+python-string-utils===0.6.0
+automaton===1.16.0
+os-service-types===1.6.0
+keyring===18.0.0
+testscenarios===0.5.0
+sphinxcontrib-pecanwsme===0.9.0
+sadisplay===0.4.9
+enum34===1.1.6
+packaging===19.0
+flask-keystone===0.2
+nose-exclude===0.5.0
+psutil===5.6.0
+py===1.8.0
+txaio===18.8.1
+python-qinlingclient===2.1.0
+elasticsearch===2.4.1
+django-nose===1.4.6
+XStatic-JQuery.TableSorter===2.14.5.1
+pifpaf===2.2.2
+pysmi===0.3.3
+blockdiag===1.5.4
+testtools===2.3.0
+Parsley===1.3
+XStatic-tv4===1.2.7.0
+XStatic-JSEncrypt===2.3.1.1
+python-cinderclient===4.2.1
+keystonemiddleware===6.0.0
+django-formtools===2.1
+python-ceilometerclient===2.9.0
+XStatic-Spin===1.2.5.2
+openshift===0.8.6
+tap-as-a-service===3.0.0
+os-traits===0.11.0
+SecretStorage===2.3.1;python_version=='2.7'
+SecretStorage===3.1.1;python_version=='3.4'
+SecretStorage===3.1.1;python_version=='3.5'
+SecretStorage===3.1.1;python_version=='3.6'
+opentracing===1.3.0
+XStatic-Rickshaw===1.5.0.0
+iso8601===0.1.12
+tooz===1.64.2
+linecache2===1.0.0
+oauth2client===4.1.3
+idna===2.8
+python-karborclient===1.2.0
+weakrefmethod===1.0.3;python_version=='2.7'
+PuLP===1.6.9
+crc16===0.1.1
+protobuf===3.7.0
+os-dpm===1.1.0
+sushy===1.8.2
+python-neutronclient===6.12.1
+pika===0.13.0
+oslo.cache===1.33.3
+WebTest===2.0.33
+openstack.nose-plugin===0.11
+os-collect-config===10.3.1
+python-qpid-proton===0.27.0
+python-octaviaclient===1.8.1
+pysaml2===4.6.5
+requests-oauthlib===1.2.0
+oslo.reports===1.29.2
+ceilometermiddleware===1.4.0
+python-nss===1.0.1
+testrepository===0.0.20
+sympy===1.3
+sphinxmark===0.1.19
+PyNaCl===1.3.0
+osc-lib===1.12.1
+python-consul===1.1.0
+Faker===1.0.2
+more-itertools===5.0.0;python_version=='2.7'
+more-itertools===6.0.0;python_version=='3.4'
+more-itertools===6.0.0;python_version=='3.5'
+more-itertools===6.0.0;python_version=='3.6'
+seqdiag===0.9.6
+numpy===1.16.2
+msgpack===0.6.1
+Sphinx===1.8.4
+oslo.config===6.8.2
+tempest===20.0.0
+django-floppyforms===1.7.0
+openstackdocstheme===1.29.2
+osc-placement===1.5.0
+zake===0.2.2
+python-rsdclient===0.1.3
+python-magic===0.4.15
+python-solumclient===2.8.0
+PyMySQL===0.9.3
+kubernetes===8.0.1
+httplib2===0.12.1
+bottle===0.12.16
+betamax===0.8.1
+construct===2.8.22
+pyparsing===2.3.1
+dogpile.cache===0.7.1
+python-barbicanclient===4.8.1
+salt===2019.2.0
+tricircleclient===0.5.0
+WSME===0.9.3
+proboscis===1.2.6.0
+fortiosclient===0.0.3
+oslo.upgradecheck===0.2.1
+stevedore===1.30.1
+botocore===1.12.108
+xmltodict===0.12.0
+pyasn1===0.4.5
+oslo.rootwrap===5.15.3
+Django===1.11.20;python_version=='2.7'
+Django===2.0.13;python_version=='3.4'
+Django===2.0.13;python_version=='3.5'
+Django===2.0.13;python_version=='3.6'
+pexpect===4.6.0
+cmd2===0.8.9
+redis===3.2.0
+jmespath===0.9.4
+atomicwrites===1.3.0
+docker-pycreds===0.4.0
+XStatic-smart-table===1.4.13.2
+kuryr-lib===0.9.0
+scrypt===0.8.13
+jsonpatch===1.23
+python-daemon===2.2.3
+typed-ast===1.3.1;python_version=='3.4'
+typed-ast===1.3.1;python_version=='3.5'
+typed-ast===1.3.1;python_version=='3.6'
+os-testr===1.0.0
+cotyledon===1.7.3
+stomp.py===4.1.21
+xattr===0.9.6
+systemd-python===234
+python-memcached===1.59
+openstacksdk===0.27.0
+six===1.12.0
+dulwich===0.19.11
+kombu===4.4.0
+distro===1.4.0
+betamax-matchers===0.4.0
+yaql===1.1.3
+requestsexceptions===1.4.0
+testresources===2.0.1
+falcon===1.4.1
+subprocess32===3.5.3;python_version=='2.7'
+etcd3gw===0.2.4
+Flask-RESTful===0.3.7
+GitPython===2.1.11
+python-ironicclient===2.7.3
+XStatic===1.0.2
+XStatic-Angular-FileUpload===12.0.4.0
+python-openstackclient===3.18.1
+pyzmq===18.0.1
+oslo.db===4.45.0
+simplegeneric===0.8.1
+python-pcre===0.7
+yappi===1.0
+abclient===0.2.3
+pymemcache===2.1.1
+wrapt===1.11.1
+oslo.privsep===1.32.2
+sphinxcontrib-apidoc===0.3.0
+oslo.policy===2.1.2
+python-muranoclient===1.2.0
+hvac===0.7.2
+pyeclib===1.5.0
+wsgi-intercept===1.8.0
+ndg-httpsclient===0.5.1;python_version=='2.7'
+repoze.lru===0.7
+rfc3986===1.2.0
+tenacity===5.0.3
+python-designateclient===2.11.0
+future===0.17.1
+Paste===3.0.7
+jaeger-client===3.13.0
+XStatic-Json2yaml===0.1.1.0
+boto===2.49.0
+functools32===3.2.3.post2;python_version=='2.7'
+os-vif===1.15.2
+python-masakariclient===5.4.0
+Werkzeug===0.14.1
+backports.functools-lru-cache===1.5;python_version=='2.7'
+pyasn1-modules===0.2.4
+entrypoints===0.3
+APScheduler===3.5.3
+monotonic===1.5
+python-smaugclient===0.0.8
+python-troveclient===2.17.1
+etcd3===0.8.1
+XStatic-Bootstrap-Datepicker===1.3.1.0
+CouchDB===1.2
+netifaces===0.10.9
+cachetools===3.1.0
+ws4py===0.5.1
+backports-abc===0.5;python_version=='2.7'
+keystoneauth1===3.13.1
+statsd===3.3.0
+XenAPI===1.2
+python-keystoneclient===3.19.0
+ceilometer===12.0.0
+demjson===2.2.4
+diskimage-builder===2.30.0
+heat-translator===1.3.1
+python-magnumclient===2.12.0
+docker===3.7.0
+qpid-python===1.36.0.post1;python_version=='2.7'
+contextlib2===0.5.5;python_version=='2.7'
+XStatic-Angular-lrdragndrop===1.0.2.4
+python-congressclient===1.12.0
+ovsdbapp===0.15.0
+aniso8601===5.1.0
+rjsmin===1.0.12
+icalendar===4.0.3
+configparser===3.7.3;python_version=='2.7'
+decorator===4.3.2
+cffi===1.12.2
+futurist===1.8.1
+jsonschema===2.6.0
+python-blazarclient===2.1.0
+alembic===1.0.8
+glance-store===0.28.0
+sphinxcontrib-programoutput===0.13
+sphinx-testing===1.0.0
+dnspython===1.15.0;python_version=='3.4'
+dnspython===1.15.0;python_version=='3.5'
+dnspython===1.15.0;python_version=='3.6'
+dnspython===1.16.0;python_version=='2.7'
+oauthlib===3.0.1
+Babel===2.6.0
+logutils===0.3.5
+scandir===1.9.0;python_version=='2.7'
+sphinxcontrib-fulltoc===1.2.0
+smmap2===2.0.5
+greenlet===0.4.15
+XStatic-Angular-Vis===4.16.0.0
+confluent-kafka===0.11.6
+xvfbwrapper===0.2.9
+futures===3.2.0;python_version=='2.7'
+tosca-parser===1.4.0
+Flask===1.0.2
+happybase===1.1.0;python_version=='2.7'
+marathon===0.11.0
+fasteners===0.14.1
+sortedcontainers===2.1.0
+python-tackerclient===0.15.0
+python-heatclient===1.17.0
+kafka-python===1.4.4
+oslo.utils===3.40.5
+python-editor===1.0.4
+gitdb2===2.0.5
+requests-kerberos===0.12.0
+itsdangerous===1.1.0
+XStatic-jquery-ui===1.12.1.1
+monasca-statsd===1.11.0
+python-dateutil===2.8.0
+virtualenv===16.4.3
+colorama===0.4.1
+ironic-lib===2.16.4
+pytz===2018.9
+XStatic-D3===3.5.17.0
+actdiag===0.5.4
+sysv-ipc===1.0.0
+scikit-learn===0.20.3
+horizon===15.2.0
+uwsgi===2.0.17.1
--- a/snap-overlay/bin/set-default-config
+++ b/snap-overlay/bin/set-default-config
@ -28,7 +28,7 @@ snapctl set \
 # Passwords, certs, etc.
 snapctl set \
        config.credentials.os-password=keystone \
-        config.credentials.key-pair=id_microstack \
+        config.credentials.key-pair="/home/{USER}/snap/{SNAP_NAME}/common/.ssh/id_microstack" \
        config.credentials.nova-password=nova \
        config.credentials.neutron-password=neutron \
        config.credentials.placement-password=placement \
@ -37,7 +37,7 @@ snapctl set \

 # Host optimizations and fixes.
 snapctl set \
-        config.host.ip-forwarding=true \
+        config.host.ip-forwarding=false \
        config.host.check-qemu=true \
        ;

--- a/snap-overlay/bin/setup-rabbit
+++ b/snap-overlay/bin/setup-rabbit
@ -0,0 +1,8 @@
+#!/bin/bash
+
+set -ex
+
+export HOME=$SNAP_COMMON/lib/rabbitmq
+
+$SNAP/usr/sbin/rabbitmqctl add_user openstack rabbitmq || true
+$SNAP/usr/sbin/rabbitmqctl set_permissions openstack ".*" ".*" ".*"
--- a/snap-wrappers/ovs/ovs-wrapper
+++ b/snap-wrappers/ovs/ovs-wrapper
@ -13,4 +13,9 @@ mkdir -p ${OVS_LOGDIR}
 mkdir -p ${OVS_RUNDIR}
 mkdir -p ${OVS_SYSCONFDIR}/openvswitch

+# if ! snapctl is-connected openvswitch-support; then
+#    echo "openvswitch-support is not connected. Exiting."
+#    exit 0
+# fi
+
 exec $@
--- a/snap/hooks/install
+++ b/snap/hooks/install
@ -4,6 +4,8 @@ set -ex
 # Initialize config
 set-default-config

+mkdir -p $SNAP_DATA/lib/libvirt/images
+
 # MySQL snapshot for speedy install
 # snapshot is a mysql data dir with
 # rocky keystone,nova,glance,neutron dbs.
--- a/snapcraft.yaml
+++ b/snapcraft.yaml
@ -5,172 +5,206 @@ summary: OpenStack on your laptop.
 description: |
  Microstack gives you an easy way to develop and test OpenStack
  workloads on your laptop.
-grade: stable
-confinement: classic
+grade: devel
+confinement: strict
 environment:
-  # Edit the following lines with tools/update_path.py
-  LD_LIBRARY_PATH: $SNAP/lib:$SNAP/lib/$SNAPCRAFT_ARCH_TRIPLET:$SNAP/usr/lib:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET:$SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/pulseaudio
-  PATH: $SNAP/usr/sbin:$SNAP/usr/bin:$SNAP/sbin:$SNAP/bin:$PATH
  LC_ALL: C
+  PATH: $SNAP/usr/sbin:$SNAP/usr/bin:$SNAP/sbin:$SNAP/bin:$PATH
  OS_PLACEMENT_CONFIG_DIR: $SNAP/etc/nova/

+layout:
+  # Libvirt/Qemu libs
+  /usr/lib/$SNAPCRAFT_ARCH_TRIPLET/ceph:
+    bind: $SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/ceph
+  /usr/lib/$SNAPCRAFT_ARCH_TRIPLET/qemu:
+    bind: $SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/qemu
+  /usr/lib/$SNAPCRAFT_ARCH_TRIPLET/pulseaudio:
+    bind: $SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/pulseaudio
+  /usr/lib/libvirt:
+    bind: $SNAP/usr/lib/libvirt
+  /usr/share/libvirt:
+    bind: $SNAP/usr/share/libvirt
+  /usr/share/qemu:
+    bind: $SNAP/usr/share/qemu
+  # Keystone really wants to look here for its fernet keys for some reason
+  /etc/keystone/fernet-keys:
+    bind: $SNAP_COMMON/fernet-keys
+
 apps:
  # Openstack
  openstack:
    command: openstack-wrapper openstack
-#    plugs:
-#      - network
+    plugs:
+      - network

  # OpenStack Service Configuration
  init:
-    # This and other instances of /usr/bin/env are workarounds for
-    # https://bugs.launchpad.net/snapd/+bug/1860369. We force snapcraft
-    # to put us in a wrapper.
-    command: /usr/bin/env microstack_init
-#    plugs:
-#      - network
+    command: microstack_init
+    plugs:
+      - network
+      - mount-observe  # rabbitmq-ctl
+      - network-bind  # rabbitmq-ctl?
+      - firewall-control  # iptables
+      - network-control  # iptables
+      - ssh-keys # write ssh key
+      - system-observe  # rabbitmq ?
+      - hardware-observe  # rabbitmq ?

  remove:
-    command: /usr/bin/env microstack_remove
+    command: microstack_remove

  # Keystone
  keystone-uwsgi:
-    command: /usr/bin/env snap-openstack launch keystone-uwsgi
+    command: snap-openstack launch keystone-uwsgi
    daemon: simple
-#    plugs:
-#      - network-bind
+    plugs:
+      - network-bind
+      - network
+      - network-control
  keystone-manage:
-    command: /usr/bin/env snap-openstack launch keystone-manage
-#    plugs:
-#      - network
+    command: snap-openstack launch keystone-manage
+    plugs:
+      - network

  # Nova
  nova-uwsgi:
-    command: /usr/bin/env snap-openstack launch nova-uwsgi
+    command: snap-openstack launch nova-uwsgi
    daemon: simple
-#    plugs:
-#      - network-bind
+    plugs:
+      - network
+      - network-bind
+      - network-control
  nova-api:
-    command: /usr/bin/env snap-openstack launch nova-api-os-compute
+    command: snap-openstack launch nova-api-os-compute
    daemon: simple
-#    plugs:
-#      - network-bind
+    plugs:
+      - network
+      - network-bind
+      - network-control
  nova-conductor:
-    command: /usr/bin/env snap-openstack launch nova-conductor
+    command: snap-openstack launch nova-conductor
    daemon: simple
-#    plugs:
-#      - network
+    plugs:
+      - network
+      - network-control
  nova-scheduler:
-    command: /usr/bin/env snap-openstack launch nova-scheduler
+    command: snap-openstack launch nova-scheduler
    daemon: simple
-#    plugs:
-#      - network
+    plugs:
+      - network
+      - network-bind
+      - network-control
  nova-compute:
-    command: /usr/bin/env snap-openstack launch nova-compute
+    command: snap-openstack launch nova-compute
    daemon: simple
-#    plugs:
-#      - network-bind
-#      - network-control
-#      - firewall-control
-#      - hardware-observe
-#      - libvirt
-#      - openvswitch
+    plugs:
+      - network
+      - network-bind
+      - network-control
+      - firewall-control
+      - hardware-observe
  nova-api-metadata:
-    command: /usr/bin/env snap-openstack launch nova-api-metadata
+    command: snap-openstack launch nova-api-metadata
    daemon: simple
-#    plugs:
-#      - network-bind
-#      - firewall-control
+    plugs:
+      - network
+      - network-bind
+      - firewall-control
  nova-manage:
-    command: /usr/bin/env snap-openstack launch nova-manage
-#    plugs:
-#      - network
+    command: snap-openstack launch nova-manage
+    plugs:
+      - network

  # Neutron
  neutron-api:
-    command: /usr/bin/env snap-openstack launch neutron-server
+    command: snap-openstack launch neutron-server
    daemon: simple
-#    plugs:
-#      - network-bind
+    plugs:
+      - network
+      - network-bind
+      - network-control
  neutron-openvswitch-agent:
-    command: /usr/bin/env snap-openstack launch neutron-openvswitch-agent
+    command: snap-openstack launch neutron-openvswitch-agent
    daemon: simple
-#    plugs:
-#      - network-bind
-#      - network-control
-#      - network-observe
-#      - firewall-control
-#      - process-control
-#      - system-observe
-#      - openvswitch
+    plugs:
+      - network
+      - network-bind
+      - network-control
+      - network-observe
+      - firewall-control
+      - process-control
+      - system-observe
  neutron-l3-agent:
-    command: /usr/bin/env snap-openstack launch neutron-l3-agent
+    command: snap-openstack launch neutron-l3-agent
    daemon: simple
-#    plugs:
-#      - network-bind
-#      - network-control
-#      - network-observe
-#      - firewall-control
-#      - process-control
-#      - system-observe
-#      - openvswitch
+    plugs:
+      - network
+      - network-bind
+      - network-control
+      - network-observe
+      - firewall-control
+      - process-control
+      - system-observe
  neutron-dhcp-agent:
-    command: /usr/bin/env snap-openstack launch neutron-dhcp-agent
+    command: snap-openstack launch neutron-dhcp-agent
    daemon: simple
-#    plugs:
-#      - network
-#      - network-bind
-#      - network-control
-#      - network-observe
-#      - process-control
-#      - system-observe
-#      - openvswitch
+    plugs:
+      - network
+      - network-bind
+      - network-control
+      - network-observe
+      - process-control
+      - system-observe
  neutron-metadata-agent:
-    command: /usr/bin/env snap-openstack launch neutron-metadata-agent
+    command: snap-openstack launch neutron-metadata-agent
    daemon: simple
-#    plugs:
-#      - network
-#      - network-bind
-#      - network-control
+    plugs:
+      - network
+      - network-bind
+      - network-control
  neutron-ovs-cleanup:
-    command: /usr/bin/env snap-openstack launch neutron-ovs-cleanup
-#    plugs:
-#      - network
-#      - network-control
-#      - openvswitch
+    command: snap-openstack launch neutron-ovs-cleanup
+    plugs:
+      - network
+      - network-control
  neutron-netns-cleanup:
-    command: /usr/bin/env snap-openstack launch neutron-netns-cleanup
-#    plugs:
-#      - network
-#      - network-control
+    command: snap-openstack launch neutron-netns-cleanup
+    plugs:
+      - network
+      - network-control
  neutron-db-manage:
-    command: /usr/bin/env snap-openstack launch neutron-db-manage
-#    plugs:
-#      - network
+    command: snap-openstack launch neutron-db-manage
+    plugs:
+      - network

  # Glance
  glance-api:
-    command: /usr/bin/env snap-openstack launch glance-api
+    command: snap-openstack launch glance-api
    daemon: simple
-#    plugs:
-#      - network-bind
+    plugs:
+      - network
+      - network-bind
+      - mount-observe
+      - network-control
  registry:
-    command: /usr/bin/env snap-openstack launch glance-registry
+    command: snap-openstack launch glance-registry
    daemon: simple
-#    plugs:
-#      - network
-#      - network-bind
+    plugs:
+      - network
+      - network-bind
+      - network-control
  glance-manage:
-    command: /usr/bin/env snap-openstack launch glance-manage
-#    plugs:
-#      - network
+    command: snap-openstack launch glance-manage
+    plugs:
+      - network

  # Openstack Shared Services
  nginx:
-    command: /usr/bin/env snap-openstack launch nginx
+    command: snap-openstack launch nginx
    daemon: forking
-#    plugs:
-#      - network-bind
+    plugs:
+      - network
+      - network-bind

  # Openvswitch
  ovs-vswitchd:
@ -178,149 +212,174 @@ apps:
    stop-command: ovs-wrapper $SNAP/share/openvswitch/scripts/ovs-ctl --no-ovsdb-server stop
    after: [ovsdb-server]
    daemon: forking
-#    plugs:
-#      - network
-#      - network-bind
-#      - network-control
-#      - openvswitch-support
-#      - process-control
-#      - system-trace
+    plugs:
+      - network
+      - network-bind
+      - network-control
+      - openvswitch-support
+      - process-control
+      - system-trace
  ovsdb-server:
    command: ovs-wrapper $SNAP/share/openvswitch/scripts/ovs-ctl --no-ovs-vswitchd --no-monitor --system-id=random start
    stop-command: ovs-wrapper $SNAP/share/openvswitch/scripts/ovs-ctl --no-ovs-vswitchd stop
    daemon: forking
-#    plugs:
-#      - network
-#      - network-bind
-#      - network-control
-#      - openvswitch-support
-#      - process-control
-#      - system-trace
+    plugs:
+      - network
+      - network-bind
+      - network-control
+      - openvswitch-support
+      - process-control
+      - system-trace
  ovs-vsctl:
-    command: ovs-wrapper $SNAP/bin/ovs-vsctl
-#    plugs:
-#      - network
+    command: ovs-wrapper bin/ovs-vsctl
+    plugs:
+      - network
  ovs-appctl:
-    command: ovs-wrapper $SNAP/bin/ovs-appctl
-#    plugs:
-#      - network
+    command: ovs-wrapper bin/ovs-appctl
+    plugs:
+      - network
  ovs-ofctl:
-    command: ovs-wrapper $SNAP/bin/ovs-ofctl
-#    plugs:
-#      - network
+    command: ovs-wrapper bin/ovs-ofctl
+    plugs:
+      - network
  ovs-dpctl:
-    command: ovs-wrapper $SNAP/bin/ovs-dpctl
-#    plugs:
-#      - network
+    command: ovs-wrapper bin/ovs-dpctl
+    plugs:
+      - network

  external-bridge:
    command: wait-on-init setup-br-ex
    daemon: oneshot
    after: [ovs-vswitchd]
-#    plugs:
-#      - network
+    plugs:
+      - network
+      - network-control

  # Libvirt/Qemu
  libvirtd:
-    command: /usr/bin/env libvirtd
+    command: usr/sbin/libvirtd --pid $SNAP_DATA/libvirt.pid
    daemon: simple
+    plugs:
+      - network
+      - network-bind
+      - network-control
+      - netlink-connector
+      - netlink-audit
  virtlogd:
-    command: /usr/bin/env virtlogd
+    command: virtlogd --pid $SNAP_DATA/virtlogd.pid
    daemon: simple
+    plugs:
+      - network
+      - network-bind
+      - network-control
  virsh:
-    command: /usr/bin/env virsh
+    command: virsh

  # MySQL
  mysqld:
    command: mysql-start-server
    daemon: simple
-#    plugs:
-#      - process-control
-#      - network
-#      - network-bind
+    plugs:
+      - process-control
+      - network
+      - network-bind
  mysql:
    command: mysql-start-client
-#    plugs:
-#      - process-control
-#      - network
+    plugs:
+      - process-control
+      - network

  # RabbitMQ
  rabbitmq-server:
-    command: /usr/bin/env rabbitmq-server
+    command: rabbitmq-server
    daemon: simple
-#    plugs:
-#      - network-bind
+    plugs:
+      - network-bind
+      - network
+      - mount-observe
+      - log-observe
    environment:
      HOME: $SNAP_COMMON/lib/rabbitmq
  rabbitmqctl:
-    command: /usr/bin/env rabbitmqctl
-#    plugs:
-#      - network
+    command: rabbitmqctl
+    plugs:
+      - network
+      - mount-observe
    environment:
      HOME: $SNAP_COMMON/lib/rabbitmq
  rabbitmq-plugins:
-    command: /usr/bin/env rabbitmq-plugins
+    command: rabbitmq-plugins
    environment:
      HOME: $SNAP_COMMON/lib/rabbitmq

  # Memcached
  memcached:
-    command: /usr/bin/env memcached -u root -v
+    command: memcached -u root -v
    daemon: simple
-#    plugs:
-#      - network-bind
+    plugs:
+      - network
+      - network-bind

  # Cinder
  cinder-uwsgi:
-    command: /usr/bin/env snap-openstack launch cinder-uwsgi
-#    daemon: simple
-#    plugs:
-#      - network-bind
+    command: snap-openstack launch cinder-uwsgi
+    daemon: simple
+    plugs:
+      - network
+      - network-bind
  cinder-backup:
-    command: /usr/bin/env snap-openstack launch cinder-backup
-#    daemon: simple
-#    plugs:
-#      - network
+    command: snap-openstack launch cinder-backup
+    daemon: simple
+    plugs:
+      - network
+      - mount-observe
  cinder-manage:
-    command: /usr/bin/env snap-openstack launch cinder-manage
-#    plugs:
-#      - network
+    command: snap-openstack launch cinder-manage
+    plugs:
+      - network
+      - mount-observe
  cinder-scheduler:
-    command: /usr/bin/env snap-openstack launch cinder-scheduler
-#    daemon: simple
-#    plugs:
-#      - network
+    command: snap-openstack launch cinder-scheduler
+    daemon: simple
+    plugs:
+      - network
+      - mount-observe
  cinder-volume:
-    command: /usr/bin/env snap-openstack launch cinder-volume
-#    daemon: simple
-#    plugs:
-#      - network
+    command: snap-openstack launch cinder-volume
+    daemon: simple
+    plugs:
+      - network
+      - mount-observe

  # Horizon
  horizon-uwsgi:
-    command: /usr/bin/env snap-openstack launch horizon-uwsgi
+    command: snap-openstack launch horizon-uwsgi
    daemon: simple
-#    plugs:
-#      - network-bind
+    plugs:
+      - network
+      - network-bind

  # Utility to launch a vm. Creates security groups, floating ips,
  # and other necessities as well.
  launch:
-    command: /usr/bin/env microstack_launch
-#    plugs:
-#      - network
+    command: microstack_launch
+    plugs:
+      - network

  # Cluster
  cluster-server:
-    command: /usr/bin/env flask run -p 10002 --host=0.0.0.0 # TODO: run as a uwsgi app
+    command: flask run -p 10002 --host 0.0.0.0 # TODO: run as a uwsgi app
    daemon: simple
    environment:
      LC_ALL: C.UTF-8  # Makes flask happy
      LANG: C.UTF-8  # Makes flask happy
-      FLASK_APP: ${SNAP}/lib/python3.6/site-packages/cluster/daemon.py
+      FLASK_APP: $SNAP/lib/python3.6/site-packages/cluster/daemon.py
+    plugs:
+      - network
+      - network-bind

  join:
-    command: /usr/bin/env python3 ${SNAP}/lib/python3.6/site-packages/cluster/client.py
+    command: python3 $SNAP/lib/python3.6/site-packages/cluster/client.py


  filebeat:
@ -357,9 +416,10 @@ parts:
    python-version: python3
    after:
      - uca-sources
+      - libvirt
      - patches
    constraints:
-      - https://raw.githubusercontent.com/openstack/requirements/stable/stein/upper-constraints.txt
+      - ${SNAPCRAFT_STAGE}/patches/upper-constraints.txt
    source: http://tarballs.openstack.org/keystone/keystone-stable-stein.tar.gz
    python-packages:
      - libvirt-python
@ -375,18 +435,23 @@ parts:
      - python-cinderclient
      - python-openstackclient
      - python-swiftclient
+      # Workaround for https://bugs.launchpad.net/snapcraft/+bug/1860768
+      # This is a package required for the launch script, per the bug
+      # above. We include it here so that we don't try to rebuild dist
+      # stuff:
+      - petname
    build-packages:
      - gcc
      - git
      - libffi-dev
      - libssl-dev
-      - libvirt-dev
      - libxml2-dev
      - libxslt1-dev
    stage-packages:
      - conntrack
      - coreutils
      - haproxy
+      - libmagic1
      - python3-dev
      - python3-systemd
    override-prime: |
@ -401,14 +466,11 @@ parts:
          --input "$patch"
        echo
      done
-      # Uncomment the below line for better python-libvirt debugging output.
-      # patch --batch --forward --strip 1 --input ../project/patches/nova/nova_log_error.patch
      # Now that everything is built, let's disable user site-packages
      # as stated in PEP-0370
      sed -i usr/lib/python3.6/site.py -e 's/^ENABLE_USER_SITE = None$/ENABLE_USER_SITE = False/'
-      # This is the last step, let's now compile all our pyc files.
-      # Ignore errors due to syntax issues in foobar python 2.
-      ./usr/bin/python3.6 -m compileall . || true
+      # Uncomment the below line for better python-libvirt debugging output.
+      # patch --batch --forward --strip 1 --input ../project/patches/nova/nova_log_error.patch
    organize:
      lib/python3.6/site-packages/openstack_dashboard/local/local_settings.py.example: lib/python3.6/site-packages/openstack_dashboard/local/local_settings.py

@ -582,13 +644,46 @@ parts:
      done
      snapcraftctl build

+  nginx:
+    source: http://www.nginx.org/download/nginx-1.13.0.tar.gz
+    plugin: autotools
+    after:
+      - patches
+    configflags:
+      - --prefix=/usr
+      - "--http-log-path=/var/snap/$SNAPCRAFT_PROJECT_NAME/common/log/nginx-access.log"
+      - "--error-log-path=/var/snap/$SNAPCRAFT_PROJECT_NAME/common/log/nginx-error.log"
+      - "--lock-path=/var/snap/$SNAPCRAFT_PROJECT_NAME/common/lock/nginx.lock"
+      - "--pid-path=/var/snap/$SNAPCRAFT_PROJECT_NAME/common/run/nginx.pid"
+      - "--http-client-body-temp-path=/var/snap/$SNAPCRAFT_PROJECT_NAME/common/lib/nginx_client_body"
+      - "--http-proxy-temp-path=/var/snap/$SNAPCRAFT_PROJECT_NAME/common/lib/nginx_proxy"
+      - "--http-fastcgi-temp-path=/var/snap/$SNAPCRAFT_PROJECT_NAME/common/lib/nginx_fastcgi"
+      - "--http-uwsgi-temp-path=/var/snap/$SNAPCRAFT_PROJECT_NAME/common/lib/nginx_uwsgi"
+      - "--http-scgi-temp-path=/var/snap/$SNAPCRAFT_PROJECT_NAME/common/lib/nginx_scgi"
+      - --with-http_ssl_module
+    build-packages:
+      - libpcre3-dev
+      - libssl-dev
+      - python-six
+    override-build: |
+      # Apply patches
+      for patch in ${SNAPCRAFT_STAGE}/patches/nginx/*.patch; do
+        echo "Applying $(basename "$patch") ..."
+        patch \
+          --batch \
+          --forward \
+          --strip 1 \
+          --input "$patch"
+        echo
+      done
+      snapcraftctl build
+
  # libvirt/qemu
  qemu:
    source: .
    source-subdir: qemu-3.1+dfsg
    plugin: autotools
    after:
-      - openstack-projects
      - uca-sources
    build-environment:
      # Workaround for https://bugs.launchpad.net/snapcraft/+bug/1860766
@ -596,6 +691,7 @@ parts:
    stage-packages:
      - seabios
      - ipxe-qemu
+      - freeglut3 # provides libglut.so.3
      - try:
        - libnuma1
        - libspice-server1
@ -628,6 +724,7 @@ parts:
      - libvorbis0a
      - libvorbisenc2
      - libx11-6
+      - libxi6
      - libxau6
      - libxcb1
      - libxdmcp6
@ -668,7 +765,7 @@ parts:
      - gcc
    configflags:
      - --disable-blobs
-      - --prefix=/snap/$SNAPCRAFT_PROJECT_NAME/current
+      - --prefix=/usr
      - --localstatedir=/var/snap/$SNAPCRAFT_PROJECT_NAME/common
      - --sysconfdir=/var/snap/$SNAPCRAFT_PROJECT_NAME/common
      - --firmwarepath=/snap/$SNAPCRAFT_PROJECT_NAME/current/usr/share/seabios:/snap/$SNAPCRAFT_PROJECT_NAME/current/usr/share/qemu:/snap/$SNAPCRAFT_PROJECT_NAME/current/usr/lib/ipxe/qemu
@ -677,18 +774,12 @@ parts:
      - --enable-system
      - --target-list=x86_64-softmmu
    override-build: |
-      # I think that snapcraft is creating this dir, which breaks the below.xo
+      # I think that snapcraft is creating this dir, which breaks the below.
      rm -Rf qemu-3.1+dfsg
      apt source qemu
      # Fix issue with running apt source as root
      dpkg-source --before-build qemu-3.1+dfsg
      snapcraftctl build
-    organize:
-      # Hack to shift installed qemu back to root of snap
-      # required to ensure that pathing to files etc works at
-      # runtime
-      # * is not used to avoid directory merge conflicts
-      snap/microstack/current/: ./

  kvm-support:
    plugin: nil
@ -699,7 +790,6 @@ parts:
    source: .
    source-subdir: libvirt-5.0.0
    after:
-      - openstack-projects
      - qemu
      - uca-sources
    plugin: autotools
@ -730,6 +820,7 @@ parts:
    stage-packages:
    - dmidecode
    - dnsmasq
+    - iptables
    - libxml2
    - libyajl2
    - try: [libnuma1]
@ -755,7 +846,7 @@ parts:
    - --without-storage-rbd
    - --without-storage-lvm
    - --without-selinux
-    - --prefix=/snap/$SNAPCRAFT_PROJECT_NAME/current
+    - --prefix=/usr
    - --localstatedir=/var/snap/$SNAPCRAFT_PROJECT_NAME/common
    - --sysconfdir=/var/snap/$SNAPCRAFT_PROJECT_NAME/common
    - DNSMASQ=/snap/$SNAPCRAFT_PROJECT_NAME/current/usr/sbin/dnsmasq
@ -770,12 +861,6 @@ parts:
      # Fix issue with running apt source as root
      dpkg-source --before-build libvirt-5.0.0
      snapcraftctl build
-    organize:
-      # Hack to shift installed libvirt back to root of snap
-      # required to ensure that pathing to files etc works at
-      # runtime
-      # * is not used to avoid directory merge conflicts
-      snap/microstack/current/: ./

  # MySQL
  mysql-server:
@ -832,6 +917,11 @@ parts:
    requirements:
      - requirements.txt  # Relative to source path, so tools/init/req...txt
    source: tools/init
+    override-build: |
+      # Hack to avoid conflict between openstack-projects'
+      # site-customize and this one.
+      snapcraftctl build
+      rm $SNAPCRAFT_PART_INSTALL/usr/lib/python3.6/sitecustomize.py

  # Launch script
  launch:
@ -839,10 +929,12 @@ parts:
    python-version: python3
    requirements:
      - requirements.txt
-    stage-packages:
-      # Workaround for https://bugs.launchpad.net/snapcraft/+bug/1860768
-      - petname
    source: tools/launch
+    override-build: |
+      # Hack to avoid conflict between openstack-projects'
+      # site-customize and this one.
+      snapcraftctl build
+      rm $SNAPCRAFT_PART_INSTALL/usr/lib/python3.6/sitecustomize.py

  # Clustering client and server
  cluster:
@ -850,7 +942,14 @@ parts:
    python-version: python3
    requirements:
      - requirements.txt
+    constraints:
+      - ${SNAPCRAFT_STAGE}/patches/upper-constraints.txt
    source: tools/cluster
+    override-build: |
+      # Hack to avoid conflict between openstack-projects'
+      # site-customize and this one.
+      snapcraftctl build
+      rm $SNAPCRAFT_PART_INSTALL/usr/lib/python3.6/sitecustomize.py

  ### LMA stack ###
  filebeat:
@ -910,3 +1009,12 @@ parts:
    source: ./checks
    organize:
      check_systemd.py: usr/lib/nagios/plugins/check_systemd.py
+
+
+hooks:
+  install:
+    plugs: [network]
+  configure:
+    plugs: [network]
+  post-refresh:
+    plugs: [network]
--- a/tests/framework.py
+++ b/tests/framework.py
@ -96,8 +96,8 @@ class Host():
            snap = self.snap
        print("Installing {}".format(snap))

-        check(*self.prefix, 'sudo', 'snap', 'install', '--classic',
-              '--{}'.format(channel), snap)
+        check(*self.prefix, 'sudo', 'snap', 'install',
+              '--devmode', '--{}'.format(channel), snap)

    def init(self, flag='auto'):
        print("Initializing the snap with --{}".format(flag))
--- a/tools/init/init/main.py
+++ b/tools/init/init/main.py
@ -38,7 +38,7 @@ import sys
 from functools import wraps

 from init.config import log
-from init.shell import default_network, check, check_output
+from init.shell import default_network, call, check, check_output

 from init import questions

@ -103,11 +103,25 @@ def process_init_args(args):
    return auto


+def find_missing_plugs():
+    missing = []
+    if not call('snapctl', 'is-connected', 'openvswitch-support'):
+        missing.append("microstack:openvswitch-support")
+    return missing
+
+
@requires_sudo
 def init() -> None:
    args = parse_init_args()
    auto = process_init_args(args)

+    # missing_plugs = find_missing_plugs()
+    # for plug in missing_plugs:
+    #    log.critical("Missing {plug}. Please run snap connect "
+    #                 "{plug} to continue".format(plug=plug))
+    # if missing_plugs:
+    #   sys.exit(1)
+
    question_list = [
        questions.Clustering(),
        questions.Dns(),
@ -124,7 +138,6 @@ def init() -> None:
        questions.NovaControlPlane(),
        questions.NeutronControlPlane(),
        questions.GlanceSetup(),
-        questions.KeyPair(),
        questions.SecurityRules(),
        questions.PostSetup(),
        questions.ExtraServicesQuestion(),
--- a/tools/init/init/questions/init.py
+++ b/tools/init/init/questions/init.py
@ -27,8 +27,8 @@ import json
 from time import sleep
 from os import path

-from init.shell import (check, call, check_output, shell, sql, nc_wait,
-                        log_wait, restart, download)
+from init.shell import (check, call, check_output, sql, nc_wait, log_wait,
+                        restart, download)
 from init.config import Env, log
 from init.questions.question import Question
 from init.questions import clustering, network, uninstall  # noqa F401
@ -104,7 +104,7 @@ class Clustering(Question):
        # Turn off cluster server
        # TODO: it would be more secure to reverse this -- only enable
        # to service if we are doing clustering.
-        check('systemctl', 'disable', 'snap.microstack.cluster-server')
+        check('snapctl', 'stop', '--disable', 'microstack.cluster-server')


 class ConfigQuestion(Question):
@ -174,6 +174,10 @@ class NetworkSettings(Question):
    def yes(self, answer):
        log.info('Configuring networking ...')

+        # OpenvSwitch services may not have started up properly
+        restart('ovsdb-server')
+        restart('ovs-vswitchd')
+
        network.ExtGateway().ask()
        network.ExtCidr().ask()

@ -267,6 +271,7 @@ class RabbitMq(Question):
    config_key = 'config.services.control-plane'

    def _wait(self) -> None:
+        restart('rabbitmq-server')  # Restart server for plugs
        rabbit_port = check_output(
            'snapctl', 'get', 'config.network.ports.rabbit')
        nc_wait(_env['control_ip'], rabbit_port)
@ -279,9 +284,7 @@ class RabbitMq(Question):
        (actions may have already been run, in which case we fail silently).
        """
        # Configure RabbitMQ
-        call('microstack.rabbitmqctl', 'add_user', 'openstack', 'rabbitmq')
-        shell(
-            'microstack.rabbitmqctl set_permissions openstack ".*" ".*" ".*"')
+        check('{SNAP}/bin/setup-rabbit'.format(**_env))

    def yes(self, answer: str) -> None:
        log.info('Waiting for RabbitMQ to start ...')
@ -293,7 +296,7 @@ class RabbitMq(Question):

    def no(self, answer: str):
        log.info('Disabling local rabbit ...')
-        check('systemctl', 'disable', 'snap.microstack.rabbitmq-server')
+        check('snapctl', 'stop', '--disable', 'microstack.rabbitmq-server')


 class DatabaseSetup(Question):
@ -361,7 +364,7 @@ class DatabaseSetup(Question):
              '--keystone-group', 'root')
        check('snap-openstack', 'launch', 'keystone-manage', 'db_sync')

-        restart('keystone-*')
+        restart('keystone-uwsgi')

        log.info('Bootstrapping Keystone ...')
        self._bootstrap()
@ -379,7 +382,7 @@ class DatabaseSetup(Question):
        check('snapctl', 'set', 'database.ready=true')

        log.info('Disabling local MySQL ...')
-        check('systemctl', 'disable', 'snap.microstack.mysqld')
+        check('snapctl', 'stop', '--disable', 'microstack.mysqld')


 class NovaHypervisor(Question):
@ -405,7 +408,7 @@ class NovaHypervisor(Question):

    def no(self, answer):
        log.info('Disabling nova compute service ...')
-        check('systemctl', 'disable', 'snap.microstack.nova-compute')
+        check('snapctl', 'stop', '--disable', 'microstack.nova-compute')


 class NovaControlPlane(Question):
@ -465,10 +468,6 @@ class NovaControlPlane(Question):
        # list automagically.
        for service in [
                'microstack.nova-api',
-                'microstack.nova-api-metadata',
-                'microstack.nova-conductor',
-                'microstack.nova-scheduler',
-                'microstack.nova-uwsgi',
        ]:
            check('snapctl', 'start', service)

@ -488,7 +487,16 @@ class NovaControlPlane(Question):

        check('snap-openstack', 'launch', 'nova-manage', 'db', 'sync')

-        restart('nova-*')
+        restart('nova-api')
+        restart('nova-compute')
+
+        for service in [
+                'microstack.nova-api-metadata',
+                'microstack.nova-conductor',
+                'microstack.nova-scheduler',
+                'microstack.nova-uwsgi',
+        ]:
+            check('snapctl', 'start', service)

        nc_wait(_env['compute_ip'], '8774')

@ -501,13 +509,13 @@ class NovaControlPlane(Question):
        log.info('Disabling nova control plane services ...')

        for service in [
-                'snap.microstack.nova-uwsgi',
-                'snap.microstack.nova-api',
-                'snap.microstack.nova-conductor',
-                'snap.microstack.nova-scheduler',
-                'snap.microstack.nova-api-metadata']:
+                'microstack.nova-uwsgi',
+                'microstack.nova-api',
+                'microstack.nova-conductor',
+                'microstack.nova-scheduler',
+                'microstack.nova-api-metadata']:

-            check('systemctl', 'disable', service)
+            check('snapctl', 'stop', '--disable', service)


 class NeutronControlPlane(Question):
@ -545,7 +553,14 @@ class NeutronControlPlane(Question):
        check('snap-openstack', 'launch', 'neutron-db-manage', 'upgrade',
              'head')

-        restart('neutron-*')
+        for service in [
+                'microstack.neutron-api',
+                'microstack.neutron-dhcp-agent',
+                'microstack.neutron-l3-agent',
+                'microstack.neutron-metadata-agent',
+                'microstack.neutron-openvswitch-agent',
+        ]:
+            check('snapctl', 'restart', service)

        nc_wait(_env['control_ip'], '9696')

@ -587,12 +602,12 @@ class NeutronControlPlane(Question):

        # Disable the other services.
        for service in [
-                'snap.microstack.neutron-api',
-                'snap.microstack.neutron-dhcp-agent',
-                'snap.microstack.neutron-metadata-agent',
-                'snap.microstack.neutron-l3-agent',
+                'microstack.neutron-api',
+                'microstack.neutron-dhcp-agent',
+                'microstack.neutron-metadata-agent',
+                'microstack.neutron-l3-agent',
        ]:
-            check('systemctl', 'disable', service)
+            check('snapctl', 'stop', '--disable', service)


 class GlanceSetup(Question):
@ -652,7 +667,8 @@ class GlanceSetup(Question):

        check('snap-openstack', 'launch', 'glance-manage', 'db_sync')

-        restart('glance*')
+        restart('glance-api')
+        restart('registry')

        nc_wait(_env['compute_ip'], '9292')

@ -661,37 +677,8 @@ class GlanceSetup(Question):
        self._fetch_cirros()

    def no(self, answer):
-        check('systemctl', 'disable', 'snap.microstack.glance-api')
-        check('systemctl', 'disable', 'snap.microstack.registry')
-
-
-class KeyPair(Question):
-    """Create a keypair for ssh access to instances.
-
-    TODO: split the asking from executing of questions, as ask about
-    this up front. (This needs to run at the end, but for user
-    experience reasons, we really want to ask all the non auto
-    questions at the beginning.)
-    """
-    _type = 'string'
-    config_key = 'config.credentials.key-pair'
-
-    def yes(self, answer: str) -> None:
-
-        if 'microstack' not in check_output('openstack', 'keypair', 'list'):
-            user = check_output('logname')
-            home = '/home/{}'.format(user)  # TODO make more portable!
-
-            log.info('Creating microstack keypair (~/.ssh/{})'.format(answer))
-            check('mkdir', '-p', '{home}/.ssh'.format(home=home))
-            check('chmod', '700', '{home}/.ssh'.format(home=home))
-            id_ = check_output('openstack', 'keypair', 'create', 'microstack')
-            id_path = '{home}/.ssh/{answer}'.format(home=home, answer=answer)
-
-            with open(id_path, 'w') as file_:
-                file_.write(id_)
-            check('chmod', '600', id_path)
-            check('chown', '{}:{}'.format(user, user), id_path)
+        check('snapctl', 'stop', '--disable', 'microstack.glance-api')
+        check('snapctl', 'stop', '--disable', 'microstack.registry')


 class SecurityRules(Question):
@ -736,7 +723,8 @@ class PostSetup(Question):
        log.info('restarting libvirt and virtlogd ...')
        # This fixes an issue w/ logging not getting set.
        # TODO: fix issue.
-        restart('*virt*')
+        restart('libvirtd')
+        restart('virtlogd')

        # Start horizon
        check('snapctl', 'start', 'microstack.horizon-uwsgi')
--- a/tools/init/init/questions/network.py
+++ b/tools/init/init/questions/network.py
@ -50,3 +50,10 @@ class IpForwarding(Question):
        log.info('Setting up ipv4 forwarding...')

        check('sysctl', 'net.ipv4.ip_forward=1')
+
+    def no(self, answer: str) -> None:
+        """This question doesn't actually work in a strictly confined snap, so
+        we default to the no and a noop for now.
+
+        """
+        pass
--- a/tools/init/init/shell.py
+++ b/tools/init/init/shell.py
@ -94,29 +94,6 @@ def call(*args: List[str], env: Dict = _env) -> bool:
    return not proc.returncode


-def shell(cmd: str, env: Dict = _env) -> int:
-    """Execute a command, using the actual bourne again shell.
-
-    Use this in cases where it is difficult to compose a comma
-    separate list that will get parsed into a succesful bash
-    command. (E.g., your bash command contains an argument like ".*"
-    ".*" ".*")
-
-    :param cmd: the command to run.
-    :param env: defaults to our Env singleton; can be overriden.
-
-    """
-    proc = subprocess.Popen(cmd, env=env, stdout=subprocess.PIPE,
-                            stderr=subprocess.STDOUT, bufsize=1,
-                            universal_newlines=True, shell=True)
-    for line in iter(proc.stdout.readline, ''):
-        log.debug(line)
-    proc.wait()
-    if proc.returncode:
-        raise subprocess.CalledProcessError(proc.returncode, cmd)
-    return proc.returncode
-
-
 def sql(cmd: str) -> None:
    """Execute some SQL!

@ -159,7 +136,7 @@ def restart(service: str) -> None:
                    e.g. *rabbit*

    """
-    check('systemctl', 'restart', 'snap.microstack.{}'.format(service))
+    check('snapctl', 'restart', 'microstack.{}'.format(service))


 def disable(service: str) -> None:
@ -169,8 +146,7 @@ def disable(service: str) -> None:
                    e.g. *rabbit*

    """
-    check('systemctl', 'disable', 'snap.microstack.{}'.format(service))
-    check('systemctl', 'mask', 'snap.microstack.{}'.format(service))
+    check('snapctl', 'stop', '--disable', 'microstack.{}'.format(service))


 def download(url: str, output: str) -> None:
--- a/tools/launch/launch/main.py
+++ b/tools/launch/launch/main.py
@ -53,6 +53,35 @@ def parse_args():
    return args


+def check_keypair():
+    """
+    Check for the microstack keypair's existence, creating it if it doesn't.
+
+    """
+    key_path = check_output(
+        'snapctl', 'get', 'config.credentials.key-pair').format(**os.environ)
+
+    if os.path.exists(key_path):
+        return key_path
+
+    print('Creating local "microstack" ssh key at {}'.format(key_path))
+    # TODO: make sure that we get rid of this path on snap
+    # uninstall. If we don't, check to make sure that MicroStack
+    # has a microstack ssh key, in addition to checking for the
+    # existence of the file.
+    key_dir = os.sep.join(key_path.split(os.sep)[:-1])
+    check('mkdir', '-p', key_dir)
+    check('chmod', '700', key_dir)
+
+    id_ = check_output('openstack', 'keypair', 'create', 'microstack')
+
+    with open(key_path, 'w') as file_:
+        file_.write(id_)
+        check('chmod', '600', key_path)
+
+    return key_path
+
+
 def create_server(name, args):

    cmd = [
@ -131,6 +160,17 @@ def check_server(name, server_id, args):
 def launch(name, args):
    """Launch a server!"""

+    if args.key == 'microstack':
+        # Make sure that we have a default ssh key to hand off to the
+        # instance.
+        key_path = check_keypair()
+    else:
+        # We've been passed an ssh key with an unknown path. Drop in
+        # some placeholder text for the message at the end of this
+        # routine, but don't worry about verifying it. We trust the
+        # caller to have created it!
+        key_path = '/path/to/ssh/key'
+
    print("Launching server ...")
    server_id = create_server(name, args)

@ -157,15 +197,11 @@ def launch(name, args):
    if 'cirros' in args.image.lower():
        username = 'cirros'

-    ssh_key = '/path/to/ssh/key'
-    if args.key == 'microstack':
-        ssh_key = '$HOME/.ssh/id_microstack'
-
    print("""\
 Server {name} launched! (status is {status})

-Access it with `ssh -i {ssh_key} {username}@{ip}`\
-""".format(name=name, status=status, ssh_key=ssh_key,
+Access it with `ssh -i {key_path} {username}@{ip}`\
+""".format(name=name, status=status, key_path=key_path,
           username=username, ip=ip))

    gate = check_output('snapctl', 'get', 'config.network.ext-gateway')
--- a/tox.ini
+++ b/tox.ini
@ -8,7 +8,7 @@ install_command = pip install {opts} {packages}
 deps = -r{toxinidir}/test-requirements.txt
 setenv =
    PATH = /snap/bin:{env:PATH}
-passenv = HOME TERM DISTRO INTERACTIVE_DEBUG
+passenv = HOME TERM DISTRO INTERACTIVE_DEBUG USER
 whitelist_externals =
    sudo
    /snap/bin/snapcraft