[![Build Status](https://travis-ci.org/Mirantis/solar.svg?branch=master)](https://travis-ci.org/Mirantis/solar) [![Coverage Status](https://coveralls.io/repos/Mirantis/solar/badge.svg?branch=master&service=github)](https://coveralls.io/github/Mirantis/solar?branch=master) # Requirements ## Supported development platforms Linux or MacOS ## Additional software [VirtualBox](https://www.virtualbox.org/wiki/Downloads): 5.x [Vagrant](http://www.vagrantup.com/downloads.html): 1.7.x Note: Make sure that [Vagrant VirtualBox Guest plugin](https://github.com/dotless-de/vagrant-vbguest) is installed `vagrant plugin install vagrant-vbguest` Note: If you are using VirtualBox 5.0 it's worth uncommenting paravirtprovider setting in `vagrant-settings.yaml` for speed improvements: ```bash paravirtprovider: kvm ``` For details see Customizing `vagrant-settings.yaml` section. # Setup development env Setup environment: ```bash cd solar vagrant up ``` Login into vm, the code is available in /vagrant directory ```bash vagrant ssh solar --help ``` Get ssh details for running slave nodes (vagrant/vagrant): ```bash vagrant ssh-config ``` You can make/restore snapshots of boxes (this is way faster than reprovisioning them) with the `snapshotter.py` script: ```bash ./snapshotter.py take -n my-snapshot ./snapshotter.py show ./snapshotter.py restore -n my-snapshot ``` `snapshoter.py` to run requires python module `click`. 1. On debian based systems you can install it via `sudo aptitude install python-click-cli`, 2. On fedora 22 you can install it via `sudo dnf install python-click`, 3. If you use virtualenv or similar tool then you can install it just with `pip install click`, 4. If you don't have virtualenv and your operating system does not provide package for it then `sudo pip install click`. 5. If you don't have `pip` then [install it](https://pip.pypa.io/en/stable/installing/) and then execute command step 4. # Solar usage For now all commands should be executed from `solar-dev` machine from `/vagrant` directory. Basic flow is: 1. Create some resources (look at `solar-resources/examples/openstack/openstack.py`) and connect them between each other, and place them on nodes. 1. Run `solar changes stage` (this stages the changes) 1. Run `solar changes process` (this prepares orchestrator graph, returning change UUID) 1. Run `solar orch run-once ` (or `solar orch run-once last` to run the lastly created graph) 1. Observe progress of orch with `watch 'solar orch report '` (or `watch 'solar orch report last'`). Some very simple cluster setup: ```bash cd /vagrant solar resource create nodes templates/nodes '{"count": 2}' solar resource create mariadb_service resources/mariadb_service '{"image": "mariadb", "root_password": "mariadb", "port": 3306}' solar resource create keystone_db resources/mariadb_db/ '{"db_name": "keystone_db", "login_user": "root"}' solar resource create keystone_db_user resources/mariadb_user/ user_name=keystone user_password=keystone # another valid format solar connect node1 mariadb_service solar connect node1 keystone_db solar connect mariadb_service keystone_db '{"root_password": "login_password", "port": "login_port", "ip": "db_host"}' # solar connect mariadb_service keystone_db_user 'root_password->login_password port->login_port' # another valid format solar connect keystone_db keystone_db_user solar changes stage solar changes process # solar orch run-once # or solar orch run-once last watch 'solar orch report ' # or solar orch report last ``` You can fiddle with the above configuration like this: ```bash solar resource update keystone_db_user '{"user_password": "new_keystone_password"}' solar resource update keystone_db_user user_password=new_keystone_password # another valid format solar changes stage solar changes process solar orch run-once ``` To get data for the resource `bar` (raw and pretty-JSON): ```bash solar resource show --tag 'resources/bar' solar resource show --json --tag 'resources/bar' | jq . solar resource show --name 'resource_name' solar resource show --name 'resource_name' --json | jq . ``` To clear all resources/connections: ```bash solar resource clear_all ``` Show the connections/graph: ```bash solar connections show solar connections graph ``` You can also limit graph to show only specific resources: ```bash solar connections graph --start-with mariadb_service --end-with keystone_db ``` You can make sure that all input values are correct and mapped without duplicating your values with this command: ```bash solar resource validate ``` Disconnect ```bash solar disconnect mariadb_service node1 ``` Tag a resource: ```bash solar resource tag node1 test-tags # Remove tags solar resource tag node1 test-tag --delete ``` # Low level API ## Usage: Creating resources: ```python from solar.core.resource import composer as cr node1 = cr.create('node1', 'resources/ro_node/', 'rs/', {'ip':'10.0.0.3', 'ssh_key' : '/vagrant/tmp/keys/ssh_private', 'ssh_user':'vagrant'})[0] node2 = cr.create('node2', 'resources/ro_node/', 'rs/', {'ip':'10.0.0.4', 'ssh_key' : '/vagrant/tmp/keys/ssh_private', 'ssh_user':'vagrant'})[0] keystone_db_data = cr.create('mariadb_keystone_data', 'resources/data_container/', 'rs/', {'image' : 'mariadb', 'export_volumes' : ['/var/lib/mysql'], 'ip': '', 'ssh_user': '', 'ssh_key': ''}, connections={'ip' : 'node2.ip', 'ssh_key':'node2.ssh_key', 'ssh_user':'node2.ssh_user'})[0] nova_db_data = cr.create('mariadb_nova_data', 'resources/data_container/', 'rs/', {'image' : 'mariadb', 'export_volumes' : ['/var/lib/mysql'], 'ip': '', 'ssh_user': '', 'ssh_key': ''}, connections={'ip' : 'node1.ip', 'ssh_key':'node1.ssh_key', 'ssh_user':'node1.ssh_user'})[0] ``` To make connection after resource is created use `signal.connect`. To test notifications: ```python keystone_db_data.args # displays node2 IP node2.update({'ip': '10.0.0.5'}) keystone_db_data.args # updated IP ``` If you close the Python shell you can load the resources like this: ```python from solar.core import resource node1 = resource.load('rs/node1') node2 = resource.load('rs/node2') keystone_db_data = resource.load('rs/mariadb_keystone_data') nova_db_data = resource.load('rs/mariadb_nova_data') ``` Connections are loaded automatically. You can also load all resources at once: ```python from solar.core import resource all_resources = resource.load_all('rs') ``` ## Dry run Solar CLI has possibility to show dry run of actions to be performed. To see what will happen when you run Puppet action, for example, try this: ```bash solar resource action keystone_puppet run -d ``` This should print out something like this: ```bash EXECUTED: 73c6cb1cf7f6cdd38d04dd2d0a0729f8: (0, 'SSH RUN', ('sudo cat /tmp/puppet-modules/Puppetfile',), {}) 3dd4d7773ce74187d5108ace0717ef29: (1, 'SSH SUDO', ('mv "1038cb062449340bdc4832138dca18cba75caaf8" "/tmp/puppet-modules/Puppetfile"',), {}) ae5ad2455fe2b02ba46b4b7727eff01a: (2, 'SSH RUN', ('sudo librarian-puppet install',), {}) 208764fa257ed3159d1788f73c755f44: (3, 'SSH SUDO', ('puppet apply -vd /tmp/action.pp',), {}) ``` By default every mocked command returns an empty string. If you want it to return something else (to check how would dry run behave in different situation) you provide a mapping (in JSON format), something along the lines of: ```bash solar resource action keystone_puppet run -d -m "{\"73c\": \"mod 'openstack-keystone'\n\"}" ``` The above means the return string of first command (with hash `73c6c...`) will be as specified in the mapping. Notice that in mapping you don't have to specify the whole hash, just it's unique beginning. Also, you don't have to specify the whole return string in mapping. Dry run executor can read file and return it's contents instead, just use the `>` operator when specifying hash: ```bash solar resource action keystone_puppet run -d -m "{\"73c>\": \"./Puppetlabs-file\"}" ``` # Resource compiling You can compile all `meta.yaml` definitions into Python code with classes that derive from `Resource`. To do this run ```bash solar resource compile_all ``` This generates file `resources_compiled.py` in the main directory (do not commit this file into the repo). Then you can import classes from that file, create their instances and assign values just like these were normal properties. If your editor supports Python static checking, you will have autocompletion there too. An example on how to create a node with this: ```python import resources_compiled node1 = resources_compiled.RoNodeResource('node1', None, {}) node1.ip = '10.0.0.3' node1.ssh_key = '/vagrant/.vagrant/machines/solar-dev1/virtualbox/private_key' node1.ssh_user = 'vagrant' ``` # Higher-level API There's also a higher-level API that allows to write resource instances in more functional way, and in particular avoid `for` loops. Here's an example: ```python from solar import template nodes = template.nodes_from('templates/riak_nodes') riak_services = nodes.on_each( 'resources/riak_node', { 'riak_self_name': 'riak{num}', 'riak_hostname': 'riak_server{num}.solar', 'riak_name': 'riak{num}@riak_server{num}.solar', } ) riak_master_service = riak_services.take(0) riak_slave_services = riak_services.tail() riak_master_service.connect_list( riak_slave_services, { 'riak_name': 'join_to', } ) ``` For full Riak example, please look at `solar-resources/examples/riak/riaks-template.py`. Full documentation of individual functions is found in the `solar/template.py` file. # Customizing vagrant-settings.yaml Solar is shipped with sane defaults in `vagrant-setting.yaml_defaults`. If you need to adjust them for your needs, e.g. changing resource allocation for VirtualBox machines, you should just compy the file to `vagrant-setting.yaml` and make your modifications. # Image based provisioning with Solar * In `vagrant-setting.yaml_defaults` or `vagrant-settings.yaml` file uncomment `preprovisioned: false` line. * Run `vagrant up`, it will take some time because it builds image for bootstrap and IBP images. * Now you can run provisioning `/vagrant/solar-resources/examples/provisioning/provision.sh`