======== Metadata ======== Nova presents configuration information to instances it starts via a mechanism called metadata. These mechanisms are widely used via helpers such as `cloud-init`_ to specify things like the root password the instance should use. This metadata is made available via either a *config drive* or the *metadata service* and can be somewhat customised by the user using the *user data* feature. This guide provides an overview of these features along with a summary of the types of metadata available. .. _cloud-init: https://cloudinit.readthedocs.io/en/latest/ Types of metadata ----------------- There are three separate groups of users who need to be able to specify metadata for an instance. User provided data ~~~~~~~~~~~~~~~~~~ The user who booted the instance can pass metadata to the instance in several ways. For authentication keypairs, the keypairs functionality of the nova API can be used to upload a key and then specify that key during the nova boot API request. For less structured data, a small opaque blob of data may be passed via the :ref:`user data ` feature of the nova API. Examples of such unstructured data would be the puppet role that the instance should use, or the HTTP address of a server from which to fetch post-boot configuration information. Nova provided data ~~~~~~~~~~~~~~~~~~ Nova itself needs to pass information to the instance via its internal implementation of the metadata system. Such information includes the requested hostname for the instance and the availability zone the instance is in. This happens by default and requires no configuration by the user or deployer. Nova provides both an :ref:`OpenStack metadata API ` and an :ref:`EC2-compatible API `. Both the OpenStack metadata and EC2-compatible APIs are versioned by date. These are described later. Deployer provided data ~~~~~~~~~~~~~~~~~~~~~~ A deployer of OpenStack may need to pass data to an instance. It is also possible that this data is not known to the user starting the instance. An example might be a cryptographic token to be used to register the instance with Active Directory post boot -- the user starting the instance should not have access to Active Directory to create this token, but the nova deployment might have permissions to generate the token on the user's behalf. This is possible using the :ref:`vendordata ` feature, which must be configured by your cloud operator. .. _metadata-service: The metadata service -------------------- .. note:: This section provides end user information about the metadata service. For deployment information about the metadata service, refer to the :doc:`admin guide `. The *metadata service* provides a way for instances to retrieve instance-specific data via a REST API. Instances access this service at ``169.254.169.254`` or at ``fe80::a9fe:a9fe``. All types of metadata, be it user-, nova- or vendor-provided, can be accessed via this service. .. versionchanged:: 22.0.0 Starting with the Victoria release the metadata service is accessible over IPv6 at the link-local address ``fe80::a9fe:a9fe``. .. note:: As with all IPv6 link-local addresses, the metadata IPv6 address is not complete without a zone identifier (in a Linux guest that is usually the interface name concatenated after a percent sign). Please also note that in URLs you should URL-encode the percent sign itself. For example, assuming that the primary network interface in the guest is ``ens2`` substitute ``http://[fe80::a9fe:a9fe%25ens2]:80/...`` for ``http://169.254.169.254/...``. Using the metadata service ~~~~~~~~~~~~~~~~~~~~~~~~~~ To retrieve a list of supported versions for the :ref:`OpenStack metadata API `, make a GET request to ``http://169.254.169.254/openstack``, which will return a list of directories: .. code-block:: console $ curl http://169.254.169.254/openstack 2012-08-10 2013-04-04 2013-10-17 2015-10-15 2016-06-30 2016-10-06 2017-02-22 2018-08-27 latest Refer to :ref:`OpenStack format metadata ` for information on the contents and structure of these directories. To list supported versions for the :ref:`EC2-compatible metadata API `, make a GET request to ``http://169.254.169.254``, which will, once again, return a list of directories: .. code-block:: console $ curl http://169.254.169.254 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 2009-04-04 latest Refer to :ref:`EC2-compatible metadata ` for information on the contents and structure of these directories. .. _metadata-config-drive: Config drives ------------- .. note:: This section provides end user information about config drives. For deployment information about the config drive feature, refer to the :doc:`admin guide `. *Config drives* are special drives that are attached to an instance when it boots. The instance can mount this drive and read files from it to get information that is normally available through the metadata service. One use case for using the config drive is to pass a networking configuration when you do not use DHCP to assign IP addresses to instances. For example, you might pass the IP address configuration for the instance through the config drive, which the instance can mount and access before you configure the network settings for the instance. Using the config drive ~~~~~~~~~~~~~~~~~~~~~~ To enable the config drive for an instance, pass the ``--config-drive true`` parameter to the :command:`openstack server create` command. The following example enables the config drive and passes a user data file and two key/value metadata pairs, all of which are accessible from the config drive: .. code-block:: console $ openstack server create --config-drive true --image my-image-name \ --flavor 1 --key-name mykey --user-data ./my-user-data.txt \ --property role=webservers --property essential=false MYINSTANCE .. note:: The Compute service can be configured to always create a config drive. For more information, refer to :doc:`the admin guide `. If your guest operating system supports accessing disk by label, you can mount the config drive as the ``/dev/disk/by-label/configurationDriveVolumeLabel`` device. In the following example, the config drive has the ``config-2`` volume label: .. code-block:: console # mkdir -p /mnt/config # mount /dev/disk/by-label/config-2 /mnt/config If your guest operating system does not use ``udev``, the ``/dev/disk/by-label`` directory is not present. You can use the :command:`blkid` command to identify the block device that corresponds to the config drive. For example: .. code-block:: console # blkid -t LABEL="config-2" -odevice /dev/vdb Once identified, you can mount the device: .. code-block:: console # mkdir -p /mnt/config # mount /dev/vdb /mnt/config Once mounted, you can examine the contents of the config drive: .. code-block:: console $ cd /mnt/config $ find . -maxdepth 2 . ./ec2 ./ec2/2009-04-04 ./ec2/latest ./openstack ./openstack/2012-08-10 ./openstack/2013-04-04 ./openstack/2013-10-17 ./openstack/2015-10-15 ./openstack/2016-06-30 ./openstack/2016-10-06 ./openstack/2017-02-22 ./openstack/latest The files that appear on the config drive depend on the arguments that you pass to the :command:`openstack server create` command. The format of this directory is the same as that provided by the :ref:`metadata service `, with the exception that the EC2-compatible metadata is now located in the ``ec2`` directory instead of the root (``/``) directory. Refer to the :ref:`metadata-openstack-format` and :ref:`metadata-ec2-format` sections for information about the format of the files and subdirectories within these directories. Nova metadata ------------- As noted previously, nova provides its metadata in two formats: OpenStack format and EC2-compatible format. .. _metadata-openstack-format: OpenStack format metadata ~~~~~~~~~~~~~~~~~~~~~~~~~ .. versionchanged:: 12.0.0 Support for network metadata was added in the Liberty release. Metadata from the OpenStack API is distributed in JSON format. There are two files provided for each version: ``meta_data.json`` and ``network_data.json``. The ``meta_data.json`` file contains nova-specific information, while the ``network_data.json`` file contains information retrieved from neutron. For example: .. code-block:: console $ curl http://169.254.169.254/openstack/2018-08-27/meta_data.json .. code-block:: json { "random_seed": "yu5ZnkqF2CqnDZVAfZgarG...", "availability_zone": "nova", "keys": [ { "data": "ssh-rsa AAAAB3NzaC1y...== Generated by Nova\n", "type": "ssh", "name": "mykey" } ], "hostname": "test.novalocal", "launch_index": 0, "meta": { "priority": "low", "role": "webserver" }, "devices": [ { "type": "nic", "bus": "pci", "address": "0000:00:02.0", "mac": "00:11:22:33:44:55", "tags": ["trusted"] }, { "type": "disk", "bus": "ide", "address": "0:0", "serial": "disk-vol-2352423", "path": "/dev/sda", "tags": ["baz"] } ], "project_id": "f7ac731cc11f40efbc03a9f9e1d1d21f", "public_keys": { "mykey": "ssh-rsa AAAAB3NzaC1y...== Generated by Nova\n" }, "name": "test" } .. code-block:: console $ curl http://169.254.169.254/openstack/2018-08-27/network_data.json .. code-block:: json { "links": [ { "ethernet_mac_address": "fa:16:3e:9c:bf:3d", "id": "tapcd9f6d46-4a", "mtu": null, "type": "bridge", "vif_id": "cd9f6d46-4a3a-43ab-a466-994af9db96fc" } ], "networks": [ { "id": "network0", "link": "tapcd9f6d46-4a", "network_id": "99e88329-f20d-4741-9593-25bf07847b16", "type": "ipv4_dhcp" } ], "services": [ { "address": "8.8.8.8", "type": "dns" } ] } ::download:`Download` network_data.json JSON schema. .. _metadata-ec2-format: EC2-compatible metadata ~~~~~~~~~~~~~~~~~~~~~~~ The EC2-compatible API is compatible with version 2009-04-04 of the `Amazon EC2 metadata service`__ This means that virtual machine images designed for EC2 will work properly with OpenStack. The EC2 API exposes a separate URL for each metadata element. Retrieve a listing of these elements by making a GET query to ``http://169.254.169.254/2009-04-04/meta-data/``. For example: .. code-block:: console $ curl http://169.254.169.254/2009-04-04/meta-data/ ami-id ami-launch-index ami-manifest-path block-device-mapping/ hostname instance-action instance-id instance-type kernel-id local-hostname local-ipv4 placement/ public-hostname public-ipv4 public-keys/ ramdisk-id reservation-id security-groups .. code-block:: console $ curl http://169.254.169.254/2009-04-04/meta-data/block-device-mapping/ ami .. code-block:: console $ curl http://169.254.169.254/2009-04-04/meta-data/placement/ availability-zone .. code-block:: console $ curl http://169.254.169.254/2009-04-04/meta-data/public-keys/ 0=mykey Instances can retrieve the public SSH key (identified by keypair name when a user requests a new instance) by making a GET request to ``http://169.254.169.254/2009-04-04/meta-data/public-keys/0/openssh-key``: .. code-block:: console $ curl http://169.254.169.254/2009-04-04/meta-data/public-keys/0/openssh-key ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgQDYVEprvtYJXVOBN0XNKVVRNCRX6BlnNbI+US\ LGais1sUWPwtSg7z9K9vhbYAPUZcq8c/s5S9dg5vTHbsiyPCIDOKyeHba4MUJq8Oh5b2i71/3B\ ISpyxTBH/uZDHdslW2a+SrPDCeuMMoss9NFhBdKtDkdG9zyi0ibmCP6yMdEX8Q== Generated\ by Nova __ https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html .. _metadata-userdata: User data --------- *User data* is a blob of data that the user can specify when they launch an instance. The instance can access this data through the metadata service or config drive. Commonly used to pass a shell script that the instance runs on boot. For example, one application that uses user data is the `cloud-init `__ system, which is an open-source package from Ubuntu that is available on various Linux distributions and which handles early initialization of a cloud instance. You can place user data in a local file and pass it through the ``--user-data `` parameter at instance creation. .. code-block:: console $ openstack server create --image ubuntu-cloudimage --flavor 1 \ --user-data mydata.file VM_INSTANCE .. note:: The provided user data should not be base64-encoded, as it will be automatically encoded in order to pass valid input to the REST API, which has a limit of 65535 bytes after encoding. Once booted, you can access this data from the instance using either the metadata service or the config drive. To access it via the metadata service, make a GET request to either ``http://169.254.169.254/openstack/{version}/user_data`` (OpenStack API) or ``http://169.254.169.254/{version}/user-data`` (EC2-compatible API). For example: .. code-block:: console $ curl http://169.254.169.254/openstack/2018-08-27/user_data .. code-block:: shell #!/bin/bash echo 'Extra user data here' .. _metadata-vendordata: Vendordata ---------- .. note:: This section provides end user information about the vendordata feature. For deployment information about this feature, refer to the :doc:`admin guide `. .. versionchanged:: 14.0.0 Support for dynamic vendor data was added in the Newton release. **Where configured**, instances can retrieve vendor-specific data from the metadata service or config drive. To access it via the metadata service, make a GET request to either ``http://169.254.169.254/openstack/{version}/vendor_data.json`` or ``http://169.254.169.254/openstack/{version}/vendor_data2.json``, depending on the deployment. For example: .. code-block:: console $ curl http://169.254.169.254/openstack/2018-08-27/vendor_data2.json .. code-block:: json { "testing": { "value1": 1, "value2": 2, "value3": "three" } } .. note:: The presence and contents of this file will vary from deployment to deployment. General guidelines ------------------ - Do not rely on the presence of the EC2 metadata in the metadata API or config drive, because this content might be removed in a future release. For example, do not rely on files in the ``ec2`` directory. - When you create images that access metadata service or config drive data and multiple directories are under the ``openstack`` directory, always select the highest API version by date that your consumer supports. For example, if your guest image supports the ``2012-03-05``, ``2012-08-05``, and ``2013-04-13`` versions, try ``2013-04-13`` first and fall back to a previous version if ``2013-04-13`` is not present.