2a56aa003d
We use v5 and v5.3.0 to put different python files for CDH5 and CDH5.3.0. CDH5 is CDH5.0.0, we use the name "CDH5" instead of "CDH5.0.0" for backward support. Currently since CDH5.0.0 does not support cm_api>6, we cannot use first_run API in CDH5.0.0, so we only implemented parts of the services that we implemented in CDH5.3.0. implements bp: cdh-version-management Change-Id: I3b3058f25912ddf6206d64db88ac40138a45a53f
314 lines
22 KiB
JSON
314 lines
22 KiB
JSON
[
|
|
{
|
|
"desc": "For advanced use only, a string to be inserted into <strong>hive-site.xml</strong>. Applies to configurations of all roles in this service except client configuration.",
|
|
"display_name": "Hive Service Advanced Configuration Snippet (Safety Valve) for hive-site.xml",
|
|
"name": "hive_service_config_safety_valve",
|
|
"value": null
|
|
},
|
|
{
|
|
"desc": "Directory containing auxiliary JARs used by Hive. This should be a directory location and not a classpath containing one or more JARs. This directory must be created and managed manually on Hive CLI or HiveServer2 host.",
|
|
"display_name": "Hive Auxiliary JARs Directory",
|
|
"name": "hive_aux_jars_path_dir",
|
|
"value": null
|
|
},
|
|
{
|
|
"desc": "Comma-delimited list of groups that you want to allow the Oozie user to impersonate. The default '*' allows all groups. To disable entirely, use a string that doesn't correspond to a group name, such as '_no_group_'.",
|
|
"display_name": "Oozie Proxy User Groups",
|
|
"name": "oozie_proxy_user_groups_list",
|
|
"value": "*"
|
|
},
|
|
{
|
|
"desc": "Password for Hive Metastore database",
|
|
"display_name": "Hive Metastore Database Password",
|
|
"name": "hive_metastore_database_password",
|
|
"value": ""
|
|
},
|
|
{
|
|
"desc": "Directory name where Hive Metastore's database is stored (only for Derby)",
|
|
"display_name": "Hive Metastore Derby Path",
|
|
"name": "hive_metastore_derby_path",
|
|
"value": "/var/lib/hive/cloudera_manager/derby/metastore_db"
|
|
},
|
|
{
|
|
"desc": "Hive warehouse directory is the location in HDFS where Hive's tables are stored. Note that Hive's default value for its warehouse directory is '/user/hive/warehouse'.",
|
|
"display_name": "Hive Warehouse Directory",
|
|
"name": "hive_warehouse_directory",
|
|
"value": "/user/hive/warehouse"
|
|
},
|
|
{
|
|
"desc": "SSL keystore password.",
|
|
"display_name": "Keystore Password",
|
|
"name": "hiveserver2_keystore_password",
|
|
"value": null
|
|
},
|
|
{
|
|
"desc": "<p>\nConfigures the rules for event tracking and coalescing. This feature is\nused to define equivalency between different audit events. When\nevents match, according to a set of configurable parameters, only one\nentry in the audit list is generated for all the matching events.\n</p>\n\n<p>\nTracking works by keeping a reference to events when they first appear,\nand comparing other incoming events against the \"tracked\" events according\nto the rules defined here.\n</p>\n\n<p>Event trackers are defined in a JSON object like the following:</p>\n\n<pre>\n{\n \"timeToLive\" : [integer],\n \"fields\" : [\n {\n \"type\" : [string],\n \"name\" : [string]\n }\n ]\n}\n</pre>\n\n<p>\nWhere:\n</p>\n\n<ul>\n <li>timeToLive: maximum amount of time an event will be tracked, in\n milliseconds. Must be provided. This defines how long, since it's\n first seen, an event will be tracked. A value of 0 disables tracking.</li>\n\n <li>fields: list of fields to compare when matching events against\n tracked events.</li>\n</ul>\n\n<p>\nEach field has an evaluator type associated with it. The evaluator defines\nhow the field data is to be compared. The following evaluators are\navailable:\n</p>\n\n<ul>\n <li>value: uses the field value for comparison.</li>\n\n <li>userName: treats the field value as a userNname, and ignores any\n host-specific data. This is useful for environment using Kerberos,\n so that only the principal name and realm are compared.</li>\n</ul>\n\n<p>\nThe following is the list of fields that can be used to compare Hive events:\n</p>\n\n<ul>\n <li>username: the user performing the action.</li>\n <li>ipAddress: the IP from where the request originated.</li>\n <li>operation: the Hive operation being performed.</li> \n <li>databaseName: the database affected by the operation.</li>\n <li>tableName: the table affected by the operation.</li> \n</ul>\n\n",
|
|
"display_name": "Event Tracker",
|
|
"name": "navigator_event_tracker",
|
|
"value": null
|
|
},
|
|
{
|
|
"desc": "Action to take when the audit event queue is full. Drop the event or shutdown the affected process.",
|
|
"display_name": "Queue Policy",
|
|
"name": "navigator_audit_queue_policy",
|
|
"value": "DROP"
|
|
},
|
|
{
|
|
"desc": "Comma-delimited list of hosts where you want to allow the Oozie user to impersonate other users. The default '*' allows all hosts. To disable entirely, use a string that doesn't correspond to a host name, such as '_no_host'.",
|
|
"display_name": "Oozie Proxy User Hosts",
|
|
"name": "oozie_proxy_user_hosts_list",
|
|
"value": "*"
|
|
},
|
|
{
|
|
"desc": "The user that this service's processes should run as.",
|
|
"display_name": "System User",
|
|
"name": "process_username",
|
|
"value": "hive"
|
|
},
|
|
{
|
|
"desc": "<p>Event filters are defined in a JSON object like the following:</p>\n\n<pre>\n{\n \"defaultAction\" : (\"accept\", \"discard\"),\n \"rules\" : [\n {\n \"action\" : (\"accept\", \"discard\"),\n \"fields\" : [\n {\n \"name\" : \"fieldName\",\n \"match\" : \"regex\"\n }\n ]\n }\n ]\n}\n</pre>\n\n<p>\nA filter has a default action and a list of rules, in order of precedence.\nEach rule defines an action, and a list of fields to match against the\naudit event.\n</p>\n\n<p>\nA rule is \"accepted\" if all the listed field entries match the audit\nevent. At that point, the action declared by the rule is taken.\n</p>\n\n<p>\nIf no rules match the event, the default action is taken. Actions\ndefault to \"accept\" if not defined in the JSON object.\n</p>\n\n<p>\nThe following is the list of fields that can be filtered for Hive events:\n</p>\n\n<ul>\n <li>userName: the user performing the action.</li>\n <li>ipAddress: the IP from where the request originated.</li>\n <li>operation: the Hive operation being performed.</li> \n <li>databaseName: the databaseName for the operation.</li>\n <li>tableName: the tableName for the operation.</li>\n</ul>\n",
|
|
"display_name": "Event Filter",
|
|
"name": "navigator_audit_event_filter",
|
|
"value": "{\n \"comment\" : [\n \"Default filter for Hive services.\",\n \"Discards events generated by Hive MR jobs in /tmp directory\"\n ],\n \"defaultAction\" : \"accept\",\n \"rules\" : [\n {\n \"action\" : \"discard\",\n \"fields\" : [\n { \"name\" : \"operation\", \"match\" : \"QUERY\" },\n { \"name\" : \"objectType\", \"match\" : \"DFS_DIR\"},\n { \"name\" : \"resourcePath\", \"match\" : \"/tmp/hive-(?:.+)?/hive_(?:.+)?/-mr-.*\" }\n ]\n }\n ]\n}\n"
|
|
},
|
|
{
|
|
"desc": "Max number of reducers to use. If the configuration parameter Hive Reduce Tasks is negative, Hive will limit the number of reducers to the value of this parameter.",
|
|
"display_name": "Hive Max Reducers",
|
|
"name": "hive_max_reducers",
|
|
"value": "999"
|
|
},
|
|
{
|
|
"desc": "Enable support for encrypted client-server communication using Secure Socket Layer (SSL) for HiveServer2 connections. This is only applicable to non-Kerberos environments.",
|
|
"display_name": "Enable SSL for HiveServer",
|
|
"name": "hiveserver2_enable_ssl",
|
|
"value": "false"
|
|
},
|
|
{
|
|
"desc": "Type of Hive Metastore database. Note that Derby is not recommended and Cloudera Impala does not support Derby.",
|
|
"display_name": "Hive Metastore Database Type",
|
|
"name": "hive_metastore_database_type",
|
|
"value": "mysql"
|
|
},
|
|
{
|
|
"desc": "<p>The configured triggers for this service. This is a JSON formatted list of triggers. These triggers are evaluated as part as the health system. Every trigger expression is parsed, and if the trigger condition is met, the list of actions provided in the trigger expression is executed.</p><p>Each trigger has all of the following fields:</p><ul><li><code>triggerName</code> <strong>(mandatory)</strong> - the name of the trigger. This value must be unique for the specific service. </li><li><code>triggerExpression</code> <strong>(mandatory)</strong> - a tsquery expression representing the trigger. </li><li><code>streamThreshold</code> <strong>(optional)</strong> - the maximum number of streams that can satisfy a condition of a trigger before the condition fires. By default set to 0, and any stream returned will cause the condition to fire. </li><li><code>enabled</code> <strong> (optional)</strong> - by default set to 'true'. If set to 'false' the trigger will not be evaluated.</li></ul></p><p>For example, here is a JSON formatted trigger that fires if there are more than 10 DataNodes with more than 500 file-descriptors opened:</p><p><pre>[{\"triggerName\": \"sample-trigger\",\n \"triggerExpression\": \"IF (SELECT fd_open WHERE roleType = DataNode and last(fd_open) > 500) DO health:bad\",\n \"streamThreshold\": 10, \"enabled\": \"true\"}]</pre></p><p>Consult the trigger rules documentation for more details on how to write triggers using tsquery.</p><p>The JSON format is evolving and may change in the future and as a result backward compatibility is not guaranteed between releases at this time.</p>",
|
|
"display_name": "Service Triggers",
|
|
"name": "service_triggers",
|
|
"value": "[]"
|
|
},
|
|
{
|
|
"desc": "Maximum number of rolled over audit logs to retain. The logs will not be deleted if they contain audit events that have not yet been propagated to Audit Server.",
|
|
"display_name": "Number of Audit Logs to Retain",
|
|
"name": "navigator_audit_log_max_backup_index",
|
|
"value": "10"
|
|
},
|
|
{
|
|
"desc": "Port number of Hive Metastore database",
|
|
"display_name": "Hive Metastore Database Port",
|
|
"name": "hive_metastore_database_port",
|
|
"value": "3306"
|
|
},
|
|
{
|
|
"desc": "Name of the ZooKeeper service that this Hive service instance depends on.",
|
|
"display_name": "ZooKeeper Service",
|
|
"name": "zookeeper_service",
|
|
"value": null
|
|
},
|
|
{
|
|
"desc": "Host name of Hive Metastore database",
|
|
"display_name": "Hive Metastore Database Host",
|
|
"name": "hive_metastore_database_host",
|
|
"value": "localhost"
|
|
},
|
|
{
|
|
"desc": "Maximum size of audit log file in MB before it is rolled over.",
|
|
"display_name": "Maximum Audit Log File Size",
|
|
"name": "navigator_audit_log_max_file_size",
|
|
"value": "100"
|
|
},
|
|
{
|
|
"desc": "Prevent Metastore operations in the event of schema version incompatibility. Consider setting this to true to reduce probability of schema corruption during Metastore operations. Note that setting this property to true will also set datanucleus.autoCreateSchema property to false and datanucleus.fixedDatastore property to true. Any values set in Cloudera Manager for these properties will be overridden.",
|
|
"display_name": "Strict Hive Metastore Schema Validation",
|
|
"name": "hive_metastore_schema_verification",
|
|
"value": "true"
|
|
},
|
|
{
|
|
"desc": "Path to the SSL keystore.",
|
|
"display_name": "Keystore File Path",
|
|
"name": "hiveserver2_keystore_path",
|
|
"value": null
|
|
},
|
|
{
|
|
"desc": "HDFS path to the global policy file for Sentry authorization. This should be a relative path (and not a full HDFS URL). The global policy file must be in Sentry policy file format.",
|
|
"display_name": "Sentry Global Policy File",
|
|
"name": "hive_sentry_provider_resource",
|
|
"value": "/user/hive/sentry/sentry-provider.ini"
|
|
},
|
|
{
|
|
"desc": "For advanced use only, key-value pairs (one on each line) to be inserted into a role's environment. Applies to configurations of all roles in this service except client configuration.",
|
|
"display_name": "Hive Service Environment Advanced Configuration Snippet (Safety Valve)",
|
|
"name": "hive_service_env_safety_valve",
|
|
"value": null
|
|
},
|
|
{
|
|
"desc": "Path to the directory where audit logs will be written. The directory will be created if it doesn't exist.",
|
|
"display_name": "Audit Log Directory",
|
|
"name": "audit_event_log_dir",
|
|
"value": "/var/log/hive/audit"
|
|
},
|
|
{
|
|
"desc": "Name of Hive Metastore database",
|
|
"display_name": "Hive Metastore Database Name",
|
|
"name": "hive_metastore_database_name",
|
|
"value": "metastore"
|
|
},
|
|
{
|
|
"desc": "The class to use in Sentry authorization for user to group mapping. Sentry authorization may be configured to use either Hadoop user to group mapping or local groups defined in the policy file. Hadoop user to group mapping may be configured in the Cloudera Manager HDFS service configuration page under the Security section.",
|
|
"display_name": "Sentry User to Group Mapping Class",
|
|
"name": "hive_sentry_provider",
|
|
"value": "org.apache.sentry.provider.file.HadoopGroupResourceAuthorizationProvider"
|
|
},
|
|
{
|
|
"desc": "Use Sentry to enable role-based, fine-grained authorization. This configuration enables Sentry using policy files. To enable Sentry using Sentry service instead, add Sentry service as a dependency to Hive service. <strong>Sentry service provides concurrent and secure access to authorization policy metadata and is the recommended option for enabling Sentry. </strong> Sentry is supported only on CDH 4.4 or later deployments. Before enabling Sentry, read the requirements and configuration steps outlined in <a class=\"bold\" href=\"http://tiny.cloudera.com/sentry-guide-cm5\" target=\"_blank\">Setting Up Hive Authorization with Sentry<i class=\"externalLink\"></i></a>.",
|
|
"display_name": "Enable Sentry Authorization using Policy Files",
|
|
"name": "sentry_enabled",
|
|
"value": "false"
|
|
},
|
|
{
|
|
"desc": "This configuration <strong>overrides</strong> the value set for Hive Proxy User Groups configuration in HDFS service for use by Hive Metastore Server. Specify a comma-delimited list of groups that you want to <strong>allow access to Hive Metastore metadata</strong> and allow the Hive user to impersonate. A value of '*' allows all groups. Default value of empty inherits the value set for Hive Proxy User Groups configuration in HDFS service.",
|
|
"display_name": "Hive Metastore Access Control and Proxy User Groups Override",
|
|
"name": "hive_proxy_user_groups_list",
|
|
"value": null
|
|
},
|
|
{
|
|
"desc": "Default number of reduce tasks per job. Usually set to a prime number close to the number of available hosts. Ignored when mapred.job.tracker is \"local\". Hadoop sets this to 1 by default, while Hive uses -1 as the default. When set to -1, Hive will automatically determine an appropriate number of reducers for each job.",
|
|
"display_name": "Hive Reduce Tasks",
|
|
"name": "hive_reduce_tasks",
|
|
"value": "-1"
|
|
},
|
|
{
|
|
"desc": "Let the table directories inherit the permission of the Warehouse or Database directory instead of being created with the permissions derived from dfs umask. This allows Impala to insert into tables created via Hive.",
|
|
"display_name": "Hive Warehouse Subdirectories Inherit Permissions",
|
|
"name": "hive_warehouse_subdir_inherit_perms",
|
|
"value": "true"
|
|
},
|
|
{
|
|
"desc": "The group that this service's processes should run as.",
|
|
"display_name": "System Group",
|
|
"name": "process_groupname",
|
|
"value": "hive"
|
|
},
|
|
{
|
|
"desc": "The health test thresholds of the overall Hive Metastore Server health. The check returns \"Concerning\" health if the percentage of \"Healthy\" Hive Metastore Servers falls below the warning threshold. The check is unhealthy if the total percentage of \"Healthy\" and \"Concerning\" Hive Metastore Servers falls below the critical threshold.",
|
|
"display_name": "Healthy Hive Metastore Server Monitoring Thresholds",
|
|
"name": "hive_hivemetastores_healthy_thresholds",
|
|
"value": "{\"critical\":\"51.0\",\"warning\":\"99.0\"}"
|
|
},
|
|
{
|
|
"desc": "When set, Cloudera Manager will send alerts when the health of this service reaches the threshold specified by the EventServer setting eventserver_health_events_alert_threshold",
|
|
"display_name": "Enable Service Level Health Alerts",
|
|
"name": "enable_alerts",
|
|
"value": "true"
|
|
},
|
|
{
|
|
"desc": "The server name used when defining privilege rules in Sentry authorization. Sentry uses this name as an alias for the Hive service. It has nothing to do with any physical server name.",
|
|
"display_name": "Server Name for Sentry Authorization",
|
|
"name": "hive_sentry_server",
|
|
"value": "server1"
|
|
},
|
|
{
|
|
"desc": "The health test thresholds of the overall HiveServer2 health. The check returns \"Concerning\" health if the percentage of \"Healthy\" HiveServer2s falls below the warning threshold. The check is unhealthy if the total percentage of \"Healthy\" and \"Concerning\" HiveServer2s falls below the critical threshold.",
|
|
"display_name": "Healthy HiveServer2 Monitoring Thresholds",
|
|
"name": "hive_hiveserver2s_healthy_thresholds",
|
|
"value": "{\"critical\":\"51.0\",\"warning\":\"99.0\"}"
|
|
},
|
|
{
|
|
"desc": "For advanced use only, a list of configuration properties that will be used by the Service Monitor instead of the current client configuration for the service.",
|
|
"display_name": "Service Monitor Client Config Overrides",
|
|
"name": "smon_client_config_overrides",
|
|
"value": "<property><name>hive.metastore.client.socket.timeout</name><value>20</value></property>"
|
|
},
|
|
{
|
|
"desc": "User for Hive Metastore database",
|
|
"display_name": "Hive Metastore Database User",
|
|
"name": "hive_metastore_database_user",
|
|
"value": "hive"
|
|
},
|
|
{
|
|
"desc": "Instead of talking to Hive Metastore Server for Metastore information, Hive clients will talk directly to the Metastore database.",
|
|
"display_name": "Bypass Hive Metastore Server",
|
|
"name": "hive_bypass_metastore_server",
|
|
"value": "false"
|
|
},
|
|
{
|
|
"desc": "Automatically create or upgrade tables in the Hive Metastore database when needed. Consider setting this to false and managing the schema manually.",
|
|
"display_name": "Auto Create and Upgrade Hive Metastore Database Schema",
|
|
"name": "hive_metastore_database_auto_create_schema",
|
|
"value": "false"
|
|
},
|
|
{
|
|
"desc": "Perform DataNucleus validation of metadata during startup. <strong>Note</strong>: when enabled, Hive will log DataNucleus warnings even though Hive will function normally.",
|
|
"display_name": "Hive Metastore Database DataNucleus Metadata Validation",
|
|
"name": "hive_metastore_database_datanucleus_metadata_validation",
|
|
"value": "false"
|
|
},
|
|
{
|
|
"desc": "For advanced use only, a string to be inserted into the client configuration for <strong>navigator.client.properties</strong>.",
|
|
"display_name": "Hive Client Advanced Configuration Snippet (Safety Valve) for navigator.client.properties",
|
|
"name": "navigator_client_config_safety_valve",
|
|
"value": null
|
|
},
|
|
{
|
|
"desc": "The health test thresholds of the overall WebHCat Server health. The check returns \"Concerning\" health if the percentage of \"Healthy\" WebHCat Servers falls below the warning threshold. The check is unhealthy if the total percentage of \"Healthy\" and \"Concerning\" WebHCat Servers falls below the critical threshold.",
|
|
"display_name": "Healthy WebHCat Server Monitoring Thresholds",
|
|
"name": "hive_webhcats_healthy_thresholds",
|
|
"value": "{\"critical\":\"51.0\",\"warning\":\"99.0\"}"
|
|
},
|
|
{
|
|
"desc": "For advanced use only, a string to be inserted into <strong>sentry-site.xml</strong>. Applies to configurations of all roles in this service except client configuration.",
|
|
"display_name": "Hive Service Advanced Configuration Snippet (Safety Valve) for sentry-site.xml",
|
|
"name": "hive_server2_sentry_safety_valve",
|
|
"value": null
|
|
},
|
|
{
|
|
"desc": "Size per reducer. If the input size is 10GiB and this is set to 1GiB, Hive will use 10 reducers.",
|
|
"display_name": "Hive Bytes Per Reducer",
|
|
"name": "hive_bytes_per_reducer",
|
|
"value": "1073741824"
|
|
},
|
|
{
|
|
"desc": "Enable collection of audit events from the service's roles.",
|
|
"display_name": "Enable Collection",
|
|
"name": "navigator_audit_enabled",
|
|
"value": "true"
|
|
},
|
|
{
|
|
"desc": "When set, Cloudera Manager will send alerts when this entity's configuration changes.",
|
|
"display_name": "Enable Configuration Change Alerts",
|
|
"name": "enable_config_alerts",
|
|
"value": "false"
|
|
},
|
|
{
|
|
"desc": "Allows URIs when defining privileges in per-database policy files. <strong>Warning:</strong> Typically, this configuration should be disabled. Enabling it would allow database policy file owner (which is generally not Hive admin user) to grant load privileges to any directory with read access to Hive admin user, including databases controlled by other database policy files.",
|
|
"display_name": "Allow URIs in Database Policy File",
|
|
"name": "sentry_allow_uri_db_policyfile",
|
|
"value": "false"
|
|
},
|
|
{
|
|
"desc": "In unsecure mode, setting this property to true will cause the Metastore Server to execute DFS operations using the client's reported user and group permissions. Cloudera Manager will set this for all clients and servers.",
|
|
"display_name": "Set User and Group Information",
|
|
"name": "hive_set_ugi",
|
|
"value": "true"
|
|
},
|
|
{
|
|
"desc": "For advanced use only, a list of derived configuration properties that will be used by the Service Monitor instead of the default ones.",
|
|
"display_name": "Service Monitor Derived Configs Advanced Configuration Snippet (Safety Valve)",
|
|
"name": "smon_derived_configs_safety_valve",
|
|
"value": null
|
|
},
|
|
{
|
|
"desc": "MapReduce jobs are run against this service.",
|
|
"display_name": "MapReduce Service",
|
|
"name": "mapreduce_yarn_service",
|
|
"value": null
|
|
}
|
|
] |