Browse Source

Add Kafka devstack plugin main scripts

How to use:

[[local|localrc]]
enable_plugin kafka https://github.com/openstack/devstack-plugin-kafka

Change-Id: Iceb054f0dfab2fff1920cc7dc36a4be256c6f161
Komei Shimamura 3 years ago
parent
commit
22851398e6
6 changed files with 355 additions and 0 deletions
  1. 2
    0
      .gitignore
  2. 175
    0
      LICENSE
  3. 10
    0
      README.md
  4. 129
    0
      devstack/plugin.sh
  5. 14
    0
      devstack/settings
  6. 25
    0
      tox.ini

+ 2
- 0
.gitignore View File

@@ -0,0 +1,2 @@
1
+*.swp
2
+.tox

+ 175
- 0
LICENSE View File

@@ -0,0 +1,175 @@
1
+
2
+                                 Apache License
3
+                           Version 2.0, January 2004
4
+                        http://www.apache.org/licenses/
5
+
6
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7
+
8
+   1. Definitions.
9
+
10
+      "License" shall mean the terms and conditions for use, reproduction,
11
+      and distribution as defined by Sections 1 through 9 of this document.
12
+
13
+      "Licensor" shall mean the copyright owner or entity authorized by
14
+      the copyright owner that is granting the License.
15
+
16
+      "Legal Entity" shall mean the union of the acting entity and all
17
+      other entities that control, are controlled by, or are under common
18
+      control with that entity. For the purposes of this definition,
19
+      "control" means (i) the power, direct or indirect, to cause the
20
+      direction or management of such entity, whether by contract or
21
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
22
+      outstanding shares, or (iii) beneficial ownership of such entity.
23
+
24
+      "You" (or "Your") shall mean an individual or Legal Entity
25
+      exercising permissions granted by this License.
26
+
27
+      "Source" form shall mean the preferred form for making modifications,
28
+      including but not limited to software source code, documentation
29
+      source, and configuration files.
30
+
31
+      "Object" form shall mean any form resulting from mechanical
32
+      transformation or translation of a Source form, including but
33
+      not limited to compiled object code, generated documentation,
34
+      and conversions to other media types.
35
+
36
+      "Work" shall mean the work of authorship, whether in Source or
37
+      Object form, made available under the License, as indicated by a
38
+      copyright notice that is included in or attached to the work
39
+      (an example is provided in the Appendix below).
40
+
41
+      "Derivative Works" shall mean any work, whether in Source or Object
42
+      form, that is based on (or derived from) the Work and for which the
43
+      editorial revisions, annotations, elaborations, or other modifications
44
+      represent, as a whole, an original work of authorship. For the purposes
45
+      of this License, Derivative Works shall not include works that remain
46
+      separable from, or merely link (or bind by name) to the interfaces of,
47
+      the Work and Derivative Works thereof.
48
+
49
+      "Contribution" shall mean any work of authorship, including
50
+      the original version of the Work and any modifications or additions
51
+      to that Work or Derivative Works thereof, that is intentionally
52
+      submitted to Licensor for inclusion in the Work by the copyright owner
53
+      or by an individual or Legal Entity authorized to submit on behalf of
54
+      the copyright owner. For the purposes of this definition, "submitted"
55
+      means any form of electronic, verbal, or written communication sent
56
+      to the Licensor or its representatives, including but not limited to
57
+      communication on electronic mailing lists, source code control systems,
58
+      and issue tracking systems that are managed by, or on behalf of, the
59
+      Licensor for the purpose of discussing and improving the Work, but
60
+      excluding communication that is conspicuously marked or otherwise
61
+      designated in writing by the copyright owner as "Not a Contribution."
62
+
63
+      "Contributor" shall mean Licensor and any individual or Legal Entity
64
+      on behalf of whom a Contribution has been received by Licensor and
65
+      subsequently incorporated within the Work.
66
+
67
+   2. Grant of Copyright License. Subject to the terms and conditions of
68
+      this License, each Contributor hereby grants to You a perpetual,
69
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70
+      copyright license to reproduce, prepare Derivative Works of,
71
+      publicly display, publicly perform, sublicense, and distribute the
72
+      Work and such Derivative Works in Source or Object form.
73
+
74
+   3. Grant of Patent License. Subject to the terms and conditions of
75
+      this License, each Contributor hereby grants to You a perpetual,
76
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77
+      (except as stated in this section) patent license to make, have made,
78
+      use, offer to sell, sell, import, and otherwise transfer the Work,
79
+      where such license applies only to those patent claims licensable
80
+      by such Contributor that are necessarily infringed by their
81
+      Contribution(s) alone or by combination of their Contribution(s)
82
+      with the Work to which such Contribution(s) was submitted. If You
83
+      institute patent litigation against any entity (including a
84
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
85
+      or a Contribution incorporated within the Work constitutes direct
86
+      or contributory patent infringement, then any patent licenses
87
+      granted to You under this License for that Work shall terminate
88
+      as of the date such litigation is filed.
89
+
90
+   4. Redistribution. You may reproduce and distribute copies of the
91
+      Work or Derivative Works thereof in any medium, with or without
92
+      modifications, and in Source or Object form, provided that You
93
+      meet the following conditions:
94
+
95
+      (a) You must give any other recipients of the Work or
96
+          Derivative Works a copy of this License; and
97
+
98
+      (b) You must cause any modified files to carry prominent notices
99
+          stating that You changed the files; and
100
+
101
+      (c) You must retain, in the Source form of any Derivative Works
102
+          that You distribute, all copyright, patent, trademark, and
103
+          attribution notices from the Source form of the Work,
104
+          excluding those notices that do not pertain to any part of
105
+          the Derivative Works; and
106
+
107
+      (d) If the Work includes a "NOTICE" text file as part of its
108
+          distribution, then any Derivative Works that You distribute must
109
+          include a readable copy of the attribution notices contained
110
+          within such NOTICE file, excluding those notices that do not
111
+          pertain to any part of the Derivative Works, in at least one
112
+          of the following places: within a NOTICE text file distributed
113
+          as part of the Derivative Works; within the Source form or
114
+          documentation, if provided along with the Derivative Works; or,
115
+          within a display generated by the Derivative Works, if and
116
+          wherever such third-party notices normally appear. The contents
117
+          of the NOTICE file are for informational purposes only and
118
+          do not modify the License. You may add Your own attribution
119
+          notices within Derivative Works that You distribute, alongside
120
+          or as an addendum to the NOTICE text from the Work, provided
121
+          that such additional attribution notices cannot be construed
122
+          as modifying the License.
123
+
124
+      You may add Your own copyright statement to Your modifications and
125
+      may provide additional or different license terms and conditions
126
+      for use, reproduction, or distribution of Your modifications, or
127
+      for any such Derivative Works as a whole, provided Your use,
128
+      reproduction, and distribution of the Work otherwise complies with
129
+      the conditions stated in this License.
130
+
131
+   5. Submission of Contributions. Unless You explicitly state otherwise,
132
+      any Contribution intentionally submitted for inclusion in the Work
133
+      by You to the Licensor shall be under the terms and conditions of
134
+      this License, without any additional terms or conditions.
135
+      Notwithstanding the above, nothing herein shall supersede or modify
136
+      the terms of any separate license agreement you may have executed
137
+      with Licensor regarding such Contributions.
138
+
139
+   6. Trademarks. This License does not grant permission to use the trade
140
+      names, trademarks, service marks, or product names of the Licensor,
141
+      except as required for reasonable and customary use in describing the
142
+      origin of the Work and reproducing the content of the NOTICE file.
143
+
144
+   7. Disclaimer of Warranty. Unless required by applicable law or
145
+      agreed to in writing, Licensor provides the Work (and each
146
+      Contributor provides its Contributions) on an "AS IS" BASIS,
147
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148
+      implied, including, without limitation, any warranties or conditions
149
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150
+      PARTICULAR PURPOSE. You are solely responsible for determining the
151
+      appropriateness of using or redistributing the Work and assume any
152
+      risks associated with Your exercise of permissions under this License.
153
+
154
+   8. Limitation of Liability. In no event and under no legal theory,
155
+      whether in tort (including negligence), contract, or otherwise,
156
+      unless required by applicable law (such as deliberate and grossly
157
+      negligent acts) or agreed to in writing, shall any Contributor be
158
+      liable to You for damages, including any direct, indirect, special,
159
+      incidental, or consequential damages of any character arising as a
160
+      result of this License or out of the use or inability to use the
161
+      Work (including but not limited to damages for loss of goodwill,
162
+      work stoppage, computer failure or malfunction, or any and all
163
+      other commercial damages or losses), even if such Contributor
164
+      has been advised of the possibility of such damages.
165
+
166
+   9. Accepting Warranty or Additional Liability. While redistributing
167
+      the Work or Derivative Works thereof, You may choose to offer,
168
+      and charge a fee for, acceptance of support, warranty, indemnity,
169
+      or other liability obligations and/or rights consistent with this
170
+      License. However, in accepting such obligations, You may act only
171
+      on Your own behalf and on Your sole responsibility, not on behalf
172
+      of any other Contributor, and only if You agree to indemnify,
173
+      defend, and hold each Contributor harmless for any liability
174
+      incurred by, or claims asserted against, such Contributor by reason
175
+      of your accepting any such warranty or additional liability.

+ 10
- 0
README.md View File

@@ -0,0 +1,10 @@
1
+# devstack-plugin-kafka
2
+
3
+The devstack plugin for installing and configuring Apache Kafka
4
+
5
+## How to Use
6
+
7
+in your local.conf file
8
+
9
+        [[local|localrc]]
10
+        enable_plugin devstack-plugin-kafka https://github.com/openstack/devstack-plugin-kafka.git

+ 129
- 0
devstack/plugin.sh View File

@@ -0,0 +1,129 @@
1
+#!/bin/bash
2
+
3
+# devstack/plugin.sh
4
+# Functions to install and configure Apache Kafka
5
+#
6
+# Dependencies:
7
+#
8
+#   - ``functions`` file
9
+#
10
+# ``stack.sh`` calls the entry points in this order:
11
+#
12
+#   - download_kafka
13
+#   - install_kafka
14
+#   - configure_kafka
15
+#   - init_kafka
16
+#   - stop_kafka
17
+#   - cleanup_kafka
18
+
19
+# Save trace setting
20
+XTRACE=$(set +o | grep xtrace)
21
+set +o xtrace
22
+
23
+# Functions
24
+# ------------
25
+# download_kafka() - downloading kafka
26
+function download_kafka {
27
+    if [ ! -f ${FILES}/kafka_${SCALA_VERSION}-${KAFKA_VERSION}.tgz ]; then
28
+        wget ${KAFKA_BASEURL}/${KAFKA_VERSION}/kafka_${SCALA_VERSION}-${KAFKA_VERSION}.tgz \
29
+            -O ${FILES}/kafka_${SCALA_VERSION}-${KAFKA_VERSION}.tgz
30
+    fi
31
+}
32
+
33
+# install_kafka() - installing Kafka with Scala and Zookeeper
34
+function install_kafka {
35
+
36
+    local scala_version=${SCALA_VERSION}.0
37
+
38
+    if is_ubuntu; then
39
+        sudo apt-get install -y scala
40
+    elif is_fedora; then
41
+
42
+        is_package_installed java-1.8.0-openjdk-headless || install_package java-1.8.0-openjdk-headless
43
+
44
+        if [ ! -f ${FILES}/scala-${scala_version}.tar.gz ]; then
45
+            wget ${SCALA_BASEURL}/scala-${scala_version}.tgz \
46
+                -O ${FILES}/scala-${scala_version}.tar.gz
47
+        fi
48
+
49
+        if [ ! -d ${FILES}/scala-${scala_version} ]; then
50
+            tar -xvzf ${FILES}/scala-${scala_version}.tar.gz -C ${FILES}
51
+        fi
52
+
53
+        if [ ! -d /usr/lib/scala-${scala_version} ]; then
54
+            sudo mv ${FILES}/scala-${scala_version} /usr/lib
55
+            sudo ln -s /usr/lib/scala-${scala_version} /usr/lib/scala
56
+            export PATH=$PATH:/usr/lib/scala/bin
57
+        fi
58
+    fi
59
+
60
+    download_kafka
61
+
62
+    if [ ! -d ${FILES}/kafka_${SCALA_VERSION}-${KAFKA_VERSION} ]; then
63
+        tar -xvzf ${FILES}/kafka_${SCALA_VERSION}-${KAFKA_VERSION}.tgz -C ${FILES}
64
+    fi
65
+    if [ ! -d ${KAFKA_DEST}/kafka ]; then
66
+        mkdir -p ${KAFKA_DEST}
67
+        mv ${FILES}/kafka_${SCALA_VERSION}-${KAFKA_VERSION} ${KAFKA_DEST}/kafka
68
+    fi
69
+}
70
+
71
+# init_kafka() - starting Kafka and Zookeeper processes
72
+function init_kafka {
73
+    # start Zookeeper process before starting Kafka
74
+    ${KAFKA_DEST}/kafka/bin/zookeeper-server-start.sh -daemon ${KAFKA_DEST}/kafka/config/zookeeper.properties
75
+    ${KAFKA_DEST}/kafka/bin/kafka-server-start.sh -daemon ${KAFKA_DEST}/kafka/config/server.properties
76
+}
77
+
78
+# configure_kafka() - configuring Kafka service
79
+function configure_kafka {
80
+    # currently a no op
81
+    :
82
+}
83
+
84
+# stop_kafka() - stopping Kafka process
85
+function stop_kafka {
86
+    ${KAFKA_DEST}/kafka/bin/kafka-server-stop.sh
87
+    ${KAFKA_DEST}/kafka/bin/zookeeper-server-stop.sh
88
+}
89
+
90
+# cleanup_kafka() - removing Kafka files
91
+# make sure this function is called only after calling stop_kafka() function
92
+function cleanup_kafka {
93
+    rm ${FILES}/kafka_${SCALA_VERSION}-${KAFKA_VERSION}.tgz
94
+    rm -rf ${KAFKA_DEST}/kafka
95
+}
96
+
97
+# check for kafka service
98
+if is_service_enabled devstack-plugin-kafka; then
99
+    if [[ "$1" == "source" ]]; then
100
+        # Initial source
101
+        source $TOP_DIR/lib/kafka
102
+
103
+    elif [[ "$1" == "stack" && "$2" == "install" ]]; then
104
+        # Perform installation of service source
105
+        echo_summary "Installing kafka"
106
+        install_kafka
107
+
108
+        echo_summary "Initializing kafka"
109
+        init_kafka
110
+
111
+    elif [[ "$1" == "unstack" ]]; then
112
+        # Shut down kafka services
113
+        echo_summary "Shut down kafka service"
114
+        stop_kafka
115
+
116
+    elif [[ "$1" == "clean" ]]; then
117
+        # Remove state and transient data
118
+        # Remember clean.sh first calls unstack.sh
119
+        echo_summary "Clean up kafka service"
120
+        cleanup_kafka
121
+    fi
122
+fi
123
+
124
+# Restore xtrace
125
+$XTRACE
126
+
127
+## Local variables:
128
+## mode: shell-script
129
+## End:

+ 14
- 0
devstack/settings View File

@@ -0,0 +1,14 @@
1
+# Devstack settings
2
+
3
+enable_service kafka
4
+
5
+# Kafka installation directory
6
+KAFKA_DEST=${KAFKA_DEST:-/opt/stack/devstack-plugin-kafka}
7
+
8
+# Specify Kafka version
9
+KAFKA_VERSION=${KAFKA_VERSION:-0.9.0.1}
10
+KAFKA_BASEURL=${KAFKA_BASEURL:-http://www.us.apache.org/dist/kafka}
11
+
12
+# Specify Scala version
13
+SCALA_VERSION=${SCALA_VERSION:-2.10}
14
+SCALA_BASEURL=${SCALA_BASEURL:-http://www.scala-lang.org/files/archive}

+ 25
- 0
tox.ini View File

@@ -0,0 +1,25 @@
1
+[tox]
2
+minversion = 1.6
3
+skipsdist = True
4
+envlist = bashate
5
+
6
+[testenv]
7
+usedevelop = False
8
+install_command = pip install {opts} {packages}
9
+
10
+[testenv:bashate]
11
+deps = bashate
12
+whitelist_externals = bash
13
+commands = bash -c "find {toxinidir}          \
14
+         -not \( -type d -name .?\* -prune \) \
15
+         -not \( -type d -name doc -prune \)  \
16
+         -type f                              \
17
+         -not -name \*~                       \
18
+         -not -name \*.md                     \
19
+         \(                                   \
20
+          -name \*.sh -or                     \
21
+          -name \*rc -or                      \
22
+          -name functions\* -or               \
23
+          -wholename \*/lib/\*                \
24
+         \)                                   \
25
+         -print0 | xargs -0 bashate -v"

Loading…
Cancel
Save