From 914dec824bbff69dcf3e1f8623f26dea8fccaca8 Mon Sep 17 00:00:00 2001 From: tomohiro-maruyama Date: Fri, 29 Aug 2014 15:39:06 +0900 Subject: [PATCH] Modify imagebuild.sh and README of the pi-montecarlo application Change-Id: Id857828cd7049c853686205bb542a2f17a7d56ac --- tools/sample-apps/pi-montecarlo/README.md | 200 ++++++++++-------- tools/sample-apps/pi-montecarlo/app/picalc.py | 0 tools/sample-apps/pi-montecarlo/app/picalc.sh | 0 .../pi-montecarlo/app/picalc_receiver.sh | 0 .../pi-montecarlo/app/picalc_sender.sh | 0 tools/sample-apps/pi-montecarlo/app/put.py | 0 .../pi-montecarlo/app/websocket_client.py | 0 tools/sample-apps/pi-montecarlo/imagebuild.sh | 9 +- .../{requirement.txt => requirements.txt} | 0 9 files changed, 110 insertions(+), 99 deletions(-) mode change 100644 => 100755 tools/sample-apps/pi-montecarlo/app/picalc.py mode change 100644 => 100755 tools/sample-apps/pi-montecarlo/app/picalc.sh mode change 100644 => 100755 tools/sample-apps/pi-montecarlo/app/picalc_receiver.sh mode change 100644 => 100755 tools/sample-apps/pi-montecarlo/app/picalc_sender.sh mode change 100644 => 100755 tools/sample-apps/pi-montecarlo/app/put.py mode change 100644 => 100755 tools/sample-apps/pi-montecarlo/app/websocket_client.py mode change 100644 => 100755 tools/sample-apps/pi-montecarlo/imagebuild.sh rename tools/sample-apps/pi-montecarlo/{requirement.txt => requirements.txt} (100%) diff --git a/tools/sample-apps/pi-montecarlo/README.md b/tools/sample-apps/pi-montecarlo/README.md index 5bcfc46..786e348 100644 --- a/tools/sample-apps/pi-montecarlo/README.md +++ b/tools/sample-apps/pi-montecarlo/README.md @@ -1,113 +1,129 @@ -# PI Calculation Rack Sample Program -##1. Getting Start -This sample program, parallel distributed processing the approximate value of pi calculated by Monte Carlo method using the RACK-API. -The program has two parameters as "number of trials" and "number of processes". -The number of trials, it is processing by dividing specified in the Number of processes, and outputs to the "Swift Storage" by summing the results of each process. +# RACK Sample Program - PI calculation -##2. Setup Example -The following is a setup scenario for PI Caluculation Program. +##Overview +This application calculates the approximate value of PI using the "Monte Carlo method". +The Monte Carlo is a method to calculate iteratively to gain an approximate result. +The more you iterate the calculation, the more precise result will be. +You can specify the "number of trials" and "number of processes(instances)" in this application. +You will get more precise PI with "a larger number of trials", but it takes longer to get the result. +Then you can make it faster with larger "number of process", as more instance performs distributed processing. +Ultimately, you can obtain a more precise value of PI faster with RACK. -* Create Image - - Sample program execution image that you create using your "CentOS-6.5" boot image. -* Proxy Create - - Proxy process, relaying of inter-process communication and hold the endpoint information. -* Proxy Update - - "Proxy Update", registers the container name and Swift endpoint. +##Preparation Procedure +This chapter describes the preparation and setting up of RACK. +After completing this chapter, you will have one RACK API, one glance image and one RACK proxy. + +### Deploy RACK API +RACK API is required to control RACK Proxy. +Please refer to [How to use RACK](https://github.com/stackforge/rack/tree/master/tools/setup) to deploy RACK API for detailed procedure. + +###Create an Image +We provide `imagebuild.sh` to build an image from CentOS 6.x instance. We tested it on CentOS6.5. +First, launch an instance from CentOS6.5 image and log in to it as the root user and install git and rack.git. +Then execute `imagebuild.sh` to build the sample application's image. - ###2-1. Create Image - * In OpenStack Dashboard, please launch an instance from your "CentOS-6.5" boot image. - * Please copy the following file into the user's home directory of launched instance. ``` -./rack/tools/sample-apps/pi-montecarlo/imagebuild.sh +# yum install -y git +# git clone https://github.com/stackforge/rack.git +# cd ./rack/tools/sample-apps/pi-montecarlo/ +# ./imagebuild.sh ``` - * From the shell of the launched instance, please run the imagebuild.sh that you copied. -Will build the environment required to run. -``` -$ sh ./imagebuild.sh -``` -In OpenStack Dashboard, please create a snapshot of this instance. -Image ID is specified in the argument of the sample program. -(Section 3) +When it is finished, shutdown the instance and save a snapshot. - ###2-2. Proxy Create - * Please edit the required parameters of the group-init.conf file. - - ./rack/tools/group-init.conf. +###Deploy RACK Proxy +Before deploying an application, you have to deploy RACK Proxy. +Please log in to RACK API and edit `group-init.conf`. +This is a configuration file for `rack-client`. +Fill in the blanks depending on your environment. + +####./rack/tools/group-init.conf. ``` -[group] -name = ### ex: pi_test +[group] +name = {Group name} ### ex: pi_test [network] -cidr = ### ex: 10.0.2.0/24 -ext_router_id = +cidr = {Network address in CIDR format} ### ex: 10.0.2.0/24 +ext_router_id = {Your rooter ID} [securitygroup] -securitygrouprules = protocol=tcp,port_range_max=80,port_range_min=80,remote_ip_prefix=0.0.0.0/0 protocol =tcp,port_range_max=5000,port_range_min=5000,remote_ip_prefix=0.0.0.0/0 protocol=tcp,port_range_max=8088,port_range_min=8088,remote_ip_prefix=0.0.0.0/0 protocol=tcp,port_range_max=8888,port_range_min=8888,remote_ip_prefix=0.0.0.0/0 +securitygrouprules = + protocol=tcp,port_range_max=80,port_range_min=80,remote_ip_prefix=0.0.0.0/0 + protocol=tcp,port_range_max=5000,port_range_min=5000,remote_ip_prefix=0.0.0.0/0 + protocol=tcp,port_range_max=8088,port_range_min=8088,remote_ip_prefix=0.0.0.0/0 + protocol=tcp,port_range_max=8888,port_range_min=8888,remote_ip_prefix=0.0.0.0/0 [proxy] -nova_flavor_id = 3 -glance_image_id = ``` -The proxy image, you need to create in advance by the "Rack" Image Build Tool (imagebuid.sh) . - * Please execute the following command. -``` -$ export OS_USERNAME= -$ export OS_TENANT_NAME= -$ cd ./rack/tools -$ python rack_client.py --url http://:8088/v1/ group-init --config-file group-init.conf +nova_flavor_id = {Any Flavor ID you like(We recommend a flavor with at least 2GB memory)} +glance_image_id = {Your proxy image ID} ``` - * When you run a group-init command, The following dumps are output. -Please paste the --gid option of proxy-update command in the section 2-3. +Then execute the following command. +This command creates a new keypair, a new security group, a new network, assigns group id(`gid`) and process id(`pid`). + ``` -gid=b60bdb69-1751-4b88-b129-d41f044c858 -keypaire_id=f19db0a3-6ab3-4a88-ad72-f90d5aa9e0f4 -securitygroup_id=12aac225-662f-4a06-82c9-ad06b546b9a8 - ： +# export OS_USERNAME=demo_user +# export OS_TENANT_NAME=demo_tenant +# rack_client group-init --config-file /path/to/group-init.conf +gid: 8612d17d-737e-4050-be93-cd49b1574e1e +keypair_id: da6bde2d-1753-410a-9f2a-3a99c88584f6 +securitygroup_id: 1cd37c45-f38e-4407-9bd6-cb2222ad928d +network: 8c627394-e4cd-4c9e-972c-fad704323966 +pid: f08d3e1b-5b51-4fd6-849a-5b7983f35c0d ``` - ###2-3. Proxy Update - * Please execute the following command. -``` -$ cd ./rack/tools -$ python rack_client.py proxy-update --url http://:8088/v1/ --gid --fs_endpoint {\"host\":\"\",\"name\":\"\"}` -``` -`ex. --fs_endpoint {\"host\": \"http://192.168.100.19:5000/v2.0\",\"name\": \"picalc_result\"}` +Note that variables, `OS_USERNAME` and `OS_TENANT_NAME` will be registered on RACK database as someone who executed the command. +It's not related to OpenStack username and tenant name. +Note values above and bring it to the following commands as option. +Now you can see a further instance launched on your OpenStack environment. +Ensure the RACK Proxy is deployed successfully by running the following command. - Update is successful. The information of the proxy will be response in JSON format. +``` +# rack_client --url http://{proxy's IP address}:8088/v1/ proxy-show --gid {gid} +{"proxy": {"status": "ACTIVE", "userdata": null, "ppid": null, "user_id": "demo_user","name": "pro-f08d3e1b-5b51-4fd6-849a-5b7983f35c0d", "ipc_endpoint": {"host": "10.0.50.2", "port": "8888"}, "app_status": "ACTIVE", "pid": "f08d3e1b-5b51-4fd6-849a-5b7983f35c0d", "args": {"roles": "ipc/shm/api/proxy"}, "fs_endpoint": null, "gid": "8612d17d-737e-4050-be93-cd49b1574e1e", "keypair_id": "da6bde2d-1753-410a-9f2a-3a99c88584f6", "nova_flavor_id": 3, "shm_endpoint":{"host": "10.0.50.2", "port": "6379"}, "project_id": "demo_user", "glance_image_id": "1725f6c0-264a-4202-8886-7998dfe4457b"}} +``` -##3. Running Sample Program -* Please copy the following file into the user's home directory of proxy instance. -``` -./rack/tools/sample-apps/pi-montecarlo/app/picalc.sh -``` +It's okay if you get a JSON response like above. -* From the shell of the proxy instance, please execute the following command. -``` -$ export OS_USERNAME= -$ export OS_PASSWORD= -$ export OS_TENANT_NAME= -$ ./picalc.sh 10000000 10 -``` - Usage : picalc.sh IMAGE_ID NUMBER_OF_TRIALS NUMBER_OF_PROCESS - - When the process is complete, result.txt file will be output to your container. -Please see the result.txt file. - +###Register Swift endpoint +As the sample application stores result on `File System`, you need to register keystone credentials on RACK Proxy so that RACK Proxy can store the result on Object Storage(Swift). +Please register your credentials on RACK Proxy by executing the following commands. -+ Results such as the following will be output. ``` -------------------------------------------------------------------` -picalc result -------------------------------------------------------------------` - trials : 10000000 - process : 10 - within circle : 7855266 -------------------------------------------------------------------` - pi : 3.14159265359 - result : 3.1421064 - error : 0.000513746410207 -------------------------------------------------------------------` - time : 00 d 00 h 00 m 26 s -------------------------------------------------------------------` -``` \ No newline at end of file +# export OS_USERNAME={openstack username} +# export OS_PASSWORD={openstack password} +# export OS_TENANT_NAME={openstack tenant name} +# export OS_AUTH_URL={openstack keystone endpoint} +# rack_client --url http://{proxy's IP address}:8088/v1/ proxy-update --gid {gid} \ + --fs_endpoint {\"host\":\"$OS_AUTH_URL\"\,\"name\":\"picalc_result\"} --app_status ACTIVE +``` + +And you also need to create a Swift container with the name of `gid` value, in this example "8612d17d-737e-4050-be93-cd49b1574e1e". + +##Run Sample Program +Execute the following command to run the program. + +``` +# OS_USERNAME={openstack username} +# OS_PASSWORD={openstack password} +# OS_TENANT_NAME={openstack tenant name} +# /path/to/rack/tools/sample-apps/pi-montecarlo/app/picalc.sh {SampleProgramImageID} {number of trials} {number of process} +``` + +When the process is complete, you will have `result.txt` at the swift container. + +####result.txt +``` +------------------------------------------------------------------` +picalc result +------------------------------------------------------------------` + trials : 10000000 + process : 10 + within circle : 7855266 +------------------------------------------------------------------` + pi : 3.14159265359 + result : 3.1421064 + error : 0.000513746410207 +------------------------------------------------------------------` + time : 00 d 00 h 00 m 26 s +------------------------------------------------------------------` +``` + +Please check the `error` and `time` vary as you change the `{number of trials}` and `{number of process}`. diff --git a/tools/sample-apps/pi-montecarlo/app/picalc.py b/tools/sample-apps/pi-montecarlo/app/picalc.py old mode 100644 new mode 100755 diff --git a/tools/sample-apps/pi-montecarlo/app/picalc.sh b/tools/sample-apps/pi-montecarlo/app/picalc.sh old mode 100644 new mode 100755 diff --git a/tools/sample-apps/pi-montecarlo/app/picalc_receiver.sh b/tools/sample-apps/pi-montecarlo/app/picalc_receiver.sh old mode 100644 new mode 100755 diff --git a/tools/sample-apps/pi-montecarlo/app/picalc_sender.sh b/tools/sample-apps/pi-montecarlo/app/picalc_sender.sh old mode 100644 new mode 100755 diff --git a/tools/sample-apps/pi-montecarlo/app/put.py b/tools/sample-apps/pi-montecarlo/app/put.py old mode 100644 new mode 100755 diff --git a/tools/sample-apps/pi-montecarlo/app/websocket_client.py b/tools/sample-apps/pi-montecarlo/app/websocket_client.py old mode 100644 new mode 100755 diff --git a/tools/sample-apps/pi-montecarlo/imagebuild.sh b/tools/sample-apps/pi-montecarlo/imagebuild.sh old mode 100644 new mode 100755 index 89cb44f..1e8da8e --- a/tools/sample-apps/pi-montecarlo/imagebuild.sh +++ b/tools/sample-apps/pi-montecarlo/imagebuild.sh @@ -1,12 +1,7 @@ #!/usr/bin/env sh -APP_PATH=~/rack/tools/sample-apps/pi-montecarlo -yum install git -y +yum update -y easy_install pip -pip install -U setuptools -git clone https://github.com/stackforge/rack.git ~/rack -pip install -r $APP_PATH/requirement.txt - +pip install -r ./requirements.txt curl -o /usr/bin/jq http://stedolan.github.io/jq/download/linux64/jq chmod +x /usr/bin/jq -chmod +x $APP_PATH/app/*.sh diff --git a/tools/sample-apps/pi-montecarlo/requirement.txt b/tools/sample-apps/pi-montecarlo/requirements.txt similarity index 100% rename from tools/sample-apps/pi-montecarlo/requirement.txt rename to tools/sample-apps/pi-montecarlo/requirements.txt