Browse Source

Add simple guide how to debug service

Add new guide document about debuggin service in container.
heat-engine is used as example.

Change-Id: I3b0a25b5cd668a807d8bf3f467a224ee398ac377
Sergey Kraynev 2 years ago
parent
commit
9a0f569b23
2 changed files with 132 additions and 0 deletions
  1. 131
    0
      doc/source/debugging.rst
  2. 1
    0
      doc/source/index.rst

+ 131
- 0
doc/source/debugging.rst View File

@@ -0,0 +1,131 @@
1
+.. debugging:
2
+
3
+==================================
4
+Debugging microservice/application
5
+==================================
6
+
7
+This part of the documentation contains some practice recommendations, which
8
+can be used for debugging some issues in service code.
9
+
10
+Problem description
11
+===================
12
+
13
+Workable service is perfect, but sometimes user may be in situation, when
14
+application does not work as expected or fails with some unknown status.
15
+Obviously if service can not provide clear traceback or logs, there is no
16
+another option except debug this service. Let's take a look on some useful
17
+how to do it and use heat-engine service as example.
18
+
19
+How to debug
20
+============
21
+
22
+#. Create a local copy of the source code related project.
23
+
24
+   ::
25
+
26
+    cd /tmp
27
+    git clone http://github.com/openstack/heat
28
+
29
+#. Do all necessary changes, i.e. add breakpoint and etc., in this source code.
30
+
31
+#. Update global configuration file by using local source for heat service.
32
+
33
+   ::
34
+
35
+    sources:
36
+      openstack/heat:
37
+        source_dir: /tmp/heat
38
+
39
+#. Build new image and re-deploy heat service:
40
+
41
+   ::
42
+
43
+    ccp build -c heat-engine
44
+    ccp deploy -c heat-engine
45
+
46
+#. Login in container and enjoy debugging.
47
+
48
+.. NOTE:: This approach is really pure for understanding, but has one issue.
49
+          If you want to change code again you need to repeat all operations
50
+          from building image again.
51
+
52
+Another way to debug
53
+====================
54
+
55
+The idea is to run new process with necessary changes in code (breakpoints)
56
+inside already created container for current service. Execute
57
+follow commands to run bash in container with heat-engine service:
58
+
59
+::
60
+
61
+ kubectl get pods | grep heat-engine
62
+ kubectl exec -it <id of pod from previous command> bash
63
+
64
+So now bash is run in container. There are two new issues here:
65
+
66
+#. It's not possible to change service source files, because we are logged
67
+   as ``heat`` user.
68
+
69
+#. If heat-engine process be killed, Kubernetes detect it and re-create
70
+   container.
71
+
72
+Both issues can be solved by changing Docker image and Service definition.
73
+
74
+- First of all change user ``heat`` used in container to ``root``. It should be
75
+  done in file: ``fuel-ccp-heat/docker/heat-engine/Dockerfile.j2``.
76
+
77
+.. NOTE:: There is also alternative way to obtain root access is container:
78
+
79
+       #. find node where pod with heat-engine was run
80
+
81
+          ::
82
+
83
+           kubectl get pods -o wide | grep heat-engine
84
+       #. ssh to this node with heat-engine pod
85
+       #. find id of container with heat-engine and run bash as root user
86
+
87
+          ::
88
+
89
+           docker ps | grep heat-engine
90
+           docker exec -it -u root <heat-container-id> bash
91
+
92
+- The next step is to change run command in service definition:
93
+  ``fuel-ccp-heat/service/heat-engine.yaml``. Find key word ``command``,
94
+  comment it and write follow code:
95
+
96
+  ::
97
+
98
+    command: sleep 1h
99
+
100
+  It will allow to run container for 1 hour without real heat-engine process,
101
+  it's usually enough for leisurely debugging.
102
+
103
+
104
+To ship new container to the CCP is necessary to build new image and then
105
+re-deploy it:
106
+
107
+::
108
+
109
+ ccp build -c heat-engine
110
+ ccp deploy -c heat-engine
111
+
112
+When re-deploy is finished, run bash in new container again.
113
+The source code is placed in follow directory:
114
+
115
+::
116
+
117
+ /var/lib/microservices/venv/lib/python2.7/site-packages/
118
+
119
+Change it by adding necessary breakpoints.
120
+
121
+.. NOTE:: Text editor ``vim`` can work incorrect from container. For fixing it
122
+          try to execute command: export TERM=xterm
123
+
124
+The last step is to run updated service code. Execute command, which was
125
+commented in service definition file, in the current example it's:
126
+
127
+::
128
+
129
+ heat-engine --config-file /etc/heat/heat.conf
130
+
131
+Now patched service is active and can be used for debugging.

+ 1
- 0
doc/source/index.rst View File

@@ -49,6 +49,7 @@ Developer docs
49 49
    app_def_guide
50 50
    docker
51 51
    dsl
52
+   debugging
52 53
 
53 54
 Design docs
54 55
 -----------

Loading…
Cancel
Save