EN master
Change Logs
Tensorflow
Example of Tensorflow framework with CUSTOM endpoints
Overview
In this example, we demonstrate Konduit-Serving with a complete pipeline step consist of :
- 1.Pre-processing step
- 2.Running a deep learning model
- 3.Post-processing step, expressing the output in a way human can understand
Adding package to the classpaths
Let's add the main package of Konduit-Serving so that the notebook can load all the required libraries that need to be used by Jupyter Notebook kernel.
1
%classpath add jar ../../konduit.jar
Copied!
Classpaths can be considered similar to
site-packages in the python ecosystem where each library that's to be imported to your code is loaded from.Starting a server
Before starting a server, let's check if there is a running server with id
tensorflow-mnist and stop it. This command may use once the server finished.1
%%bash
2
konduit stop tensorflow-mnist
Copied!
You'll get the following message if there is no server running with mentioned id.
1
No konduit server exists with an id: 'tensorflow-mnist'.
Copied!
Or, if you have a running server you'll received as following.
1
Stopping konduit server 'tensorflow-mnist'
2
Application 'tensorflow-mnist' terminated with status 0
Copied!
Now, let's start the server with an id of
tensorflow-mnist using tensorflow.json as a configuration file in the background without creating the manifest jar file before launching the server.1
%%bash
2
konduit serve -id tensorflow-mnist -c tensorflow.json -rwm --background
Copied!
You'll be able to view a similar message like below.
1
Starting konduit server...
2
Using classpath: /root/konduit/bin/../konduit.jar
3
INFO: Running command /root/miniconda/jre/bin/java -Dkonduit.logs.file.path=/root/.konduit-serving/command_logs/tensorflow-mnist.log -Dlogback.configurationFile=/tmp/logback-run_command_5fd1b7c309d448ea.xml -jar /root/konduit/bin/../konduit.jar run --instances 1 -s inference -c tensorflow.json -Dserving.id=tensorflow-mnist
4
For server status, execute: 'konduit list'
5
For logs, execute: 'konduit logs tensorflow-mnist'
Copied!
View the logs for the last 1000 lines
-l for a given id by using the konduit logs command.1
%%bash
2
konduit logs tensorflow-mnist -l 1000
Copied!
The output of log is similar as following.
1
08:23:16.423 [main] INFO a.k.s.c.l.command.KonduitRunCommand - Processing configuration: /root/konduit/demos/4-tensorflow-mnist/tensorflow.json
2
.
3
.
4
.
5
​
6
####################################################################
7
# #
8
# | / _ \ \ | _ \ | | _ _| __ __| | / | / #
9
# . < ( | . | | | | | | | . < . < #
10
# _|\_\ \___/ _|\_| ___/ \__/ ___| _| _|\_\ _) _|\_\ _) #
11
# #
12
####################################################################
13
​
14
08:23:17.982 [vert.x-worker-thread-0] INFO a.k.s.v.verticle.InferenceVerticle - Pending server start, please wait...
15
.
16
.
17
.
18
08:23:18.145 [vert.x-eventloop-thread-0] INFO a.k.s.v.p.h.v.InferenceVerticleHttp - Inference HTTP server is listening on host: '0.0.0.0'
19
08:23:18.145 [vert.x-eventloop-thread-0] INFO a.k.s.v.p.h.v.InferenceVerticleHttp - Inference HTTP server started on port 9008 with 4 pipeline steps
Copied!
Sending an input to served model
We can display all the available image for the inference result of the model. You'll be able to see the picture from zero to nine.
1
%%html
2
<div style="display: flex; justify-content: center; align-items: center; border: 1px solid black;">
3
<div style="display: inline-block; margin: 2px">
4
<img src="test_files/test_input_number_0.png"/>
5
</div>
6
​
7
<div style="display: inline-block; margin: 10px">
8
<img src="test_files/test_input_number_1.png"/>
9
</div>
10
​
11
<div style="display: inline-block; margin: 10px">
12
<img src="test_files/test_input_number_2.png"/>
13
</div>
14
15
<div style="display: inline-block; margin: 10px">
16
<img src="test_files/test_input_number_3.png"/>
17
</div>
18
19
<div style="display: inline-block; margin: 10px">
20
<img src="test_files/test_input_number_4.png"/>
21
</div>
22
23
<div style="display: inline-block; margin: 10px">
24
<img src="test_files/test_input_number_5.png"/>
25
</div>
26
​
27
<div style="display: inline-block; margin: 10px">
28
<img src="test_files/test_input_number_6.png"/>
29
</div>
30
​
31
<div style="display: inline-block; margin: 10px">
32
<img src="test_files/test_input_number_7.png"/>
33
</div>
34
35
<div style="display: inline-block; margin: 10px">
36
<img src="test_files/test_input_number_8.png"/>
37
</div>
38
39
<div style="display: inline-block; margin: 10px">
40
<img src="test_files/test_input_number_9.png"/>
41
</div>
42
43
</div>
Copied!
Let's take one of the testing images and send it to the served model in Konduit-Serving. With the help of the pipeline considered in the configuration, we could translate the image into an array and feed it into the model.
1
%%bash
2
konduit predict tensorflow-mnist --input-type multipart "[email protected]_files/test_input_number_9.png"
Copied!
Thus, giving a result straight forward with the label of number classification based on prediction probabilities.
1
{
2
"output_layer/Softmax" : [ [ 3.0811898E-7, 6.085964E-6, 1.1470697E-4, 1.5436264E-9, 0.0023717284, 1.7763212E-12, 6.587209E-11, 0.99487466, 4.904844E-11, 0.0026325122 ] ],
3
"prob" : 0.9948746562004089,
4
"index" : 7,
5
"label" : "7"
6
}
Copied!
Last modified 1yr ago