Basic

Search…
EN master
Basic
Simple example to demonstrate Konduit-Serving
In the first example, we’ll use normal operations as a model, but it is deploying on Konduit-Serving. You can give input and get the return of output from the server. This will make your model more straightforward to understand by humans as it can provide direct results.
Viewing directory structure
Let’s run cells with bash in a sub-process by using cell magic command and view files in the current directory that will be used in this demonstration.
%%bash
echo "Current directory $(pwd)" && tree
The following files are present in our simple python script demo.
Current directory /root/konduit/demos/0-python-simple
.
├── init_script.py
├── python-simple.ipynb
├── python.yaml
└── run_script.py
​
0 directories, 4 files
Viewing Python script content
The scripts contain a simple initialization script for an add function which loads the main function in the init_script.py and executes the incoming input through run_script.py.
%%bash
less init_script.py
You’ll be able to see the following.
def add_function(x, y):
    return x + y
Once again, let’s browse through the python script for the calling function from init_script.py.
%%bash
less run_script.py
You’ll notice the script only has a line of code.
c = add_function(a, b)
Viewing the main configuration file
The main configuration should define the inputs as a and b and the output as c, just as we've showed in the run_script.py.
%%bash
less python.yaml
The YAML script file is as follows.
---
host: "0.0.0.0"
pipeline:
  steps:
  - '@type': "PYTHON"
    python_config:
      append_type: "BEFORE"
      extra_inputs: {}
      import_code_path: "init_script.py"
      python_code_path: "run_script.py"
      io_inputs:
        a:
          python_type: "float"
          secondary_type: "NONE"
          type: "DOUBLE"
        b:
          python_type: "float"
          secondary_type: "NONE"
          type: "DOUBLE"
      io_outputs:
        c:
          python_type: "float"
          secondary_type: "NONE"
          type: "DOUBLE"
      job_suffix: "konduit_job"
      python_config_type: "CONDA"
      python_path: "1"
      environment_name: "base"
      python_path_resolution: "STATIC"
      python_inputs: {}
      python_outputs: {}
      return_all_inputs: false
      setup_and_run: false
port: 8082
protocol: "HTTP"
Using the configuration to start a server
Now we can use the konduit serve command to start the server in background with the given files and configurations.
%%bash
konduit serve -rwm --config python.yaml -id server --background
You’ll get the message like this.
Starting konduit server...
Expected classpath: /root/konduit/bin/../konduit.jar
INFO: Running command /root/miniconda/jre/bin/java -Dkonduit.logs.file.path=/root/.konduit-serving/command_logs/server.log -Dlogback.configurationFile=/tmp/logback-run_command_13ccd5e27dfe43b1.xml -cp /root/konduit/bin/../konduit.jar ai.konduit.serving.cli.launcher.KonduitServingLauncher run --instances 1 -s inference -c python.yaml -Dserving.id=server
For server status, execute: 'konduit list'
For logs, execute: 'konduit logs server'
Listing the servers
We can list the created servers with konduit list command
%%bash
konduit list
The ID’s server lists like below, giving the status of the server.
Listing konduit servers...
​
 #   | ID                             | TYPE       | URL                  | PID     | STATUS     
 1   | server                         | inference  | 0.0.0.0:8082         | 421     | started    
Viewing logs
Logs can be viewed for the server with an ID of server through running konduit logs server .. command.
%%bash
konduit logs server --lines 1000
Logs output of started server:
09:44:17.852 [main] INFO  a.k.s.c.l.command.KonduitRunCommand - Processing configuration: /root/konduit/demos/0-python-simple/python.yaml
.
.
.
09:44:19.436 [vert.x-worker-thread-0] INFO  a.k.s.v.verticle.InferenceVerticle - 
​
####################################################################
#                                                                  #
#    |  /   _ \   \ |  _ \  |  | _ _| __ __|    |  /     |  /      #
#    . <   (   | .  |  |  | |  |   |     |      . <      . <       #
#   _|\_\ \___/ _|\_| ___/ \__/  ___|   _|     _|\_\ _) _|\_\ _)   #
#                                                                  #
####################################################################
​
09:44:19.436 [vert.x-worker-thread-0] INFO  a.k.s.v.verticle.InferenceVerticle - Pending server start, please wait...
.
.
.
09:44:19.589 [vert.x-eventloop-thread-0] INFO  a.k.s.v.p.h.v.InferenceVerticleHttp - Inference HTTP server is listening on host: '0.0.0.0'
09:44:19.589 [vert.x-eventloop-thread-0] INFO  a.k.s.v.p.h.v.InferenceVerticleHttp - Inference HTTP server started on port 8082 with 1 pipeline steps
Sending inputs
Now we’ll be able to send the inputs for inferring the output.
%%bash
konduit predict server '{"a":1,"b":2}'
The output result of the function deployed from the server.
{
  "c" : 3.0
}
Stopping the server
Stop the server by giving the ID’s we want to terminate. 
%%bash
konduit stop server
Status of the server will be printed out as below.
Stopping konduit server 'server'
Application 'server' terminated with status 0
As you can see from this example, we only use a simple function to deploy in Konduit-Serving. Next, we'll deploy the model in Konduit-Serving.
Examples - Previous
IPython Notebook
ONNX
Last modified 1yr ago
Copy link
On this page
Viewing directory structure
Viewing Python script content
Viewing the main configuration file
Using the configuration to start a server
Listing the servers
Viewing logs
Sending inputs
Stopping the server