In this section, we provide guidance on how to demonstrate Konduit-Serving with Docker. Konduit-Serving is a platform to serve the ML/DL models directly through a single line of code. We'll start building and installing the Konduit-Serving from docker image and follow by deploying the trained model in Konduit-Serving.
Prerequisites
You will need following prerequisites to follow along
To ensure Konduit-Serving works properly, install these prerequisites. We’ll be using Konduit-Serving to deploy machine/deep learning pipelines easier using a few lines of code. We've prepared a Github repository to simplify showcasing Konduit-Serving.
Introduction to Repository
The repository contains simple examples of deploying a pipeline using different model types. To clone the repository, run the following command:
Build the CPU version of docker image by running the following command in the root directory:
bash build.sh CPU
After a successful build, run docker image with docker-compose at current working directory:
docker-compose up
Now, open in the browser.
Explore Konduit-Serving in Repository
Let's take a look inside the directory from . Each folder inside the demos folder demonstrate serving a different kind of model, using a configuration file in either JSON or YAML, through the Konduit-Serving CLI.
The examples use different frameworks, including Keras, Tensorflow, Pytorch, and DL4J. These examples can be run in IPython Notebook (.ipynb) with Java-based kernel. Konduit-Serving provides a platform for users to take advantage of models using konduit CLI such as serve, list, logs, predict and stop. You can also build your model and start using Konduit-Serving.
Let's look at training and serving a model with Konduit-Serving.
Build your model
These are steps to train a model for Keras and DL4J from scratch:
First, import library that will be use:
import keras
from keras.models import Sequential
from keras.layers import Dense
from keras.optimizers import Adam
from sklearn.preprocessing import LabelEncoder
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix
import seaborn as sns
import pandas as pd
import numpy as np
Declare the variables that will be used in the training process (this code starts under main body):
int numLinesToSkip = 0;
char delimiter = ',';
int batchSize = 150; // Iris data set: 150 examples total. We are loading all of them into one DataSet (not recommended for large data sets)
int labelIndex = 4; // index of label/class column
int numClasses = 3;
int seed = 1234;
int epochs = 800;
double learningRate = 0.1;
int nHidden = 4;
Load Iris data set from resources file:
File inputFile = new ClassPathResource("datavec/iris.txt").getFile();
FileSplit fileSplit = new FileSplit(inputFile);
Get data set using record reader (to handle loading or parsing):
RecordReader recordReader = new CSVRecordReader(numLinesToSkip, delimiter);
recordReader.initialize(fileSplit);
DataNormalization normalizer = new NormalizerStandardize();
normalizer.fit(trainingData);
normalizer.transform(trainingData);
normalizer.transform(testData);
Configure and initiate the model that will be used:
MultiLayerConfiguration config = new NeuralNetConfiguration.Builder()
.seed(seed)
.weightInit(WeightInit.XAVIER)
.activation(Activation.TANH)
.updater(new Nesterovs(learningRate, Nesterovs.DEFAULT_NESTEROV_MOMENTUM))
.l2(1e-4)
.list()
.layer(0, new DenseLayer.Builder()
.nIn(labelIndex)
.nOut(nHidden)
.build())
.layer(1, new DenseLayer.Builder()
.nOut(nHidden)
.build())
.layer(2, new OutputLayer.Builder(LossFunctions.LossFunction.MCXENT)
.activation(Activation.SOFTMAX)
.nOut(numClasses)
.name("output")
.build())
.backpropType(BackpropType.Standard)
.build();
MultiLayerNetwork model = new MultiLayerNetwork(config);
model.init();
Display in UI of training process which can be seen when open in browser (optional):
StatsStorage storage = new InMemoryStatsStorage();
UIServer server = UIServer.getInstance();
server.attach(storage);
model.setListeners(new StatsListener(storage, 10));
Train the model using training data:
for (int i=0; i < epochs; i++) {
model.fit(trainingData);
}
Evaluate the model by using testing data (save the model if satisfied):
Evaluation eval = new Evaluation(3);
eval.eval(model.output(testData.getFeatures()),testData.getLabels());
System.out.println(eval.stats());
Then, save the model at the location you want to keep with the name in Zip format. The model is ready to be used (for example, the model saved in the current working directory):
File locationToSave = new File("./dl4j_iris_model.zip");
boolean saveUpdater = true;
ModelSerializer.writeModel(model,locationToSave,saveUpdater);
System.out.println("******PROGRAM IS FINISHED******");
The model is now ready to be deployed.
Deploy your model in Konduit-Serving
Now, you are ready to deploy a model in Konduit-Serving by using konduit CLI. This step needs a saved model file (h5 or zip file) and JSON/YAML configuration file. Let's begin with:
Create a new folder in the demos directory (for example, 10-iris-model):
Drag the model file into the demos directory folder and create an IPython Notebook file with Java kernel (for example, iris-model.ipynb).
In this notebook, we will use konduit CLI.
Check the version of Konduit-Serving, and either is installed or not:
%%bash
konduit -version
Create the JSON/YAML configuration file by using konduit config command:
YAML configuration:
%%bash
konduit config -p keras -o iris-keras.yaml -y
Or, you can try this command to get a JSON configuration file:
