# YAML configurations

## YAML components

A Konduit Serving YAML configuration file has three top-level entities:

1. `serving`
2. `steps`
3. `client`

The following is a sample YAML file for serving a Python script located at `simple.py` which takes a NumPy array `first` as input and returns a NumPy array `second` as output:

```yaml
serving:
  http_port: 1337
  output_data_format: NUMPY
  log_timings: True
  extra_start_args: -Xmx8g
steps:
  python_step:
    type: PYTHON
    python_path: .
    python_code_path: ./simple.py
    python_inputs:
      first: NDARRAY
    python_outputs:
      second: NDARRAY
client:
    port: 1337
```

### Serving

The server configuration, `serving` takes the following arguments:

* `http_port`: specify the port number 
* `listen_host`: the host of the Konduit Serving instance. Defaults to `http://localhost`. 

Additional arguments include:

* `uploads_directory`: Directory to store file uploads. Defaults to `'file-uploads/'`.
* `jar_path`: Path to the Konduit Serving uberjar. Defaults to the `KONDUIT_JAR_PATH` environment variable, or if unavailable, `~/.konduit/konduit-serving/konduit.jar`. 
* `log_timings`: Whether to log timings for this config. Defaults to `False`.
* `extra_start_args`: Java Virtual Machine (JVM) arguments. In this case, `-Xmx8g` specifies that the maximum memory allocation for the JVM is 8GB. 

Refer to the [Server](/0.1.0-snapshot/server/inference.md) documentation for other arguments.

### Client

The client configuration takes the following arguments:

* `port`: specify the same HTTP port as `serving`. 
* `host`: defaults to `http://localhost`. Ignore this argument for local instances.

Typically it is sufficient to specify the `port` and `host`as the remaining attributes are obtained from the Server. Refer to the [Client](/0.1.0-snapshot/client/python-client.md) documentation for details.

* `input_names`, `output_names`: names of the first and final nodes of the Konduit Serving pipeline configuration defined in the Server. 
* `output_data_format`: the format of the server's input and output. Specify one of the following: `JSON`, `NUMPY`, `ARROW`, `IMAGE`, `ND4J`. 

### Steps

#### Python steps

Python steps run code specified in the `python_code` (a string) or `python_code_path` (a `.py` script) argument. Python steps defined in the YAML configuration default to input name and output name `"default"`.

Importantly, the `python_inputs` argument maps each input and output variable to the respective data type. Accepted data types are `INT`, `STR`, `FLOAT`, `BOOL`, `NDARRAY`.

```yaml
steps: 
  python_step: 
    type: PYTHON
    python_code: |
      first += 2
      second = first
    python_inputs:
      first: NDARRAY
    python_outputs:
      second: NDARRAY
```

If no Python path is specified, NumPy will still be available in the environment where the Python step is run.

To further customize Python steps, refer to the YAML configuration section of the Python pipeline steps page.

{% content-ref url="/pages/-LtNeobIL9\_JE8lXIz6S" %}
[Python pipeline steps](/0.1.0-snapshot/steps/python.md)
{% endcontent-ref %}

A more comprehensive example is available on the following page:

{% content-ref url="/pages/-LuptfSTVbseUO9gX9uP" %}
[Open Neural Network Exchange (ONNX)](/0.1.0-snapshot/examples/python/onnx.md)
{% endcontent-ref %}

#### Model steps

Use model steps when you want to use pre-packaged modules such as TensorFlow, DL4J and PMML for inference.

```yaml
steps:
  tensorflow_step:
    type: TENSORFLOW
    model_loading_path: ../data/mnist/mnist_2.0.0.pb
    input_names:
      - input_layer
    output_names:
      - output_layer/Softmax
    input_data_types:
      input_layer: FLOAT
```

The following parameters should be specified:

* `type`: one of `TENSORFLOW`, `KERAS`, `COMPUTATION_GRAPH`, `MULTI_LAYER_NETWORK`, `PMML`, `SAMEDIFF`;
* `model_loading_path`: location of your model file; 
* `input_names`: list of the names of input nodes of your model file;
* `output_names`: list of the names of output nodes of your model file;
* `input_data_types`: map each of the input nodes to one of the following data types using the input names as keys: `INT`, `STR`, `FLOAT`, `BOOL`, `NDARRAY`. 

Refer to the model-specific example for details on configuring model steps.

{% content-ref url="/pages/-Lupw-aLrZSPXWeB9hMA" %}
[TensorFlow (1.x)](/0.1.0-snapshot/examples/python/tensorflow-model-serving.md)
{% endcontent-ref %}

{% content-ref url="/pages/-Lv4jqWJf9I0LtMWQkJ3" %}
[Keras (TensorFlow 2.0)](/0.1.0-snapshot/examples/python/keras.md)
{% endcontent-ref %}

{% content-ref url="/pages/-LuptbQpkK76qsdqbAd6" %}
[Broken mention](broken://pages/-LuptbQpkK76qsdqbAd6)
{% endcontent-ref %}

## Usage

On the **server**, start a Konduit Serving instance by:

1. creating a Server object using `server_from_file`, 
2. starting the server using the `.start()` method. 

```python
from konduit.load import server_from_file

konduit_yaml_path = "../yaml/simple.yaml"

server = server_from_file(konduit_yaml_path)
server.start()
```

```
Starting server..

Server has started successfully.
```

After the server has started, on the **client**:

1. create a Client object using `client_from_file`; and
2. use the `.predict()` method to perform inference on a NumPy array (note that the input name of this Server configuration is `default`, therefore we can pass a NumPy array directly to the `.predict()` method.).

```python
import numpy as np 
import os
from konduit.load import client_from_file

konduit_yaml_path = "../yaml/simple.yaml"
input_arr = np.array(33)

client = client_from_file(konduit_yaml_path)
print(client.predict(input_arr))
```

```
[35]
```

Finally, stop the server with the `.stop()` method:

```python
server.stop()
```

This can also be run in the **command line**. In the root folder of [konduit-serving-examples](https://github.com/KonduitAI/konduit-serving-examples), initialize the Konduit Serving instance with

```bash
konduit serve --config yaml/simple.yaml
```

Send the NPY file to the server for inference with

```bash
konduit predict-numpy --config yaml/simple.yaml --numpy_data data/simple/input_arr.npy
```

and finally, stop the server with

```bash
konduit stop-server --config yaml/simple.yaml
```

## Resources

Some resources on the YAML format:

* <https://gettaurus.org/docs/YAMLTutorial/>
* <https://docs.saltstack.com/en/latest/topics/yaml/>
* <http://jessenoller.com/blog/2009/04/13/yaml-aint-markup-language-completely-different>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://serving.konduit.ai/0.1.0-snapshot/yaml-configurations.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.