Deployment and Serving

With all the lifecycle actions simply at our fingertips, let’s now focus on deploying the project as a ForML application making it available for serving.

Creating and Publishing the Application Descriptor

ForML applications are implemented in form of a descriptor instance defined within a Python module. Due to the potentially non-exclusive nature of the project-application relationship, this module might in general need to be maintained out of the project scope but for the sake of this tutorial let’s assume a direct 1:1 relationship so that we can keep it along with the project sources in a module called application.py. Our top-level project structure is then going to look as follows:

$ ls -1p forml-tutorial-titanic
application.py
notebooks/
pyproject.toml
tests/
titanic/

For simplicity, we choose to implement the application in the most basic (yet full-featured) manner by reusing the existing application.Generic descriptor and passing its instance to application.setup() for registration. Note this simplistic setup requires the application name to match the project name (in order to select the relevant assets from the model registry):

application.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
"""
Titanic application descriptor.

Using the basic ready-to-use application.Generic descriptor provides
the following features:

* loading the *latest* model generation of the project *matching*
  the application name
* attempting to decode the payload using any of the available decoders
  based on the *declared content-type*
* returning the predictions encoded using any of the available encoders
  based on the *requested content-type*
"""

from forml import application

application.setup(application.Generic('forml-tutorial-titanic'))

That’s all it takes to implement a simple application descriptor. It can now be deployed by the means of publishing into a platform-configured application inventory:

$ forml application put application.py
$ forml application list
forml-tutorial-titanic

Serving

The easiest way to expose our model for serving is to spin up a particular serving gateway provider linked through the platform configuration to the same inventory and registry holding our application and models respectively.

The configured gateway (the rest.Gateway in our case) can be started simply using the CLI:

$ forml application serve
INFO:     Started server process [568798]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8080 (Press CTRL+C to quit)

Note the gateway is capable of serving any application in the linked inventory.

Let’s explore the capabilities using manual curl queries:

  1. Querying a non-existent application foobarbaz:

    $ curl -X POST http://127.0.0.1:8080/foobarbaz
Application foobarbaz not found in Dispatch-registry

  2. Querying our forml-titanic-example application using a JSON encoded payload:

    $ curl -X POST -H 'Content-Type: application/json' -d '[{"Pclass":1, "Name":"Foo", "Sex": "male", "Age": 34, "SibSp": 3, "Parch": 2, "Ticket": "13", "Fare": 10.1, "Cabin": "123", "Embarked": "S"}]' http://127.0.0.1:8080/forml-tutorial-titanic
[{"c0":0.3459976655}]

  3. Making the same query but requesting the result to be encoded as CSV:

    $ curl -X POST -H 'Content-Type: application/json' -H 'Accept: text/csv' -d '[{"Pclass":1,"Name":"Foo", "Sex": "male", "Age": 34, "SibSp": 3, "Parch": 2, "Ticket": "13", "Fare": 10.1, "Cabin": "123", "Embarked": "S"}]' http://127.0.0.1:8080/forml-tutorial-titanic
c0
0.34599766550668526

  4. Making the same query but sending the payload in the pandas-split (JSON) format and requesting the result as (JSON) values:

    $ curl -X POST -H 'Content-Type: application/json; format=pandas-split' -H 'Accept: application/json; format=pandas-values' -d '{"columns": ["Pclass", "Name", "Sex", "Age", "SibSp", "Parch", "Ticket", "Fare", "Cabin", "Embarked"], "data": [[1, "Foo", "male", 34, 3, 2, 13, 10.1, "123", "S"]]}' http://127.0.0.1:8080/forml-tutorial-titanic
[[0.3459976655]]

That concludes this Titanic Challenge tutorial, from here you can continue to the other available tutorials or browse the general ForML documentation.