Deployment and Serving¶
With all the lifecycle actions simply at our fingertips, let’s now focus on deploying the project as a ForML application making it available for serving.
Creating and Publishing the Application Descriptor¶
ForML applications are implemented in form of a descriptor
instance defined within a Python module. Due to the potentially non-exclusive nature of the
project-application relationship, this module might in general
need to be maintained out of the project scope but for the sake of this tutorial let’s assume a
direct 1:1 relationship so that we can keep it along with the project sources in a module called
application.py
. Our top-level project structure is then going to look as follows:
$ ls -1p forml-tutorial-titanic
application.py
notebooks/
setup.py
tests/
titanic/
For simplicity, we choose to implement the application in
the most basic (yet full-featured) manner by reusing the existing application.Generic
descriptor and passing its instance to application.setup()
for registration. Note this simplistic setup requires the application
name to match the project name (in order to select the relevant assets from the model
registry):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
That’s all it takes to implement a simple application descriptor. It can now be deployed by the means of publishing into a platform-configured application inventory:
$ forml application put application.py
$ forml application list
forml-tutorial-titanic
Serving¶
The easiest way to expose our model for serving is to spin up a particular serving gateway provider linked through the platform configuration to the same inventory and registry holding our application and models respectively.
The configured gateway (the rest.Gateway
in our case)
can be started simply using the CLI:
$ forml application serve
INFO: Started server process [568798]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:8080 (Press CTRL+C to quit)
Note the gateway is capable of serving any application in the linked inventory.
Let’s explore the capabilities using manual curl
queries:
Querying a non-existent application
foobarbaz
:$ curl -X POST http://127.0.0.1:8080/foobarbaz Application foobarbaz not found in Dispatch-registry
Querying our
forml-titanic-example
application using a JSON encoded payload:$ curl -X POST -H 'Content-Type: application/json' -d '[{"Pclass":1, "Name":"Foo", "Sex": "male", "Age": 34, "SibSp": 3, "Parch": 2, "Ticket": "13", "Fare": 10.1, "Cabin": "123", "Embarked": "S"}]' http://127.0.0.1:8080/forml-tutorial-titanic [{"c0":0.3459976655}]
Making the same query but requesting the result to be encoded as CSV:
$ curl -X POST -H 'Content-Type: application/json' -H 'Accept: text/csv' -d '[{"Pclass":1,"Name":"Foo", "Sex": "male", "Age": 34, "SibSp": 3, "Parch": 2, "Ticket": "13", "Fare": 10.1, "Cabin": "123", "Embarked": "S"}]' http://127.0.0.1:8080/forml-tutorial-titanic c0 0.34599766550668526
Making the same query but sending the payload in the pandas-split (JSON) format and requesting the result as (JSON) values:
$ curl -X POST -H 'Content-Type: application/json; format=pandas-split' -H 'Accept: application/json; format=pandas-values' -d '{"columns": ["Pclass", "Name", "Sex", "Age", "SibSp", "Parch", "Ticket", "Fare", "Cabin", "Embarked"], "data": [[1, "Foo", "male", 34, 3, 2, 13, 10.1, "123", "S"]]}' http://127.0.0.1:8080/forml-tutorial-titanic [[0.3459976655]]
That concludes this Titanic Challenge tutorial, from here you can continue to the other available tutorials or browse the general ForML documentation.