Providers Library¶

ForML providers are plugins implementing particular functionality defined within the framework using an abstract interface to decouple itself from specific technologies allowing for greater operational flexibility.

Providers become available for runtime operations after being properly configured within the given platform.

Custom Provider Setup¶

In addition to the existing providers, users might find themselves requiring to implement their bespoke instances. To avoid having to always release and deploy these providers as true code artifacts, ForML allows to alternatively treat them in a rather more configuration-like manner.

For this purpose, the standard config directories are valid locations for hosting python modules with bespoke provider implementations available to the particular runtime platform.

Following is an example of a custom Feed setup (even though this one could well be solved using the existing generic Alchemy Feed or - given the particular dataset - even more easily using the openlake.Lite feed):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

 from forml import io
 from forml.provider.feed.reader.sql import alchemy
 from openschema import kaggle
 import sqlalchemy


 class Baz(io.Feed):
     """Custom feed example."""

     class Reader(alchemy.Reader):
         """Using the existing SQLAlchemy reader."""

     @property
     def sources(self):
         """This feed can serve just one and only dataset - the ``titanic`` table mapped to
            the ``kaggle.Titanic`` schema."""

         return {kaggle.Titanic: sqlalchemy.table('titanic')}

This custom foobar:Baz feed provider can now be added to the platform config:

 [FEED.foobar]
 provider = "foobar:Baz"
 connection = "sqlite:////tmp/foobar.db"

Model Registries¶

ForML delegates responsibility for model persistence to model registry providers implementing the abstract forml.io.asset.Registry base class.

Runners¶

The actual execution of the ForML workflows is performed by the pipeline runner providers implementing the forml.runtime.Runner base class.

Feeds¶

To decouple projects from any physical data sources, ForML is using a generic query DSL working with logical schemas that only at runtime get resolved to actual data provided by the platform-configured set of feeds implementing the forml.io.Feed base class.

External Providers

Sinks¶

Reciprocally to the Feeds system, ForML is using sink providers for submitting the workflow results according to the particular implementation of the forml.io.Sink base class.

Application Inventories¶

For managing the high-level application descriptors driving the serving layer, ForML defers to the inventory providers implementing the forml.io.asset.Inventory base class.

Gateways¶

The serving layer representing one of the possible execution mechanisms is using the gateway providers implementing the forml.runtime.Gateway base class.