Data Source DSL¶

To allow projects to declare their data requirements in a portable way independently of any particular storage technology and data formats, ForML comes with its custom Data Source DSL (domain-specific language) which gets interpreted at runtime by the feeds subsystem performing the content resolution routine to deliver the requested datasets.

Conceptually, it is an internal DSL (i.e. within Python grammar) based on a declarative style of specifying the data profiles using the following two main constructs:

schema definition syntax for logical representation of individual datasets
query statement notation for declaring the project data requirements

Important

Do not confuse the DSL with an ORM framework. The DSL entities are not used to manage any data directly. Its sole purpose is to describe the data sources independently of the data access mechanism in the same way the workflow expression describes the processing logic independently of the execution mechanism. In both cases, the abstract descriptions get transcoded to runtime-specific instructions when launched.

The DSL agenda is divided into the following chapters: