Architecture

Metamorf is a Python Software that orchestrates, controls and transforms data. From a metadata entry, Metamorf process the metadata to full its system and in exchange grants a series of functionalities that allows to control all your datawarehouse.

Metamorf stores all the metadata historically, it means that the users can see the status of their datawarehouse in the past at any time and know how the data was processed.

The main benefit for use a framework to build all the data process, is to standarize all the processes, build modular and reusable data models and control all the data flow.

Metadata and data can be on the same database or in separate ones, thus allowing it to be adaptaded to a greateer number of different architectures and needs.

Databases

Actually Metamorf supports the following databases for both data and metadata:

• Snowflake
• SQLite
• PostgreSQL
• MySQL

How does it work?

Metamorf downloads as a python package, thus facilitating its installation in different environments. Its operation depends on a configuration file in YML format and a specific directory generated by the application itself. Once the environment has been prepared, any command can be executed through the shell.

Once the developper has entered the metadata, the metadata needs to be processed. After a previous validation, the metadata is promoted on an exploitable metadata model. This final metadata permits to generate SQL models.

Metamorf works with the concept of owner: it can be a user or a use case. All actions executed by metamorf are done selecting an owner, indicated on the configuration file. It can be useful for teamwork using the same Metadata Database and isolating each user or use case under development. Is up to you.

It can fit with other technologies as dbt, giving the code ready to execute.