Monitoring of ETL / ELT processes in the Business Intelligence environment

PTA identifies relevant or appropriate metrics to monitor the ETL or ELT processes. For this purpose, the metadata is extracted from the Oracle Repository Database of the ETL/ELT tool Oracle Data Integrator (12c) at regular intervals and imported as metrics into the designated timeseries database InfluxDB. Analogous to the existing data warehouse architecture in the form of a Business Object Model (BOM), the PTA creates dedicated Grafana dashboards based on the metrics for all components. In addition, each component is visualised as a tile in an overview dashboard, which shows the current status based on the metrics and defined threshold values in green, yellow or red. The detailed dashboard of the respective component or sub-components can be called up at any time via drill-down.