Identification of technically incorrect entries in a LIMS database and user notification
Brief description
When using a LIMS for data collection in the routine operations of a biopharmaceutical manufacturer, incorrect data entries occur regularly. The errors are identified by analyzing the data in a data warehouse. The check is performed regularly, and if errors are found, the responsible users are automatically notified by email. The application is developed as custom software within the customer's existing architecture (Databricks). This ensures the quality and integrity of the customer's database entries.
Supplement
The ETL pipeline is implemented by PTA in Python and SQL and hosted on the customer's Databricks tenant. SQL queries are used to identify incorrect entries in a Postgres warehouse, which are then converted into suitable Polars DataFrames and further processed so that they can ultimately be sent to the appropriate recipients via an SMTP server.
Subject description
The customer has precise requirements for storing research data in the warehouse. Previously, incorrect entries were displayed in a dashboard. The software developed by PTA replaces this dashboard. This allows automated notification of the responsible persons without the need for manual checking of the dashboard. In addition, targeted emailing ensures that only the relevant persons are informed about the errors.