Develop complex data pipelines with Argo workflows

Requests from the business departments often require the development of complex data pipelines, which are difficult to realize with common ETL/ELT tools such as Oracle Data Integrator. The orchestration of procedures from the field of data science to complex workflows often requires flexibility in development, which classic ETL/ELT tools rarely offer. For this reason, the customer has chosen 'Argo Workflows'. Argo Workflows is a container-native open source workflow engine for orchestrating parallel jobs on Kubernetes. With Argo Workflows, process flows (workflows) can be defined where each step in the workflow is a container. This allows workflows to be developed whose steps use different versions of libraries or technologies. Argo Workflows can also be used to model multi-step workflows as a sequence of steps, including their interdependencies, using directed acyclic graphs (DAG).