Data Engineer – Azure Databricks (SOX‑Governed Financial Reporting)

About the role

We are seeking a Data Engineer to join a high‑impact transformation project that will build a fully automated, SOX‑governed financial reporting pipeline. The role combines deep technical work on Azure Databricks Lakehouse architecture with close collaboration with finance stakeholders.

Key responsibilities

• Design, develop and optimise production‑grade ETL pipelines using Azure Databricks, PySpark and Python.
• Build and manage Delta Lake structures and Unity Catalog governance frameworks.
• Create Bronze, Silver and Gold Lakehouse pipelines with clear lineage and auditability.
• Implement financial data transformation logic, KPI‑to‑GL mappings and reconciliation processes.
• Develop automated data‑quality frameworks (validation, completeness, duplicate detection, thresholds).
• Build REST and SOAP API integrations with robust error handling and monitoring.
• Implement SOX‑compliant controls, audit trails, approval workflows and governance processes.
• Support testing, UAT, reconciliation activities and production monitoring.
• Produce operational dashboards, reporting layers and technical documentation for long‑term success.

Required profile

• Bachelor’s degree in Computer Science, Data Engineering, IT, Mathematics or a related field.
• 3+ years of experience in Data Engineering or Analytics Engineering.
• Strong ability to take ownership, drive delivery and bridge technical‑business conversations.
• Excellent documentation and communication skills.

Required skills

• Azure Databricks
• Databricks Workflows
• Delta Lake
• Unity Catalog
• PySpark
• Python
• SQL
• Azure Data Lake Storage Gen2 (ADLS)
• Azure Key Vault
• Azure Data Warehouse environments
• REST and SOAP API integrations