Data Quality Engineer

Data Quality Engineer (Databricks)Contract6 Months35,000 - 37,000 AED a monthCandidates must be Immediately Available OverviewWe are seeking a Data Quality Engineer to design, implement, and manage an enterprise-scale data quality framework within a Databricks environment. The role will focus on profiling, validation, monitoring, cleansing, remediation, and reporting across a large and complex data estate.Key ResponsibilitiesData Quality Framework & PlatformConfigure and maintain Databricks, Delta Lake, Unity Catalog, and PySpark frameworks.Establish scalable data quality controls and deployment standards.Optimise platform performance for large-scale profiling and rule execution.Data Profiling & Quality AssessmentDevelop profiling solutions to assess completeness, accuracy, consistency, validity, timeliness, and uniqueness.Analyse data quality issues including nulls, duplicates, invalid values, schema drift, and outliers.Produce baseline quality assessments and recommendations.Rule Development & AutomationBuild a reusable Data Quality Rule Factory using PySpark.Develop scalable, parameterised validation rules.Integrate quality controls into data pipelines and implement automated quality gates.Data Cleansing & RemediationCreate automated cleansing, standardisation, enrichment, and deduplication processes.Utilise machine learning techniques for anomaly detection and duplicate identification.Support exception handling, failed record management, and reprocessing workflows.Monitoring & ReportingDevelop quality metrics, dashboards, and reporting datasets.Monitor data quality performance and implement automated alerts.Produce Data Quality Index (DQI) scores, compliance metrics, and operational reporting outputs.Governance & Stakeholder ManagementCollaborate with data governance, modelling, and engineering teams.Maintain documentation, audit trails, and quality artefacts.Support governance reviews and stakeholder workshops.Required ExperienceStrong experience implementing enterprise Data Quality frameworks.Advanced Databricks and PySpark expertise.Experience with Delta Lake, Unity Catalog, Databricks Jobs/Workflows, and Databricks SQL.Proven track record building scalable validation and quality rule frameworks.Knowledge of ML-driven data quality monitoring and anomaly detection.Experience integrating quality metrics into reporting tools such as Power BI.Strong understanding of data governance, metadata management, and modern cloud/lakehouse architectures.Key DeliverablesDatabricks-based Data Quality framework and rule engine.Automated profiling, validation, cleansing, and remediation solutions.Data quality monitoring, reporting, and alerting capabilities.Governance and compliance documentation supporting enterprise data management initiatives