Atos is seeking an Azure Databricks Developer to design, develop, and support scalable data engineering solutions using Azure Databricks and the broader Azure data platform.
In this role, you will build robust data ingestion, transformation, and processing pipelines that enable advanced analytics, artificial intelligence, machine learning, and business intelligence capabilities. You will work closely with Data Architects, Data Engineers, Data Analysts, and Solution Architects to deliver trusted, high-quality data products while ensuring compliance with security, governance, and operational standards.
Key Responsibilities
- Design, develop, and optimise scalable data pipelines using Azure Databricks, PySpark, Spark SQL, and Delta Lake.
- Build robust ETL/ELT solutions and data ingestion frameworks supporting batch and streaming workloads.
- Integrate data from Azure SQL Database, Azure Data Lake Storage Gen2, Microsoft Fabric, APIs, SaaS platforms, and on-premise systems.
- Implement data quality, validation, monitoring, and governance controls across the data lifecycle.
- Support platform engineering activities, including CI/CD implementation, environment promotion, performance optimisation, and operational support.
Essential Skills
- Expertise in Azure Databricks, Apache Spark, PySpark, and Spark SQL.
- Experience building enterprise-scale ETL/ELT pipelines within Azure environments.
- Strong Python and SQL development skills.
- Experience with Azure Data Lake Storage Gen2 and Azure Data Factory or Microsoft Fabric Data Factory.
- Git version control and software engineering best practices.
Desirable Skills
- Experience with Microsoft Fabric, Azure Synapse Analytics, Azure Event Hubs, or Kafka.
- Knowledge of Azure DevOps, Terraform, Unity Catalog, MLflow, or Azure Machine Learning.
Additional Requirements
- Eligibility to obtain and maintain SC and NPPV3 clearance.
- Proven experience working with large-scale data processing and distributed computing technologies.
- Experience working within Agile delivery teams.
Location
UK-based. Remote working is available, with national mobility required.
Benefits
- 25 days paid leave plus national holidays
- Pension scheme with contributions matched up to 10%
- Private medical cover
- Income protection
- Life assurance
- Flex Benefits Program
Atos is a care leaver friendly and Disability Confident employer committed to diversity, inclusion, and reasonable adjustments throughout the application process.