GCP Data Engineer

Role Summary

We are looking for an experienced Data Engineer to join our Data Ops team. This hands-on engineering role is responsible for delivering scalable data solutions while supporting DataOps practices across the data platform.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT solutions using Python and PySpark.
• Build and optimize data processing solutions on Google Cloud Platform (GCP).
• Develop and maintain workflow orchestration using Apache Airflow, Cloud Composer, or Astronomer.
• Design and optimize complex SQL queries for analytical workloads.
• Lead and support DataOps activities, including deployment automation, CI/CD, monitoring, operational support, incident resolution, and continuous improvement.
• Support infrastructure provisioning and configuration using Terraform.
• Work with containerized environments and Google Kubernetes Engine (GKE) to support data platform deployments.
• Monitor pipeline health, data quality, performance, and reliability, implementing proactive alerting and operational best practices.
• Collaborate with Data Engineers, Analytics teams, Product Owners, and business stakeholders.
• Troubleshoot production issues and drive root cause analysis.
• Contribute to engineering standards, code reviews, documentation, and Data Engineering and DataOps best practices.

Required Skills

• 6+ years of experience in Data Engineering with hands-on development experience.
• Strong proficiency in Python, PySpark, and advanced SQL.
• Experience designing and implementing production-grade ETL/ELT pipelines.
• Hands-on experience with Google Cloud Platform, including BigQuery, Cloud Storage, Dataflow, Cloud Composer, and Secret Manager.
• Experience with CI/CD pipelines and DataOps or DevOps practices.
• Working knowledge of Terraform and Infrastructure as Code.
• Familiarity with Google Kubernetes Engine or containerized application deployments.
• Experience monitoring, troubleshooting, and supporting production data platforms.
• Strong analytical, debugging, and problem-solving skills.
• Ability to independently own technical deliverables from development through production support.

Preferred Skills

• Prior American Express experience.
• Experience with enterprise-scale cloud data platforms.
• Knowledge of data warehousing, data modeling, and data governance principles.
• Experience implementing observability, monitoring, logging, and operational automation for data platforms.
• Experience working in Agile delivery teams.