Senior Data Engineer

Job Summary

A company is looking for a Senior Data Engineer to take ownership of data infrastructure and build scalable data pipelines.

Key Responsibilities

• Architect and build scalable ELT/ETL pipelines using Databricks, Spark, and Python
• Lead and mentor a team of data engineers, providing technical direction and support
• Ensure data integrity through comprehensive testing frameworks and data validation rules

Required Qualifications

• Bachelor's or master's degree in computer science, engineering, or a related field
• 7+ years of hands-on experience in data engineering, focusing on Databricks and Apache Spark
• Expert-level proficiency in Python and a strong understanding of data structures and algorithms
• Deep expertise in SQL and Spark SQL, with experience in optimizing complex analytical queries
• Hands-on experience with the Databricks ecosystem, including Delta Lake and Data frame APIs