Staff Data Engineer

Job Summary

A company is looking for a Staff Data Engineer.

Key Responsibilities

• Define and drive software architecture development across engineering teams, focusing on data pipeline efficiency and scalability
• Lead the design and implementation of robust data solutions and microservices for real-time and batch processing
• Automate and streamline deployments, managing cloud-based workflows and data pipeline orchestration

Required Qualifications

• Bachelor's degree in Computer Science or Computer Engineering
• 5 years of experience in developing and optimizing large-scale microservices and APIs using Scala, Java, or Python
• 5 years of experience with Spark or similar tools for data pipeline development
• 5 years of experience in setting up CI/CD pipelines using tools like GitHub, GitLab, or Jenkins
• Experience with Docker for containerization and handling data processing of at least 100 million records daily