Senior Data Collection Engineer

Job Summary

A company is looking for a Senior Data Collection Engineer based in Spain.

Key Responsibilities

• Design and build robust web crawlers for high-scale data extraction
• Enhance and maintain infrastructure, including CI/CD pipelines and centralized spider management
• Ensure data integrity and accuracy through validation mechanisms and collaboration with internal teams

Required Qualifications

• Experience with Git workflows, code reviews, and CI/CD pipelines
• Proficiency in cloud infrastructure, particularly AWS
• Familiarity with monitoring systems such as Grafana and Sentry
• Knowledge of web standards and networking protocols
• Experience in developing scalable web crawlers using Python and Scrapy