Job Summary
A company is looking for a Software Engineer, Data Collection to join their Data Engineering team.
Key Responsibilities
- Build high-volume, fault-tolerant scraping services to enhance data extraction infrastructure
- Manage large-scale event processing systems handling millions of events daily
- Design and improve algorithms for optimizing scraping efficiency and accuracy
Required Qualifications
- Experience with distributed systems and data pipelines
- Strong programming skills in Python or TypeScript
- Expertise in SQL for querying and transforming data
- Hands-on experience in a cloud-native environment (AWS preferred)
- Familiarity with CI/CD pipelines and automated testing
Comments