This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Ingestion Engineer based in the United States.
This role sits at the heart of a large-scale data ingestion and processing ecosystem powering social video intelligence products used by leading global media organizations. You will be responsible for building and evolving end-to-end ETL pipelines that process high-volume, multi-platform data across sources such as YouTube, Instagram, Facebook, and Twitch. The position combines backend engineering, distributed systems design, and data infrastructure ownership in a fast-moving, product-driven environment. You will work closely with product and analytics teams to design new capabilities, improve data reliability, and expand system scalability. The environment is highly collaborative and remote-first, requiring strong autonomy, technical leadership, and a focus on engineering excellence. This is an opportunity to shape systems that directly influence how media companies understand audience behavior and content performance at scale.
Accountabilities:
This role is responsible for designing, building, and maintaining scalable ingestion pipelines and distributed data systems that power analytics and product features across multiple platforms.
- Design, build, and maintain end-to-end data ingestion and ETL pipelines processing large-scale social video data
- Develop and optimize distributed systems using technologies such as Python, Apache Spark, and Apache Kafka
- Ensure data quality, reliability, and performance across ingestion workflows and downstream systems
- Collaborate with product and engineering teams to define requirements and deliver new data-driven features
- Work with relational databases and data storage systems, including SQL-based architectures and cloud storage (e.g., AWS S3)
- Contribute to architecture decisions and advocate for best practices in scalability, performance, and maintainability
- Lead technical initiatives and own complex engineering projects end-to-end across multiple teams
- Participate in on-call rotations to ensure system reliability and operational stability
- Mentor junior engineers and contribute to a strong engineering culture focused on quality and collaboration
- Continuously improve ingestion frameworks and data processing workflows for efficiency and scalability
Requirements:
This role requires strong backend engineering experience with a focus on data systems, distributed processing, and scalable pipeline development. The ideal candidate is proactive, collaborative, and comfortable owning complex technical systems.
- 5+ years of experience as a software or data engineer with strong proficiency in Python
- Solid understanding of relational databases and SQL
- Experience working with distributed systems such as Apache Kafka and/or Apache Spark (preferred)
- Proven ability to lead technical projects and collaborate across engineering and product teams
- Strong experience building or maintaining data pipelines, ETL systems, or large-scale data infrastructure
- Familiarity with cloud environments and data storage systems (e.g., AWS, S3, Hive Metastore, or similar)
- Strong communication skills with the ability to align stakeholders and drive technical consensus
- Experience working in remote-first environments and strong self-management skills
- Comfortable participating in on-call rotations and production system support
- Experience with AI-assisted development tools (e.g., Cursor, Claude Code) is a plus
- Mentorship experience and a collaborative, knowledge-sharing mindset
Benefits:
- Fully remote-first role within the United States
- Opportunity to work on large-scale, high-volume social video data systems
- Exposure to modern data engineering stack including Python, Spark, Kafka, and AWS-based infrastructure
- High-impact role influencing product features used by leading global media brands
- Collaborative engineering culture with strong emphasis on ownership and technical excellence
- Opportunity to lead projects end-to-end and grow into technical leadership responsibilities
- Flexible work environment supporting autonomy and work-life balance
- Competitive compensation package (based on experience and qualifications)
- Inclusive and diverse workplace culture focused on continuous learning and improvement.