Senior/Staff Software Engineer, Search & Retrieval Infrastructure

Jobgether · US

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior/Staff Software Engineer, Search & Retrieval Infrastructure based in the United States.

This role sits at the core of next-generation AI knowledge systems, building the infrastructure that powers how modern applications retrieve, understand, and synthesize information at scale. You will design and develop high-performance backend systems that enable semantic search, hybrid retrieval, and LLM-augmented knowledge workflows across massive datasets. The work focuses on transforming both structured and unstructured data into reliable, queryable knowledge for AI-driven applications. You will own key architectural decisions spanning indexing pipelines, retrieval orchestration, and distributed backend services. The environment is deeply technical, fast-moving, and centered on large-scale AI infrastructure and system performance. Your contributions will directly impact the quality, speed, and reliability of retrieval systems used by thousands of developers and enterprises. This is a high-ownership role with significant influence on the future of AI-powered search.

Accountabilities:

Design and build scalable search and retrieval infrastructure, including semantic search, hybrid retrieval, metadata-aware querying, and LLM-driven query planning systems.
Develop high-throughput indexing pipelines for both structured and unstructured data, ensuring performance, reliability, and scalability.
Build and maintain backend services that support retrieval orchestration, knowledge synthesis, and agentic AI workflows.
Improve retrieval quality through robust evaluation frameworks, observability systems, and experimentation on ranking and relevance.
Design clean, intuitive, and scalable APIs for both internal systems and external developer and agentic use cases.
Optimize system performance across latency, throughput, and cost in large-scale distributed environments.
Drive technical direction for reliability, security, and architectural evolution of core retrieval systems.
Collaborate closely with cross-functional teams to align infrastructure capabilities with product and AI application needs.
Contribute to system-level design decisions that shape long-term platform scalability and extensibility.

Requirements

6+ years of experience building production-grade backend systems in large-scale distributed environments.
Strong expertise in system architecture, with a focus on high throughput, low latency, and scalable design principles.
Experience building or working with search systems, including semantic search, vector databases, hybrid retrieval, or platforms like Elastic or OpenSearch.
Deep understanding of retrieval-augmented generation (RAG), embedding pipelines, and LLM-based orchestration patterns.
Proficiency in at least one major programming language such as Go, Rust, C++, Java, or Python.
Experience with data engineering and building large-scale indexing pipelines for diverse data types.
Familiarity with modern infrastructure tools such as Kubernetes, cloud-native architectures, observability systems, and IaC tools like Terraform or Pulumi.
Strong product mindset with the ability to design developer-friendly and agent-friendly APIs.
Comfortable working in ambiguous, high-growth environments with significant ownership expectations.
Strong problem-solving skills and a bias toward building robust, long-term systems rather than short-term fixes.
Bonus: experience with multi-tenant SaaS systems, retrieval evaluation frameworks, or agentic query planning systems.

Benefits

Comprehensive medical, dental, vision, and mental health coverage
401(k) retirement plan
Equity compensation package
Flexible PTO policy
Paid parental leave
Annual company retreat and offsites
Home office equipment stipend
Inclusive and collaborative engineering culture focused on AI innovation
Opportunity to work on cutting-edge AI search and retrieval infrastructure at scale.

Software pay context

Based on 7,869 disclosed Software salaries on RoleSuite, the role pays a median of $158K/year, with most offers between $124K and $200K (10th–90th percentile: $102K–$235K).

See the full Software salary breakdown →

Apply →

Senior/Staff Software Engineer, Search & Retrieval Infrastructure

Accountabilities:

Requirements

Benefits

Software pay context

Other roles at Jobgether

More Software roles