Member of Technical Staff - Site Reliability Engineer, Crawler

About the role

On Perplexity's Crawler team, you will improve throughput, plan capacity and build operational reliability through safer deployments, better observability and durable fixes. The role keeps crawlers productive by handling queues, server health and fleet-wide bottlenecks.

Key responsibilities

• Operate and optimize Kubernetes workloads and supporting infrastructure for crawler services, including fetching and rendering fleets.
• Build and maintain infrastructure as code and CI/CD pipelines, with safe rollouts, validation and fast rollback.
• Define SLOs and build monitoring, alerting and observability for availability, latency, throughput and resource saturation.
• Own capacity planning and resource efficiency; diagnose queue backlogs, network failures and resource bottlenecks with service owners.
• Take part in the on-call rotation, lead incident response and turn blameless postmortems into reliability improvements.
• Write production software to automate diagnosis and routine remediation, with validation and approval gates for high-impact actions.

Required profile

• 5+ years of experience in SRE, production engineering or infrastructure-heavy backend roles, with strong programming skills in Python, C++, Rust or Go.
• Hands-on experience with cloud infrastructure, production Kubernetes, infrastructure as code such as Terraform and automated deployment workflows.
• Proven experience diagnosing distributed-system failures, leading production incidents and improving system performance or resource efficiency.
• Nice to have: experience operating web crawlers, search infrastructure or browser fleets at scale, and experience with distributed queues, backpressure, workload isolation or multi-region recovery.

Required skills

• Strong Linux and networking fundamentals, including TCP/IP, DNS, TLS, HTTP and load balancing.
• Experience building observability with tools such as Datadog, Prometheus or Grafana, and using SLOs to guide reliability improvements.