Member of Technical Staff (Site Reliability Engineer, Crawler)

Perplexity · London · posted 14 Nov 2026
Full-timeIT & Software5+ years

About the job

Keeping a crawler running is not the same as keeping it productive. Queues can grow while servers look healthy, and a small change can become a fleet-wide bottleneck. On Perplexity's Crawler team, you'll improve throughput, plan capacity, and build operational reliability through safer deployments, better observability, and durable fixes. Responsibilities • Operate and optimize Kubernetes workloads and supporting infrastructure for crawler services, including fetching and rendering fleets. • Build and maintain infrastructure as code and CI/CD pipelines, with safe rollouts, validation, and fast rollback. • Define SLOs and build monitoring, alerting, and observability for availability, latency, throughput, and resource saturation. • Own capacity planning and resource efficiency; diagnose queue backlogs, network failures, and resource bottlenecks with service owners. • Participate in the team's on-call rotation, lead incident response, and turn blameless postmortems into durable reliability improvements. • Write production software to automate diagnosis and routine remediation, with validation and approval gates for high-impact actions. Qualifications • 5+ years of experience in SRE, production engineering, or infrastructure-heavy backend roles, with strong programming skills in Python, C++, Rust, or Go. • Hands-on experience with cloud infrastructure, production Kubernetes, infrastructure as code such as Terraform, and automated deployment workflows. • Strong Linux and networking fundamentals, including TCP/IP, DNS, TLS, HTTP, and load balancing. • Experience building observability with tools such as Datadog, Prometheus, or Grafana, and using SLOs to guide reliability improvements. • Proven experience diagnosing distributed-system failures, leading production incidents, and improving system performance or resource efficiency. Nice to have • Experience operating web crawlers, search infrastructure, or browser fleets at scale. • Experience with…

Read the full description on Perplexity's site ↗

Skills

Upload your CV on LitCareer to see which of these you match.

Similar jobs

Z
Data Scientist
Zego · London · 16 Nov 2026
ML Data & Platform Engineer
Speechmatics · London · 16 Nov 2026
Senior Data Analyst, Marketing Measurement
Airalo · United Kingdom · 16 Nov 2026
Software Developer - Senior Consultant (Contract)
Opencast · Hybrid - Newcastle, Edinburgh, Glasgow, Leeds, Manchester, Birmingham, Bristol, London · 16 Nov 2026
Software Developer - Senior Consultant
Opencast · Hybrid - Newcastle, Edinburgh, Glasgow, Leeds, Manchester, Birmingham, Bristol, London · 16 Nov 2026
Software Developer - Consultant
Opencast · Hybrid - Newcastle, Edinburgh, Glasgow, Leeds, Manchester, Birmingham, Bristol, London · 16 Nov 2026

Collected from Perplexity's official careers site by LitCareer. Always apply on the employer's own site; LitCareer never charges you to apply.