Sr. Software Engineer, AI / ML Inference Platform

Your role

We are hiring a Senior Software Engineer to build the shared AI / ML platform that takes Dialpad’s model-backed capabilities from training through production inference.

The AI / ML Platform team builds and operates GPU training infrastructure, model evaluation and lifecycle tooling, and production inference systems running on NVIDIA GPUs in GCP. We provide the common engineering foundations that allow ASR, NLP, and other AI teams to train, evaluate, release, operate, and continually improve models at enterprise scale.

Inference is an important center of gravity for this role: turning trained models into reliable, observable, efficient production services. The work is intentionally end-to-end, however, because production outcomes are shaped by decisions made throughout the model lifecycle. You will work across training clusters, model artifacts, evaluation and release workflows, serving runtimes, production operations, and feedback loops.

You will also serve as a senior engineering partner to ASR and NLP scientists. You will help teams reason about reproducibility, evaluation, scalability, hardware and runtime constraints, latency, reliability, cost, and release safety while there is still time to influence the design. You will not be expected to conduct original ML research, but you must understand training, data, evaluation, and model behavior well enough to help translate scientific work into dependable enterprise ML systems.

This is an implementation-heavy engineering role, not an operations support position. You will build systems directly, lead substantial technical work, and improve the shared practices by which Dialpad moves AI capabilities from experimentation into production.

What you’ll do

• Design, build, and improve shared platform capabilities spanning model training, evaluation, artifact management, release, production inference, and operational feedback.
• Build and operate shared GPU training infrastructure that provides scientists with reliable, reproducible, and efficient environments for model development and experimentation.
• Improve training-cluster scheduling, workload isolation, capacity management, storage, networking, observability, and accelerator utilization.
• Develop production-serving pathways for low-latency, high-throughput, and highly available inference workloads.
• Integrate and adapt model-training frameworks and inference runtimes to meet Dialpad’s requirements for automation, observability, security, and operational control.
• Improve the performance and efficiency of GPU workloads by reasoning across compute, memory, storage, networking, batching, concurrency, and workload scheduling.
• Partner with ASR and NLP scientists to translate evolving model capabilities into scalable production designs.
• Counsel scientific teams on production concerns including reproducibility, evaluation coverage, artifact design, resource requirements, serving feasibility, failure modes, and quality–performance trade-offs.
• Improve how models and related artifacts are versioned, traced, validated, compared, promoted, deployed, and rolled back across environments.
• Enable safe releases through representative evaluation, automated quality and performance checks, shadow traffic, staged rollouts, candidate-versus-incumbent comparisons, and fast rollback.
• Build benchmarking and evaluation infrastructure that measures model quality alongside latency, throughput, saturation behavior, reliability, resource utilization, and cost.
• Strengthen telemetry, structured logging, tracing, dashboards, alerting, and diagnostic tooling across training and production environments.
• Use performance data, incidents, developer feedback, and production model behavior to identify and deliver high-value improvements across the AI lifecycle.
• Reduce recurring manual work by building self-service workflows, clear interfaces, and practical standards that other AI teams can adopt.
• Lead technical projects from design through production operation, contribute to architectural decisions, and mentor other engineers.

Skills you’ll bring

• Production engineering experience: Seven or more years of professional software engineering experience, with demonstrated ownership of backend, infrastructure, distributed, or ML platform systems in production.
• ML systems experience: Experience building or operating systems that support model training, model inference, or the lifecycle connecting them.
• Strong software fundamentals: Proficiency in Python, Go, or another backend-oriented language, with a record of producing maintainable production software and well-designed interfaces.
• Cloud and Kubernetes fluency: Hands-on experience with Linux, containers, Kubernetes, cloud infrastructure, CI/CD, deployment automation, and production operations.
• Accelerated-computing knowledge: Experience operating GPU workloads and reasoning about utilization, memory, storage, networking, scheduling, and workload performance.
• Training familiarity: Working knowledge of modern model-training workflows, including datasets, experiments, distributed execution, checkpoints, reproducibility, and model artifacts.
• Applied data-science fluency: An understanding of dataset quality, evaluation design, experimental validity, error analysis, model-quality metrics, and production model behavior sufficient to collaborate effectively with applied scientists.
• Systems and performance judgment: The ability to find bottlenecks across system boundaries and make reasoned trade-offs among model quality, latency, throughput, reliability, capacity, and cost.
• Operational judgment: A strong instinct for observability, repeatability, release safety, failure containment, rollback, and whole-system resilience.
• Technical leadership: The ability to independently lead ambiguous projects, communicate clearly across disciplines, mentor engineers, and influence decisions through sound technical reasoning.

Particularly relevant experience

You do not need experience with every technology or domain listed below. Experience in several of these areas would be especially valuable:

• ASR, speech processing, NLP, large language models, or other production model-backed systems.
• GPU-based or distributed model training.
• Model-serving runtimes such as vLLM, Triton, TGI, or comparable systems.
• Model-development frameworks such as PyTorch or JAX.
• Kubernetes-based GPU scheduling and workload management.
• GCP infrastructure, particularly GKE and related storage, networking, and observability services.
• Experiment tracking, model evaluation, artifact registries, or production model monitoring.
• Building internal platforms for scientific and engineering users.

How we work

We treat an enterprise ML capability as more than a trained model. It includes the data and evaluation evidence behind the model; the infrastructure used to train it; the artifact and release process; the runtime and hardware on which it operates; and the telemetry, safeguards, and feedback loops required to operate and improve it.

We favor practical, incremental improvements over unnecessary platform expansion. We build shared capabilities where they remove recurring friction, improve reliability, or create meaningful leverage while preserving the flexibility scientists need to explore and iterate.

Success in this role means that scientists can move from an idea to a production-ready capability with less manual work and stronger evidence—and that deployed models become easier to understand, operate, and improve over time.