Staff / Principal Machine Learning Engineer, Serving - USA
Location
Mountain View, California, USA
Employment Type
Full time
Location Type
Hybrid
Department
ML Engineering
Overview
Application
About Inworld
Inworld is a research lab of top researchers and engineers, building the world’s top-ranked realtime voice models.
Today our models are the #1 ranked realtime voice models in the world. They are used to power the largest consumer-facing AI applications available, across categories like health, fitness, learning, therapy, companions, customer experience and media; representing 100s of millions of end users. Our work spans areas like research and development of state-of-the-art models, optimizing realtime inference, and creating best-in-class APIs and products that allow developers to engage their users.
We’ve raised more than $125M from Lightspeed, Section 32, Kleiner Perkins, Microsoft’s M12 venture fund, Founders Fund, Meta and Stanford, among others. Our technology has powered experiences from companies such as NVIDIA, Microsoft Xbox, Niantic, Logitech Streamlabs, Wishroll, Little Umbrella and Bible Chat. We’ve also been recognized by CB Insights as one of the 100 most promising AI companies globally and have been named one of LinkedIn’s Top 10 Startups in the USA.
Who We're Looking For
A year ago, reliably working agentic systems and sub-second multimodal inference at scale barely existed. Nobody has a decade of experience here. So we're not screening for a resume template — we're looking for strong people from varied backgrounds who learn fast, thrive in ambiguity, and can show us what they've built, broken, and understood.
Experience We Find Useful
You don't need all of this. But you need enough to make a case.
Inference Optimization. Deep understanding of modern serving frameworks and techniques like vLLM or TRT-LLM.
Model Acceleration. Hands-on experience with quantization, distillation, caching strategies , continuous batching, paged attention, and speculative decoding.
High-Performance Systems. Proficiency in C++, CUDA, Rust, or highly optimized Python. You know how to profile code and squeeze every ounce of performance out of NVIDIA GPUs.
Distributed Systems & Scaling. Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and reliably handling thousands of concurrent connections.
Public work. Non-trivial systems programming projects, open-source contributions to major inference engines, or deep-dive technical write-ups.
Full-cycle ownership. You can take a model from the research team, containerize it, optimize its serving, and ensure it runs reliably in production.
Background. PhD in CS, Physics, Math, or equivalent practical experience building backend or ML systems.
Who Thrives Here
You don’t need a roadmap to start walking; you’re comfortable picking a direction and building the map as you go.
You believe engineering isn't finished until it’s shipped and stable. You have a bias for impact over purely theoretical optimizations.
You don't just ship code; you obsess over the why. You’re the first to question an architecture if you think there’s a better way to solve the core latency or throughput problem.
You aren't satisfied with "the PM said so." You thrive on deep context and want to understand the fundamental logic behind every decision we make.
What Working Here Is Like
We hand you unclear problems and expect you to make them clear. We value engineers who say "I don't know yet" and then design the benchmark or prototype that finds out. We treat performance, latency, and reliability as first-class product features, not a box to check before launch. Impact comes before everything else, though we support sharing work and open-source contributions that move the field forward. Your work should be visible. Flat structure, fast iterations, minimal process theater.
We believe in the power of in-person collaboration to solve the hardest problems and foster a strong team culture. We offer relocation assistance and look forward to you joining us in our Mountain View office.
The base salary range for this full-time position is $270,000 - $500,000+ bonus + equity + benefits.
Inworld Jobs Privacy
Apply for this Job
Powered by
Privacy PolicySecurityVulnerability Disclosure
Listed by Inworld AI for a position based in the United States. Employers on this board attest they are hiring domestically.
Engineering
155 days ago
Staff / Principal Machine Learning Engineer, Serving - USA
About Inworld
Inworld is a research lab of top researchers and engineers, building the world’s top-ranked realtime voice models.
Today our models are the #1 ranked realtime voice models in the world. They are used to power the largest consumer-facing AI applications available, across categories like health, fitness, learning, therapy, companions, customer experience and media; representing 100s of millions of end users. Our work spans areas like research and development of state-of-the-art models, optimizing realtime inference, and creating best-in-class APIs and products that allow developers to engage their users.
We’ve raised more than $125M from Lightspeed, Section 32, Kleiner Perkins, Microsoft’s M12 venture fund, Founders Fund, Meta and Stanford, among others. Our technology has powered experiences from companies such as NVIDIA, Microsoft Xbox, Niantic, Logitech Streamlabs, Wishroll, Little Umbrella and Bible Chat. We’ve also been recognized by CB Insights as one of the 100 most promising AI companies globally and have been named one of LinkedIn’s Top 10 Startups in the USA.
Who We're Looking For
A year ago, reliably working agentic systems and sub-second multimodal inference at scale barely existed. Nobody has a decade of experience here. So we're not screening for a resume template — we're looking for strong people from varied backgrounds who learn fast, thrive in ambiguity, and can show us what they've built, broken, and understood.
Experience We Find Useful
You don't need all of this. But you need enough to make a case.
- Inference Optimization. Deep understanding of modern serving frameworks and techniques like vLLM or TRT-LLM.
- Model Acceleration. Hands-on experience with quantization, distillation, caching strategies , continuous batching, paged attention, and speculative decoding.
- High-Performance Systems. Proficiency in C++, CUDA, Rust, or highly optimized Python. You know how to profile code and squeeze every ounce of performance out of NVIDIA GPUs.
- Distributed Systems & Scaling. Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and reliably handling thousands of concurrent connections.
- Public work. Non-trivial systems programming projects, open-source contributions to major inference engines, or deep-dive technical write-ups.
- Full-cycle ownership. You can take a model from the research team, containerize it, optimize its serving, and ensure it runs reliably in production.
- Background. PhD in CS, Physics, Math, or equivalent practical experience building backend or ML systems.
Who Thrives Here
- You don’t need a roadmap to start walking; you’re comfortable picking a direction and building the map as you go.
- You believe engineering isn't finished until it’s shipped and stable. You have a bias for impact over purely theoretical optimizations.
- You don't just ship code; you obsess over the why. You’re the first to question an architecture if you think there’s a better way to solve the core latency or throughput problem.
- You aren't satisfied with "the PM said so." You thrive on deep context and want to understand the fundamental logic behind every decision we make.
What Working Here Is Like
We hand you unclear problems and expect you to make them clear. We value engineers who say "I don't know yet" and then design the benchmark or prototype that finds out. We treat performance, latency, and reliability as first-class product features, not a box to check before launch. Impact comes before everything else, though we support sharing work and open-source contributions that move the field forward. Your work should be visible. Flat structure, fast iterations, minimal process theater.
We believe in the power of in-person collaboration to solve the hardest problems and foster a strong team culture. We offer relocation assistance and look forward to you joining us in our Mountain View office.
The base salary range for this full-time position is $270,000 - $500,000+ bonus + equity + benefits.
Inworld Jobs Privacy https://inworld.ai/jobs-privacy
Listed by Inworld AI for a position based in the United States. Employers on this board attest they are hiring domestically.
Inworld is a research lab of top researchers and engineers, building the world’s top-ranked realtime voice models.
Today our models are the #1 ranked realtime voice models in the world. They are used to power the largest consumer-facing AI applications available, across categories like health, fitness, learning, therapy, companions, customer experience and media; representing 100s of millions of end users. Our work spans areas like research and development of state-of-the-art models, optimizing realtime inference, and creating best-in-class APIs and products that allow developers to engage their users.
We’ve raised more than $125M from Lightspeed, Section 32, Kleiner Perkins, Microsoft’s M12 venture fund, Founders Fund, Meta and Stanford, among others. Our technology has powered experiences from companies such as NVIDIA, Microsoft Xbox, Niantic, Logitech Streamlabs, Wishroll, Little Umbrella and Bible Chat. We’ve also been recognized by CB Insights as one of the 100 most promising AI companies globally and have been named one of LinkedIn’s Top 10 Startups in the USA.
About the role
Join our team as a Staff / Principal Platform Engineer and take end-to-end ownership of building, securing, and scaling our AI products. You'll be the driving force behind our cloud infrastructure, partnering with engineers across the organization to deploy and evolve services across major cloud providers using Terraform, ArgoCD, and other tooling. In this high-impact role, you'll identify what needs to be done and move it forward, directly shaping how we operate and innovate.
What you’ll do
Work closely with engineers to design, deploy, and maintain reliable, high-performance, and secure cloud infrastructure for our and .
Drive engineering velocity by identifying and building AI-powered tooling and workflows that improve how our teams develop and deploy software.
Facilitate a "you build it, you run it" culture by providing the necessary tools and processes for monitoring the reliability, availability, and performance of services.
Manage pipelines to ensure smooth and efficient code integration and deployment.
Conduct root cause analysis to identify critical issues and develop automated solutions to prevent recurrence.
Expected experience
8-10 years of experience in software engineering.
3+ years of experience with infrastructure-as-code.
Proficiency in managing Kubernetes clusters and applications, including creating Kustomize manifests/Helm charts for new applications.
Experience in creating and maintaining CI/CD pipelines for both applications and infrastructure deployments (using tools like Terraform/Terragrunt, ArgoCD, GitHub Actions, Ansible, etc.).
Deep knowledge of at least one major cloud provider (Google Cloud Platform, Microsoft Azure, Oracle Cloud).
Proficient in at least one backend programming/scripting languages such as Golang, Python, and Bash.
Candidates must be based in the SF Bay Area or willing to relocate (you will be working on-site in our South Bay office a few days a week).
The US base salary range for this full-time position is $280,000 - $350,000. In addition to base pay, total compensation includes equity and benefits. Within the range, individual pay is determined by work location, level, and additional factors, including competencies, experience, and business needs. The base pay range is subject to change and may be modified in the future.
About Inworld
Inworld is a research lab of top researchers and engineers, building the world’s top-ranked realtime voice models.
Today our models are the #1 ranked realtime voice models in the world. They are used to power the largest consumer-facing AI applications available, across categories like health, fitness, learning, therapy, companions, customer experience and media; representing 100s of millions of end users. Our work spans areas like research and development of state-of-the-art models, optimizing realtime inference, and creating best-in-class APIs and products that allow developers to engage their users.
We’ve raised more than $125M from Lightspeed, Section 32, Kleiner Perkins, Microsoft’s M12 venture fund, Founders Fund, Meta and Stanford, among others. Our technology has powered experiences from companies such as NVIDIA, Microsoft Xbox, Niantic, Logitech Streamlabs, Wishroll, Little Umbrella and Bible Chat. We’ve also been recognized by CB Insights as one of the 100 most promising AI companies globally and have been named one of LinkedIn’s Top 10 Startups in the USA.
About the role
Join our team as a Staff / Principal Platform Engineer and take end-to-end ownership of building, securing, and scaling our AI products. You'll be the driving force behind our cloud infrastructure, partnering with engineers across the organization to deploy and evolve services across major cloud providers using Terraform, ArgoCD, and other tooling. In this high-impact role, you'll identify what needs to be done and move it forward, directly shaping how we operate and innovate.
What you’ll do
- Work closely with engineers to design, deploy, and maintain reliable, high-performance, and secure cloud infrastructure for our TTS https://inworld.ai/tts and LLM Router https://inworld.ai/router.
- Drive engineering velocity by identifying and building AI-powered tooling and workflows that improve how our teams develop and deploy software.
- Facilitate a "you build it, you run it" culture by providing the necessary tools and processes for monitoring the reliability, availability, and performance of services.
- Manage pipelines to ensure smooth and efficient code integration and deployment.
- Conduct root cause analysis to identify critical issues and develop automated solutions to prevent recurrence.
Expected experience
- 8-10 years of experience in software engineering.
- 3+ years of experience with infrastructure-as-code.
- Proficiency in managing Kubernetes clusters and applications, including creating Kustomize manifests/Helm charts for new applications.
- Experience in creating and maintaining CI/CD pipelines for both applications and infrastructure deployments (using tools like Terraform/Terragrunt, ArgoCD, GitHub Actions, Ansible, etc.).
- Deep knowledge of at least one major cloud provider (Google Cloud Platform, Microsoft Azure, Oracle Cloud).
- Proficient in at least one backend programming/scripting languages such as Golang, Python, and Bash.
Candidates must be based in the SF Bay Area or willing to relocate (you will be working on-site in our South Bay office a few days a week).
The US base salary range for this full-time position is $280,000 - $350,000. In addition to base pay, total compensation includes equity and benefits. Within the range, individual pay is determined by work location, level, and additional factors, including competencies, experience, and business needs. The base pay range is subject to change and may be modified in the future.
Inworld Jobs Privacy https://inworld.ai/jobs-privacy
Listed by Inworld AI for a position based in the United States. Employers on this board attest they are hiring domestically.
Inworld is a research lab of top researchers and engineers, building the world’s top-ranked realtime voice models.
Today our models are the #1 ranked realtime voice models in the world. They are used to power the largest consumer-facing AI applications available, across categories like health, fitness, learning, therapy, companions, customer experience and media; representing 100s of millions of end users. Our work spans areas like research and development of state-of-the-art models, optimizing realtime inference, and creating best-in-class APIs and products that allow developers to engage their users.
We’ve raised more than $125M from Lightspeed, Section 32, Kleiner Perkins, Microsoft’s M12 venture fund, Founders Fund, Meta and Stanford, among others. Our technology has powered experiences from companies such as NVIDIA, Microsoft Xbox, Niantic, Logitech Streamlabs, Wishroll, Little Umbrella and Bible Chat. We’ve also been recognized by CB Insights as one of the 100 most promising AI companies globally and have been named one of LinkedIn’s Top 10 Startups in the USA.
About the role:
Inworld recently launched a few exciting new products (, , and) for consumer AI applications, and we're looking for an ambitious and capable Staff/Principal Backend Engineer to join us and help take the Inworld AI platform even farther. Here is what you are going to work on:
Inworld Router: an intelligent routing layer that gives developers a single API to access 200+ LLMs. You'll own core systems for multi-provider failover, cost/latency-based routing, live A/B experimentation, and real-time observability at massive scale.
Realtime API
API-based model services: Our custom TTS/STT models and API includes free instant voice cloning. Learn more and hear examples at. Better yet, sign up yourself at, try out the premade voices, clone your own voice in just a few seconds, and let us know what you think! Beyond TTS, there is also LLM, Knowledge/RAG, STT, and more.
New exciting products, ambitious and large-scale, in the lineup for the launch later this year.
Services for control and optimization. We're just getting started on these deeper capabilities.
Finally complicated and exciting Infrastructural projects: platformization of new product upcoming offering, development and integration of best development tools, projects like system-wide billing and so on.
As a Staff/Principal Software Engineer, you would be a significant part of one or more of these areas. The key challenges are:
Shipping quickly. AI is evolving weekly, so there's a ton of opportunity to be had. We want to move fast to capture those opportunities while they are still fresh and full of potential.
Zero to one. The platform is not a simple copycat. We have a vision for a deep platform/suite of capabilities that make it dramatically simpler for developers to scale and evolve their AI.
Realtime, online. As consumer applications become more capable of listening and talking, performance will matter, and AI has to adapt in realtime as well. These are bold but exciting challenges.
Multi-provider complexity at scale. Inworld Router must intelligently route across hundreds of models and providers while handling failover, sticky sessions, cost optimization, and conditional logic, all with minimal latency overhead. You'll design systems where every millisecond and every routing decision matters.
Finally, almost everything here is a collaboration with our sibling ML teams, since ML and AI are critical to providing the learning and adaptability central to this vision.
Please note: This is an IC-focused role. We are looking for someone who loves direct technical contribution alongside very capable peers.
What you’ll do:
Establish significant scope: Collaborate with the PMs, engineers and leads to determine the biggest product needs to focus on now.
Operate with technical autonomy: You have considerable leeway to suggest how to address a given focus area, including bringing in new technical dependencies or standards where it's the best choice.
Collaborate, execute, deliver: This is the core of the building loop. We aim to optimize for both speed and quality, despite it being decidedly non-obvious how to manage that tradeoff exceptionally well.
Reflect and drive improvements: Especially as a Staff Engineer, advocate for and realize system improvements, both related to and independent of key features.
Expected experience:
Must Haves
Excellent programming skills and experience in a statically typed backend programming language, preferably Go, Python, C++ or Rust
Experience developing and deploying cloud-based services to at least hundreds of qps (preferably more)
Experience with relational databases (PostgreSQL or MySQL)
Hands-on experience with caching (Redis or Memcached), pubsub/queues, data pipelines (Flink, Beam), and Cloud storage
Excellent verbal and written communication skills, can collaborate and coordinate with other roles and engineer with ease, trusted and well-regarded teammate
Bonus Qualifications
Experience building API gateways, routing/proxy layers, or multi-provider orchestration systems
Experience with analytics or timeseries databases (ClickHouse, Timescale, InfluxDB)
Experience with OpenTelemetry
Experience with C++
Candidates must be based in the SF Bay Area or willing to relocate (you will be working on-site in our South Bay office a few days a week).
The US base salary range for this full-time position is $280,000 - $350,000. In addition to base pay, total compensation includes equity and benefits. Within the range, individual pay is determined by work location, level, and additional factors, including competencies, experience, and business needs. The base pay range is subject to change and may be modified in the future.
About Inworld
Inworld is a research lab of top researchers and engineers, building the world’s top-ranked realtime voice models.
Today our models are the #1 ranked realtime voice models in the world. They are used to power the largest consumer-facing AI applications available, across categories like health, fitness, learning, therapy, companions, customer experience and media; representing 100s of millions of end users. Our work spans areas like research and development of state-of-the-art models, optimizing realtime inference, and creating best-in-class APIs and products that allow developers to engage their users.
We’ve raised more than $125M from Lightspeed, Section 32, Kleiner Perkins, Microsoft’s M12 venture fund, Founders Fund, Meta and Stanford, among others. Our technology has powered experiences from companies such as NVIDIA, Microsoft Xbox, Niantic, Logitech Streamlabs, Wishroll, Little Umbrella and Bible Chat. We’ve also been recognized by CB Insights as one of the 100 most promising AI companies globally and have been named one of LinkedIn’s Top 10 Startups in the USA.
ABOUT THE ROLE:
Inworld recently launched a few exciting new products (Inworld TTS https://inworld.ai/tts, Inworld STT https://inworld.ai/speech-to-text, Speech-to-Speech / Realtime API https://inworld.ai/realtime-api and Inworld Router https://inworld.ai/router) for consumer AI applications, and we're looking for an ambitious and capable Staff/Principal Backend Engineer to join us and help take the Inworld AI platform even farther. Here is what you are going to work on:
- Inworld Router: an intelligent routing layer that gives developers a single API to access 200+ LLMs. You'll own core systems for multi-provider failover, cost/latency-based routing, live A/B experimentation, and real-time observability at massive scale.
- Realtime API
- API-based model services: Our custom TTS/STT models and API includes free instant voice cloning. Learn more and hear examples at https://inworld.ai/ttsinworld.ai/tts http://inworld.ai/tts. Better yet, sign up yourself at https://platform.inworld.ai/platform.inworld.ai http://platform.inworld.ai, try out the premade voices, clone your own voice in just a few seconds, and let us know what you think! Beyond TTS, there is also LLM, Knowledge/RAG, STT, and more.
- New exciting products, ambitious and large-scale, in the lineup for the launch later this year.
- Services for control and optimization. We're just getting started on these deeper capabilities.
- Finally complicated and exciting Infrastructural projects: platformization of new product upcoming offering, development and integration of best development tools, projects like system-wide billing and so on.
As a Staff/Principal Software Engineer, you would be a significant part of one or more of these areas. The key challenges are:
- Shipping quickly. AI is evolving weekly, so there's a ton of opportunity to be had. We want to move fast to capture those opportunities while they are still fresh and full of potential.
- Zero to one. The platform is not a simple copycat. We have a vision for a deep platform/suite of capabilities that make it dramatically simpler for developers to scale and evolve their AI.
- Realtime, online. As consumer applications become more capable of listening and talking, performance will matter, and AI has to adapt in realtime as well. These are bold but exciting challenges.
- Multi-provider complexity at scale. Inworld Router must intelligently route across hundreds of models and providers while handling failover, sticky sessions, cost optimization, and conditional logic, all with minimal latency overhead. You'll design systems where every millisecond and every routing decision matters.
Finally, almost everything here is a collaboration with our sibling ML teams, since ML and AI are critical to providing the learning and adaptability central to this vision.
Please note: This is an IC-focused role. We are looking for someone who loves direct technical contribution alongside very capable peers.
WHAT YOU’LL DO:
- Establish significant scope: Collaborate with the PMs, engineers and leads to determine the biggest product needs to focus on now.
- Operate with technical autonomy: You have considerable leeway to suggest how to address a given focus area, including bringing in new technical dependencies or standards where it's the best choice.
- Collaborate, execute, deliver: This is the core of the building loop. We aim to optimize for both speed and quality, despite it being decidedly non-obvious how to manage that tradeoff exceptionally well.
- Reflect and drive improvements: Especially as a Staff Engineer, advocate for and realize system improvements, both related to and independent of key features.
EXPECTED EXPERIENCE:
Must Haves
- Excellent programming skills and experience in a statically typed backend programming language, preferably Go, Python, C++ or Rust
- Experience developing and deploying cloud-based services to at least hundreds of qps (preferably more)
- Experience with relational databases (PostgreSQL or MySQL)
- Hands-on experience with caching (Redis or Memcached), pubsub/queues, data pipelines (Flink, Beam), and Cloud storage
- Excellent verbal and written communication skills, can collaborate and coordinate with other roles and engineer with ease, trusted and well-regarded teammate
Bonus Qualifications
- Experience building API gateways, routing/proxy layers, or multi-provider orchestration systems
- Experience with analytics or timeseries databases (ClickHouse, Timescale, InfluxDB)
- Experience with OpenTelemetry
- Experience with C++
Candidates must be based in the SF Bay Area or willing to relocate (you will be working on-site in our South Bay office a few days a week).
The US base salary range for this full-time position is $280,000 - $350,000. In addition to base pay, total compensation includes equity and benefits. Within the range, individual pay is determined by work location, level, and additional factors, including competencies, experience, and business needs. The base pay range is subject to change and may be modified in the future.
Inworld Jobs Privacy https://inworld.ai/jobs-privacy
Listed by Inworld AI for a position based in the United States. Employers on this board attest they are hiring domestically.
Select a role
The full posting opens here — pay, setting and the full description, without leaving the list.