Together AI runs a cloud built for AI engineers, offering high-performance inference, fine-tuning and reinforcement learning, and large-scale pre-training around a marketplace of open models. The company is hiring an AI Infrastructure System Engineer in Bangalore to build and operate one of the largest GPU fleets used for frontier model training and inference. This is a software-first infrastructure role for engineers who would rather write code than click through dashboards.
What you will build
You will design and build fleet automation systems that provision, validate, deploy, upgrade, repair, and retire GPU clusters with minimal human intervention. You will build AI infrastructure agents that automate deployment, trace failures to their root cause, triage incidents, and remediate problems on their own.
You will develop fleet intelligence platforms that continuously monitor hardware health, firmware, networking, storage, thermals, and workload performance, so failures can be predicted before they reach customers. You will build software that pushes GPU availability, utilization, performance, and reliability higher across thousands of accelerators.
You will create automated validation systems for GPUs, InfiniBand and RoCE fabrics, NVLink and NVSwitch, storage, and distributed AI workloads. You will build internal platforms and developer tools so infrastructure is managed through software rather than manual operations, and you will keep improving deployment velocity, reliability, and operational efficiency through automation, working closely with hardware, networking, platform, and AI teams.
What they are looking for
- 3 or more years building distributed systems, infrastructure platforms, or large-scale backend software.
- Strong software engineering skills in Python, Go, or Rust.
- Experience building platforms, automation systems, or developer infrastructure.
- Experience with Linux, Kubernetes, Terraform, Ansible, or similar infrastructure technologies.
- Strong systems thinking, with the ability to follow problems across hardware and software.
- A passion for solving complex infrastructure challenges through software, and an automation-first mindset where a repeated task is a prompt to build a system that removes it.
Bonus experience
- GPU infrastructure, CUDA, NCCL, and NVLink and NVSwitch.
- InfiniBand or RoCE networking, and bare-metal provisioning and lifecycle management.
- Large-scale AI training or inference clusters.
- Hardware health monitoring and predictive failure detection.
- Distributed storage systems, and AI agents and autonomous infrastructure operations.
