Senior Software Engineer, Agent Infrastructure

Cohere is a leading enterprise AI company building frontier foundation models and end-to-end products that solve real business problems. The company works with enterprises deploying AI systems at scale, with a global presence including offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul. Cohere's mission centers on making AI widely adoptable through cutting-edge research and engineering.

This Senior Software Engineer role sits within the Agent Infrastructure team, which is designing the platform that powers autonomous AI agents at scale. You'll tackle forward-looking infrastructure challenges at the intersection of machine learning systems, distributed infrastructure, and developer experience, working on problems including secure code execution, agent state management, model routing, identity and authentication, and resource management for long-running workflows.

What you'll do

• Design and build secure execution environments where agents can run generated code safely

• Develop identity, authentication, and trust boundary systems for agent interactions

• Create model routing and orchestration layers that work across different model types and deployment environments

• Implement rate limiting, quota systems, and resource management for agent workflows

• Build state management, memory, and filesystem abstractions that agents depend on

• Convert emerging ML research concepts into production-ready infrastructure components

• Establish core platform capabilities for execution, storage, and state operations

• Prototype and evaluate new technologies, then help determine what moves into production

• Collaborate with research teams to ensure infrastructure meets the needs of future agent systems

What you'll bring

• Production experience building ML infrastructure with solid systems fundamentals

• Hands-on experience working with agentic systems, multi-agent workflows, or agent frameworks

• Knowledge of model routing and LLM provider frameworks across different model types

• Experience designing scalable, fault-tolerant distributed systems and working with Kubernetes

• Demonstrated ability to move quickly on prototypes and make sound productionization decisions

Nice to have

• Experience across on-premises, private cloud, and public cloud environments

• Knowledge of storage systems, embedded databases, or filesystem abstractions

• Familiarity with code execution sandboxes like gVisor, Firecracker, Kata, or WASM runtimes

• Open-source contributions to LLM or agent infrastructure projects

• Background in identity systems, workload authentication, or capability-based security

What they offer

• Weekly lunch stipend of $75 or local currency equivalent

• Comprehensive health and dental coverage with separate mental health budget

• Retirement benefits including RRSP matching, 401K, or Pension Scheme

• Full parental leave top-up for six months for either parent

• Six weeks of paid vacation annually plus annual enrichment budget for arts, culture, fitness, education and workspace improvements

• $500 home office setup stipend plus co-working benefits if remote

• Travel budget for visiting other offices and annual company offsite