Principal AI Engineer - Agentic Systems

1w1 week ago

TMS, LLC

Irvine, US · Full-time · $230,000 – $290,000

About this role

This Principal AI Engineer role focuses on Agentic Systems and LLM Platforms in a 6+ month hybrid position. Build advanced AI solutions using Python and production LLM applications including tool calling, structured outputs, and RAG. Expertise in agentic systems and multi-agent workflows is required.

Day-to-day work involves designing agentic AI systems with planner-executor and multi-agent orchestration. Develop RAG pipelines for enterprise data like policies, rules, and documents. Integrate LLMs with ML models for fraud, risk, and decisioning systems.

Implement AI safety and guardrails addressing PII, prompt injection, auditability, and HITL. Drive LLMOps/MLOps including CI/CD, monitoring, telemetry, and evaluation pipelines. Leverage distributed systems, APIs like REST, RPC, Kafka, SQS, Pub/Sub, and vector databases.

Collaborate with cross-functional teams in a cloud environment using AWS, Azure, GCP, Docker, and Kubernetes. Mentor engineers while applying data engineering fundamentals like SQL and data modeling. Advance LLM evaluation through A/B testing, adversarial testing, and eval frameworks.

Requirements

Strong programming in Python (Java/Go/TypeScript is a plus)
Hands-on experience with LLM applications in production (tool calling, structured outputs, RAG, evaluation)
Experience with agentic systems / multi-agent workflows
Strong in distributed systems & APIs (REST, RPC, Kafka, SQS, Pub/Sub)
Knowledge of vector databases, hybrid search, knowledge graphs
Cloud experience (AWS/Azure/GCP) + Docker, Kubernetes
Solid data engineering fundamentals (SQL, data modeling)
Experience with LLM evaluation (A/B testing, adversarial testing, eval frameworks)

Responsibilities

Design and build agentic AI systems (planner-executor, multi-agent orchestration)
Develop RAG pipelines for enterprise data (policies, rules, documents)
Integrate LLMs with ML models (fraud, risk, decisioning systems)
Implement AI safety & guardrails (PII, prompt injection, auditability, HITL)
Drive LLMOps/MLOps (CI/CD, monitoring, telemetry, evaluation pipelines)
Collaborate with cross-functional teams and mentor engineers