AI/ML Evaluation Engineer

AI/ML Evaluation EngineerThe Opportunity:

As an experienced engineer, you know that machine learning (ML) and AI evaluation are critical to understanding and operationalizing massive datasets in support of public health and safety missions. Your ability to evaluate, optimize, and deploy AI-driven systems makes you an integral part of delivering mission-focused solutions for scientists, analysts, and leadership teams.

In this role, you’ll help define and implement scientific AI evaluation and enablement initiatives by translating advanced AI capabilities into practical, mission-specific workflows. You’ll collaborate with a large community of ML engineers, data scientists, architects, and product teams to design scalable ML and Generative AI solutions, including AI agents, retrieval-augmented generation (RAG) pipelines, and enterprise AI evaluation frameworks.

You’ll apply technical expertise across AI evaluation, retrieval optimization, memory and state management, and AI agent architecture to support real-time insights and decision-making. Your work will also contribute to enterprise MLOps capabilities, data governance standards, and ethical AI practices within regulated public health environments. This position is located in Atlanta, GA.

What You'll Work On

• Build and maintain scalable data pipelines using PySpark and Palantir Foundry to support AI, analytics, and scientific evaluation workflows.
• Design and implement ML and Generative AI workflows, including AI agents, RAG pipelines, and AI evaluation frameworks.
• Integrate advanced AI technologies such as Codex and Claude, to support mission-specific workflows, real-time insights, and decision-making capabilities.
• Develop evaluation strategies covering model quality, retrieval optimization, benchmarking, and performance monitoring for deployed AI systems.
• Design AI architectures supporting memory, state management, structured and unstructured public health data interaction, and sc...