Case study — Dayforce
How Dayforce improved its AI agents
Dayforce brought in Clarity by Epistemic Me “to provide much needed expertise for building and improving our AI agents.” Lynn Chyi of Dayforce says human expert alignment scores for the agents improved by nearly 60%.
This page uses the words of the Dayforce team. Lynn Chyi, AI Experience Architect at Dayforce, tells the story in the video below.
Standing up agents is easy. Trusting them is hard.
Lynn Chyi says “getting them beyond a demo and ready for the scale that we needed was not that straightforward.” The Dayforce agents had to do complex work. They needed to “read documents and extract specialized domain knowledge in a way that makes sense to the business, both with high recall and high precision.”
The goal, in Lynn Chyi's words: not to compromise on the accuracy of a human, and to “reduce the number of hours it took for them to do it by about half or more.” That is the goal Dayforce set. It is not a result.
This meant the agents needed to work with judgment close to that of a human expert.
Building, measuring and improving the agents
Lynn Chyi says Dayforce brought Clarity in to “tell us how agents are measured and evaluated and how to improve them.”
For the general method Clarity uses in evals consulting, read how Clarity approaches AI agent evals. That page describes the method in general. It does not list the steps used at Dayforce.
Human expert alignment scores, up
improvement in human expert alignment scores
Before
- ×building agents just by guessing its system prompts and skills and tools with somewhat inconsistent results
After
- ✓deploying reliable agents that delivered real business value
Both quotes are from Lynn Chyi, Dayforce.
What the Dayforce team says
Becoming AI native
Justin Li, Sr. Product Manager at Dayforce, says Dayforce needed “the right framework in place to help our product team work more efficiently and more at scale.”
“Clarity helped us achieve that by improving our human expert alignment scores by nearly 60%.”
Read the full transcript
Hi, my name is Lynn, and I'm an AI experience architect here at Dayforce. Our team brought in Clarity to provide much needed expertise for building and improving our AI agents. As you know, standing up agents, well, that's the easy part. And getting them beyond a demo and ready for the scale that we needed was not that straightforward. So specifically, we brought them in to tell us how agents are measured and evaluated and how to improve them. And as anyone who has ever built agents before, you know, this is not that trivial. The agents that we were building had to do really complex work. Now, these are agents that need to read documents and extract specialized domain knowledge in a way that makes sense to the business, both with high recall and high precision. And we didn't want to compromise on the accuracy we'd get from a human, but we wanted to reduce the number of hours it took for them to do it by about half or more. So this meant that we needed agents to operate with judgment nearly indistinguishable from a human expert. And Clarity helped us achieve that by improving our human expert alignment scores by nearly 60%. So we went from building agents just by guessing its system prompts and skills and tools with somewhat inconsistent results to deploying reliable agents that delivered real business value. So, look, I can't understate how valuable it is to have someone like Clarity and bring them into your team to provide the industry's best practices on how to ship agents that you can really measure and trust.
“his ability to help us navigate this interesting transition period of understanding what it takes to become AI native has been extremely helpful.”
“Robert has been an incredible resource as we build out our AI and Agentic Product strategy.”
Want agents you can measure and trust?
Clarity by Epistemic Me offers evals consulting and AI-native services. Read how evals consulting works, or scope an agent engagement with us.