Discover · microsoft.com

Your best AX fixes are hiding in agent trajectories

Microsoft for Developers microsoft.com

An eval score tells you whether the agent finished the task. To improve how agents work with your product, you need to see where they struggled, and that’s in the trajectory, the record of every step the agent took and why. Agents are persistent. They guess another URL after a 404 and try values until an error goes away, so the struggle rarely shows up in the score.

We saw this when we evaluated Dev Proxy, a local proxy that intercepts your app’s HTTP requests. We asked GitHub Copilot Chat with Claude Sonnet 5.5 and no extensions, five times, to write a CI script that uses the Dev Proxy API to record which GitHub APIs a Node.js tool calls. All five scripts worked. Yet the trajectories showed 15 problems, and each run took 29 to 39 requests and up to 1.2 million tokens.

More ways to use this
Open on another device
Scan this QR code to open the same public page

Scan to open this page. Your saved notes and account access are not included.

Download QR code

Open the page link

In the piece

Named in it

Keep going

Each of these is read from what this estate already kept: the feed a piece came through, the publisher behind it, the credits and topics the publisher printed. None of it is a recommendation, an identity or a record kept anywhere else.

More like this

Also from Microsoft for Developers

  1. Do AI coding agents care about SemVer?Waldek Mastykarz microsoft.com
  2. Does ‘eval’ in the workspace path change what your agent does?Waldek Mastykarz microsoft.com
  3. Publish Visual Studio Marketplace extensions securely with Trusted PublishingLynn Dai microsoft.com
  4. PostgreSQL Like A Pro: Build, query, and troubleshoot without leaving VS CodeCynthia Zanoni microsoft.com
  5. Can a local git remote replace GitHub in your agent eval?Waldek Mastykarz microsoft.com
  6. Introducing the Agent Experience (AX) Practitioner PlaybookWaldek Mastykarz microsoft.com