Discover · microsoft.com

Introducing the Agent Experience (AX) Practitioner Playbook

Microsoft for Developers microsoft.com

The AX Practitioner Playbook is our method for evaluating what AI coding agents do with your technology, diagnosing why they get it wrong, and fixing it at the source. It puts everything we’ve learned from hundreds of agent sessions in one place. Download it at aka.ms/ax-playbook.

Ask a coding agent to build something with the technology you work on, and watch what happens. It generates code that looks reasonable and might even compile. Look closer and you’ll find the wrong SDK version, a deprecated authentication pattern, or a setup nobody on the product team would recommend. The agent didn’t make a random mistake. It did exactly what its training data told it to do.

More ways to use this
Open on another device
Scan this QR code to open the same public page

Scan to open this page. Your saved notes and account access are not included.

Download QR code

Open the page link

Names and references

  1. Experience — person

In the piece

From a series of articles to one methodSince fall 2025, we in DevRel at Microsoft have measured how agents perform with Azure, Cosmos DB, SharePoint Framework (SPFx), and Microsoft 365 Copilot extensions, using the same kinds of prompts real developers use. We’ve shared what we learned piece by…OneDriveWhat you get from itThe playbook follows an evaluation from setup to shipped fix.OneDriveResults you can defendAn evaluation can lie to you convincingly. We’ve seen perfect scores for code that never compiled, and a “used Platform X” check that passed whether or not the agent used Platform X. The playbook’s evaluation model sets out what a result needs before…OneDriveCriteria that measure what mattersWriting criteria is where your domain expertise becomes measurable, and it’s the hardest step. The playbook shows how to write criteria a judge rules on the same way every time, calibrate them before you trust them, and version them when your product…OneDriveA shorter path from symptom to causeA readout tells you what failed while the trajectory tells you why. You’ll learn to tell apart an extension that never loaded, one that loaded but was never called, and one that was called but applied wrong, because each needs a different fix. Nine failure…OneDriveFixes that shipAn evaluation has no impact until someone changes the source of the behavior. The playbook covers how to test each proposed change as a hypothesis before you ship it, and how to bring evidence to the owning team, whether that’s your own team, a team you work…OneDriveExamples from real evaluationsEvery step in the playbook is grounded in scenarios we ran on Cosmos DB, SPFx, and other technologies. Those evaluations led to dozens of shipped fixes to docs and agent extensions, including 46 improvements to the Azure Cosmos DB Agent Kit. The SPFx project…OneDriveWho it’s forOneDrive

Named in it

Keep going

Each of these is read from what this estate already kept: the feed a piece came through, the publisher behind it, the credits and topics the publisher printed. None of it is a recommendation, an identity or a record kept anywhere else.

Also from Microsoft for Developers

  1. Do AI coding agents care about SemVer?Waldek Mastykarz microsoft.com
  2. Your best AX fixes are hiding in agent trajectoriesWaldek Mastykarz microsoft.com
  3. Does ‘eval’ in the workspace path change what your agent does?Waldek Mastykarz microsoft.com
  4. Publish Visual Studio Marketplace extensions securely with Trusted PublishingLynn Dai microsoft.com
  5. PostgreSQL Like A Pro: Build, query, and troubleshoot without leaving VS CodeCynthia Zanoni microsoft.com
  6. Can a local git remote replace GitHub in your agent eval?Waldek Mastykarz microsoft.com