“You cannot use old maps to explore a new world.” — Albert Einstein
“AI quality is fundamentally a design problem.” This was my conclusion at our design deep dive on our AI-enabled software development lifecycle (SDLC).
We hear every day about how organizations are setting up AI SDLCs with new tools, agents, and automated workflows. What gets discussed less often is how to improve the quality and consistency of AI output as these practices scale across a team.
My reflection on the last 12 months of applying an AI SDLC is that many product design skills we use to design for people can also help us design better systems for AI agents. Empathy helps us anticipate what people need and where they may struggle. Information architecture helps us organize complex knowledge so it can be understood and applied. Interaction design helps us determine when the system should proceed, ask a question, explain an assumption, or involve a person. Together, these skills can help agents perform more reliably and consistently.
I plan to share our learnings and progress every month. This first article is about setting up the floor: treating AI quality as a design problem and building the foundation for it.
Moving beyond vibe coding
Vibe coding and prompt experimentation were fun ways to get started. They helped us discover what AI could do and made the potential of AI-assisted design tangible. The limitation we found is that these workflows are not easy to scale across a team. Each person provided different context, used different prompts, and made different assumptions.
We realized that we needed shared infrastructure that could give agents the right knowledge, guide their reasoning, and involve people at the right moments. We started building this foundation incrementally, learning through trial and error and celebrating small successes along the way.
An aha moment
While my team was already making steady progress, hearing other UX leaders’ perspectives was invaluable to me. At the Shift UX 2026 conference, a talk about architecting organizational understanding helped me articulate what our team had been building. Our work could be understood through three familiar design concepts::
- Information architecture, or the knowledge foundation: What should the agent be able to know?
- Guided inquiry, or the reasoning structure: What should the agent figure out before proceeding?
- Conversation design, or the human interaction structure: When and how should the system involve a person?
Although these concepts are not new, applying them in practice leads to tangible progress. We thoughtfully incorporated them into the shared repositories that connect to Claude Code, so that design questions are embedded throughout our context, skills, workflows, and quality guardrails.
Information architecture: what the agent should know
How do we structure layers of key information for agents? What is the agent’s entry point, and how should we route it to relevant knowledge? To answer these questions, we audited existing documents and made implicit knowledge explicit. We drafted and structured Markdown files covering our company and product context alongside our key accessibility goals.
Guided inquiry: what the agent should figure out first
We designed the underlying decision model that governs the conversation with the Claude Code agent in the IDE. For each type of task, it defines what Claude must know, which facts it can retrieve from context and which must come from the user, which unknowns block progress, which assumptions are safe, and when there is enough information to proceed.
Conversation design: when to involve a person
Conversation design is the surface where the decision model appears to the people who request a prototype. Depending on the situation, the agent asks one focused question, offers a few relevant options, explains a detected conflict, or states a low-risk assumption and continues. Before a consequential action, it pauses for approval.
Bringing it together in the repository
We then translated this model into the repository. CLAUDE.md acts as a thin, persistent router that identifies the task and loads the relevant skill. Each SKILL.md defines what Claude Code needs to determine for that particular job: the required knowledge, guided inquiry, conversation rules, and validation.
Applying it to prototyping
Another pattern we observed was that everyone, including PMs and engineers, started creating their own prototypes. While prototyping should be open to everyone, we started categorizing prototypes by type and intention to manage the situation and optimize token usage. We classify prototypes into three types:
- Ideation prototypes explore possibilities and help teams select a direction.
- Validation prototypes test assumptions and produce evidence.
- Reference prototypes express the intended experience and preserve design intent as work moves toward implementation.
Each prototype type requires different instructions for fidelity, design system alignment, accessibility, and human involvement. The agent should not treat every prototype as if it serves the same purpose. This is why we embedded a conversation design pattern for when the agent is asked to create a prototype without a specification. Since then, we have observed that more prototypes are made with clear intention. This pattern also might help teams move from prototype to specification to development with the necessary details and evidence.
Where we are going next
We have an ambitious goal to reduce the time to create a validation prototype from roughly one week to one day while improving interactivity and consistency. This accelerates creation, not judgment. Testing, validation, accessibility review, and decision-making still follow, based on the prototype’s purpose and the risk of the decision.
Our next step is implementing an AI heuristic review that provides a quick evaluation and surfaces what needs to be tested through thorough evaluative research. We are also looking for ways to connect customer and market signals to the ideas that feed prototyping. Our aim is to help intent travel more clearly from an idea, through a prototype, into validation and implementation.
Setting up the floor
All of this led me to a broader conclusion: AI quality does not come from the model alone. It emerges from the system we design around it. We are no longer designing only the experience presented to users. We are also designing the knowledge, instructions, decision points, and guardrails that help AI produce that experience with greater quality and consistency.
When we proudly presented our progress to product and technology leaders, the first question was: “Where is design craft going? Are we delegating everything to AI?” My answer is that this workflow actually helps us bring more design craft, taste, and customer-centric thinking to the experience. This workflow sets up the floor. With a solid floor, we can raise the ceiling.
What a higher ceiling looks like, and where design craft is going, is something we will explore over the coming months.
