Imagine a familiar conversation in a product team.
“We need to understand whether customers will find this idea useful.”
“Let's organize a focus group.”
It sounds reasonable. The term focus group itself suggests that gathering people in a room will help us focus on the right problem. But something interesting happens when people come together to evaluate an idea.
The most vocal participants begin shaping the conversation. Others adjust their opinions based on what they've heard. Some thoughtful participants remain quiet.
We may walk away with a consensus, but consensus isn't necessarily evidence of how individuals will behave when using a product.
After all, products are experienced by individuals, even when those individuals work within larger teams and organizations.
And now that AI is changing how quickly we can create working prototypes, this distinction matters more than ever.
From expensive prototypes to inexpensive experiments
For years, building a working prototype required meaningful investment. Product teams researched an opportunity, designers developed concepts, engineers evaluated feasibility, and stakeholders debated which ideas deserved investment. A prototype often represented weeks of effort.
AI is changing that equation.
Today, we can use AI-assisted design and development tools to explore multiple directions, generate functional interfaces, simulate interactions, and create prototypes in a fraction of the time previously required. At Intelligaia, we're exploring how these capabilities can help teams move from abstract discussions to tangible experiences earlier in the product development process.
- Instead of asking customers to imagine how a proposed AI assistant might work, we can put a prototype in front of them.
- Instead of describing an agentic workflow in a presentation, we can let someone interact with a simulated version of it.
- And instead of debating five possible approaches in a conference room, we can create several alternatives and evaluate the assumptions behind them.
This creates an important business opportunity.
AI doesn't just reduce the cost of building prototypes. It can reduce the cost of discovering that we're wrong.
But only if we use those prototypes to learn.
The problem with testing everything
When experiments become inexpensive to create, it's tempting to run more of them. More concepts. More variations. More interfaces. More features.
But there's a hidden cost.
Consider a sales representative using an enterprise application every day.
- On Monday, they're introduced to a new AI assistant.
- On Wednesday, they're invited to evaluate a redesigned opportunity dashboard.
- On Friday, they're testing an autonomous agent that recommends next actions.
Each experiment might be valuable on its own. But together, they create an environment in which the user is constantly adapting to something new.
Every change demands attention. Every unfamiliar interaction introduces uncertainty. Every interrupted workflow has a cost. Over time, even well-intentioned experimentation can damage the relationship between customers and the product.
The cost of creating an experiment may approach zero. The cost of interrupting someone's work never does.
This changes the challenge for product leaders.
The question is no longer simply, “How many ideas can we test?”
It becomes, “Which ideas deserve our customers' attention?”
Observe what individuals do. Understand what they expected.
There's another opportunity that faster prototyping creates. We can go beyond asking whether someone likes an idea. We can observe how they interact with it and explore the thinking behind their behavior.
Imagine someone testing an AI-powered customer health dashboard. They receive an AI-generated recommendation to escalate an account. They understand the recommendation. They know where to click. They successfully complete the task.
From a conventional usability perspective, the interaction might appear successful.
- But what if the user doesn't trust the recommendation?
- What if they follow it only because the interface makes it seem authoritative?
- What if they would never take the same action in their actual working environment?
Task completion alone doesn't reveal these concerns.
This is where qualitative techniques such as sentence completion can help.
Ask the participant to finish statements such as:
- “I would trust this recommendation more if…”
- “What I expected to happen next was…”
- “I would hesitate to use this when…”
- “I would rather make this decision myself if…”
These prompts can reveal expectations, uncertainties, and mental models that may not be apparent through observation alone. They aren't a replacement for observing real behavior. What people say and what they do can differ. But together, behavioral observation and thoughtful questioning create a richer understanding of the experience.
This becomes especially important for AI products.
A user can understand an interface without trusting the system behind it.
They can appreciate an AI-generated recommendation without wanting to delegate the decision.
And they can complete a task successfully while still feeling uncertain about the outcome.
For AI experiences, we need to understand not only whether the interaction works, but whether the user knows what the system is doing, why it's doing it, and when human judgment is required.
The new experimentation loop: Observe, Understand, Adapt, Validate
Traditionally, usability testing often involved preparing a prototype, conducting research sessions, analyzing findings, and returning to the design team for revisions.
That process remains valuable.
But AI-assisted prototyping creates an opportunity to shorten the distance between an observation and the next experiment.
We can think about this as four connected activities.
1. Observe behavior
Put a realistic prototype in front of an individual. Give them a meaningful task and observe what happens.
Where do they hesitate? What do they ignore? What do they attempt that the design didn't anticipate?
2. Understand expectations
Explore why the participant behaved that way.
What did they expect? What information was missing? What made them uncertain? What would have helped them make a confident decision?
3. Adapt the experience
Use what we've learned to create alternative interactions.
Perhaps the AI recommendation needs supporting evidence. Perhaps the user needs an approval step. Perhaps the system should offer assistance rather than take action automatically.
4. Validate the alternative
Test whether the revised interaction actually improves the experience.
Whenever possible, validate with additional participants rather than relying exclusively on someone who has already seen the earlier version.
The objective isn't to demonstrate how quickly we can modify a prototype.
It's to discover whether a change improves task performance, comprehension, confidence, or another meaningful outcome.
The speed that matters is the time between an assumption and reliable evidence.
Not every experiment belongs in a customer's workflow
As the volume of potential experiments increases, organizations need a more deliberate system for moving ideas toward production.
One useful approach is to separate experimentation into five stages.
Internal exploration
Explore without disrupting customers.
Individual validation
Observe real behaviour.
Controlled pilots
Test with consenting users.
Workflow validation
Measure outcomes and unintended effects.
Governed rollout
Expand with oversight and rollback.
Stage 1: Internal exploration
Use AI to create prototypes and evaluate different approaches without exposing customers to unnecessary changes.
Challenge assumptions, explore edge cases, and assess technical feasibility.
Stage 2: Individual validation
Invite selected participants to work with prototypes in controlled research sessions.
Observe behavior, investigate expectations, and identify the assumptions that need further testing.
Stage 3: Controlled pilots
Introduce promising ideas to small groups of consenting users in realistic environments.
Evaluate the effects on actual workflows, including the possibility that a new feature creates more work than it eliminates.
Stage 4: Workflow and business validation
Measure whether the experiment improves the outcome it was designed to address.
For an enterprise AI assistant, that might mean reduced task completion time, fewer escalations, improved decision quality, or less manual rework.
Also measure unintended consequences, including errors, interruptions, and loss of user confidence.
Stage 5: Progressive productization
Expand availability when evidence supports the decision.
Provide appropriate onboarding, monitoring, human oversight, and rollback mechanisms.
The goal is to ensure that experiments earn their way into the product rather than becoming permanent features simply because they were easy to build.
Are we testing the right problems?
There's a more fundamental question beneath all this. Do we understand the customer's job well enough to know which experiments are worth running?
AI can help us produce dozens of prototypes. But if our understanding of the customer's problem is incomplete, we may simply generate dozens of variations of the wrong solution.
Consider an enterprise product team trying to improve sales productivity. The initial request might be to build an AI assistant that summarizes account information. That's a plausible solution.
But individual research might reveal that sales representatives aren't struggling to find information. They're struggling to decide which information is reliable enough to act upon.
Now the problem changes.
Instead of building a better summarization interface, the team might explore source transparency, conflicting data, recommendation confidence, and decision support. The second direction could create substantially more value, even if the first was easier to prototype.
This is why understanding the customer's job to be done remains essential.
AI can accelerate solution exploration.
It cannot eliminate the need to identify the right problem.
From prototype velocity to learning velocity
Product organizations have traditionally measured delivery through metrics such as sprint velocity, release frequency, and time to market. AI-assisted development makes it possible to move faster on many of these dimensions. But faster delivery doesn't necessarily produce better business outcomes.
We should also be asking:
- How quickly can we identify an incorrect assumption?
- How much reliable evidence does each experiment generate?
- How many low-value ideas can we eliminate before engineering investment?
- How much customer disruption does our experimentation create?
- How effectively do research findings influence product decisions?
These questions shift attention from the number of prototypes produced to the quality of decisions those prototypes enable. They also introduce a different way of thinking about return on investment.
A prototype that prevents six months of unnecessary development may be more valuable than one that accelerates the delivery of a feature nobody needs. And an experiment that reveals why customers distrust an AI recommendation may be more valuable than another round of interface refinements.
Designing the experimentation system
As AI changes the economics of product development, experience design has an opportunity to influence more than individual screens and interactions. We need to help design the system through which organizations discover problems, evaluate possibilities, gather evidence, and make investment decisions.
That means connecting customer research, prototyping, product strategy, engineering, and business outcomes. It means protecting customers from unnecessary disruption while creating more opportunities for meaningful learning. And it means recognizing that experimentation is not an end in itself.
At Intelligaia, we see this as an important direction for human-centered product development in the AI age: using faster prototyping to make better decisions, not merely to produce more software.
Perhaps the next competitive advantage won't belong to the organization that generates the most ideas or builds the most prototypes. It will belong to the organization that can distinguish a promising idea from a costly distraction before either reaches production.
If AI allows us to test almost anything, how will we decide what's worth testing with our customers?
Intelligaia helps enterprises and emerging businesses explore, design, and validate digital product experiences—connecting human-centered design, AI-enabled prototyping, and measurable business outcomes.
Validate high-risk assumptions with AI-assisted prototyping.
Curious where AI-assisted prototyping could de-risk your roadmap? Talk to us before you commit to full-scale development.