What nobody tells you about leading AI experiments

There is a part of this work that I hadn't named yet in the earlier articles. It isn't technical. It doesn't show up in a flow diagram or in a token schema. But it is present every day of the project: the feeling of moving forward without knowing exactly where to.

I'm living it right now. On the Harness Design System project and on the AI application in the upstream, we are making progress. There are concrete results: the token governance I described in the previous article is real, it works, and the team can operate it. But real progress coexists with an unpredictability that doesn't shrink as the project advances. It changes shape.

The problem with an experiment is that it's an experiment

When you're building something that has been built before, the scope is relatively stable. You may not know how long it will take, but you know what you'll find along the way. Someone has already mapped the terrain.

When you're testing something that no one in your context has done before, the scope is a hypothesis. Every week reveals something that changes what you thought you knew. In the article about tokens, I said the estimate was to close in one week, and it didn't close. That wasn't a planning failure. It was the project showing its real complexity as we got into it. Typography got stuck in a way no one had predicted. The bind required a path that wasn't in the original plan. The question of where the JSON should live turned into an infrastructure discussion that took real time.

That is what an experiment does. It reveals what you didn't know you didn't know.

The problem is that teams, companies, and stakeholders are generally not calibrated for this kind of work. There is an implicit expectation, sometimes an explicit one, that you should know what will happen before you do it. And when you don't, it looks like a lack of preparation.

What this produces in whoever leads

There is a human cost in this that is worth naming, because it is real and rarely talked about.

Leading an AI experiment in a company that depends on predictable results creates a permanent tension. On one side, the project asking for attention to what is happening now, to the course correction, to the discovery that arrived on Wednesday and changed what you were going to present on Thursday. On the other, the legitimate pressure to report progress, defend priority, justify the time allocated.

That tension produces anxiety. And anxiety produces a specific temptation: to promise more than you know in order to look like you're in control. To anticipate results that don't exist yet. To present the plan as more solid than it is so you don't look like you're improvising.

I understand that temptation. I've felt it. And I learned that it is the shortest path to a bigger problem.

The only way out I've found

Transparency isn't a communication strategy. It is the only posture that works when you're in genuinely new territory.

What that means in practice: bringing leadership into the conversation before you have answers, not after. Not arriving with the result, but with the map of what is being discovered. Talking about what worked and what didn't with the same weight. Protecting the team from the pressure to deliver certainties that the project doesn't yet allow.

This takes courage of a specific kind: not the courage to move forward despite the risk, but the courage to be vulnerable in front of people who expect you to know what you're doing. To say "I don't know yet" to someone with a legitimate interest in knowing.

On our project, what has worked is keeping stakeholders in the loop continuously, not just at milestones. When the scope changes, and it will, they already have enough context to understand why. When a result isn't the one expected, it arrives as learning inside a conversation that was already happening, not as a surprise in a presentation.

This doesn't eliminate the tension. But it stops being about credibility and becomes about the real problem: how to move forward on terrain that no one has mapped yet.

What remains

I don't have a ready-made method for managing unpredictability in AI projects. I don't think anyone does yet, at least not one that works in every context.

What I have is a posture: protect the team from promising what can't be promised, keep leadership inside the reality of what is happening, and treat every discovery, including the ones that complicate the path, as valid information, not as a failure to hide.

Experiments are like that. The result isn't only what works at the end. It is everything that was learned along the way.