Skip to content
Marketing leader insights Optimise

From experimentation to operation: making AI work in practice

Sapphire Smith
Sapphire Smith

This post represents anonymised insights from a marketing leaders roundtable discussion. All participant information has been removed to protect individual privacy while preserving valuable learnings for the broader Changemakers community.

 

The current landscape: the messy middle

Most marketing teams have AI switched on. Copilot (or equivalent) is rolled out, a handful of agents exist, and someone is quietly wondering whether any of it is actually solving anything. That’s the messy middle: enough activity to look busy, not enough structure to know if it’s working.

Further along in the AI journey, a different picture emerges: teams with token usage they can actually measure, an AI source of truth they can trust, and a KPI story that survives contact with the board. Getting from one state to the other doesn’t need a bigger AI budget. It needs a plan.

Four themes define that journey: knowing when to stop and audit rather than build another agent, a simple framework for choosing where to start, the governance work that makes AI answers trustworthy, and the measurement question every board keeps asking.

Theme 1: Audit before you automate

The instinct across most marketing functions right now is to keep building. Tone of voice agents, market research agents, content agents: the tools multiply faster than anyone can judge whether they’re earning their keep. The more useful instinct is to stop, audit what’s actually broken, and only then decide whether AI is the fix.

Recognise FOMO for what it is

Director, design and engineering software company: “We are in this total AI crazy FOMO, everyone wants 5,000 agents for this and that.”

Speed pressure is real, but it’s driving teams to build volume rather than value. The mandate to “do something” with AI is not the same as a strategy for what to do with it.

Automation solves more than people expect

CMO, HR and payroll software company: “It wasn’t really solving problems. We were just using AI for AI’s sake.”

Several leaders pointed out that a chunk of what gets labelled an “AI problem” is actually an automation problem, and a much cheaper one to fix. The advice: audit first on what’s actually causing friction, then work out whether the right answer is AI, automation, or neither.

Replace deployment thinking with problem thinking

Marketing leader, financial services organisation: “Think about the problem that you’re trying to solve, and how AI will solve it, not where you can deploy AI.”

Maturity isn’t measured by how many tools are live. It’s measured by how clearly a team can explain why each one exists.

Theme 2: Building purposeful agents

Once the audit question is settled, the next challenge is prioritisation: with limited time and a long list of possible use cases, what actually goes first? One practical framework gaining traction breaks this down into six questions, known by the acronym MOTIVE: manual effort, ownership, tedium, impact, variability, and existing data. Score a potential process against all six and the highest-scoring, lowest-friction option becomes the obvious starting point.

Start the race before trying to win it

AI implementation practitioner, marketing consultancy: “You’re not going to get to the finish line unless you’ve started the race.”

Analysis paralysis is as common a failure mode as reckless deployment. A small, well-scoped pilot beats a perfect plan that never ships.

Pick the process people already dislike doing

In one example, a sales team’s cold outreach process scored highly on every measure: high manual effort, clear ownership, genuine tedium, high potential impact, low variability, and usable existing data in the form of case studies and account histories. That combination made it the obvious low-risk, high-value starting point, not because it was the most ambitious idea on the list, but because it was the easiest to build and the most likely to stick.

Measure adoption, not ambition

The resulting tool was built and piloted with a small group over eight weeks. A year on, it’s still in daily use, with no requests to change or switch it off. The lesson generalises: success for a first AI project looks like sustained use and positive feedback from the people doing the work, not a headline ROI figure.

Theme 3: Trust is the real bottleneck

Access to AI tools is rarely the constraint anymore. Trust in what they produce is. Once an AI-generated answer, a stat, a claim about revenue, a fact about a client, gets copied into an email or a deck, an untruth stops being a minor error and starts being a liability.

Assume plausible isn’t the same as true

AI implementation practitioner, marketing consultancy: “Those plausible answers are getting us in trouble, potentially, down the road.”

Every team with broad AI access is exposed to this risk, whether or not they’ve noticed it yet. The fix isn’t restricting access, it’s building a layer of truth AI can be pointed at.

Build a source of truth, not just a skill

Teams that had tried building truth into individual AI skills found it kept drifting. The more durable fix was a governance layer that ingests everything (documents, transcripts, case studies, video), de-duplicates conflicting figures, and lets any answer be traced back to its original source. Once that layer exists, new use cases (events, tone of voice, client-facing tools) can be built on top of it with far less risk.

Guardrails enable adoption, they don’t slow it

Marketing leader, financial analytics software company: “It’s built in, so everyone felt they could use it.”

Rolling out guardrails first, before wider adoption, was consistently described as the thing that made people comfortable using AI in the first place, not a barrier that held them back.

Theme 4: Measuring AI performance to the board

Every AI investment eventually gets a KPI question from the board. The teams making progress have stopped trying to force AI into a single time-saving or cost-saving metric, and started building a broader measurement story instead.

Know what’s actually moving the metric

One board-level AI metric, tracked closely as adoption picked up, moved in the wrong direction entirely, not because AI wasn’t helping, but because there was no reliable way to isolate its effect on a genuinely messy dataset. A single proxy metric, chosen for simplicity, can end up telling the wrong story entirely.

Let the tech stack review drive the restructure

One team’s AI measurement approach centred on pipeline per marketing employee, benchmarked against industry ranges from efficient through to elite performance, alongside SDR ramp time and infrastructure cost savings identified through a full tech stack review. That review, using AI to assess what the stack could lose and where the savings should be reinvested, became the basis for restructuring the marketing organisation itself.

Redistribute skills before you cut headcount

Not every AI-driven change is a restructure. In one case, a centre of excellence handling research, account planning and content for an ABM programme moved that work into an orchestration layer instead, without reducing the team. The people who’d been doing manual research and analysis were repositioned to manage, validate and refine the AI agents. Headcount stayed flat; what people spent their time on changed completely.

Treat token cost as a planning input, not an afterthought

As more organisations move to enterprise-tier AI licensing, token consumption becomes a real budget line, not just a technical detail. One team’s early data suggested typical usage was a small fraction of the allowance provided per employee, but the more important shift was building tokens into the ROI model from the start, rather than treating them as a cost to worry about later.

Key takeaways

1. Audit first

Most AI problems are audit problems in disguise, and a chunk of what looks like an AI gap is actually an automation gap.

2. Score before you build

A simple framework across manual effort, ownership, tedium, impact, variability and existing data turns a long wish list into an obvious starting point.

3. Governance is the unlock, not the brake

A traceable source of truth is what lets teams scale AI use with confidence, not what slows them down.

4. Pick metrics that survive scrutiny, and know what’s driving them

Pipeline per employee, ramp time, and infrastructure savings tell a more honest story than a single proxy metric ever will, and restructuring or redistributing skills both count as legitimate ways to act on that story.

None of this is really about technology. The tools will keep getting faster and more capable regardless of what any one team does. What separates teams making real progress from those stuck in the messy middle is discipline: the willingness to audit before building, to start small and prove value before scaling, and to be honest about what a metric is actually measuring. That’s a mindset shift as much as a technical one, and it’s available to any team, regardless of how many licences they’ve already bought.

From insight to action

Too often, AI ends up as a solution looking for a problem: agents built because the tools exist, not because a process needed fixing. The teams making genuine progress are doing the opposite, starting with a clear-eyed audit of where the friction actually is, then building a prioritised, governed path from pilot to scale.

Modern works alongside marketing leaders as a peer-level partner: auditing where AI genuinely fits your current workflows, building the roadmap and governance to get there safely, and developing solutions your team will actually use, not just switch on.

If you want to find out more, get in touch. 

If you'd like to join the conversation, sign up to the community below:

Join the Changemakers community.

Share this post