Real life Examples and Practical Tips on how to Identify Your Organization’s Innovation Blind Spots
Written By Ken Field • April 2026 Strategy & Innovation Consultant
There’s a slide in almost every corporate innovation presentation right now that talks about “test and learn.” About running fast experiments, staying close to the customer, and letting data drive decisions. We’ve all seen the case studies: Booking.com’s 1,000 concurrent experiments, Netflix’s A/B-testeverything culture. It’s a compelling story, and for a certain kind of organization, it’s working.
But if you manage innovation or insights for a company that sells through independent dealers, franchise operators, or a distributed retail network, you’ve probably sat through those presentations with a quietly nagging feeling that none of it quite applies to you. It’s a problem Arima works on directly, and it’s more structurally stubborn than most organizations realize.The examples are always direct-to-consumer brands or tech platforms. The advice always assumes you can see your customer and experiment freely.
But, what if you can’t?
I want to spend some time on that question, because I think the innovation conversation has a significant blind spot. And it’s costing organizations that operate through distributed retail models more than they realize. To make the problem concrete, I’ll keep returning to a case I find instructive: Home Hardware, Canada’s largest dealer-owned hardware cooperative. Close to 1,100 independently owned stores, three banners, and a business model that gives individual dealers significant autonomy over their customer relationships. That autonomy is the cooperative’s competitive strength against big-box chains like Rona and Home Depot. It is also, as we’ll see, the source of a structural data problem that experimentalist innovation playbooks aren’t designed to address.
The case for moving fast and testing in market is solid. NIQ BASES data¹ shows that 52% of new items with national distribution grow in their second year, and that the divergence between products that grow and products that don’t show up within the first four weeks after launch. In-market signals tend to be both strong and fast, which means getting to market sooner gives you more time to read signals and adjust strategy accordingly.
The venture world has internalized this completely. Amazon’s two-pizza teams, Google’s OKR culture, the widespread adoption of Agile outside software: all of it points in the same direction. Stefan Thomke’s research² at Harvard Business School documents how the most innovative organizations treat experimentation not as a project but as a capability, something built into how they operate every day. It’s a great story. And it rests on a foundation that a lot of organizations simply don’t have: the infrastructure to conduct experiments and collect data on their success.
The Structural Problem That Experimentalists Tend to Overlook
For large organizations selling through dealer, franchise, or retail networks, there’s a problem that comes before data: the people you need to run experiments with don’t want to be experimented on.
Dealers, franchise operators, and independent retailers have invested real time, money, and local reputation into their customer relationships. They joined a cooperative or franchise network to run their own stores and serve their local markets, not to serve as test sites for head office initiatives. Asking a dealer to change how they merchandise, price, or promote a product in order to test a hypothesis that the brand team developed is asking them to take on operational disruption and some measure of risk in service of someone else’s learning agenda. That’s a hard ask, and a reasonable person in their position would want a compelling case before saying yes.
Which brings us to the second problem: that compelling case is very hard to make, because the data needed to build it doesn’t exist.
In dealer-owned or franchise-based retail, the customer relationship belongs to the point of sale, not to the brand or the head office. Purchase history, basket composition, transaction frequency, trade area behaviour: all of that typically sits with the individual dealer, if it’s collected at all. The parent organization can observe sell-in (what moves from the warehouse to the dealer) but not sell-through (what consumers actually buy, when, and in response to what). This data imbalance is intrinsic to the business model and not just a quirk of the tech stack.
Home Hardware illustrates the bind clearly. Consider BeautiTone, the cooperative’s private label paint brand, positioned in the company’s own words as “the number one Canadian-owned retail paint brand.” The brand team can track awareness and some aggregate sales data, but they have almost no visibility into who is actually buying BeautiTone, in which markets, or in response to what triggers. So, if some markets see a dip in their BeautiTone sales, corporate needs to somehow connect their BeautiTone brand metrics to those local market trends and disentangle the macro-economic factors that could account for the move. And if corporate can’t come up with a more compelling story as to what caused the challenges than the local dealers, what credibility do they have in proposing a solution?
And yes, this is the problem loyalty programs are supposed to help solve. And yes, Home Hardware has one (Scene+) like most retailers today. But these programs are hamstrung by high costs and low participation rates that can limit their utility beyond directly activating engaged program members.
So, the head office can’t easily ask dealers to run experiments, and they can’t easily build the evidence that might make dealers willing to participate. Each problem reinforces the other.
You might think the solution is more corporate-controlled channels that head office can experiment on to their heart’s content. Home Hardware has made meaningful moves here too, through a Kibo Commerce implementation that enables inventory visibility, in-store pickup, and ship-from-DC fulfillment. But as Digital Commerce 360 reported in 2024, online penetration in hardware and home improvement sits around 10% of total category sales, dominated by big-box players. For a cooperative without centralized e-commerce scale, the digital signal is partial at best. And this is a recurring challenge for corporate-owned channels: they tend to be marginal and atypical. There’s significant value that dealers, franchises, and retailers bring to the relationship, after all.
This dynamic isn’t unique to Home Hardware. Any organization that sells through independent dealers, franchise operators, or third-party retailers faces some version of it. The customer belongs to the channel, and the organizations in that channel have their own interests and their own legitimate reasons to be cautious about how the information gap gets closed.
The stakes of getting it wrong are also real. True Value’s Chapter 11 filing in October 2024, in which the 75-year-old hardware wholesaler sold substantially all of its assets to rival Do it Best for $153 million, is a reminder of what can happen when dealer-affiliated organizations can’t keep pace with the data-driven merchandising and supply chain efficiency of consolidated competitors. True Value’s filing cited COVIDera supply chain disruptions and sustained liquidity challenges as the proximate causes, and those were real factors. But the deeper structural lesson is that wholesaler-cooperative models face compounding pressure when they can’t match the demand intelligence of chains that own their customer relationship end-to-end.
The Instinctive Response to Invest In Research Makes the Problem Worse
When organizations face uncertainty about their consumers, the natural response is to commission research. And the research industry has built a comprehensive set of products to fill every perceived gap: brand health trackers, concept tests, usage and attitude studies, ethnographic deep dives, segmentation models.
None of that is without value. But in practice, these tools tend to form a sequential, dependency-heavy process, a research waterfall, that isn’t well-matched to the speed requirements of contemporary retail competition.
A typical innovation research program, from initial brief through concept test to product development to launch, runs 18 to 24 months in a large CPG or retail organization³. Each stage requires sign-off. Each sign-off requires research. Each research project requires a briefing cycle, a vendor selection process, fieldwork, analysis, a report, and a stakeholder readout. By the time a concept clears the waterfall, the market has often moved on. Competitors have launched. Shelf space has been reallocated.
The deeper issue is the accountability architecture that makes the waterfall so persistent. Most large organizations govern innovation through stage-gate processes⁴, originally designed to impose capital discipline on product development. The logic is sound. In practice, the gates have become research procurement checkpoints.
Here’s what tends to happen. At each gate, the review committee asks for evidence that the concept will work. The safest form of evidence is a piece of commissioned research: a concept test, a conjoint analysis, a usage and attitude study. These are not because these are always the best inputs to the decision but because they provide defensible documentation. If the product fails, the manager who commissioned the research can point to the positive concept test and say, credibly, that the decision was made on evidence. As a result, the research protects the decision-maker, even when it doesn’t improve the decision.
This dynamic has several consequences worth spelling out. First, it systematically favours pre-market research over in-market learning. A stage-gate review can evaluate a concept test score. It is much less equipped to evaluate the messy, ambiguous signal from a live market experiment where some stores showed lift, others didn’t, and the reasons aren’t fully understood. The gate process rewards clean, unambiguous inputs, and concept tests are designed to produce exactly that: a single number that says go or no-go. In-market experiments produce more nuanced results (as all real-world results will), which are harder to present in a gate review and harder still to defend if the outcome is mixed. So, the process selects for the research method that produces the clearest internal justification, not the one that produces the most accurate prediction of market behaviour.
Second, it extends timelines in ways that compound. Each gate typically requires its own research workstream. A concept gate requires a concept test. A business case gate requires a volume forecast. A launch readiness gate requires a tracking study design. Each of these has its own briefing cycle, vendor selection, fieldwork window, and reporting timeline. The dependencies are sequential, not parallel, because each gate’s output is supposed to inform the next stage’s work. The result is that organizations routinely take 18 to 24 months to move from insight to launch, in an environment where NIQ data shows that meaningful market signal arrives within four weeks of a product hitting shelves. The stagegate process is optimized for a world where getting the decision right before launch matters more than learning quickly after it. We see this argued for explicitly from research industry leaders like NielsenIQ who advocate for even longer more research-intensive pre-market evaluation than currently exists⁵. This is almost certainly because longer pre-market cycles benefit their business model.
Third, it creates a structural preference for incremental innovation. Genuinely novel products produce ambiguous research results because consumers struggle to evaluate things they haven’t experienced. Concept tests for truly new products tend to produce middling scores with wide confidence intervals. Line extensions of existing products, by contrast, test well because respondents can easily imagine a slightly different version of something they already know. The stage-gate process, by requiring strong research scores to advance, systematically filters out the novel in favour of the familiar. Mintel’s 2026 CPG innovation report⁶ found that only 35% of global CPG launches in 2024 were genuinely new products, the lowest share since 1996. The rest were line extensions and reformulations. That statistic is what you’d expect from an industry where the innovation process is optimized to minimize the appearance of risk.
There’s also a structural issue with how the research supplier ecosystem is organized. According to ESOMAR’s 2024 Global Market Research report⁷ , the global insights industry generated approximately $142 billion in revenue in 2023, with the top 10 firms, including NielsenIQ/GfK, Kantar, and Ipsos, accounting for more than half of total market research turnover. The business model of large research suppliers rewards frequency and volume of research engagements. I’m not being critical of the people who work in the industry; it’s a structural observation about incentives. The waterfall is, for the supplier ecosystem, a feature: more time spent on more research is good for the bottom line.
For organizations in distributed retail specifically, the standard research toolkit has some failure modes that don’t get talked about enough:
- Brand health trackers measure consumer awareness and affinity at a national or regional level. They can tell you that a brand’s awareness is up three points in Ontario. They can’t tell you why one dealer market is outperforming another, or which markets have the right demographic conditions to support a price premium.
- Concept tests typically use nationally representative samples. In a cooperative with close to 1,100 stores serving distinct local markets, urban, suburban, rural, with different regional housing stock and different renovation triggers, nationally representative samples wash out exactly the kind of heterogeneity that matters most for deployment decisions.
None of these tools are useless. But they answer questions about the national market when the decisions that matter are local. And they feed a stage-gate process that rewards documentation over learning.
What a Data Layer Actually Looks Like
Ace Hardware offers a useful reference point here, not as a template to copy, but as evidence that the distributed retail data problem is solvable in principle. Ace is also a dealer cooperative, the world’s largest, with over 5,900 stores in approximately 60 countries, and they’ve made the key structural move: building a first-party data asset at the cooperative level rather than trying to aggregate dealer-level data.
Their Ace Rewards loyalty program⁸ , now with over 73 million members, links customer identity to purchase behaviour across independently owned stores in a way that head office can work with directly. From that foundation they’ve built out an impressive demand intelligence capability, including a retail media network (RedVest Media, launched 2025⁹), real-time pricing and inventory visibility through a VusionGroup digital shelf label partnership¹⁰, and Planalytics¹¹ weather-driven demand analytics for hyperlocal inventory and promotional planning.
The piece worth paying attention to is not the loyalty program itself. Loyalty programs at this scale are expensive to build and sustain, and the analytics value alone almost certainly doesn’t justify the cost. The instructive part is what happens once the data layer exists. When a cooperative can tell an individual dealer that demand for exterior paint in their trade area is likely to spike three weeks earlier than usual because of an atypically warm spring forecast, the conversation between head office and dealer changes fundamentally. It’s no longer corporate asking dealers to trust them. It’s corporate demonstrating that they understand what’s happening in that dealer’s specific market, and that their recommendations are grounded in something concrete. That’s the dynamic that unlocks experimentation: not asking for cooperation on faith but offering evidence that makes cooperation rational.
Not every cooperative has Ace’s scale or capital. But the direction is instructive: the organizations finding a way through the distributed retail data problem are building cooperative-level intelligence assets, not trying to aggregate dealer-level ones.
A Different Question
The organizations I find most interesting right now aren’t asking “how do we do more research faster?” They’re asking something different: how do we build a demand intelligence layer that tells us where to test, what to test, and how to interpret the signal from a small-scale in-market experiment with confidence?
For distributed retail organizations, that layer needs to work independently of dealer transaction data, because that data isn’t yours to use. What you can work with is population-level intelligence: a model of consumer behaviour at a geographic and demographic level that is rich enough to identify where a given product or positioning is likely to resonate.
But there’s a second, less obvious benefit, and it goes back to the stakeholder problem. When you can model with some credibility which markets have the right consumer profile for a particular initiative, you can have a fundamentally different conversation with your dealer partners. Instead of asking them to participate in an experiment on your behalf, you can show them specifically why you think the experiment is likely to work in their market and what they stand to gain. That changes the dynamic from “we’d like to test this in your stores” to “here’s the analysis that makes us confident this is worth trying, and here’s what the upside looks like for you.” It’s not a guarantee of cooperation, but it’s a much stronger foundation for it.
Synthetic population modelling, the approach Arima has built its platform around, is one of the more credible ways to build this kind of layer at a cost and speed that makes genuine experimentation viable.The intuition is straightforward: if you can model a sufficiently realistic synthetic population, one that reflects real demographics, purchase behaviour patterns, category relationships, and trade area dynamics, you can run analytical experiments on that population before committing to expensive in-market tests. The tests you do run can then be modelled across the synthetic population to provide a much more nuanced interpretation of the results. And, if you can also calibrate that model against your existing brand health and attitudinal data, then you get something extremely useful: a demand intelligence layer that is specific to your brand’s context, not just a generic market model.
The tactical implementation of this solution is necessarily bespoke to each company. Although some interesting playbooks may be the subject of a future article. They could range from some well-structured dashboards supporting partnership engagement across the network to full-on data science initiatives. But the point here is simpler: the question worth asking isn’t how to do the current research process better. It’s whether the current research process is actually solving the problem you have, including the problem of getting your channel partners to the table.
Closing Thought
The experimentalist agenda in corporate innovation is still alive and well. But experimentation requires the active consent of all participants, which is harder when you don’t own the platform top to bottom. In distributed retail, your participants are independent dealers and franchise operators with real money and reputations on the line. You earn their cooperation by demonstrating you understand their market well enough to make the ask credible.
This is the potential of a demand intelligence layer enabled by technologies like Arima’s Synthetic Society. In solving a data problem, it can also help solve a trust problem, supporting productive collaboration and partnership across the network. And in doing so, it also addresses most of what’s wrong with the modern research waterfall: the long timelines, conservative evaluations, bias toward documentation over learning, and the national benchmarks that wash out local reality.
So, it’s worth asking, is your innovation program designed to find truth, or to document the search for it?
¹NielsenIQ, “The CPG Innovator’s Guide to Vitality” (2023). https://nielseniq.com/global/en/insights/report/2023/the-cpginnovators-guide-to-vitality/
²Stefan Thomke, “Building a Culture of Experimentation,” Harvard Business Review (March 2020). https://hbr.org/2020/03/building-a-culture-of-experimentation
³Retail TouchPoints, “Maximizing Profitability: How to Improve 4 Key Areas of the CPG Product Lifecycle.” https://www.retailtouchpoints.com/features/executive-viewpoint/maximizing-profitability-how-to-improve-4-key-areas-ofthe-cpg-product-lifecycle
⁴Robert G. Cooper, “Perspective: The Stage-Gate Idea-to-Launch Process,” Journal of Product Innovation Management 25:3 (2008). https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1540-5885.2008.00296.x
⁵NielsenIQ, “The CPG Innovator’s Guide to Vitality” (2023). https://nielseniq.com/global/en/insights/report/2023/the-cpginnovators-guide-to-vitality/
⁶ Mintel, “Most Innovative 2026: Recognizing the CPG Products Redefining the Future.” https://www.mintel.com/presscentre/mintel-most-innovative-2026-recognizing-the-cpg-products-redefining-the-future/
⁷ ESOMAR, “Global Market Research 2024.” https://www.esomar.org/uploads/public/research-resources/ESOMAR_GlobalMarket-Research-2024.pdf
⁸Ace Hardware, “Ace Hardware Launches RedVest Media, The Helpful Network” (2025). https://newsroom.acehardware.com/ace-hardware-launches-redvest-media-the-helpful-network-offering-brand-partners-apowerful-new-retail-media-platform/
⁹Retail Dive, “Ace Hardware Launches Retail Media Network RedVest Media” (2025). https://www.retaildive.com/news/acehardware-launches-retail-media-network-redvest-media/758125/
¹⁰Business Wire, “Ace Hardware Selects VusionGroup’s Innovative Solutions to Enhance Retail Experience” (August 2024). https://www.businesswire.com/news/home/20240807051295/en/Ace-Hardware-Selects-VusionGroups-Innovative-Solutionsto-Enhance-Retail-Experience
¹¹Planalytics, “Ace Hardware and Weather-Driven Retail Strategy.” https://www.planalytics.com/ace-hardware-and-weatherdriven-retail-strategy/
