The Right Way to Pilot Your Retail Innovation, Right from the Start

If you lead retail innovation initiatives, then you know just how much effort can go into getting a concept to the pilot stage. So when you actually do get the greenlight for a pilot, the last thing you want is to squander the opportunity.

“Retailers today have a lot on their plate. They may not have the time or the knowledge to run a pilot that uses specific standards and formalized testing. But if you really want to understand the performance of an innovation, you need a strategic pilot process from the outset,” says John Cloe, CEO at ProductivityONE, a consultancy firm specializing in retail optimization.

In the following, we’ll explore what that process should look like by:

1. Addressing crucial considerations for launching a preliminary pilot.
2. Explaining how to effectively broaden that pilot, with an emphasis on the store selection process.

(Note: For helpful insights on getting an innovation to the pilot stage, check out 4 Questions to Determine if Your Retail Innovation Is Pilot-Ready.)

Stage 1: Launch Your Pilot with a Questioning Mind 

For the initial pilot stage, you’re understandably not yet prepared to begin formalized testing or a full trial. Before you can reach that point, Cloe says your focus should be on achieving the following:

1. Reaching stakeholder consensus on the guidelines for implementing the innovation
2. Establishing a thorough how-to process for conducting the pilot itself 

Let’s consider the key steps to accomplishing these preliminary goals. 

1. Answering the Unknowns Through a Discovery Process

You initiate the pilot with the knowledge you currently have, which will be based largely on your research in the run up to the pilot’s approval.

Meanwhile, you need to be conducting a discovery process, says Cloe. “You’re going into this with identified unknowns that you want answered, and you’re staying alert to anything that you haven’t accounted for yet.”

Although questions will vary in relation to your specific innovation, here are some important examples:

• Have you identified all the SKUs and/or mechanical components needed?
• Can everything feasibly be delivered and installed in stores?
• What are your contingency plans should any step in the process be interrupted?
• Will there be new demands put on store associates and customers?
• Will the innovation displace existing store products, fixtures, and equipment?

You also want to continue this discovery process by gathering info in the following areas.

2. Defining the Win by Establishing Stakeholders’ Preferred Metrics

You need to establish the metrics that will prove if the innovation is indeed a win. Stakeholders have likely provided a degree of input on this already. Now, you want to verify that you’re including all the metrics they expect to see. You also then need to formulate the methodology for data collection.

When it comes to establishing the pilot’s metrics, Cloe emphasizes the importance of being thorough. “The last thing you want is to have a stakeholder later ask why you didn’t measure this or that factor. In a worst-case scenario, that can actually hold up a pilot’s progress.”

3. Formulating a Solid Framework for Measures and Practices 

The initial pilot involves installing and implementing the actual innovation in only a select number of stores. The full complement of pilot stores won’t be finalized until you expand your pilot in the next stage (discussed below).

For now, you’re focusing on formulating a framework for measures and practices, and establishing best practices in relation to needed time, resources, and equipment.

4. Demonstrating the Innovation in a Real Store Environment

Before you get too far into the pilot process, you’ll want to demonstrate the actual in-store innovation to key stakeholders. Flagship or showcase stores can work well for this. They’re typically located near corporate offices, which means they can be visited conveniently by upper management and others.

Note that Cloe cautions against using stores like these for formal testing because they usually have store events and other innovations continually running. “That creates a lot of competing customer traffic drivers,” he explains, “which prevent you from reliably quantifying data on your specific innovation.”

5. Removing the Guesswork with a Detailed Timeline 

Once critical information like the above has been gathered, it’s time to generate a timeline of scheduled events for the overall pilot.

An effective schedule takes away the guesswork, prevents unwanted surprises, and helps ensure everyone involved knows what their expected responsibilities are AND understands when they’re expected to execute them.

Additional timeline considerations from Cloe include the following:

Is it calendar-sensitive? If your innovation pilot is calendarized in relation to, for example, varied expectations through seasons and/or holidays, then this will likely stretch the duration for testing.

Does it directly involve a consumer packaged good (CPG)? If the innovation has a direct effect on a specific CPG, it could require a longer timeframe because of the timing of item promotions or, similarly, the peak of demand in relation to the time of year.

Is the innovation more equipment-oriented? If the pilot involves testing new equipment or fixtures, the timeline could be shorter, so long as you have a range of volume activity. That can be accomplished at various times of the year just by looking at days of the week.

Stage 2. Lay the Foundation for a Successful Broader Pilot

How will you know you’re ready to move from the pilot’s preliminary stage to a broader version for formalized testing?

Cloe says once you’re confident that you’ve incorporated insights from the discovery process (see above), you can then formalize the complete game plan for implementing the innovation and executing the pilot.

Important elements you want to confirm in order to broaden the pilot include the following:

• All the innovation’s necessary products and/or parts can be delivered.
• The in-store implementation of the innovation is feasible.
• Contingency plans have been put in place for potential problems.
• The visual reviews from executives and other stakeholders align with their expectations.
• Reaction from other key parties such as associates and shoppers is generally positive.
• All metrics and testing methods are in place.

In addition, you’ll need to finalize the stores to include. For your broader pilot to yield data that’s reliable and valuable, this critical component must be done effectively. Let’s take a closer look at the two major elements for this.

1. Calculating Just How Big Your Store Sample Size Should Be

Although numerous criteria need to go into answering this question, Cloe notes there are some rules of thumb he’s derived from decades of store pilot consulting. Here are some examples: 

• Innovations that are SKU-specific and/or are intended to influence a direct purchase generally require a higher number of test stores.
  
• The more intense the competitive landscape is for the retailer, the larger the sample size should be.
  
• On the other hand, if the pilot is testing the durability or labor impact of a specific fixture or other device, typically fewer stores are necessary.
  
• Similarly, if timed studies are the primary metric, such as for innovations in policy procedures or operational changes, fewer stores may be adequate.

“A sample size can be underdone, and I suppose in practice it can be overdone. But you can’t really overdo it because the larger the sample size the better,” says Cloe.

That said, he adds that a “carefully filtered” (see below) sample of 5 to 10 stores can still yield valuable results, regardless of an organization’s size. That’s especially true if the pilot is measuring an innovation’s impact on, for example, an entire department product category rather than on a specific product SKU.

2. Selecting Stores by Using Reasoned Criteria 

A careful, thorough store selection process is crucial for an effective pilot expansion. “You don’t want to select stores randomly, nor do you want to lean on piloting in stores that are, say, more conveniently located to the retailer or the innovation suppliers.”

So what should be used to drive your store selection process? Here are two major factors to consider:

Ruling out stores with red flags. You’ve already designated your metrics at this point. However, the data they provide will be less reliable if you’re running your pilots in stores that are not, in Cloe’s words, “clean and free of various data noise.”

The following is just a sampling of possible factors that could exclude a store from a pilot: 

• It’s new or recently remodeled, so year-over-year data is either unavailable or unreliable.
• Another recent innovation is still creating an uptick in customer traffic.
• It’s adjacent to new development that is likely increasing foot traffic.
• It’s near recent major road construction, which could have driven customers away.
• It’s in an area experiencing an economic downturn.
• A competitor’s recently built store is influencing your store’s current customer counts.

Getting an apples to apples comparison for test and control. A pilot may analyze how an innovation affects, for example, customer demographics, general customer traffic, and/or operational costs. In addition, depending on the retailer and the nature of the innovation, a pilot could differentiate performance in, say, rural, suburban, and urban stores.

But to acquire helpful information on these and myriad other potential data points, you also need control stores. These are stores which don’t have the innovation and therefore are used for statistical comparison. They also need to be selected with the utmost care.

Ideally, you want the test and control stores to offset one another, which prevents skewed data. “That’s why you’re trying to get an apples to apples comparison between your test stores and your control stores,” says Cloe.

For example, if a pilot store offers fuel and pharmacy, you want the control store to have those same offerings because those are significant customer traffic drivers. Cloe says, “There are literally dozens of considerations to make like that” as you select your test and control stores.

Leverage Outside Expertise for Success

You’ve just gotten some helpful insights on conducting an effective pilot. But we’ve only scratched the surface. In fact, depending on the scope and importance of your pilot, you may require expertise outside of your internal team. That’s why support from retail optimization experts like John Cloe can certainly help.

Of course, developing innovations to pilot in the first place isn’t easy. Don’t underestimate the importance of the right vendors. The most effective do more than just provide a product; they take the time to understand your current challenges and partner with you on your path to innovative solutions.

Marmon Retail Solutions and its family of retail-focused brands can help. We provide integrated retail products and services that enable your innovations to be more strategic, more cost-effective, and ultimately more successful.

Learn more about how these retail solutions can work for you.