The Risks of Launching Right Before the Holidays
Why do professional sports teams require a preseason? Players and coaches who have spent their entire lives playing the game, and then months preparing for a particular season, still need more time. Each team in the NFL plays four preseason games. In the NBA, they play eight. And in MLB, they play 30 preseason games. Why all the extra time? Shouldn’t they be ready to play?
Coaches need to see their players in action. Tens of millions of dollars depend on coaches making the right decisions and practices are just too slow. Internal scrimmages don’t tell you how players will react in the moment and under real pressure. In the NFL, coaches need to whittle down their roster from 90 players to 53 before game one. Imagine if you had to cut half your team? Four games barely seem like enough time to decide.
Your Challenge
Luckily, no commission or board is telling you to cut half your team before the holidays. Instead, your challenge has to do with technology. That means fewer emotional/human entanglements, but, still, our expectations and understanding of what to do in this situation don’t match reality. Reframing the way we think about timelines can be a real challenge, but if we can’t change our frame of mind, we’re bound to take unnecessary risks. In this article, we’ll talk about the risks, how to change your mindset, how to calculate a reasonable amount of time required to get “holiday-ready,” whether or not to consider beta testing, and what to do if you must launch after October.
*Download Blue Acorn iCi’s Complete Customer Experience Report to learn how to create an unforgettable, personalized customer experience. *
How did you get here?
There are many reasons that a launch date comes down to the wire. Whether your development was drawn out by unexpected challenges or your decision to replatform was made too late, you now are looking at a launch date somewhere around November. You may think you have just slipped into the game by the skin of your teeth. Well, I’m here to tell you that your skin is still on the line. Your technology has not been tested, and you are leaping from practice right into the playoffs. QA and UAT may have allowed you to catch glaring errors, but nothing competes with the real thing. Simulating every experience enough times to uncover every issue doesn’t happen until you go live.
Think you’ve invested too much to be dealing with such issues? Take a look at Toyota and other manufacturers like Honda who were affected by the Takata airbags recall. Earlier this year, the Japanese car manufacturer discovered that, instead of saving lives, a handful of airbags installed in its vehicles have, on multiple occasions, hurled lethal metal fragments at drivers and passengers. You can bet that if a massive manufacturer, the thirteenth largest company in the world, can overlook an error so grand in 63 million vehicles, you and your QA partner might overlook a few things, too.
Let’s start with the good news, though. No one is going to be playing catch with shrapnel because you launch two days before Black Friday. So let’s define the risks.
Lost Orders – Looking at the Risks
UX Risks – Customer Loyalty
Customer loyalty is the number one casualty of a pre-holiday launch. You put loyalty at risk in a number of ways when you launch just before the holidays, but let’s start with what happens with your UX.
Picture your loyal customer, or to use a loftier term, your brand ambassador. It’s the season of giving and this ambassador is ready to share you with the world. They have purchased on your site many times before. Perhaps they have had to get over a crappy user experience to do so, but they love your product so much, they have familiarized themselves with the ins and outs of your intricate UX.
Before they can ignore their family’s wish list and do all their holiday shopping on your site, you launch a new experience. You’ve replaced the labyrinth with a fluid experience, but for some reason, there is a tiny issue with the navigation. It isn’t loading properly on their particular browser, resolution, and/or operating system. The pressure of the holidays is on and you have introduced unfamiliar friction. They begin to view the change as negative. “What else has changed?” they wonder. Doubts proliferate under stress. Maybe now the wish list looks like a safer bet. Today, they may not buy. After the holidays, they may reevaluate their loyalty. As a cautious ecommerce manager, you have to ask yourself, “How often does this or a similar scenario have to happen before I start losing revenue?”
Integration Risks – Communication Breakdown
With UX risks you are losing orders because they are not being placed. With integration risks, you are losing orders because they aren’t being received. Following your launch, it may take time to identify situations that create dropped or duplicate orders. Once again, there are long-term consequences. If you fail to fulfill an order that happens to be an important gift, you can bet that you will lose that customer, loyal or not.
SEO Risks – Where did you go?
There are a number of SEO issues that may arise after you launch, and a good partner will account for most of them before hitting the big red button. Still, it’s not uncommon to see a dip in organic traffic after you launch even if you follow all the best practices. While your loyal customers will still find you, many new customers won’t. You have to ask yourself, “How much organic traffic am I willing to forgo during my busiest time of year?”
What you can do
Change your mindset.
If you could launch several months earlier, you would I’m sure. But, if you’re reading this article, I
imagine that is not an option. And chances are, you don’t want to wait until January when your customers won’t be able to benefit from the new UX and you won’t be able to benefit from the smooth business processes your new platform offers.
Given that time is inflexibly linear, you don’t really have any other options besides launching before, on, or after the holidays. So that can’t change. What can change, however, is your mindset. Launch date, that is the date you are able to push a functioning site live, is not the most important date to pay attention to. Knowing that you missed the date by more than you previously thought puts less pressure on you to rush things.
For example, if you know you are going to be late to a doctor’s appointment by five minutes, you may speed. But if you know that you are going to be late by an hour, you may call and reschedule, therefore saving yourself wasted time and preventing unnecessary, risk-taking behavior.
When we look at launch date, we are essentially looking at the time we will leave for our appointment instead of the time we will arrive at our destination. To be realistic means taking into account “travel” time and looking at our “arrival” time. The trouble is, a lot of people think that you can leave for Black Friday on November 15th. The next section should help us understand a reasonable travel time between launch and Black Friday.
Calculate a reasonable time
The formula for figuring out a good launch date is easy, but figuring out the value of each variable ahead of time is hard. You need to learn the time it takes time to identify issues and fix issues and subtract that from your holiday-ready date (eg. Cyber Monday).
Holiday-ready date – days it takes to identify issues – days it takes to fix issues = latest possible launch date
Those two variables will determine the length between launch and the holidays. They will depend on the complexity of integrations, the size and scope of the project, and your traffic.
Identifying issues
Early on, you can guess how long it will take to identify issues by examining your monthly unique visitors. Fewer regular visitors means more time required to get a full picture of the funnel in analytics. Once you have a full view of the funnel, you can identify choke points and make corrections in the UX. Monitoring tools like Pingdom and New Relic, which identify performance issues, can help you quickly discover and identify revenue impacting issues on your site.
In case you weren’t tired of sports metaphors, let’s go back to the one we introduced at the beginning of this article. An NFL coach requires only four preseason games to identify a player’s strengths and weaknesses because football players often go through more than 30 plays per game. Together that’s 120 plays in preseason. In MLB, coaches require 30 preseason games to identify a lineup because baseball players only appear at the plate about four times per game. Total, that’s 120 plate appearances in the preseason. It seems for coaches, 120 is the magic number. For you, identifying weaknesses isn’t about plate appearances or plays, but sessions.
4 games * 30 plays = 120 plays
30 games * 4 plate appearances = 120 plate appearances
So what’s the magic number of sessions? Expect at least 6000 sessions minimum before you can identify enough issues in the funnel. Using historical data, you can determine how long it takes for your site to see 6000 sessions, and thus how long it should take to identify issues. If your company sees a lot of traffic, you won’t have to wait long. But if you see only a little bit of traffic each month, it could take a lot longer.
Remediating issues
The second variable is harder to ascertain. You won’t know exactly how long it will take you to be “holiday-ready” until after you identify the magnitude of issues that need to be addressed. Here’s some advice though for once you do have that list of issues:
Because technology continually updates, things will continue to break ad infinitum. That’s the price of innovation. If you think you have solved every problem on your site, it means you have not adequately searched for issues or looked in the past week. With that in mind, a good goal is to solve 85% of the revenue-impacting issues you initially identify after 6000 sessions (obviously, more is better). Afterward, you or your ecommerce partner can determine a reasonable estimation of how long it takes to fix those issues, giving you a date to call yourself “holiday-ready.”
Have a team ready
Now is probably a good time to remind you that having a 24/7 SLA (service-level agreement) in place for after your launch is an absolute necessity. A team that built the site will probably get things patched up much faster than an alternative (yes, this is a shameless plug for Blue Acorn; we build, protect, and optimize sites for our clients).
Speaking of Blue Acorn, our business development managers have this discussion often. It’s hard to know exactly how long you need, but we almost never take on a client who demands a launch between November 1 and December 25. If the traditional holiday shopping season is your peak, launching anything later than early November is asking for trouble.
Now you may be considering beta testing during the holidays. You may think, “If I can’t launch, maybe I can start beta testing so I can launch right after the holidays.” We examine below.
Define and Prepare for a Beta-Testing Period
One of the best things you can do post-launch is beta testing. If you aren’t familiar, beta testing allows you to open up your site to a small segment of real customers. It’s a huge leap beyond QA and UAT. You can start with 5% of your customers and dial it up slowly until you feel comfortable that the new site can handle your needs. You can also test incremental changes to that small segment of customers.
Unfortunately, while it is a best practice, beta testing a new platform isn’t easy or cheap. It’ll require custom development. But if you do decide to perform beta testing, there are some things you should know.
Beta testing during the holidays
You might think that beta testing during the holidays is a good alternative to a full launch during peak season. Well, that may not be the case. Check out the following snippet on beta testing from a previous blog post, “Redesigning with Intelligence,” written by our CEO:
“During beta testing (and forever after), you should use advanced analytics tracking to pinpoint where customers are running into trouble. Then you can remove roadblocks, add confidence builders, and tweak the design until it has a more intuitive layout. Also, make sure that you’re not jumping to any conclusions with your data. Two days is not a sufficient amount of time to gather accurate information. Make sure you compile regular reports for several weeks, account for big email blasts, PPC campaigns, or big discounts when counting important metrics (traffic, conversions, etc.).”
In case you missed it, here’s the real important piece again: “account for email blasts, PPC campaigns, and big discounts.” Now imagine how difficult it’s going to be to account for those things during your big holiday push. How regular is your report going to be? Will you really be able to trust the insights? If you can, then I recommend going ahead and beta testing. If your data abilities are lacking in the slightest, then beta testing may not be worth the time and effort.
Do you want more content like this delivered directly to your inbox on a monthly basis? Subscribe to our newsletter, Blue Acorn iCi Monthly Digital Digest!