Testing: Before, During, and After Launching a New Site Design

Testing: Before, During, and After Launching a New Site Design

Launching a new version of an ecommerce website is no easy feat, especially if it’s an entirely new design. It takes time, as well as a great deal of strategy and planning. When it all comes together, the best you can hope for is that your customers and prospective customers will follow the journeys you laid out for them. Unfortunately, we also know that consumers are nothing but unpredictable. Without testing, your data and analytics will show that they are finding their own way, in most cases the path of least resistance.

All change comes with risk. Testing throughout a transition or cutover mitigates risk and introduces more predictability. The following approaches and opportunities come from the brains of our internal optimization and UX strategists Meredith Eads and Erin McElwee.

Testing and Benchmarking Before a New Design

Whether you’re considering a full design cutover or iterative changes, you’ll want to start the process by identifying major areas of change. With this in mind, the strategy should consider prototyping, user testing, and optimization (like A/B testing).

Identifying Major Areas of Change

Change can come in the form of simple fixes like designing a more user-friendly navigation, cutting steps out of your current customer experience based on where they fall off, or adding automated upsells and cross-sells to your homepage. Of course, more revolutionary changes are often required. If you’re migrating to a new ecommerce platform, you can save an incredible amount of time and money by basing the design on the platform’s native functionality.


There are a lot of tools available to prototype what these changes could or would look like. Here at Blue Acorn iCi we use Invision and Sketch, and a few others, but the options are seemingly endless. Regardless of the tool, prototyping allows you to test the usability of a concept or experience without engaging in development. Without putting forth too much effort, you’ll get a more accurate picture before steaming ahead.

User Testing

If you have an existing web property, getting quantitative data is like shooting fish in a barrel, but qualitative data isn’t as easy to gather. User testing can go a long way for capturing usability feedback. To be effective, you’ll need a working production deployment of the feature you want to test. If you want to get crafty, you could test on a competitor’s site.


At Blue Acorn iCi we refer to optimization as the incremental approach, or where you take what is currently in place and find ways to squeeze more out of it (whether it’s to increase conversions, reduce bounce rate, click-through-rate, etc.). On the other hand, optimization can also be used to test larger, riskier changes on your production site in order to determine if a feature should move on to a redesign as well. This allows you to test against any hunches that you may have for why customers may be falling off their purchase journey, and avoid bringing it to a new version of the site.

Testing During a Redesign

Before a site launch, testing makes perfect sense, as it does after you cutover, but what about during? Absolutely. Now, we know what you’re thinking. Testing in the middle of your project will lead to concerns about your current direction, potentially affecting your budget, and throwing your deadlines off. That may occur if you’re testing a core component of the new site, but while progress is being made you can focus on smaller, feature-level changes or business decisions to validate.

One test example may be focused on the presentation and content of a promotional banner, or even the decision to display on-model photography rather than the standard flat products on a category page.

It’s similar to the prototyping design decision, but depending on the scope, you can usually leverage the current production site to evaluate changes like promo messaging, product photography, sizing guides, product page content, low stock messaging, exit intent, data-driven, or event curated upsell/cross-sell opportunities.

According to Erin and Meredith, there is one caveat on this. If the functionality and data don’t yet exist to test against on a web property, a test will cost you more time and effort. With that in mind, it may be better suited to test site changes after a cutover takes place.

Testing After a Redesign

Now that you have insights and metrics from tests conducted before and during the cutover, it’s time to bask in your results! Ok, not so fast, there is still some more work to be done here. Just like a marathon, it’s all about the journey to get to the end, and just when you think you’re there, you’ve actually got one last mile to knock out. If you’re on a platform that doesn’t allow much wiggle room for customizations, you’ll be hard pressed to add many new features post-launch, but you can still test against some of these concepts. For starters though, you’ll want to ensure you have an optimization or personalization platform/tool in mind such as Optimizely.

Maintenance and Feature Development

Post cutover, the next step is to build out a plan for ongoing feature development and improvements. Technology in our space moves surprisingly slow, but that doesn’t mean the platform you’re on won’t roll out something on a quarterly basis that is worth implementing. For example, Blue Acorn iCi tested a variation that increased the visibility of the search on RMS Beauty’s site, which improved interactions with the field and increased search engagement and completion. That one test resulted in a 28.57% lift in revenue per visitor and improved conversions.

In addition to lack of search visibility, RMS Beauty also suffered from an overbearing header that took up a third of the screen on mobile devices. Blue Acorn redesigned the header to show more valuable content while increasing clarity on the banner, account, search, and cart. As a result, RMS Beauty experienced a 9.7% increase in conversion rate.

Business Initiatives

During the holidays, most brands see an influx of traffic, but not all are willing to risk design or optimization changes to their site. Leading up to the major holidays, you can conduct smaller split tests, but with a similar promotion or feature.

For example, our UX team tested shipping messaging for Victoria Emerson during the holidays. By adding a message that simply said “Get it in time for the Holidays” to their product detail page (PDP) they saw an increase to their revenue per user by as much as 20% and a 4.6% increase to add to cart. We also lobbed off the zeros from their price listings ($20 versus $20.00) and got some interesting results. From this, Victoria Emerson found that messages that indicated urgency on the PDP resulted in more add to cart actions and drove revenue by 43.5%.


If AI isn’t the biggest buzzword you’ve heard this year, personalization is likely second on that list. We’ve heard it for years, but ecommerce platforms and marketing platforms are getting smarter at using the behaviors of customers to create a more personal experience. Less retargeted ads of items you just bought versus upsells, side-sells, and subscription re-ups later down the road.

Can’t Stop, Won’t Stop

Of the tests and processes suggested here, this really only scratches the surface. Each site and brand will have their own set of priorities, and their consumer data will have a different story to tell. Not sure where to start? If you’d like to pick the brains of some of Blue Acorn iCi’s experts, feel free to reach out here.