“Testing” is a buzzword in the world of online work. If you have a company that interacts with customers in the digital sphere, it’s likely that you want to consider testing in some form. And that’s a good thing. Testing gives you insights into consumer behavior and helps you make smarter decisions on your webshop.
Web developers and marketers often use A/B testing to experiment how different web layouts influence consumer behavior. When consumers land on a site, they are randomly redirected to an A or B page, each of which has the same goal in mind but which varies in messaging or design. Developers and marketers then evaluate differences in conversion or bounce rate between the two pages to determine which performed best.
Just like web developers, retailers can use testing to learn how prices influence the overall conversion rate for products. You can then use these insights to optimize your display price for margins and conversions — a crucial part of your company’s marketing plan. But testing prices online is not as easy as testing a website design.
To start, several factors influence the validity of your data, and testing at the product level is time-consuming and inefficient. As a result, A/B testing prices on the product level is a risky venture, and retailers should approach it with caution.
So what’s a better way to test your pricing method? Especially since it's a crucial stage in the Five Steps to Successfully Implement Dynamic Pricing? Read on to learn everything you need to know about price testing and the best way to manage your tests.
Testing pricing online is difficult
Testing prices on a product level is difficult for several reasons, regardless of which method you use. Some significant barriers to online testing include:
1. Cross-elasticity
Product prices don’t live in isolation; instead, they are part of a broader system of price elasticity. Raising prices of Product A can increase the demand for the substitute Product B. It could also decrease the demand of the complementary Product C.
There’s no way to be certain that cross-elasticity didn’t impact your test. But if you don’t consider this effect, your results will skew to one side.
2. The marketing effect
Pricing and marketing are interconnected in modern e-commerce, but many companies don’t treat them as two halves of a whole. Marketing teams and pricing teams have different KPIs and success indicators, so they ultimately have different goals when it comes to displaying the product online. If you want to test prices, you also need to consider the impact marketing will have on the product.
For example, say your pricing team wants to test a price cut on Product A but doesn’t tell the marketing team. Without that knowledge, the marketing team might aggressively push that product on a platform like Google Shopping and generate a high volume of sales. However, in this case, it’s impossible to tell if that increase in sales resulted from the marketing campaign or the lowered price.
3. Statistical validity
Testing hinges on one key component: you need a high volume of sales to gather enough information about the test. This is especially true for A/B tests, where the sample size for Price A and Price B need to be significant.
There are also several other validity threats to account for in your testing, such as selection bias, the effect of seasonality, and more. Because of these factors, many small- and mid-sized retailers won’t generate enough traffic to make sure their data is valid.
4. Competitor pricing
Much like cross-elasticity, your product’s price doesn’t exist in a vacuum on the internet. The uncontrollable factor of your competition’s price for the same (or a similar) product will affect your ability to test a price. If you are priced higher than your competition for a test, consumers will naturally choose the lower price. As a result, you miss out on both valuable sales and important testing data.
Additionally, your price and its relation to your competition impacts your overall price perception. This is a massive driver of consumer action and one you should consider carefully.
Why you shouldn’t use A/B tests to evaluate your pricing method
Just because it is difficult to test your pricing doesn’t mean that you shouldn’t do the tests. Since price is one of the best ways to improve your overall earnings, it’s worth taking the time to experiment and determine the right spot for your products.
However, A/B tests of prices offer a unique set of challenges, each of which makes the venture risky.
1. Consumer resentment
Consumers see the act of charging different prices to different people as discriminatory, regardless of how the prices were assigned. This has been the case for years: in 2000 Amazon created a PR-nightmare when they tested optimal prices on different customers. Consumers were outraged, and the company ultimately refunded money to 6,900 customers.
Today, consumers understand that prices will change throughout the day. But with social media just a few taps away, consumers will find out quickly whether or not their price for a product differs from someone else’s. Any discrepancy they find will embroil their resentment toward a company.
This sentiment is the same reason personalized pricing remains unpopular among consumers. Many feel the practice is predatory and discriminatory.
2. Data privacy issues
In line with consumer resentment, the general public associates price discrimination with the unfavorable usage of customer data. As an increasingly important topic on the global stage, and it‘s at the forefront of many consumers' minds.
Data integrity is crucial to controlling customer resentment. And while you might be able to truthfully say that A/B tests and prices were assigned randomly, customers will grow to mistrust your organization.
3. Decrease in customer loyalty
When consumers resent your company for price differences, and when they feel their personal data was somehow used to determine their display price, their loyalty to your organization will waver. And just like consumers communicate about price online, they also communicate about their satisfaction with a company through social media.
In the age where customer experience is essential, A/B testing could tip the balance to create unhappy customers. Disgruntled customers are not only detrimental to your long-term business goals, but they can also harm your reputation.
4. Inefficiency
Finally, A/B testing is time-consuming and inefficient. For an A/B test to be accurate, you need to monitor the tests on a product level. If you’re doing the tests manually, this will eat up significant amounts of time, energy, and money to the testing process.
Additionally, if you run an A/B test on a product, but carry other similar products, your test results will skew. For example, let’s say you want to test a pricing method on televisions. You could carry out an A/B test on Model X television to see which pricing method works best. But if your store sells more than one model of television, consumers could just buy a different TV. They could even buy a tablet to use instead of a TV — the functionality is essentially the same.
The better way to test your prices online
Considering testing is already tricky, and that A/B testing has significant downsides, what’s a better way to evaluate your pricing method?
The answer is to look broader than individual products and expand to categories. Instead of testing a price change across a single product, it’s better to test your price methods on different products categories. As long as these categories are similar in elasticity and customer perception, you can evaluate which pricing method works best for your store.
Let’s take an electric kettle and a toaster as an example: both serve the consumer in a single functional way (a kettle serves boiling water, and a toaster serves toast - no more, no less). They are also both kitchen appliances, so they both serve the same customer segment. Even though they are entirely different products, consumers view them in the same regard, so they have similar elasticities and price points.
This strategy helps reduce the general risks of online testing and eliminates the risks of A/B testing:
- Individual customers won’t receive different prices for the same product
- You can track the effects of marketing more easily
- You can better control for cross-elasticity
Getting started with this testing method is pretty easy with a user-friendly tool:
1. Select the products to test
Select two similar product groups, such as electric kettles and toasters. What’s important is that the products have similar functionalities, price perceptions, and elasticities.
You can use historical pricing and conversion data to guess which products have similar elasticities, but an easier way is to use a tool to calculate the elasticity for you.
2. Implement the pricing strategy across these products
There are several types of pricing strategies you can test. Once you choose the ones you’d like to test for effectiveness, you can apply the new prices to those products and market them as usual.
It’s important to remember though that you can’t control your competition’s prices. As a result, when you change your price online, you’ll also change your relationship to your competitors. One way to control this aspect is to use automation that locks in your price position on the market.
3. Test, monitor and analyze
After you’ve implemented the new pricing strategies, you can run your products normally and compare conversion data. It will take some time to see results and the data depends on the size of your assortment, but you should be able to evaluate the test after 2-3 months.
A world without price testing
Testing your pricing strategy across similar products gives you insights on which pricing method you should use. However, it’s also a time-consuming process and requires a significant investment to track manually.
Software like Omnia can provide a solution that eliminates your need to test pricing strategy at the product level. Our Dynamic Pricing algorithm optimizes prices for every product and makes testing less relevant.
Read more: The Ultimate Guide to Dynamic Pricing
The algorithm looks at each product’s elasticity, historical conversion data, and competitor prices then combines that data with internal factors such as stock levels and your commercial strategy to determine the optimal price. It then automatically adjusts the product price online to reduce the manual labor of making price changes. Additionally, fully utilizing the Omnia algorithm enables your prices to fluctuate predictively based on market insights.
Final thoughts
When it comes to testing a pricing strategy, A/B tests won’t make the cut. To make sure your data is valid and reduce the risk to your company, you need a more complicated test. It's possible to create one of these analyses manually, but it costs a significant amount of time, energy, and money to execute correctly.
Tools like Omnia make testing easier. By taking over the manual labor and helping you test more effectively, Omnia frees up time for you to focus on strategy instead of monitoring.
Interested in learning more? Click the button below to sign up for a free demo of the software today.
