In recent years, data has surpassed oil in being the most valuable commodity on Earth. In just the four years between 2016 - 2020, the data market in the US grew in value from €129 billion to €211 billion. In a nutshell, data is how we understand something on an intricate level without bias and subjectivity, and within the world of e-commerce and retail, it is the cog in the machine that’s indispensable. In this four-part series, Omnia shares the process a potential customer will enter into once they decide to choose our pricing software solutions. In early August, we shared part one, which included the technical pre-requirements a customer needs to begin their pricing journey. Today, we are delving into how we collect data as one of the initial and most vital parts of the process.

Where does the data come from?

Speaking to David Gengenbach, a developer at Omnia, and Berend van Niekerk, Omnia’s Head of Product, it is impressive to see how much time and attention goes into getting detailed - and most importantly, correct - data for the pricing strategies of customers. There are two types of data that provides everything needed to reprice an entire online store or marketplace: Internal data and market data.

Internal data

Internal data includes information that comes directly from the customer. “Everything from sales data, purchase prices, stock data, performance measures, information on champion and non-champion products, new and old products, categories, seasonal products… it’s important to provide as much information as possible,” says Berend. Insights from Google Analytics and additional plugins can also be used to understand where traffic is going and which products are most popular. If your online store is using Shopify, Magento, Shopware, Plentymarkets or JTL, you can make use of our "Pricemonitor plugins" which you can find in the respective app store. The plugin allows you to connect easily to our database, without having to involve your IT department. 

After the initial data is connected to Omnia, the customer has the ability to modify all information within the tool. They can clean the data, change the formatting and add additional logic and calculations to the data. In this way the user can do the required modifications, without having to bother their IT department. “There’s less hassle for the customer this way,” says Berend.

External data

Market data, or external data, comes in two categories, according to Berend: "Data from marketplaces and comparison websites, like Google or Amazon, and data that is directly collected from competitors' websites." Typically our customers use a combination of both. The data from marketplaces and comparison websites will provide a good view of all competitors selling the product, where data that is scraped directly from your competitors will ensure you the most up-to-date and complete overview of your main competitors. The data from marketplaces and comparison websites includes highest and lowest pieces, reviews, delivery times and many other features.

How do we ensure data quality?

Vetting data is also part of the scraping and collection process. David, who specialises in competitor data, shares that there are four aspects to data quality:

• Finding the right competitor prices by ensuring that the competitor prices are for the exact same product as you are selling. For example, if we were checking the prices on Google Shopping for the iPhone 13, we would not consider the prices for second hand iPhones, where many websites advertise on Google. These prices would not be included.

• Within those prices, making sure that we identify any outliers. For example, perhaps Google grouped the products incorrectly and there is a very high or low price in the grouping.

• The timeliness of the data: Making sure that we update the prices on a particular schedule, so that we collect any price updates quickly.

• Data quantity plays a role too. If we conduct a product search, and there are 10% less products today compared to yesterday, we need to investigate what may be causing that.

Superior pricing strategies are informed by our data

Within the retail and e-commerce landscape, there is no successful web shop or marketplace without a comprehensive dynamic pricing strategy. And, in turn, there is no complete dynamic pricing strategy without data. However, it is up to the customer how much of their internal data they are willing to give. The more data we have, the more we can create a profitable and competitive pricing strategy for each customer.

Stay posted for our next part of the series on what customers can do with this data.