How do we measure whether your store is really performing better with the model?
Measuring a store’s performance correctly is an extremely complex task for humans. Most owners rely on instinct and very simple comparisons – “was revenue higher than last month?” – while not taking into account market effects: seasonal fluctuations, inflation, consumer sentiment, external events.
Without these aspects, the measurement almost always gives a distorted picture: it may look like the store is performing better or worse, while in reality only the environment has changed.
In our measurements, we treat your store’s own performance and the market effects together, and we use professional, statistically well-founded methods that filter out randomness and external noise. We apply hypothesis testing based on complex probability distributions so that we can “see out of” the information noise and determine whether a growth is due to randomness or is in fact the effect of the optimization.
One of the biggest advantages of our new measurement technique is that we can measure the effect even when the entire product range is already under optimization – in other words, there is no need for a separate control group, and you do not have to keep any products “untouched” artificially just for measurement purposes.
The result: real, demonstrable performance indicators suitable for business decisions, even when the market or your costs are changing.
Net profit per day (calculated from orders)
Profit stability (reduction in volatility)
Relative performance compared to the market
if the market is going down, how much less you decline,
if the market is growing, how much more you grow.
Performance adjusted to the market and cost environment
if supplier costs increase but you achieve the same or better profit, that is objectively stronger performance,
our growth calculation takes this into account and does not “penalize” you just because of cost increases.
How do we filter out the “noise of the market”?
We use two mutually reinforcing methods, now without maintaining a control group:
Before–after (temporal) measurement with permutation hypothesis testing
we compare the profit of periods of equal length before and after optimization,
the permutation test tells us how likely it is that the observed difference is purely random,
the calculation also takes into account market movements and changes in the cost environment.
Dynamic, control-group-free comparison
there is no need to maintain a separate “non-optimized” product group,
we analyze the behaviour of the entire store and use statistical methods to separate
what comes from the market and what is truly the effect of the pricing model,
this allows us to measure even when every product is already running in the model-optimized environment.
What do we ask for clean measurement?
Do not turn off advertising in the period before optimization and during the measurement period.
As far as possible, do not change logistics, discount policies, category offering.
Ideally: at least 1 month of stable baseline before starting.
If supplier costs change in the meantime, the measurement handles this – but every other factor should be kept as stable as possible.
We do not measure based on “is revenue higher than last time,” but show in a market-adjusted, statistically proven way how much the model brings – even in a negative market, with rising costs.
For a store, the greatest value is knowing exactly what works and what doesn’t – this certainty is what we provide.