The complex of issues surrounding demand, product line and pricing intelligence relies on a fact-based analysis of market data through algorithm-supported data crawling. Valuable insights can be generated for various areas of decision-making (marketing, sales, product development, etc.) by analysing the data collected.

Figure 1: Overview of the main advantages of a data-driven analysis


  • Overall market potential

The determination of overall market potential is a pivotal input for a fact-based business case. Data crawling makes it possible to ascertain the overall potential of selected product categories – on Amazon, for example – with a very high degree of accuracy and validity, both on the basis of revenues and units sold. Thanks to this market transparency, providers can evaluate the actual level of market attractiveness very effectively.

  • Competitors’ top sellers

Crawling data enables up-to-the-minute evaluations at product level. As such, it also allows us to determine the top selling products amongst all competitors in particular categories, which in turn provides important information about main drivers of a top ranking. In light of the fact that the distribution of sales by ranking is anything but linear, this is a crucial area of investigation. At Amazon, for example, depending on the category, up to 90% of revenue might be accounted for by the top ten-selling products, which makes clear the necessity of systematically pursuing a high ranking. Furthermore, the database of top sellers is an important starting point for further value-adding analyses.

  • Identifying product line gaps: By comparing the provider’s product line with top sellers, we can examine the extent to which gaps in the product line need to be filled.
  • Inspiration for product development: By evaluating customer reviews, we can find out which product features customers really like as well as the extent to which features are missing and further development is needed.
  • Findings for sales & marketing: By analysing pricing and customer evaluations, we can ascertain the customer’s willingness to pay, enabling the right conclusions to be drawn for pricing and marketing strategy
  • Possible areas of action for listings optimisation: An analysis of the quality of individual product listings with regard to media content, HTML content with product descriptions, etc. provides information on possible areas of action in this regard.


  • Competitor market shares

Thanks to crawling data (e.g. on Amazon), we can determine the market shares of all competitors for selected product categories. On this basis, it’s possible not only to draw up and verify strategic goals (e.g a market share of at least 35% by 2025), but also to implement active performance management. For example, we can measure how market shares have changed over the course of a specific marketing campaign. This is then used to derive a calculation of market efficiency.

Figure 2: Example of an evaluation of competitor market shares by revenues and units sold


  • Competitor pricing

Data collection through crawling provides daily updated pricing information at a product level. Over a particular monitoring period, a very accurate picture of competition prices and the associated price strategies can be built up. On this basis of this, we can then derive important findings for the development of a dedicated pricing strategy.


  • Product line performance

The market figures give a clear indication of the current product line performance, e.g. in the form of the average revenue per product listing. On the basis of this performance indicator – and especially in regard to comparison with competitors – appropriate optimisation measures can be derived for the provider’s product portfolio.


In general, the market transparency described above can be utilised both in the context of a (one-time) study and on an ongoing basis, through the installation of a dedicated performance cockpit or dashboard. This can also be linked to internal controlling data via corresponding interfaces, thus ensuring that transparency is maintained on a continuous, long-term basis.

Figure 3: Overview of data crawling sources and expansion possibilities


The main advantages of an analysis based on crawling data are:

  • Meaningful business planning through tailored choice of categories

Typically, both top-down and bottom-up business planning are based on general market studies, which are only roughly indicative of the product categories that are truly relevant for a particular company. This dilution with irrelevant market data results in huge uncertainties early on in the process. The advantage of crawling data-based analyses lies in the fact that the analysis is tailor-made and that only relevant data will be obtained, thus avoiding uncertainties of this nature.

  • Meaningful data basis for up-to-the minute evaluation at a product level

Crawling data is based on daily tracking of transactions at an product level. This detailed data is then aggregated at a higher level, e.g. for product categories or brands. As such, it is possible both to carry out evaluations at the product level and to obtain an overall snapshot.


For further information on our market intelligence services and a comprehensive sample project, please download our white paper.


Please provide your name and email address to receive an email with the corresponding PDF file free of charge.

    Click here for more information about data protection.

Contact one of our experts

Markus Fost, MBA, is an expert in e-commerce, online business models and digital transformation, with broad experience in the fields of strategy, organisation, corporate finance and operational restructuring.

Learn more

Markus Fost

Managing Partner
Markus Fost, MBA, is an expert in e-commerce, online business models and digital transformation, with broad experience in the fields of strategy, organisation, corporate finance and operational restructuring.

FOSTEC & Company is known from

View more

FOSTEC & Company GmbH

Marienstraße 17, D-70178 Stuttgart

+49 (0) 711 995857-0

+49 (0) 711 995857-99

Contact us now