Three steps to monitor your competitors in E-commerce

General, Tutorial  · 

Competitor price monitoring is a game-changing process for E-commerce companies. In this post, we are going to see how to easily monitor your competitors using a Google Sheet and a simple formula.

Most of the time, price monitoring involved scraping, and because webscraping is often difficult we recently launched the simplest web scraping API around here. But don’t worry, you won’t need such a tool if you follow this post :).

This won’t work for every websites, but it’s a nice and easy way to track competitors when you don’t have a lot of URLs to scrape.

Your margin is my opportunity – Jeff Bezos

This post is going to be a little technical so fasten your seatbelt!

Step 1: Importing your products in a Google Sheet

The first step is to import your product catalog into a Google Sheet. Whatever E-commerce platform you use, you should be able to export your catalog into a CSV file and import this file into Google Sheet.

I recommend to skip every field and only import the product name, SKU, and price.

Your own product catalog in Google Sheet

Step 2: Matching your product against your competitors

Once you’ve imported your own catalog, you need to create a column for each competitor you want to monitor.

If you don’t know who are your top 5 competitors, here are some things you should try:

Once you’ve identified your top competitors, you will need to match your product catalog with their products. If you don’t have many products, this shouldn’t take long using Google Shopping or by directly going to your competitors’ website and using their search box.

This step is the most time-consuming. If you have hundreds/thousands of products, you can literally spend hours doing this.

That’s why at PricingBot we have a feature called “Smart Import” that matches product semi-automatically with your competitors’ catalog.

Step 3: Extracting your competitors’ prices

As an example, we’re going to extract this Sephora product price inside our Google Sheet:

https://www.sephora.com/product/the-dewy-skin-cream-P441101

Sephora Product

In order to extract your competitors’ prices directly inside Google Sheet, we are going to use a magical function called IMPORTXML.

=IMPORTXML("mycompetitor.com/product/some-product1", "//xpath_expression")

Import XML is a function accepting two parameters. The first one is a URL, in our case your competitor product URL, and the second one is an XPath expression. Xpath is a query language that we can use to select specific HTML tags in a webpage. We can test XPath expressions inside Chrome Dev tools, so it is time to fire up Chrome.

To do so, right click on the web page -> inspect and then cmd + f on a Mac or ctrl + f on other systems, then you can enter an Xpath expression, and the match will be highlighted in the Dev tool.

An easy way to get the correct XPath expression is to right click on the right html tag, then copy > XPath expression, but unfortunately it won’t work in our example, we will have to create a custom XPath expression.

XPath expression

You can find more about XPath here.

In our example, the XPath expression to get the price is simple:

=IMPORTXML("https://www.sephora.com/product/the-dewy-skin-cream-P441101?icid2=mostwanted_skincare0419_lp_desktop_dryness_product1_image_us_ufe","//div[@data-comp='Price Box']")

You can now add a column “Sephora.com” to our spreadsheet, and enter the formula:

Product and competitors's Google sheet

And that’s it! You now have a spreadsheet that automatically scrapes your competitors’ prices. You could go even further, and write a script to automatically refresh the data, see here.

Limitations

This technique works pretty well but there are serious limitations. First of all, if the target websites use too much Javascript on their website, it won’t work. Google sheet won’t fetch too many URLs either. So if you have hundreds of URLs to monitor, this might break.

In those cases, don’t hesitate to create a PricingBot account and let us do the hard work.