Bulk checking Canonical URLs & Hreflang Tags for SEO on an eCommerce Store (With Screaming Frog)

On By AndyIn Screaming Frog, seo

TLDR – Easiest way to check hreflang is to just go to reports > Hreflang > non200 Hreflang URLs:

Filter “Source” column to your current domain and then you should see all the hreflang that 300 or 404.

  • However – you ideally want to get all of your indexable, 200 status code URLs – to have the relevant hreflang tags.
  • So first find all the indexable, 200, URLs with exact-match canonicals
  • Then check the hreflang on these only
  • Get all the URLs that 200 and are indexable from “internal_all” report
  • Paste in list mode, all of these URLs – check hreflang tab
  • Perform a Crawl Analysis
  • Check The overview panel

hreflang screaming Frog
  • Order the Hreflang tab by occurrences to double check tall URLs have at least 1 hreflang

Checklist here – https://docs.google.com/spreadsheets/d/1IHRZNFravVKaZ5T9uOEOg_zQ5ee8dHcRP8m92vkn4KY/edit#gid=0

Configuration – spider – crawl and store Hreflang

Find URLs that should have a hreflang tag – Indexable URLs

  • Crawl the website with Screaming Frog – in configuration crawl and store Hreflang
  • Download the “internal” report on first tab
  • Filter relevant columns to find all URLs that return 200 (Status Code), are indexable (Indexability)

Check Canonical URLs are Correct and Indexable

  • From the internal Report – filter status code to 200 only and Indexable
  • Paste the filtered columns into a new tab/sheet and delete irrelevant columns so you just have URL and Canonical columns
  • Use =Exact formula to find all URLs that match the canonical URL – add an extra column for this and call it “exact?”
  • For canonical checks – filter to “FALSE” and see which URLs don’t have self ref canonicals

Back in Internal – Original Spreadsheet

  • unfilter the internal report.
  • Filter to show all the canonical URLs. Canonical Column – filter – untick “blanks”
  • Copy and paste into Screaming Frog , list mode – check status code of all these canonical URLs

  • For Hreflang – Filter Canonical URL – “exact?” column to “true”
  • You should be left with all URLs that are indexable (and should therefore have hreflang if an international site) – i.e. 200 status codes and URLs that exactly match canonical URLs.

Hreflang Checks

For the indexable URLs found above, check hreflang using:

https://technicalseo.com/tools/hreflang/

https://www.hreflangs.com/?utm_source=Weglot&utm_medium=blog&utm_campaign=hreflang-tag
  • Go to screaming frog GUI – and the Hreflang Window

Order by Occurrences – check if any URLs have less hreflang then they should have

  • Check all the reports in the drop down menu on the left

Non-200 Hreflang URLs – URLs within the rel=”alternate” markup that don’t result in a 200 status code

Download the non-200 hreflang report – by going to the “hreflang” tab in Screaming Frog.

Delete all the columns except those containing the actual hreflang URLs

Then in the column adjacent to all the hreflang URLs add the formula:
=INDEX($A$1:$G$126,CEILING(ROW()/7, 1),MOD(ROW()-1,7)+1)

The formula above would be used if the hreflang URLs are in columns A to G, and the last row that is populated in number 126.
You’ll also need to update the number 7 in the formula – for example if you have 5 columns of hreflang URLs, then change 7 to 5.

Drag this formula down until you start seeing #REF! errors, indicating you’ve covered all the data in your original columns

  • You should now have all the hreflang URLs listed in the non-200 hreflang report
  • To find the specific URLs that are non-200 – paste the last column containing all the URLs, into screaming frog in “List” mode

Export the List mode ‘crawl’ from Screaming Frog and filter by status code

hreflang-columns-formula-1Download

  • Unlinked Hreflang URLS – Page sthat conain one or more hrelgang tag / URL that’s only linked to by a hreflang tag and not in the actual webpages

  • Missing Return Links – Hreflang should be recipricol.
  • Inconsistent Language & Region Return Links – This filter includes URLs with inconsistent language and regional return links to them. This is where a return link has a different language or regional value than the URL is referencing itself
  • Non Canonical Return Links – URLs with non canonical hreflang return links. Hreflang should only include canonical versions of URLs.
  • Noindex Return Links – Return links which have a ‘noindex’ meta tag. All pages within a set should be indexable, 
  • Incorrect Language & Region Codes – This simply verifies the language (in ISO 639-1 format) and optional regional (in ISO 3166-1 Alpha 2 format) code values are valid
  • Missing Self Reference – URLs missing their own self referencing rel=”alternate” hreflang annotation. It was previously a requirement to have a self-referencing hreflang, but Google has updated their guidelines to say this is optional. It is however good practice and often easier to include a self referencing attribute.
  • Not Using Canonical – URLs not using the canonical URL on the page, in it’s own hreflang annotation. Hreflang should only include canonical versions of URLs.
  • Missing – URLs missing an hreflang attribute completely. These might be valid of course, if there aren’t multiple versions of a page.
  • Outside <head> – Pages with an hreflang link element that is outside of the head element in the HTML. The hreflang link element should be within the head element, or search engines will ignore it.

Official documentation from Screaming Frog here.

You can also use this Google sheet to check hreflang further

Leave a comment