Auditing Canonicals With Screaming Frog [2024]

  • TL;DR – Go to the “issues” reports – Bulk Report – Issues – All –> Export into a folder
  • View canonicals canonicalized reports

Some of the names of the reports can be confusing:

  • The “canonicals missing inlinks” report – is a list of the pages missing canonical URLs, and the inlinks to those pages
  • Canonicals Missing report – as you’d expect – shows you the pages without canonical URLs/tags
  • Canonicals Canonicalised – has pages with canonical to a different URL. So you might have example.com/help/contact – canonicalised to example.com/help – which may or may not be a problem.
  • Canonicals canonicalised inlinks – those pages with canonicals different to their own URL – the inlinks to those pages.

  • Check canonical with JS turned off (using Developer Chrome Extension) & check it remains the same
  • Check the View Source Code with and Without JS turned on – check canonical remains the same
  • Check canonical is not added using JS – this is not idea – more info here
  • Check for multiple canonical URLs using Screaming Frog and check visually in the view source code (JS rendering might be required to see all cononicals)

  • For paginated pages – check if you want each page indexed, that each different page has it’s own canonical URL
  • Check that faceted/filters on pages don’t change the canonical URL (generally you dont want them to)

Exclude these/filter out by, in this instance adding a filter in Excel – does not contain – +

Check this blog post too about auditing canonicals and Hreflang tags

Tools & Checklist for Schema [2023]

I’ve put together a spreadsheet for SEO schema checks here:

https://docs.google.com/spreadsheets/d/1dDqJ9_qrTVoJOZRxieJDp9XE3Q1TCq4ixNVudVtgp2M/edit#gid=0

Use https://classyschema.org/Visualisation and import a URL – test using different URLs, especially for eCommerce websites – like homepage, about page, product page, category pages

Test also with:

https://search.google.com/test/rich-results – add any issues in column C of sheet

and

https://validator.schema.org/ – add issues in Column D

  • Put your “to do” list in columns E and F

When using validator, make sure you’re schema has all the relevant parameters.

The separate sheets on the Google Sheet linked above, has some examples for you to check against – written in JSON-LD

Finally, if the site is already live, check the schema reports in Search Console for any errors

If you have Screaming Frog, I’d also recommend running a crawl –

Go to Configuration – Spider – Crawl – and tick all the “Structured Data” boxes:

crawl schema in screaming frog

Then when the crawl has run – go to Reports – Structured Data and download each report

Finally, you can also check individual pages using the Ryte Structured Data Helper for Chrome –

Check with CHrome plugin – Ryte structured data helper
https://chrome.google.com/webstore/detail/ryte-structured-data-help/ndodccbbcdpcmabmiocobdnfiaaimgnk/related

CollectionPage Schema Markup Example (eCommerce)

Here’s an eCommerce example, that I found on StackOverflow:

<script type="application/ld+json">
{
  "@context" : "http://schema.org",
  "@type": "CollectionPage",
  "name": "Shopify Apps",
  "url": "https://sherpas.design/pages/shopify-apps",
  "description": "We build apps that function and feel natively Shopify",
  "image": "https://cdn.shopify.com/s/files/1/0085/8515/0560/files/logox2_500x500.png?v=1555661781",
  "mainEntity" : {
    "@type":"ItemList",
    "itemListElement":[
      {
        "@type":"ListItem",
        "position":1,
        "url":"http://example.com/coffee_cake.html",
         "name":"coffee cake"

      },
      {
        "@type":"ListItem",
        "position":2,
        "url":"http://example.com/apple_pie.html",
        "name":"apple pie"
      },
      {
        "@type":"ListItem",
        "position":3,
        "url":"http://example.com/blueberry-pie.html"
         "name":"blueberry pie"
      }
    ]
  }
}
</script>

(Thanks stackoverflow and sherpa.design)

You can add an image URL below “name” too^

Here’s an example with elatedLink and IsPartOf schema elements:

<script type="application/ld+json">

{
@context: "http://schema.org",
@type: "CollectionPage",
name: "Shopify Apps",
url: "https://sherpas.design/pages/shopify-apps",
description: "We build apps that function and feel natively Shopify",
image: "https://cdn.shopify.com/s/files/1/0085/8515/0560/files/logox2_500x500.png?v=1555661781",
isPartOf: {
@type: "WebSite",
name: "Sherpas Design",
url: "https://sherpas.design"
},
relatedLink: [
https: //sherpas.design/about-us,
https: //sherpas.design/contact
],
mainEntity: {
@type: "ItemList",
itemListElement: [{
@type: "ListItem",
position: 1,
url: "http://example.com/coffee_cake.html",
name: "Coffee Cake",
image: "http://example.com/images/coffee_cake.jpg"
}, {
@type: "ListItem",
position: 2,
url: "http://example.com/apple_pie.html",
name: "Apple Pie",
image: "http://example.com/images/apple_pie.jpg"
}, {
@type: "ListItem",
position: 3,
url: "http://example.com/blueberry-pie.html",
name: "Blueberry Pie",
image: "http://example.com/images/blueberry_pie.jpg"
}]
}
}
</script>

Here’s an example with a “description” and “brand” – please note – Brand should really be used with Product Schema:

<script type="application/ld+json">

{
"@context": "http://schema.org",
"@type": "CollectionPage",
"name": "Shopify Apps",
"url": "https://sherpas.design/pages/shopify-apps",
"description": "We build apps that function and feel natively Shopify",
"image": "https://cdn.shopify.com/s/files/1/0085/8515/0560/files/logox2_500x500.png?v=1555661781",
"isPartOf": {
"@type": "WebSite",
"name": "Sherpas Design",
"url": "https://sherpas.design"
},
"relatedLink": [
"https://sherpas.design/about-us",
"https://sherpas.design/contact"
],
"mainEntity": {
"@type": "ItemList",
"itemListElement": [
{
"@type": "ListItem",
"position": 1,
"url": "http://example.com/coffee_cake.html",
"name": "Coffee Cake",
"image": "http://example.com/images/coffee_cake.jpg",
"description": "product is yummy",
"brand": "NanesBakes"
},
{
"@type": "ListItem",
"position": 2,
"url": "http://example.com/apple_pie.html",
"name": "Apple Pie",
"image": "http://example.com/images/apple_pie.jpg",
"description": "product is yummy",
"brand": "MumsCakesYo"
},
{
"@type": "ListItem",
"position": 3,
"url": "http://example.com/blueberry-pie.html",
"name": "Blueberry Pie",
"image": "http://example.com/images/blueberry_pie.jpg",
"description": "product is yummy",
"brand": "NanesBakes"
}
]
}
}
</script>

You can add products as the ‘listitems”, although I’m not 100% sure this is best practice:

<script type="application/ld+json">

{
"@context": "http://schema.org",
"@type": "CollectionPage",
"name": "Shopify Apps",
"url": "https://sherpas.design/pages/shopify-apps",
"description": "We build apps that function and feel natively Shopify",
"image": "https://cdn.shopify.com/s/files/1/0085/8515/0560/files/logox2_500x500.png?v=1555661781",
"isPartOf": {
"@type": "WebSite",
"name": "Sherpas Design",
"url": "https://sherpas.design"
},
"relatedLink": [
"https://sherpas.design/about-us",
"https://sherpas.design/contact"
],
"mainEntity": {
"@type": "ItemList",
"itemListElement": [
{
"@type": "Product",
"position": 1,
"url": "http://example.com/coffee_cake.html",
"name": "Coffee Cake",
"image": "http://example.com/images/coffee_cake.jpg",
"description": "product is yummy",
"brand": "NanesBakes"
},
{
"@type": "Product",
"position": 2,
"url": "http://example.com/apple_pie.html",
"name": "Apple Pie",
"image": "http://example.com/images/apple_pie.jpg",
"description": "product is yummy",
"brand": "MumsCakesYo"
},
{
"@type": "Product",
"position": 3,
"url": "http://example.com/blueberry-pie.html",
"name": "Blueberry Pie",
"image": "http://example.com/images/blueberry_pie.jpg",
"description": "product is yummy",
"brand": "NanesBakes"
}
]
}
}
</script>

If you are going to embed “Product” schema within itemlist – validator suggests removing the “position” element:

<script type="application/ld+json">

{

"@context": "http://schema.org",

"@type": "CollectionPage",

"name": "Shopify Apps",

"url": "https://sherpas.design/pages/shopify-apps",

"description": "We build apps that function and feel natively Shopify",

"image": "https://cdn.shopify.com/s/files/1/0085/8515/0560/files/logox2_500x500.png?v=1555661781",

"isPartOf": {

"@type": "WebSite",

"name": "Sherpas Design",

"url": "https://sherpas.design"

},

"relatedLink": [

"https://sherpas.design/about-us",

"https://sherpas.design/contact"

],

"mainEntity": {

"@type": "ItemList",

"itemListElement": [

{

"@type": "Product",

"url": "http://example.com/coffee_cake.html",

"name": "Coffee Cake",

"image": "http://example.com/images/coffee_cake.jpg",

"description": "product is yummy",

"brand": "NanesBakes"

},

{

"@type": "Product",

"url": "http://example.com/apple_pie.html",

"name": "Apple Pie",

"image": "http://example.com/images/apple_pie.jpg",

"description": "product is yummy",

"brand": "MumsCakesYo"

},

{

"@type": "Product",

"url": "http://example.com/blueberry-pie.html",

"name": "Blueberry Pie",

"image": "http://example.com/images/blueberry_pie.jpg",

"description": "product is yummy",

"brand": "NanesBakes"

}

]

}

}

</script>






Here’s is another example – but with more “Product” schema categories/elements – This is now pretty comprehensive, but we could also include “Review”review” “aggregateRating” and “PriceValidUntil”

<script type="application/ld+json">

{
"@context": "http://schema.org",
"@type": "CollectionPage",
"name": "Cakes",
"url": "https://food.com/all-cakes",
"description": "We build cakes that function and feel natively welsh",
"image": "https://cdn.shopify.com/s/files/1/0085/8515/0560/files/logox2_500x500.png?v=1555661781",
"isPartOf": {
"@type": "WebSite",
"name": "Sherpas Design",
"url": "https://sherpas.design"
},
"relatedLink": ["https://food.design/about-us", "https://food.design/contact"],
"mainEntity": {
"@type": "ItemList",
"itemListElement": [{
"@type": "Product",
"url": "http://example.com/coffee_cake.html",
"name": "Coffee Cake",
"image": "http://example.com/images/coffee_cake.jpg",
"description": "product is yummy",
"brand": "NanesBakes",
"sku": "sku1",
"manufacturer": "Manufacturer 1",
"offers": {
"@type": "Offer",
"price": "99.99",
"priceCurrency": "USD",
"availability": "https://schema.org/InStock"
}
}, {
"@type": "Product",
"url": "http://example.com/apple_pie.html",
"name": "Apple Pie",
"image": "http://example.com/images/apple_pie.jpg",
"description": "product is yummy",
"brand": "MumsCakesYo",
"sku": "sku2",
"manufacturer": "Manufacturer 2",
"offers": {
"@type": "Offer",
"price": "99.99",
"priceCurrency": "USD",
"availability": "https://schema.org/InStock"
}
}, {
"@type": "Product",
"url": "http://example.com/blueberry-pie.html",
"name": "Blueberry Pie",
"image": "http://example.com/images/blueberry_pie.jpg",
"description": "product is yummy",
"brand": "NanesBakes",
"sku": "sku3",
"manufacturer": "Manufacturer 3",
"offers": {
"@type": "Offer",
"price": "99.99",
"priceCurrency": "USD",
"availability": "https://schema.org/InStock"
}
}]
}
}
</script>

SEMRush Guide to the Basics (2023)

SEMRush Tech Audit

Use the tech audit to keep an eye on your website.

You’ll need a Google Search Console account to do an SEMRush tech audit.

Go to “site Audit” near the bottom of the side-menu on the left and then click the “+project” button.

Once the site audit has been carried out, you’ll get a load of metrics that score the health of your website from a tech SEO perspective.

Image from SEMRush.com

Check the notices, warnings and errors, to see what you need to fix.

More info in this blog post.

SEMRush Keyword Gap

Keyword gap – add competitors, see what they are ranking for, that you are not.

Filter by search volume of keywords and keyword difficulty.

Keyword gap is also good for Google ads insights. If a keyword is les than a few dollars per click, but is transactional and difficult to rank for organically, it might be worth bidding on in Google ads.

On Page SEO Checker

Gives you ideas including top pages to optimize.

suggests keywords to include on specific pages

suggests backlinks to go after

UX issues can be flagged here too – connect Google analytics. Check out pages with high bounce rate and short time on site.

Organic Traffic insights

Connect Google analytics and search console

Here, you can see pages with good and bad UX metrics and the keywords driving traffic

You can also see which keywords each page ranks for.

Use this report to decide if you should include other keywords on a given page – in the page’s existing content, or create new, more specific pages.

Domain overview

Key metrics, shown at the top include bounce rate and average time on site.

You can see your own, or competitors top pages in terms of traffic

See traffic sources- what percentage of traffic is direct, coming from social media etc. (you may have to click on “Traffic Analytics” in the side menu to see this)

You can compare domain metrics to another competitors, or your own website.

You can use Bulk Analysis to enter up to 100 domains to examine.

Keyword overview tool

Good for finding head/seed keywords.

Can view the Google SERPs and see what the competition for a given keyword is like

See what other keywords competitors are ranking for, on a given URL:

You can see questions related to/containing a given keyword

You can see international search volume for a keyword.

Related keywords are shown too – e.g. “football supplies” is related to “football equipment”

See SERP Features – what rich snippets etc are showing in the SERPs.

Keyword Magic tool

Start with a seed keyword

by default the report will give you broad match keywords.

Be sure to check out “related” match types (all keywords is also helpful)

For KW research – click on one of the main head KWs – and see what keywords your competitors rank for
(clicking on the keyword takes you to the Keyword Overview tool – scroll down to SERP Analysis)

You can add your selected keywords to a list.

More info in the semrush blog post about the tool.

Content marketing

topic research – add a keyword to get topic ideas

For example, if you enter “football equipment” (in the US) the tool gives lots of article ideas around “shoulder pads”, “football helmets”, “football cleats” etc.

Sometimes you get random ideas that you would never have thought of, like the searches around the dog called “pickle” when you look for topic ideas about Pickleball

  • There is loads more you can do with SEMRush, including position tracking. I’ll cover these additional tools in dedicated blog posts.

Keyword Manager

If you have a big list of keywords you want to get search volume for, I like to click “create a regular list” and then name the list and click “Add keywords” in the top right of the screen

Also handy if you’ve done keyword research for one country and need the search volumes for another country

Chrome Develop Tools – for SEO (2023) (inspect element)

For SEO checks you can use in a tech audit, please see table newr the bottom of this post.

Notes from this course on Udemy

Introduction to the Elements Panel

You can view code in Chrome, by right-clicking on a webpage and clicking “view page source”

You can see the code of the page, but you can’t do much with it.

Instead, you can right click and choose “developer tools”

The elements panel is normally the default panel that you see:

As you hover over items in the Elements tab, it will highlight the relevant part of the webpage too.

You can right-click and choose “Inspect Element” and the Dev Tools window will open up, with the relevant piece of HTML code highlighted.
For example, if you click the main header of a page, and choose “Inspect Element”, there’s a good chance you’ll be taken to the elements tab and have a “<h1>” highlighted.

You can change the text if you like, and the style – by typing and editing it in the styles panel on the right:

You can click the “+” symbol to add a new style (I couldn’t be arsed putting this screen snippet into Paint to used proper type-text):

  • You can right click > Inspect an element and click the delete key to get rid of a style
  • Click on the colour picker to get a eye-dropped tool and other colour-picker options to change colours used on the webpage:

Chrome Developer Tools

Click the mobile icon – to see a website previewed on a mobile phone – you can change the phone model with the drop down menu

Console Tab

You can type console.log javascript code. This allows you to see the output of JavaScript code.

Web developers log messages for 2 general reasons:

Sources Tab

Files and folders that have been loaded by the webpage, can be found in the Sources tab – e.g. type Jquery to see if a page is using jquery

It’s often used for debugging JavaScript:

“Top” – The top-level, such as top on the screenshot above, represents an HTML frame. You’ll find top on every page that you visit. top represents the main document frame. (more info here)

The second-level, such as developers.google.com on the screenshot above, represents an origin.

The third-level, fourth-level, and so on, represent directories and resources that were loaded from that origin. For example, on the screenshot above, the full path to the resource devsite-googler-button is developers.google.com/_static/19aa27122b/css/devsite-googler-button.

  • You can click individual files to view them in the preview pane in the middle:
  • Edit CSS and JavaScript

You can edit code and see the changes immediately on the webpage

Image from Developer.chrome.com

CSS changes take effect immediately, no save needed. For JavaScript changes to take effect, press Command+S (Mac) or Control+S (Windows, Linux). DevTools doesn’t re-run a script, so the only JavaScript changes that take effect are those that you make inside of functions.

  • You can save snippets of code and run them on any page

Application Tab

See local storage – Open the local storage section – you can see, well the local storage:

I have no fucking idea what any of this stuff means at the time of writing. I think the {“url” etc is some type of JSON

Screenshot from Mozilla

  • Cookies

You can also see what cookies are loaded:

Screenshot source

Network Tab

Shows a timeline of all the assets that are loaded

Security Tab

You can view security info, like info about the secure certificate/server/https thing

Audits Tab – This doesn’t exist anymore – use Lighthouse/Performance Insights Tab

SEO Checks Using Chrome Dev Tools

Notes from SearchEngineJournal article

1. Check If You Content Is Accessible

The DOM – can be seen in the Elements Tab of Inspect-Element, allows you to:

  • Review various content elements includng meta title
  • Verify if Google can crawl it
  • A key thing to look for here is the differences between source-code and the contents of the elements tab.

    Fucking JavaScript
    When auditing websites that have heavy JavaScript usage, content – such as page headers – sometimes appears fine in the HTML source code. But if injected via JavaScript, it may just show as a JS block in the Elements tab.

This can potentially be an issue, later verified through performing a site:example.com/page “content you’re looking for” advanced search and checking the cached version of the page.

The blog heading here, appears to be fine and stuff:

2. Check on mobile devices

Use the mobile icon on the top menu-bar, next to the “elements | Console” tab options, to see the page in different mobile devices

3. Site Speed

This can really be useful if you’ve blocked your staging website and 3rd party tools can’t access your URL

From the three-dot burger menu in the top right of the screen, you can access Network Conditions (found under More Tools).

you can throttle the network to slow 3G etc

Disable broswer cache

You can also select from a number of user agents ranging from Googlebot Smartphone through to various Microsoft Edge UAs, and Opera.

Once you have setup the setting for the network – go to the NetWork Tab, and see the page load

4. Render Blocking Resources

As mentioned previously, from the Network tab you can identify which JavaScript and CSS resources are being loaded before the DOM.

While this isn’t necessarily an issue on all websites, having resources load ahead of the DOM can potentially lead to it being blocked, which is one of the more common and resolvable site speed issues.

These can be identified by filtering for JS and CSS in the Network tab:

  • You can also see which elements, JS, CSS etc have a 200 status code, and which 404 or are blocked

Notes from another SearchEngineJournal Article

Switch Your User-Agent

In order to switch your user-agent in Chrome, use the Network Conditions tab in the console drawer.

Sometimes security measures on a website might stop Googlebot from crawling

Using the User-Ageny tool in dev tools, it’s possible to see if bots are getting blocked

Diagnosing Core Web Vitals in DevTools

Within the timings section, users can see flags for things like FCP (First Contentful Paint), and LCP (Largest Contentful Paint), one of our Core Web Vitals.

In hovering over the flag for LCP, we can actually see the piece of content flagged to be the largest contentful paint during the page load.

In order to help detect layout shift, use the Rendering tab in the console drawer:

Check the option for Layout Shift Regions, which will highlight areas of the page that undergo a layout shift as content is loaded. (when you go back to the performance tab and refresh)

Layout shifts will then be highlighted in blue as you interact with the page.

3. Double-Check Your HTTP Headers & Review Unused Code

 it’s helpful to see the HTTP response codes for every page and resource.

Go to Network Tab

After selecting a resource, the Headers sub-menu in the Network tab allows us to see a host of information regarding the selected file. (headers appears when you click and choose a file)

This includes all HTTP header information, like the Request URL, the Request Method, the Status Code, content-encoding, last-modified date, server, and so much more.

You can lean on this info to:

  • Verify GZip compression is enabled.
  • Double-check that cache-control is working as expected.
  • Verify we’re sending Googlebot and other user-agents the appropriate status code.

Review Unused Code

Use the console tab

Refresh the page

Click “Issues” section, next to the filter:

chrome dev tools - console - issues

SearchEngineLand notes

  • View the DOM in the Elements panel, to make sure Google can see it

Use this to review the content of any of the page’s elements (such as the title, meta description or page text) to verify that the desired relevant information is shown in each case, to make sure it’s accessible to Google.

When identifying these differences between the content shown in both, you can then take the appropriate steps to verify it’s being indexed correctly: reviewing the page cached version in Google, verifying if the text is being shown in Google’s results when searching for it, using the “Fetch as Googlebot” in Google Search Console and so on.

3. Find Unused JavaScript & CSS

To look for unused JavaScript & CSS open the Chrome Web Devtools, go to elements and press ctrl+shift+P and type coverage; and then select [Show Coverage]

After selecting show coverage you will see the below screen, now you have to click on reload button as you can see in the screenshot and that will finally lead you to the coverage report that will show you the JS & CSS files.

How to use Lighthouse in Google Chrome Dev Tools

Using Dev Tools Console Panel to Run JavaScript

Notes from Udemy course you can find here.

You can do a few nifty things, that seem a bit pointless at this point, like for example, calling and seeing how many paragraph <p> tags are on a page:

There are 5 paragraph tags, in the example shown above^

You can change the background colour of elements, like paragraphs:

Put the elements in a variable, and then change them with a loop^

Here i’ve changed the background colour of the paragraphs on vanilla-js.com:

Sources Tab & Basic Debugging

Right click on an element and select “inspect”

Go to the sources tab

You can click on lines of code to create a “breakpoint” so the code stops on that particular line of code

Breakpoints – pauses the code at a specific point

More info on setting up breakpoint to pause code in the Google documentation here

Line-of-code breakpoints

Use a line-of-code breakpoint when you know the exact region of code that you need to investigate. DevTools always pauses before this line of code is executed.

To set a line-of-code breakpoint in DevTools:

  1. Click the Sources tab.
  2. Open the file containing the line of code you want to break on.
  3. Go to the line of code.
  4. To the left of the line of code is the line number column. Click on it. A blue icon appears on top of the line number column.
A line-of-code breakpoint.

You can view the local and global scope of the code

The “Filesystem” panel, tells you all the files that make up the current page:

Official Google Documentation for Chrome Dev Tools

Google has their own info and tutorial stuff on dev tools here.

Some Tips from Reddit

document.body.innerHTML = [...document.getElementsByTagName("a")].map(el => el.href).join("<br>");

Note from video^

  • Elements tab – we can see HTML representation of DOM – check page elements are in the DOM
  • Network tab – see what happens between the server and our browser.
    Reload page – and see each request from the server and the response
    You can see where time is spent. We can see each element/request – we can see how long it took including the time the server and download took.
    We can also see http/response headers – make sure there’s no X-robots tags if you want it indexed
    You can click on an element/request, then click on “initiator” and see what requested it:

Network tab can also be used to disable the cache, set network speed and the user agent.

Please note that Googlebot might not always work as expected, real Googlebot will respect robots.txt and some sites might do IP lookups to see if a request is really from a data centre.

Some SEO Checks using Developer Tools:

Dev Tools Checks
Inspect main elements – are they visible in the inspect window? e.g. right click and inspect the Headings – check <head> has meta title and desc
Check on mobile devices
Check all the elements result in a 200 – view the Network tab
Network Tab – Check with Googlebot
Console tab – refresh page – what issues are flagged?
Unused JS in the elements tab – coverage
Check for client rendered links using the code above , using the console tab

You can also use a “diff tool” to check the difference between a live DOM copied from dev tools and the view source code.

Easy Way to Analyse DOM Size (For SEO Audits & Speed etc)

Add the chrome extension here – https://chrome.google.com/webstore/detail/dom-size-analyzer/mcneiimlodlbmohipgdbbglgbmaoojen

when you right click > Inspect

In the menu with the two arrows >> – you can click that and it has an option to analyze the DOM – which then gives you the number of elements and a few other bits

Just click the arrows, and then “Analyze DOM Size” and click the play icon

  • You can also right-click and inspect and type:

SEO Reports in Google Analytics 4 (GA4) [2023]

SEO Landing Page Report

Go to Reports > Engagement>Pages and screens

click the pencil icon to customize

Click Dimensions – search and add “Landing page + query string”

Delete other dimensions

Click APply

Click MEtrics

Remove “views”

Add “sessions”

Drag sessions to top.

Click Apply

Click “Add filter” on the right

Search for/add “session default channel group”

Tick “organic”

Click “OK”

Click Save on the top middle-right

Click “Save as new report”

Name is “Landing Pages – Organic” or something

To get the report to appear in the sidebar menu

Click Reports in the left-side bar and then “Library “

Under “life cycle”

click “Edit collection”

Click “Create New Topic”

On the right search for the Landing Page – Organic report

Drag it into the new SEO folder you made (sometimes this doesn’t work as GA4 has a bug at the time of writing)

It worked for me when I dropped it into the second rectangle

Click “back” on the top left

and you should now see an SEO section to the reports.

“SEO Report 2 – Devices or Organic Search Traffic

Go to the Reports>User>Tech >Tech details

Click the Pencil icon to edit the report

Add a filter

Add “session default channel group”

and then Tick “organic” (search for it, if it’s not there for some reason)

Go to Dimensions in the right side bar

Click the 3 dots next to “device category”

Choose “set as default”

Then drag “device category” to the top

Click “Apply”

Click “Save”

Save as a new report

Name it – “Tech details – Organic” or something

  • at the time of writing GA4 is buggy as f*ck, and it won’t let me save this report

Go back

Go to Reports > Library (folder icon near bottom)

Add to the Life Cycle collection

Drag the report into the SEO folder.

SEO Report 3 – Search Engines

Go to Reports>Acquisition>Traffic acquisition

Click the pencil icon to customise the report

Delete on the dimensions in the top right, except for “Session Source”

Click Apply

Add a filter – session default channel group and then select “organic Search”:

Go to reports – click “library” icon at the bottom

Edit the life cycle reports

drag in the new report to the “SEO” section you made for the first report

Google Search Console Reports

Link GA 4 to Search Console

Go to Admin (blue and white cog incon on bottom left of screen)

On the bottom right click “search console links”

Choose the account and web stream and you’re done.

You might need to wait 24 hours for the data to get imported.

Go to Reports > Library (folder icon near bottom)

Life cycle – edit collection

In the top right search for “Google”

You should see 2 reports – Google organic saerch traffic and Queries: organic Google search queries

Click Save – save current collection

SEO Behaviour Exploration Report

Go to “EXPLORE>Path exploration > Start over (top right)

Click the starting point and then select / for homepage

Filter to show organic traffic only

Go to SEGEMENTs (left hand side)

Click the + icon

Click “Session segment”

Click “Add new condition” on the left – add “session default channel group”

Add filter by clicking the box on the right – contains – organic search – click “apply”

Name is “organic search visitors” – “Save and apply”

You can’t add this report to the sidebar (sorry) – so to see it you’ll have to always click into the Explore section.

Using Get Stat to Monitor Rankings

Home – click on a website to view their rankings

Dashboard Tab

Graph at the bottom is arguably the most helpful.

  • Ranking Averages vs. Distribution Graph
  • If you use the date-picker to a date range of less than 2 weeks, you see daily fluctuations
  • It can be easier to click the # on the bottom left and view the number of KWs in the top 3

Keywords Tab

Click “show/Hide” on top right

Click “google base rank” and “ranking change” and “ranking URL”

Click on the “change” column. Those keywords with a positive change number, have gone down in rankings

Click on each Keyword to see ranking changes over a set date-range (use date-picker on top right)

Competitive Landscape

Compare latest column, to what the share of voice was 7 days ago, 30 days ago, up to 180 days ago.

Click the “site” menu on the top left column of the whole GUI – and pick a category of keywords to see specific changes

Source HTML vs Rendered HTML

Summary

For an SEO Audit

You want to check that href links and H1s etc are in the source and the rendered HTML

  • Rendered HTML, is basically HTML after all the JS has run.

You can view rendered HTML using right click and “inspect” from the drop down menu
This shows the Live DOM, which includes changes made by JavaScript.

  • Source HTML is static HTML, as it was sent from the server to your browser.

This is the raw HTML before anyclient-side JavaScript has been run/executed.

You also want to check that the rendered HTML is not too different to the source HTML.

View the site with JS turned off. Ideally images should still be there and href links should definitely still work with JS turned off.


Rendering process – start with the source HTML and CSS.

The browser parses the source html and CSS which creates the DOM and CSSOM.

The DOM is more important for SEO

The browser creates a DOM tree to identify elements on the website and where everything belongs and relates

The browsers takes the DOM and CSSOM and creates a render tree to lay everything out.

The tree is then used to create the actual pixels and layout of what you see.

JS may then be added to manipulate the DOM

You can turn the final DOM Tree, back into HTML – this is Rendered HTML

When you click View Source in your browser

This shows the SOURCE HTML

Developer Tools – Sources – SOURCE HTML

Network Tab in developer tools – SOURCE HTML

In the elements tab in Chrome in Dev Tools – you get the current DOM Content

https://support.google.com/webmasters/answer/11626894?hl=en

Overview

When you visit a page, the website sends HTML code to your browser. Often this source code includes additional resources such as scripts, which must be loaded, and which may alter the page code.

How to view Source HTML (Ctrl + U)

Right-clicking “show source” typically shows only the original page code returned to the browser, before scripts and other resources have been loaded and run. But there are many instances, particularly when troubleshooting your page, when you need to see the code of the final, rendered page on the browser, after all resources have been loaded, and any scripts run. For example:

  • To search for Google Analytics or Google Tag Manager tags used in verification.
  • To debug page loading and display (that is, to check that all libraries and other resources that you want to be loaded are).
  • To look at structured data on the served page.

How to view the rendered source (Inspect Element)

Here are a few methods to view the rendered source code for a web page:

  • In the Chrome browser: Right-click any part of the page and select Inspect to see all the HTML from the rendered page. Search for items in the rendered HTML with Control + F (Windows) or Command + F (Mac).
  • For a page on your own site:
    1. Inspect the URL, either by entering the URL directly in the URL Inspection tool, or by clicking an inspection link next to a URL shown in most Search Console reports.
    2. Click Test live URL > View tested page.
    3. The HTML tab shows the rendered HTML for the page.
  • For a page on any site, not just a site that you own:
    1. Open the Mobile-friendly Test.
    2. Enter the URL of the page. The page must be available to Google without a login, and not blocked by robots.txt.
    3. Click View tested page.
    4. The HTML tab shows the rendered HTML for the page.

Creating an Actionable 404 Report from Screaming Frog

Update – I don’t think all the process below is required.

Just download the 404 – inlinks report from Screaming Frog

Bulk Export (very top, slightly to the left on the GUI)> Response Codes > Internal > Client Error 4**s

Copy the “Destination” (column C on report) column and paste into an new Excel tab/sheet and remove duplicates

In the first sheet, copy and paste the source column into column D

In the second sheet, do a vlookup using the first destination URL, and “lookup” in the first sheet – columns C and D, to return the relevant source URL

Copy the vlookup and Paste – values – into column A into the second sheet

You can also copy and paste the anchor text and location into column C

Follow this protocol, to produce a sheet you can send to devs etc, to remove 404s

  • This will get rid of the site-wide 404s and some individual 404s

Run a crawl with Screaming Frog

Export the report –> Screaming Frog – Bulk Export  – Response Codes – Internal – Internal Client Error (4xxs)  (check 500s too)

In Excel – Copy and paste the “destination” URLs into a new sheet – into column A

Remove duplicates from the destination URLs that you’ve just copied into a new sheet

rename the colum – 404s Destination

  • Copy and paste the Source URLs and the Anchor Text into a new sheet.

Paste Source URLs in column A, and Anchor Text into column C

In cell B1 type – ” | ”

In cell D1 – give the column the heading “Source | Anchor”

In cell D2 concatenate – =CONCATENATE(A2,$B$1,C2)

Drag the formula down.

You’ll now have the anchor text and the source URL together, so you can vlookup the destination (404) URL

  • create a new sheet
  • Copy and paste all of the source URLs | Anchor Text (from the concatentate formula – paste special -values only
  • Copy & Paste Destination URLs from the original sheet into columns B and C in the new sheet you just made.


You need “destination” in column B and “Source | Anchor Text” in column C, as vlookup has to go left to right

  • So you’ll have – 404s Destination – Destination – Source | Anchor Text

Name column D in the new sheet “Example Source URL & Anchor Text” and in cell D2 enter the lookup – VLOOKUP(B2,B:C,2,0) (put “equals sign” before the V. Drag the formula down

Copy column A and paste into a new sheet. Name the sheet “Final”.

Copy column D with the vlookup and paste values into column B in the “Final Spreadsheet”

In “final”, you should now have all the unique 404s and an example of a page that links to those 404s with the anchor text.

  • You can use “text to columns” to seperate the source URLS and anchor text if you wish
  • If you’re sending the spreadsheet onto a dev or someone to fix the 404s, you are probably best sending the full sheet with all the inlinks to the 404s, plus the one you’ve just made. It depends how they go about fixing the 404s.

    Once 404s have been fixed, rerun a crawl and recheck them.

look out for 404s that are classed as  HTTP Redirects in the “type” column – these don’t seem to have a unique source URL. You may have to search for the URL in the search box in Screaming Frog and click the “inlinks” tab to see original link to the non-secure http page

If you like, before you send off the report to someone, you can double check the “destination” URLs definitely are 404s, by pasting them into screaming frog in “list” mode