Semantic Search & SEO – Using Python and Google Colab

Semantic search adds context and meaning to search results. For example, if someone is searching for “Lego” – do they want to buy Lego toys, or see a Lego movie or TV show (Ninjago is great). Another example might be “Tesla” – do people want to see the latest self-driving car, or learn more about Tesla the scientist and inventor?

  • Make sure you understand search intent and any confusing searches like Tesla(inventor or car?), Jaguar (car or animal?), etc
  • Look for structured data opportunities
  • Optimise internal links – especially if you are using a “Pillar Post” and “Cluster Page” structure
  • Follow traditional on page SEO best practices with headers, meta titles, alt tags etc

SMA Marketing have done a cool YouTube video about Semantic Search and they recommend tools including:

  • Wordlift
  • Frase
  • Advanced Custom Fields for WordPress
  • Google Colab with a SpaCy

Before you publish a post – look at the search results for the keyword(s) you are optimising the post for. Check in incognito in Chrome to remove most of the personalisation of the results.

For any answer boxes or snippets, you can click the “3 dots” to get information about the results:

As well as the snippets, you can click the 3 dots next to any organic result. Here’s another result for “MMA training program pdf” with some additional information:

With this in mind – if you are looking to rank for “MMA training program pdf” then you will want to include the search terms highlighted in the “About this result” box: mma, training, program, pdf and ideally LSI keywords “workout” and “plan”.

It’s also a good idea to scroll down to the bottom of the SERP and check out the “related searches”

Take a look too at any breadcrumb results that pull through below the organic listings. Combining all this information will give you a good idea as to what Google understands by your search query and what people are looking for too.

Semantic Search & NLP

This is a bit techy, but thankfully, the guy at SMA Marketing (thank you if you’re reading this) has put together a file/load of python code that does most of the work for us. You can find it here – https://colab.research.google.com/drive/1PI6JBn06i3xNUdEuHZ9xKPG3oSRi1AUm?usp=sharing#scrollTo=uatWEoHp5nxZ

Hover over [1] and click the play icon that appears (highlighted yellow in screenshot below)

When that section has finished loading and refreshing, scroll down to the “Installation tensorflow + transformers + pipelines” section and click the play icon there.

When that’s finished doing it’s thing, scroll down again, and add your search query to the uQuery_1: section:

add your query and then press the “play” button on the left hand side opposite the uQuery_1 line

You should then see the top 10 organic results from Google on the left hand side – in the form of a list of URLs

Next, you can scrape all the results by scrolling down to the “Scraping results with Trafilatura” section and hover over the “[ ]” and press play again:

Next, when the scraping of results is done – scroll down to “Analyze terms from the corpus of results” section and click the play button that appears when you hover over “[ ]”

Next! when that’s done click the play button on the section full of code starting with:

“df_1[‘top_result’] = [‘Top 3’ if x <= 3 else ‘Positions 4 – 10’ for x in df_1[‘position’]] # add top_result = True when position <=3 “

Finally – scroll down and click the play button on the left of the “Visualizing the Top Results” section.

On the right hand side where it says “Top Top 3” and lists a load of keywords/terms – these are frequent and meaningful (apparently) terms used in the top 3 results for your search term.

Below that, you can see the terms used in the results from 4-10

Terms at the top of the graph are used frequently in the top 3 results e.g. “Mini bands”

Terms on the right are used frequently by the results in positions 4-10

From the graph above, I can see that for the search term “resistance bands” the top 3 results are using some terms, not used by 4-10 – including “Mini bands”, “superbands” “pick bodylastics”

  • If you click on a term/keyword in the graph – a ton of information appears just below:

e.g. if I click “mini bands”

Google Colab TOol

It’s interesting that “mini bands” is not featured at all in the results positioned 4-10

If you were currently ranking in position 7 for example, you’d probably want to look at adding “mini bands” into your post or product page

You can now go to the left-side-bar and click “Top 25 Terms” and click the “play icon” to refresh the data:

Semantic SEO tool

Obviously – use your experience etc and take the results with a pinch of salt – some won’t be relevant.

Natural Language Processing

next click on “Natural Langauge Processing” in the side-menu

Click the “play” icons next to “df_entity =df_1[df_1[‘position’] < 6]” and the section below.

When they have finished running click the play icon next to “Extracting Entities”

Click “play” on the “remove duplicates” section and again on the “Visualising Data” section

This should present you with a colourful table, with more terms and keywords – although for me most of the terms weren’t relevant in this instance 😦

You can also copy the output from the “Extracting the content from Top 5” section:

Python Google Colab
Then paste it into the DEMO/API for NLP that Google have created here:

https://cloud.google.com/natural-language#section-2

You can then click the different tabs/headings and get some cool insights

Google NLP API

Remember to scroll right down to the bottom, as you’ll find some additional insights about important terms and their relevance

The Google NLP API is pretty interesting. You can also copy and paste your existing page copy into it, and see what Google categories different terms as, and how “salient” or important/relevant it thinks each term is. For some reason, it thinks “band” is an organisation in the above screenshot. You can look to improve the interpretations by adding relevant contextual copy around the term on the page, by using schema and internal links.

Quickly Scrape SERP Results for SEO

Go to Google.com

Press CTRL+D to bookmark the page

Add it to your bookmarks bar if possible

Copy the code below:

javascript:var a = document.getElementsByTagName('a'), arr = '';for(var i=0; i<a.length; i++) if (a[i].ping && !a[i].href.includes('google'))arr +=('<p>' + a[i].href + '</p>');var newWindow = window.open();newWindow.document.write(arr);newWindow.document.close();


Right click on the bookmark and click “Edit”

Where it says “URL” paste in the code

Rename the bookmark “URL Extractor”

Save the bookmark

That’s it!

Test the URL Extractor – do a Google Search and then click on the Bookmark you’ve just created

You should now have a new tab with all of the URLs listed in it

Change your Google Search settings to show 100 results – you can just copy and paste the tab full of extracted URLS into Google Sheets

Once in Google Sheets, you can easily extract the Meta Titles, Descriptions and keywords

You just need to find and replace the URL in these formulas:

=importxml("https://blackbeltwhitehat.com","//title")
=importxml("https://blackbeltwhitehat.com/","//meta[@name='description']/@content")
=importxml("https://blackbeltwhitehat.com/","//meta[@name='keywords']/@content")

=IMPORTXML("https://blackbeltwhitehat.com/","//h1")

Google Analytics 4 – Quick Notes & Introduction 

The new version of Google Analytics – Google Analytics 4 – has less pre-made reports and users are prompted to customise their own dashboards and use the search function.

You can literally ask questions, using the search bar. Which is fun.

Instead of pageviews and sessions, GA 4 is built around events.

Pre-configured reports are limited in GA 4. It’s a good idea to have Universal Analytics in addition to GA4

  • You can’t yet link GA 4 to Search Console
  • You can Can store raw data in BigQuery

when you install GA4 on your site, the reports don’t import data from GA universal – start from scratch – no historical data is passed over to GA4 when you install it on your site.

  • GA4 is Built Around Events.

Events:

You can track pageloads, elements clicked, product details, and loads more

Parameters:

Parameters are info that are sent to GA4 with the events. 

Eg. Pageview is sent to GA4 with URL of the page, page title and the referral details

Automatic Events:

First Visit – first time someone visits site (this even populates the new user report too)

Page View – same as normal pageviews

Session Start – new session after 30 mins of inactivity

User Engagement – starts whens someone on your site for at least 10 secs

Enhancement Measurement

GA4  automatically tracks:

Scrolls – 90% of a page

Outbound Clicks –

Site Search – search queries

Video Engagement – for embedded YouTube videos

File Downloads

To edit the Enhancement Measurement reports go to “Data Streams” in the Admin area:

Click on data stream and check to see that “Enhanced Measurement” is switched to the on position. If you click on the cog icon, you can then switch different measurements on and off.

They’ll be set to “on” by default.

There are advanced settings for pageviews and site search – just click “Show advanced settings”. 

Google provides a list of recommended events for all websites and apps, and then by industry.

Retail & eCommerce

Jobs, Education, Local Deals & Real Estate

Travel 

Games

Custom Events

Custom events allows you to name your events.

Google’s naming convention is probably best to copy – [action]_[object/item]

For example, if you create an event to track when people rate your website, call it:

Rate_item 

Instead of

Product-rate

Add parameters such as product, item_id and rating – so you know which product is rated and what rating the user gave it.

You’ll need to register the parameters as custom dimensions or metrics

To make a new event – in the left hand nav/side bar – click – 

Events – Create Event – Name the event

Enter the parameters to tell Google when to trigger the new event

E.g. page_location – contains – thank-you

Click “create” in top right corner.

You can create brand new events in Tag Manager – rather than basing them on existing events

Data Settings>Data Retention

By default GA4 only stores data for 2 months – in the Admin area, you can change this to 14 months.

Go to Admin>Data Settings>Data Retention and change it to 14 months on the drop down menu.

Brief Run Down of GA4 Reports (in the left hand side-menu)

Acquisition Reports – Where are visitors coming from?

Engagement Reports – What are visitors doing on your site?

Monetization – New term for eCommerce Reporting in GA 4

See what items people are buying and how much money their spending

Retention – Looks at new users/visitors and returning visitors. See what percentage of different cohorts come back to the site etc

Demographics – Where are people based and what type of people are coming to your site?

Tech – what computers, tablets and phones are people using?

EVENTS

Conversions – conversions can be customised. But typically include “Begin_checkout” and “purchase”

All Events – basically less important events – compared to conversions. Include clicks, scrolls etc.

EXPLORE

Analysis – There is a Template Gallery (top right of screen) 

You can look into funnels, acquisition etc and get cool reports

CONFIGURE 

Audiences – you can build different audiences by location, device etc and analyse them

Data Studio – Quick Tips (Advanced)

Speed Up Data Studio Reports (Significantly) – Extract Data

To speed up your reports – you can “Extract Data” and cache it.

It can help to have 2 copies of the report up – so you can see which metrics and dimensions you need to select when adding the data to extract and cache (also a good idea to test the extract data method on a copy of the report in case you faff anything up)

Go to “Add Data” in the top menu-bar

  • Click on “Extract Data”
  • Choose the data you need – eg Google Analytics
  • Add the dimensions and metrics you need for the report
  • On the Right hand side – click to turn “Auto Update” on
  • Select “daily”
  • Click “Save and Extract”

Sometimes you have to faff around a bit with the dimensions – Google Analytics doesn’t seem to like caching a dimension, but still goes super-quick if you cache the metrics only.

Edit in Bulk

If you want to edit all of the charts or tables on the page, in “Edit” mode, right click – go to “Select” and then choose “Tables on page” or whatever type of chart, scorecard or table you’ve selected.

This works instead of CTRL clicking or SHIFT clicking – but you can only change charts or visualisations of the same type at the same time. You can change the style, add a comparison date range etc.

Brand Colour Theme in Data Studio

Click on “Them and Layout” at the top of the screen and then, near the bottom right click “Extract Theme from Image” – you can then upload your logo and choose a theme with your brand colours.

If your shite at presentation like me, this is helpful.

Copy & Paste Styles

In Data Studio – If you want to copy a style from a chart or table, right click it, then choose “copy”

Click another chart/table and the right click – Paste Special – Paste Style Only

Add Chart Filters to an Entire Report

If you want to add a filter to all the data in a report, then it can be a pain going through the charts individually.

Right click on a blank part of the page –

  • Click “Current Page Settings”
  • On the right hand side – click “Create a Filter”
  • Choose or create a filter to apply to all the page

To add a filter to multiple pages

  • Right click on a blank part of the page
  • click “Report Settings”
  • click “Add a filter” in the right side-menu

Add Elements to All Pages of a Report in Data Studio

If you want to add a header and date range selector, for example, to all the pages in the report – add the elements to a page, then right click on the element – and choose “Make report-level”

Quickly Align Elements in Data Studio

Click and drag to select all the elements

Right click – choose “align” – “middle” to get everything inline horizontally

To get an equal space between all the elements, so they’re spaced evenly:

– click and drag to select the elements

– right click – select “Distribute”

– “horizontally” to space evenly across the page, or “vertically” to distribute evenly in a vertical manner.

You can also tidy up individual tables to align the columns vertically – right click and select “”Fit to data”

See our Data Studio Fields & Filters blog post – https://businessdaduk.com/2021/12/15/data-studio-fields-filters-notes/

Online Marketing for New Businesses

Started a business?

Got a website?

Okay, now you need to look at building your brand and getting traffic and engagement on your website and social media accounts.

Website Checklist

  • Trust signals – include membership badges, university logos prominently
  • Humanize the site – include images and videos of people who work for the business
  • Social Proof – include reviews and testimonials. TrustPilot and video testimonials work well
  • Site Speed – make sure the site is quick and works perfectly on mobile devices
  • EAT – Expertise Authority & Trust – show your credentials on the about us page
  • Contact Form – Make it as easy as possible to be contacted
  • Trust & Transparency – Include full contact info if possible – address, tel number, email
  • Have you got Google Analytics & Search Console installed?

You will also need to think about the colour scheme and imagery.

Think about what mood you want to portray

https://www.bedtracks.com/blog/2017/6/15/how-to-coordinate-music-with-the-colour-in-your-video

  • Strong Call To Action

You will probably want a “Call to Action” or “CTA” button, such as “Buy Now”, “Learn More” or “Contact Us”

This CTA button is generally placed “above the fold” on most pages, so that people don’t have to scroll down or look for a way to get in touch or buy from you.

“Join Free for a Month” – is the CTA on Netflix’s homepage (at the time of writing)

For more expensive, high end or though-out purchases such as – buying a car or contacting a therapist, sometimes it’s better to have the CTA below the fold. The best thing to do is test it, with Google Optimize.

For more information about “Conversion Rate Opimization” (CRO), see this article:

https://blog.hubspot.com/marketing/conversion-rate-optimization-guide

For a full SEO (Search Engine Optimization) checklist for your website – to help get visibility on Google – see this article – https://backlinko.com/seo-checklist

Google My Business

Register your website and your office with Google My Business

You can go through the steps here:

https://www.google.com/intl/en_uk/business/

Google will send out a postcard to your office (or home) address

The postcard has a code – so you can confirm you are at that address

Local Directories

Register your business with high quality, local directories such as

  • Bing Places
  • Yelp
  • Yell
  • Free Index
  • Open Di

Try and get on any local government directories too.

Social Media & Captioned Videos

If relevant, register your business on:

  • Linkedin
  • Facebook
  • Instagram
  • TikTok


    Arguably the best way to get noticed on social media at the moment, is to create videos with captions – so they can be watched on mute.

Linkedin is said to have the greatest organic reach at the moment too – meaning you can get your video, image or text-post in front of more people, without paying for ads.

Social media sites like people posting videos too – because they drive a high rate of engagement and keep people on the site for longer.

  • Do NOT post to YouTube and then post a link on social media

Instead – upload your video direct to the platform.

Social media sites will tend to kill your reach if you post a link – they don’t want people to click and leave their website

YouTube is also showing on more and more Search Results Pages on Google.

Consider creating a YouTube channel with lots of informative, helpful and entertaining content. You can then edit the videos and post to specific social media platforms.

Jab, Jab, Jab – Right Hook

General principle of content and social media marketing by Gary V.

Identify your target market

Identify their issues and pain points

Post helpful content related to their pain points and problems

Do NOT constantly promote your business – slip in the odd “Right Hook”, every 3 or 4 posts

People do not want to be sold to constantly, they want helpful, insightful and funny content.

For example.

If you target market is small business owners, take a look on Quora and Reddit and see what people are talking about. If a common theme is Facebook advertising for example, make some helpful videos and blog posts about Facebook marketing.

SEO, PPC and More

The above is just a foundation.

If you have the time and resources, you will ideally produce lots of insightful blog content, earn lots of inbound links and work your way to the top of Google.

You will also want to consider “PPC” – Pay Per Click ads on Google, Facebook and Linkedin.

One beginner mistake to avoid with ads – is sending people to your homepage.

Have a specific “landing page” for each advertising campaign.

oh – make sure you have a good looking logo too. You can use Canva or hire someone on PeoplePerHour.com

A good place to start with SEO is to check your website using an On-Page SEO Checklist.

Videos are great for social media, and YouTube is also starting to show more and more often in the Google results. I would personally have a good go at gaining an online presence using videos and social media – particularly Linkedin at the moment.

Build a Brand

Here’s a good article that some hero wrote about building your brand as a small business

  • Nail down your USP
  • Identify other propositions “why use me/us and not the competitor?”
  • Write down your brand story
  • Use high quality photography & videography (avoid stock pictures)

Consider making customer support a key element of your brand – this can help with online reviews too. Pre-purchase, purchase and post-purchase consumer stages are all opportunities to impress and help.

Get Content Ideas from Competitor Websites (SEMRush)

Requires:

  • SEMRush
  • A computer
  • The internet

If competitor has articles and blog posts inside a subfolder e.g. “buyers-guides” or “/blog” – make a note of the sub-folder name

  • Add Competitor’s homepage URL in “Search Bar” on SEMRush Homepage
  • Click “Organic Research”
  • Click on the “Positions” tab
  • Click on “Advanced Filter” and add the subfolder name e.g. “blog”
  • Export Results into Excel
  • Create a Pivot Table
  • Use the settings below – you’ll need to change position to “average” instead of “sum of”:
  • Tick the check-box for “URL” at the top
  • Drag search volume, traffic and position into “Values” box at the bottom right
  • Click the little arrow on the right of “Sum of Position” – go to “Value Field Settings” and choose “average”
  • Analyse which articles get the most traffic (approximately) and have most potential

Obvs. the URLs of my competitor have been blacked-out in the image above

If the competitor has all their articles at the root domain level e.g.

https://barbend.com/best-weightlifting-belts/

Just use SEMRush – Organic Research – Positions tab and download and pivot the pages data – no need for advanced filter

Google Ads Reporting Script

This script is no longer live, but I had it in my account, I’m pasting it here for safekeeping and future reference as it is pretty handy:

to use the script:

Change the Google sheet/doc URL and the email address

  • Sign into Google Ads
  • Click on “Tools & Settings” on the top menu near the right hand side
  • Click “scripts”
  • Click the addition (+) sign to add a new script
  • Paste in the below script – save – preview and then run

// Copyright 2015, Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

/**
 * @name Account Summary Report
 *
 * @overview The Account Summary Report script generates an at-a-glance report
 *     showing the performance of an entire Google Ads account. See
 *     https://developers.google.com/google-ads/scripts/docs/solutions/account-summary
 *     for more details.
 *
 * @author Google Ads Scripts Team [adwords-scripts@googlegroups.com]
 *
 * @version 1.1
 *
 * @changelog
 * - version 1.1
 *   - Add user-updateable fields, and ensure report row ordering.
 * - version 1.0.4
 *   - Improved code readability and comments.
 * - version 1.0.3
 *   - Added validation for external spreadsheet setup.
 * - version 1.0.2
 *   - Fixes date formatting bug in certain timezones.
 * - version 1.0.1
 *   - Improvements to time zone handling.
 * - version 1.0
 *   - Released initial version.
 */

var RECIPIENT_EMAIL = 'drewgriffiths@live.com';

var SPREADSHEET_URL = 'https://docs.google.com/spreadsheets/d/1mNjc7iJWOIq580DMLf6rYKT8xRAcl2MynnGSY29UiXY/edit#gid=3';

/**
 * Configuration to be used for running reports.
 */
var REPORTING_OPTIONS = {
  // Comment out the following line to default to the latest reporting version.
  apiVersion: 'v201809'
};

/**
 * To add additional fields to the report, follow the instructions at the link
 * in the header above, and add fields to this variable, taken from the Account
 * Performance Report reference:
 * https://developers.google.com/adwords/api/docs/appendix/reports/account-performance-report
 */
var REPORT_FIELDS = [
  {columnName: 'Cost', displayName: 'Cost'},
  {columnName: 'AverageCpc', displayName: 'Avg. CPC'},
  {columnName: 'Ctr', displayName: 'CTR'},
  {columnName: 'Impressions', displayName: 'Impressions'},
  {columnName: 'Clicks', displayName: 'Clicks'}
];

function main() {
  Logger.log('Using spreadsheet - %s.', SPREADSHEET_URL);
  var spreadsheet = validateAndGetSpreadsheet();
  spreadsheet.setSpreadsheetTimeZone(AdsApp.currentAccount().getTimeZone());
  spreadsheet.getRangeByName('account_id_report').setValue(
      AdsApp.currentAccount().getCustomerId());

  var yesterday = getYesterday();
  var date = getFirstDayToCheck(spreadsheet, yesterday);

  var rows = [];
  var existingDates = getExistingDates();

  while (date.getTime() <= yesterday.getTime()) {
    if (!existingDates[date]) {
      var row = getReportRowForDate(date);
      rows.push([new Date(date)].concat(REPORT_FIELDS.map(function(field) {
        return row[field.columnName];
      })));
      spreadsheet.getRangeByName('last_check').setValue(date);
    }
    date.setDate(date.getDate() + 1);
  }

  if (rows.length > 0) {
    writeToSpreadsheet(rows);

    var email = spreadsheet.getRangeByName('email').getValue();
    if (email) {
      sendEmail(email);
    }
  }
}

/**
 * Retrieves a lookup of dates for which rows already exist in the spreadsheet.
 *
 * @return {!Object} A lookup of existing dates.
 */
function getExistingDates() {
  var spreadsheet = validateAndGetSpreadsheet();
  var sheet = spreadsheet.getSheetByName('Report');

  var data = sheet.getDataRange().getValues();
  var existingDates = {};
  data.slice(5).forEach(function(row) {
    existingDates[row[1]] = true;
  });
  return existingDates;
}

/**
 * Sorts the data in the spreadsheet into ascending date order.
 */
function sortReportRows() {
  var spreadsheet = validateAndGetSpreadsheet();
  var sheet = spreadsheet.getSheetByName('Report');

  var data = sheet.getDataRange().getValues();
  var reportRows = data.slice(5);
  if (reportRows.length) {
    reportRows.sort(function(rowA, rowB) {
      if (!rowA || !rowA.length) {
        return -1;
      } else if (!rowB || !rowB.length) {
        return 1;
      } else if (rowA[1] < rowB[1]) {
        return -1;
      } else if (rowA[1] > rowB[1]) {
        return 1;
      }
      return 0;
    });
    sheet.getRange(6, 1, reportRows.length, reportRows[0].length)
        .setValues(reportRows);
  }
}

/**
 * Append the data rows to the spreadsheet.
 *
 * @param {Array<Array<string>>} rows The data rows.
 */
function writeToSpreadsheet(rows) {
  var access = new SpreadsheetAccess(SPREADSHEET_URL, 'Report');
  var emptyRow = access.findEmptyRow(6, 2);
  if (emptyRow < 0) {
    access.addRows(rows.length);
    emptyRow = access.findEmptyRow(6, 2);
  }
  access.writeRows(rows, emptyRow, 2);
  sortReportRows();
}

function sendEmail(email) {
  var day = getYesterday();
  var yesterdayRow = getReportRowForDate(day);
  day.setDate(day.getDate() - 1);
  var twoDaysAgoRow = getReportRowForDate(day);
  day.setDate(day.getDate() - 5);
  var weekAgoRow = getReportRowForDate(day);

  var html = [];
  html.push(
    '<html>',
      '<body>',
        '<table width=800 cellpadding=0 border=0 cellspacing=0>',
          '<tr>',
            '<td colspan=2 align=right>',
              "<div style='font: italic normal 10pt Times New Roman, serif; " +
                  "margin: 0; color: #666; padding-right: 5px;'>" +
                  'Powered by Google Ads Scripts</div>',
            '</td>',
          '</tr>',
          "<tr bgcolor='#3c78d8'>",
            '<td width=500>',
              "<div style='font: normal 18pt verdana, sans-serif; " +
              "padding: 3px 10px; color: white'>Account Summary report</div>",
            '</td>',
            '<td align=right>',
              "<div style='font: normal 18pt verdana, sans-serif; " +
              "padding: 3px 10px; color: white'>",
               AdsApp.currentAccount().getCustomerId(), '</h1>',
            '</td>',
            '</tr>',
          '</table>',
          '<table width=800 cellpadding=0 border=0 cellspacing=0>',
            "<tr bgcolor='#ddd'>",
              '<td></td>',
              "<td style='font: 12pt verdana, sans-serif; " +
                  'padding: 5px 0px 5px 5px; background-color: #ddd; ' +
                  "text-align: left'>Yesterday</td>",
              "<td style='font: 12pt verdana, sans-serif; " +
                  'padding: 5px 0px 5px 5px; background-color: #ddd; ' +
                  "text-align: left'>Two Days Ago</td>",
              "<td style='font: 12pt verdana, sans-serif; " +
                  'padding: 5px 0px 5x 5px; background-color: #ddd; ' +
                  "text-align: left'>A week ago</td>",
            '</tr>');
  REPORT_FIELDS.forEach(function(field) {
    html.push(emailRow(
        field.displayName, field.columnName, yesterdayRow, twoDaysAgoRow,
        weekAgoRow));
  });
  html.push('</table>', '</body>', '</html>');
  MailApp.sendEmail(email, 'Google Ads Account ' +
      AdsApp.currentAccount().getCustomerId() + ' Summary Report', '',
      {htmlBody: html.join('\n')});
}

function emailRow(title, column, yesterdayRow, twoDaysAgoRow, weekAgoRow) {
  var html = [];
  html.push('<tr>',
      "<td style='padding: 5px 10px'>" + title + '</td>',
      "<td style='padding: 0px 10px'>" + yesterdayRow[column] + '</td>',
      "<td style='padding: 0px 10px'>" + twoDaysAgoRow[column] +
          formatChangeString(yesterdayRow[column], twoDaysAgoRow[column]) +
          '</td>',
      "<td style='padding: 0px 10px'>" + weekAgoRow[column] +
          formatChangeString(yesterdayRow[column], weekAgoRow[column]) +
          '</td>',
      '</tr>');
  return html.join('\n');
}


function getReportRowForDate(date) {
  var timeZone = AdsApp.currentAccount().getTimeZone();
  var dateString = Utilities.formatDate(date, timeZone, 'yyyyMMdd');
  return getReportRowForDuring(dateString + ',' + dateString);
}

function getReportRowForDuring(during) {
  var report = AdsApp.report(
      'SELECT ' +
          REPORT_FIELDS
              .map(function(field) {
                return field.columnName;
              })
              .join(',') +
          ' FROM ACCOUNT_PERFORMANCE_REPORT ' +
          'DURING ' + during,
      REPORTING_OPTIONS);
  return report.rows().next();
}

function formatChangeString(newValue,  oldValue) {
  var x = newValue.indexOf('%');
  if (x != -1) {
    newValue = newValue.substring(0, x);
    var y = oldValue.indexOf('%');
    oldValue = oldValue.substring(0, y);
  }

  var change = parseFloat(newValue - oldValue).toFixed(2);
  var changeString = change;
  if (x != -1) {
    changeString = change + '%';
  }

  if (change >= 0) {
    return "<span style='color: #38761d; font-size: 8pt'> (+" +
        changeString + ')</span>';
  } else {
    return "<span style='color: #cc0000; font-size: 8pt'> (" +
        changeString + ')</span>';
  }
}

function SpreadsheetAccess(spreadsheetUrl, sheetName) {
  this.spreadsheet = SpreadsheetApp.openByUrl(spreadsheetUrl);
  this.sheet = this.spreadsheet.getSheetByName(sheetName);

  // what column should we be looking at to check whether the row is empty?
  this.findEmptyRow = function(minRow, column) {
    var values = this.sheet.getRange(minRow, column,
        this.sheet.getMaxRows(), 1).getValues();
    for (var i = 0; i < values.length; i++) {
      if (!values[i][0]) {
        return i + minRow;
      }
    }
    return -1;
  };
  this.addRows = function(howMany) {
    this.sheet.insertRowsAfter(this.sheet.getMaxRows(), howMany);
  };
  this.writeRows = function(rows, startRow, startColumn) {
    this.sheet.getRange(startRow, startColumn, rows.length, rows[0].length).
        setValues(rows);
  };
}

/**
 * Gets a date object that is 00:00 yesterday.
 *
 * @return {Date} A date object that is equivalent to 00:00 yesterday in the
 *     account's time zone.
 */
function getYesterday() {
  var yesterday = new Date(new Date().getTime() - 24 * 3600 * 1000);
  return new Date(getDateStringInTimeZone('MMM dd, yyyy 00:00:00 Z',
      yesterday));
}

/**
 * Returned the last checked date + 1 day, or yesterday if there isn't
 * a specified last checked date.
 *
 * @param {Spreadsheet} spreadsheet The export spreadsheet.
 * @param {Date} yesterday The yesterday date.
 *
 * @return {Date} The date corresponding to the first day to check.
 */
function getFirstDayToCheck(spreadsheet, yesterday) {
  var last_check = spreadsheet.getRangeByName('last_check').getValue();
  var date;
  if (last_check.length == 0) {
    date = new Date(yesterday);
  } else {
    date = new Date(last_check);
    date.setDate(date.getDate() + 1);
  }
  return date;
}

/**
 * Produces a formatted string representing a given date in a given time zone.
 *
 * @param {string} format A format specifier for the string to be produced.
 * @param {date} date A date object. Defaults to the current date.
 * @param {string} timeZone A time zone. Defaults to the account's time zone.
 * @return {string} A formatted string of the given date in the given time zone.
 */
function getDateStringInTimeZone(format, date, timeZone) {
  date = date || new Date();
  timeZone = timeZone || AdsApp.currentAccount().getTimeZone();
  return Utilities.formatDate(date, timeZone, format);
}

/**
 * Validates the provided spreadsheet URL to make sure that it's set up
 * properly. Throws a descriptive error message if validation fails.
 *
 * @return {Spreadsheet} The spreadsheet object itself, fetched from the URL.
 */
function validateAndGetSpreadsheet() {
  if ('YOUR_SPREADSHEET_URL' == SPREADSHEET_URL) {
    throw new Error('Please specify a valid Spreadsheet URL. You can find' +
        ' a link to a template in the associated guide for this script.');
  }
  var spreadsheet = SpreadsheetApp.openByUrl(SPREADSHEET_URL);
  var email = spreadsheet.getRangeByName('email').getValue();
  if ('foo@example.com' == email) {
    throw new Error('Please either set a custom email address in the' +
        ' spreadsheet, or set the email field in the spreadsheet to blank' +
        ' to send no email.');
  }
  return spreadsheet;
}

Bob’s your uncle

The code was originally on the Google Developer’s site here – https://developers.google.com/adwords/scripts/docs/solutions/ad-performance

Please Google, don’t bankrupt me for using the script on here! I’ll take it down if it upsets thee.

Emotional Intelligence for Beginners

There is much talk about emotional intelligence and the importance of it, but I rarely see any articles, posts or videos that explain how to develop it.

One thing that I personally think provides a foundation of emotional intelligence, is the ability to ‘detach from one’s thoughts’ – which normally requires a meditation practice. If you can’t stand back and observe your thoughts, it’s very difficult to evaluate their validity.

Physiological Responses & Emotions

Pre Existing Belief Bias

  • When information contradicts a pre-existing belief, our bodies generate a stress response with cortisol and adrenaline
  • When we force our opinion on someone, or someone confirms a pre-existing belief, our bodies generate a “dopamine-reward”

Consequences of Pre-Existing Belief Bias

  • Bullying – e.g. we see someone overweight – we’ve been led to belief as a child this is “wrong” or undesirable, or negative. So we bully the person
  • We dismiss people as being weird
  • We mock people with different beliefs
  • We’re brainwashed into fighting wars against people who oppose our beliefs (pretty heavy stuff!)

Examples of Pre Existing Belief Bias

When a child sees someone that is overweight – they believe people shouldn’t be overweight, so they bully that person

When someone questions if high cholesterol is the cause of heart disease – people think the statement is ridiculous, despite mounting scientific evidence and refuse to comprehend this counter-argument to mainstream ideas.

Cognitive Biases

  • Self Serving Bias
    People tend to take credit for things when they go right, but blame others when things go wrong. People may also exaggerate what the do right and exaggerate what others do wrong. A classic example, is housework – both individuals in a marriage often over-estimate their own contribution to housework and underestimate their partner’s contribution
  • Confirmation Bias
    Reaffirming an existing belief by looking for information that confirms it. For example, if someone believes coconut oil is unhealthy because of the high saturated fat content – they might search for “why coconut oil is unhealthy?” rather than a more neutral search-term such as “what are the health benefits and health risks associated with coconut oil?”

Confirmation bias and self Serving bias are often intertwined. For example, a mother who doesn’t breastfeed her children, might read about babies losing weight and becoming ill because of not getting enough volume of milk via breastfeeding. The mother globally concludes from this, that breastfeeding is bad.

  • Global Judgements based on individual or minor observations
    David may judge Tom for being a boxer – because Tom engages in violence, there is no way he could be a nice person. This is also based on a self serving bias, David is an academic who is not athletic and as a result, will discount the value of anything physical or violent when possible. I’m not sure what the proper phrase is for this type of bias :/
    Another example might be that Tim, saw Peter lose his temper once, in 1999. Tim has since then held the belief that Peter has anger management problems.

Cognitive biases are highly prevalent in martial arts. For example, someone that does a traditional martial art, may state that MMA is no good for self defense, as there are too many rules. A boxer may dismiss grappling as “rolling around on the floor” and a grappler may dismiss boxing as ineffective because “most fights end in a grapple” (true, but they also tend to start and end with punches!).


Logical Fallicies

  • Questioning the messenger instead of the message (Also known as Ad Hominem)
    You attacked your opponent’s character or personal traits in an attempt to undermine their argument.
    When someone attacks a person, instead of the person’s opinion. For example, if a vegetarian is wearing leather shoes, whilst debating the morality of vegetarianism, someone might attack the vegetarian for wearing leather shoes. This however, doesn’t address the issue being debated – is vegetarianism more ethical that a diet containing meat-products?

Ad hominem attacks can take the form of overtly attacking somebody, or more subtly casting doubt on their character or personal attributes as a way to discredit their argument. The result of an ad hom attack can be to undermine someone’s case without actually having to engage with it.

Example: After Sally presents an eloquent and compelling case for a more equitable taxation system, Sam asks the audience whether we should believe anything from a woman who isn’t married, was once arrested, and smells a bit weird.

https://yourlogicalfallacyis.com/ad-hominem
  • The Strawman Argument
    Changing the subject being debated half way through a debate.
    For example, someone might say that lots of sleep isn’t good for you, because depressed people sleep more than happy people. When it is pointed out that scientific evidence shows that too little sleep, might lead to degenerative brain conditions, and give Margaret Thatcher as an example – a strawman counter argument would be to refute this, because Margaret Thatcher achieved so much. Rather than addressing whether or not sleep is good for health, the debate changes to the achievement levels of famous people who are known not to sleep 8 hours or more.

A straw man is a form of argument and an informal fallacy of having the impression of refuting an argument, whereas the proper idea of the argument under discussion was not addressed or properly refuted. One who engages in this fallacy is said to be “attacking a straw man”. Wikipedia


Other Emotional Intelligence Issues

  • Stooping in arguments
    One major issue with debates and arguments is the willingness for one person to inflict emotional damage or to stoop more than another person.

    For example, in an office environment, where aggression is deemed inappropriate – one debator may resort to aggressiveness, whilst the other person is unwilling to become aggressive, and so let’s the aggressive individual ‘win’ the debate to prevent an esculation. When future conflicts in opinion arise, the more passive individual may not be willing to dispute or discuss anything, due to fear of an aggressive confrontation.

  • Victim Triad
    When person X complains to person Y, and person Y gives them sympathy but then person X complains to person Z and is told to look for a solution or that the sitation is not especially bad, person X will often feel like a victim of both the scenario they are complaining about and the lack of sympathy from person Z
  • Drama Triad
    If person X ridicules, bullies or is not particularly nice to person Y for a prolonged period of time – eventually person Y retaliates and tells person X to stop. Person X may feel like a victim and engage in vicious gossip about person Y for being in such a bad mood.
    Person X and person Y feel like victims in this situation.
  • Emotional Projection
    Blaming other people for your negative emotions. An individual who is subconsciously (or consciously) deemed as being weak, is usually the subject of another person’s projected emotions.

A few of the above are not directly related to emotional intelligence but an awareness of them could help to foster it, in my opinion.

emotional intelligence
https://tinybuddha.com/

Putting Emotional Intelligence into Practice

  • Have respect for other people’s points and points of view
  • Don’t globally-judge people or things in general on one point or one incidence
  • Don’t believe your impulsive thoughts – they’re not always true!
  • Feedback in private, praise in public

SEO Technical Audit Checklist (advanced)

The idea of technical SEO is to minimise the work of bots when they come to your website to index it on Google and Bing. Look at the build, the crawl and the rendering of the site.

Tools Required:

  • SEO Crawler such as Screaming Frog or DeepCrawl
  • Log File Analyzer – Screaming Frog has this too
  • Developer Tools – such as the ones found in Google Chrome – View>Developer>Developer Tools
  • Web Developer Toolbar – giving you the ability to turn off Javascript
  • Search Console
  • Bing Webmaster Tools – shows you geotargetting behaviour, gives you a second opinion on security etc.
  • Google Analytics – With onsite search tracking *

    *Great for tailoring copy and pages. Just turn it on and add query parameter

Tech SEO 1 – The Website Build & Setup

The website setup – a neglected element of many SEO tech audits.

  • Storage
    Do you have enough storage for your website now and in the near future? you can work this out by taking your average page size (times 1.5 to be safe), multiplied by the number of pages and posts, multiplied by 1+growth rate/100

for example, a site with an average page size of 1mb with 500 pages and an annual growth rate of 150%

1mb X 1.5 X 500 X 1.5 = 1125mb of storage required for the year.

You don’t want to be held to ransom by a webhost, because you have gone over your storage limit.

  • How is your site Logging Data?
    Before we think about web analytics, think about how your site is storing data.
    As a minimum, your site should be logging the date, the request, the referrer, the response and the User Agent – this is inline with the W3 Extended Format.
log file analyzer

When, what it was, where it came from, how the server responded and whether it was a browser or a bot that came to your site.

  • Blog Post Publishing
    Can authors and copywriters add meta titles, descriptions and schema easily? Some websites require a ‘code release’ to allow authors to add a meta description.
  • Site Maintenance & Updates – Accessibility & Permissions
    Along with the meta stuff – how much access does each user have to the code and backend of a website? How are permissions built in?
    This could and probably should be tailored to each team and their skillset.

    For example, can an author of a blog post easily compress an image?
    Can the same author update a menu (often not a good idea)
    Who can access the server to tune server performance?

Tech SEO 2 – The Crawl

  • Google Index

Carry out a site: search and check the number of pages compared to a crawl with Screaming Frog.

With a site: search (for example, search in Google for site:businessdaduk.com) – don’t trust the number of pages that Google tells you it has found, scrape the SERPs using Python on Link Clump:

Too many or too few URLs being indexed – both suggest there is a problem.

  • Correct Files in Place – e.g. Robots.txt
    Check these files carefully. Google says spaces are not an issue in Robots.txt files, but many coders and SEOers suggest this isn’t the case.

XML sitemaps also need to be correct and in place and submitted to search console. Be careful with the <lastmod> directive, lots of websites have lastmod but don’t update it when they update a page or post.

  • Response Codes
    Checking response codes with a browser plugin or Screaming Frog works 99% of the time, but to go next level, try using curl and command line. Curl avoids JS and gives you the response header.

Type in Curl – I and then the URL

e.g.

curl – I https://businessdaduk.com/

You need to download cURL which can be a ball ache if you need IT’s permission etc.

Anyway, if you do download it and run curl, your response should look like this:

Next enter an incorrect URL and make sure it results in a 404.

  • Canonical URLs
    Each ‘resource’ should have a single canonical address.

common causes of canonical issues include – sharing URLs/shortened URLs, tracking URLs and product option parameters.

The best way to check for any canonical issues is to check crawling behaviour and do this by checking log files.

You can check log files and analyse them, with Screaming Frog – the first 1,000 log files can be analysed with the free version (at time of writing).

Most of the time, your host will have your logfiles in the cPanel section, named something like “Raw Access”. The files are normally zipped with gzip, so you might need a piece of software to unzip them or just allow you to open them – although often you can still just drag and drop the files into Screaming Frog.

The Screaming Frog log file analyser, is a different download to the SEO site crawler – https://www.screamingfrog.co.uk/log-file-analyser/

If the log files are in the tens of millions, you might need to go next level nerd and use grep in Linux command line

Read more about all things log file analysis-y on Ian Lurie’s Blog here.

This video tutorial about Linux might also be handy. I’ve stuck it on my brother’s old laptop. Probably should have asked first.

With product IDs, and other URL fragments, use a # instead of a ? to add tracking.

Using rel-canonical is a hint, not a directive. It’s a work around rather than a solution.

Remember also, that the server header, can override a canonical tag.

You can check your server headers using this tool – http://www.rexswain.com/httpview.html (at your own risk like)


Tech SEO 3 – Rendering & Speed

  • Lighthouse
    Use lighthouse, but use in with command line or use it in a browser with no browser add-ons.If you are not into Linux, use pingdom, GTMetrix and Lighthouse, ideally in a browser with no add-ons.

    Look out for too much code, but also invalid code. This might include things such as image alt tags, which aren’t marked up properly – some plugins will display the code just as ‘alt’ rather than alt=”blah”
  • Javascript
    Despite what Google says, all the SEO professionals that I follow the work of, state that client-side JS is still a site speed problem and potential ranking factor. Only use JS if you need it and use server-side JS.

    Use a browser add-on that lets you turn off JS and then check that your site is still full functional.

  • Schema

Finally, possibly in the wrong place down here – but use Screaming Frog or Deepcrawl to check your schema markup is correct.

You can add schema using the Yoast or Rank Math SEO plugins

The Actual Tech SEO Checklist (Without Waffle)

Basic Setup

  • Google Analytics, Search Console and Tag Manager all set up

Site Indexation

  • Sitemap & Robots.txt set up
  • Check appropriate use of robots tags and x-robots
  • Check site: search URLs vs crawl
  • Check internal links pointing to important pages
  • Check important pages are only 1 or 2 clicks from homepage

Site Speed

Tools – Lighthouse, GTMetrix, Pingdom

Check – Image size, domain & http requests, code bloat, Javascript use, optimal CSS delivery, code minification, browser cache, reduce redirects, reduce errors like 404s.

For render blocking JS and stuff, there are WordPress plugins like Autoptimize and the W3 Total Cache.

Make sure there are no unnecessary redirects, broken links or other shenanigans going on with status codes. Use Search Console and Screaming Frog to check.

Site UX

Mobile Friendly Test, Site Speed, time to interactive, consistent UX across devices and browsers

Consider adding breadcrumbs with schema markup.

Clean URLs

Image from Blogspot.com

Make sure URLs – Include a keyword, are short – use a dash/hyphen –

Secure Server HTTPS

Use a secure server, and make sure the unsecure version redirects to it

Allow Google to Crawl Resources

Google wants to crawl your external CSS and JS files. Use “Fetch as Google” in Search Console to check what Googlebot sees.

Hreflang Attribute

Check that you are using and implementing hreflang properly.

Tracking – Make Sure Tag Manager & Analytics are Working

Check tracking is working properly. You can check tracking coed is on each webpage with Screaming Frog.

Internal Linking

Make sure your ‘money pages’ or most profitable pages, get the most internal links

Content Audit

Redirect or unpublish thin content that gets zero traffic and has no links. **note on this, I had decent content that had no visits, I updated the H1 with a celebrity’s name and now it’s one of my best performing pages – so it’s not always a good idea to delete zero traffic pages**

Consider combining thin content into an in depth guide or article.

Use search console to see what keywords your content ranks for, what new content you could create (based on those keywords) and where you should point internal links.

Use Google Analytics data regarding internal site searches for keyword and content ideas 💡

Update old content

Fix meta titles and meta description issues – including low CTR

Find & Fix KW cannibalization

Optimize images – compress, alt text, file name

Check proper use of H1 and H2

See what questions etc. are pulled through into the rich snipetts and answer these within content

Do you have EAT? Expertise, Authority and Trust?

https://www.semrush.com/blog/seo-checklist/

You can download a rather messy Word Doc Template of my usual SEO technical checklist here:

https://smallbusinessdad.files.wordpress.com/2021/11/drewseotemplate.docx

It uses Screaming Frog, SEMRush and Search Console

Best WordPress Plugins (Free) 2021

Well, I am cheating a bit here as it is still 2020, but it’s December 22nd so please forgive me.

I don’t have any plugins on this site, but I do on my ecommerce site – NiceMMA.com

I’ve also been asked to look into plugins and basically update myself with WordPress ready for 2021 and a slightly new job role, so a thought a blog post might make a good place to record my findings.

Rank Math SEO – WordPress Plugin

This is similar to Yoast but you get more in the free version. I’m not sure I’ll be using it in my day job though, as it asks for permission to analytics and search console.

You can update URLs, redirects and a bunch of other great things with the free version and it also gives a quick SEO audit, which is nice.

Rank Math SEO allows you to look at ‘focus keywords’ and makes it easy to update the meta details.


Duplicate Page – WordPress Plugin

If you make a specific blog layout or a contact page or anything and you don’t want to remake it from scratch, this plugin with duplicate the page or post for you and keep it as a draft so you can go in and edit it before publishing.


Manage WP – WordPress Plugin

Good for web developers, you can update all the plugins all in one go and do other stuff really easy


Empty Spam Bee

If you have a contact form or email address on your site – this should keep out most of the spam.


iThemes Security

Great security plugin. Turn off some of the notifications in the notifications center or you will get loads of emails. Check – secure the site and activate Network Group Brute Force Protection


Elementor Custom Skin

great for creating custom designs if you build WordPress sites using Elementor

This plugin works great with Custom Post Type UI

To Learn more about Elementor, subscribe to this guy’s Youtube channel –

Live Chat for WordPress

If you are looking for an ai chatbot for free, look at this plugin

For managed live chat, check out these guys and for managed live chat for WordPress, this article is worth a read

Live chat can be an important tool for customer service. It can provide 24 hour instant customer support. Email and telephone contact details are great on a website, but a user is typically in a hurry and wants instant answers, and doesn’t want to wait 24 hours for a reply via email.