Screaming Frog Tips & Tricks

This is an aide memoire for myself, but if you have found really useful Screaming Frog tips to get the most out of the tool, please leave a comment or email me at

If I include the tip, you’ll get a link back to your blog or Twitter handle.

Find unlinked mentions

If you have a long list of URLs where your brand or product is mentioned, and you want to find out which pages do or don’t link back to you, this custom filter in Screaming Frog will be useful:

<a [^>]*?href=(“|’)?http(?:s)?://(?:www\.)?YOURSITE\.com[^>]*?>

(updated to better match a link tag)

Go to Configuration and select Custom, then enter   like this:

Regular expression used in Screaming Frog custom filter
Use regex with a custom filter

Check if page is linking to your site

By using “Contains” in the Custom settings, the filter will find if the page is currently linking to your site, though in this version not whether it is nofollowed.

If you have a modified RegEx to check for nofollow links, please do share in comments.

Match anything including New Lines with RegEx

Within Screaming Frog, the expression .* matches everything except new lines and returns, so if the HTML you want to match looks like this:

<div name=”examplediv”>
<span>Extract This Text</span>

That default expression won’t work. However, if you use (?s).* instead, it will work just fine.

Regex for Inclusion or Exclusion must match full URL string

Screaming Frog only reliably matches on the full URL evaluated. This means that this regex won’t work in all cases:


What this would normally do is include or exclude all auction listing pages on that look like:

That regex should match the twelve-digit auction ID number ending the URL. However, a range of these will still be included when crawling. Updating the regex to the following, to evaluate the whole URL, will work:


Extract anchor text from links to a site

This is a bit messy, but if a page is linking to, it will extract the text from ‘’ to the next closing anchor tag </a>.

This may at times capture other HTML tags that wrap the anchor text itself inside the <a> tag, if those tags have spaces before or after as the regex currently does not handle spaces well. You can find these easily by running a =find(“<“, [cell reference]) to locate any tag opening sharp brackets, then clean up manually.


Note that only the first link’s anchor text will be extracted. If the page has more than one link to, you will need to use a script to break up the page source code and iterate through the resulting text strings.

If you have a more sophisticated working example, please share in the comments.

This Screaming Frog tips article is regularly updated

More Screaming Frog tips & tricks will be added over time.

302s in site moves? It’s OK, if you must

In John Mueller’s webmaster hangout on 6 Nov 2015, I noticed a few comments on website moves which are interesting and go against standard recommendations for SEO.

In a site move (new domain name), Google considers 302s and 301s as equivalent.

“It’s not the case that 302s do something magical to block the flow of PageRank” – John Mueller

302s in a sitemove will pass PageRank as the redirects will be seen as a moved domain, not a temporary redirect. I’m assuming the requirement is also that the change of address (site move) is set in Google Search Console under site settings:


How long it takes for the new URLs to show rather than old ones is hard to say. John Mueller says it’s from hours to maybe a day.

“What will probably happen is that for some URLs on your site, it will take a little bit longer, for some it will go a lot faster.”

This is based on how Google crawls a website, indicating that the moved page needs to be recrawled after the change of address setting & redirects are live before the move takes effect.

The full move can take up to half a year, maybe longer.

If a advanced query is used, the indexed pages won’t drop out of the SERPs entirely until the site’s indexed pages are fully recrawled. Conversely, a query will show a fast initial growth in indexed pages, and then a trickle of further URLs added over time.

Note that 301s are recommended by Google in their guidance for site moves.

“Set up server-side redirects (301-redirect directives) from your old URLs to the new ones. The Change of address tool won’t function without it.”

In other words, it’s still best practice to use 301s.

But if you’re not able to use 301 redirects, you can use 302 redirects in site moves without losing PageRank.

Search Bootcamp Ecommerce Ranking Factors Roundtable

On 22 June 2015 I led a roundtable discussion at SEO Bootcamp in London, the discussion covered various eCommerce optimisation issues, and a range of challenges the delegates were currently working on.

eCommerce Roundtable Discussion
eCommerce Roundtable Discussion
“Let’s sit down and talk about unique content, social signals, page titles, content marketing, brand entities, site speed, SSL certificates, image optimisation, re-marketing, XML sitemaps, videos, internal linking, external linking, toxic backlinks, negative SEO, search themes, local search optimisation, Google Now, crawl budget optimisation, keyword research, call tracking, information architecture, merchandising, schema markup, tracking pixels, up & cross-sell tactics, analytics, conversion tracking… feeling overwhelmed by what your ecommerce siteactually needs to perform and make more money? You’re not alone. Centred on Google and Organic Search, this is an intense 60 minute roundtable to cut through the FUD and focus on the activities that bring returns.”

The slides are a summary of a checklist I use to quickly review and prioritise outstanding issues I find with eCommerce websites. The current version of this tool is linked below:

Download the eCommerce Optimisation Checklist
Download the eCommerce Optimisation Checklist

About Search Bootcamp

Search Bootcamp is a full day set of workshops and roundtables allowing SEOs to get together and learn about new developments in search and meet the experts in person. It’s organised by the SEO Monitor team, and you should take a look at their toolset.

Many thanks to my fellow presenters Kelvin Newman(@kelvinnewman), Alexandra Tachalova (@AlexTachalova), Cosmin Negrescu (@ncosmin), Andy Cooney (@Andy_Cooney), Ann Stanley(@AnnStanley), Joe Shervell (@eelselbows), Bastian Grimm (@basgr), and Alec Bertram (@kiwialec) for making it a great event.

And thank you also to Maria, Irina, and Cosmin for inviting me to present.


eCommerce Ranking Factors Slides
eCommerce Optimisation Checklist

Speaking at eComTeam 2015, Brasov, Romania

I’m proud to be invited to run a workshop and speak at the premier eCommerce conference in Romania, eComTeam. For the conference’s 3rd year, it’s doing a 302 redirection to Brasov in central Transylvania rather than staying in Bucharest.

eComTeam 2015 website

There’s a range of fantastic speakers lined up this year, such as Lukasz Zelezny, Violeta Luca, Tamsin Fox-Davies, Alexandru Lapusan, and Judith Lewis. Full line-up here.

On Day 1, I’m presenting the closing slot and will be presenting some thoughts on SEO & eCommerce over the next few years, as well as rounding up some highlights of the day. I’m calling this “What May Come Next – A review of trends & technologies that will impact how we interact with others, use digital media and shop in 2015 to 2020”.

My workshop for Day 2 is eRetail: From average to great in 2015, from the conference programme:

“This year you can rocket your site up the rankings and boost your profits by improving search engine performance and customer appeal. With best practice across content, UX and Search Engine Optimisation, this workshop gives you a comprehensive roadmap to follow. And we’ll also share the worst SEO errors to avoid.”

In the run-up to the conference, I also took part in a Q&A for, the premier business magazine in Romania with the title Expert SEO: Tribul tau este format din 0,1% din piata. Fa-l sa te adore (in English; Expert SEO, Your Tribe Is 0.1% Of The Market. Learn To Love It).

A version in English is published at the Forward3D blog:

SEO Advice for Start-Up Businesses

I will be making materials and slideshows available here and on my Slideshare account once the conference is over.

The presentation slides are below: