Screaming Frog

10/01/2025

🚨📣 Screaming Frog Merch Giveaway Klaxon 🚨📣

These opportunities don't happen often, but once again, we're offering up some iconic Screaming Frog merch for you to win.

Head to our blog to find out how you can be in with a chance. Link in comments 👇

06/12/2024

A few photos from a brilliant night at the UK Search Awards where we took home a hat-trick of awards!

🏆 Best Local Campaign (PPC)
🏆 Best Use of Social Media in a Search Campaign
🏆 Best Use of Content Marking

06/09/2024

Black Friday is just around the corner, meaning now is the perfect time to prep your ecommerce ads to help you capitalise on that inevitable surge in interest.

From retargeting to setting budgets, here are 10 tips to make the most of Black Friday 2024:

As the old saying goes, “…in this world nothing can be said to be certain, except death and taxes.” well now you can add “ as well as a huge spike in search interest surrounding Black Friday from the end of August onwards”. Admittedly, it doesn’t quite have the same...

08/05/2024

We’re delighted to announce that Screaming Frog SEO Spider version 20.0 is now available 🐸

Buckle up, it's a big one!

https://www.screamingfrog.co.uk/seo-spider-20/

🧙‍♂️ Custom JavaScript
You’re now able to execute custom JavaScript while crawling. This means you’re able to manipulate pages or extract data, as well as communicate with APIs such as OpenAI’s ChatGPT, local LLMs, or other libraries.

You can do things like:
- Use AI to generate alt text for images.
- Trigger mouseover events.
- Scroll a page (to crawl some infinite scroll set ups).
- Download and save various content locally (like images, or PDFs etc).
- Sentiment, intent or language analysis of page content.
- Connect to SEO tool APIs that are not already integrated, such as Sistrix.
- And more!

🤖 Crawl with ChatGPT
You can select the ‘(ChatGPT) Template’ snippet, open it up in the JavaScript editor, add your OpenAI API key, and adjust the prompt to query anything you like against a page while crawling!

The possibilities are endless. Setup your own snippets, which will be saved in your own user library, and then export/import the library as JSON to share with colleagues and friends.

📱 Mobile Usability
You are now able to audit mobile usability at scale via the Lighthouse integration.

There’s a new Mobile tab with filters for common mobile usability issues such as viewport not set, tap target size, content not sized correctly, illegible font sizes and more.

🔑 N-grams Analysis
You can now analyse phrase frequency using n-gram analysis across pages of a crawl, or aggregated across a selection of pages of a website.

This analysis can help improve on-page alignment, identify gaps in keywords and also provide a new way to identify internal link opportunities.

Using n-grams you’re able to highlight a section of a website and filter for keywords in ‘Body Text (Unlinked)’ to identify link opportunities.

🌳 Carbon Footprint
The SEO Spider will now automatically calculate carbon emissions for each page using CO2.js library.

Alongside the CO2 calculation there is a carbon rating for each URL, and new ‘High Carbon Rating’ opportunity under the ‘Validation’ tab.

And lots more.

Take a look at our release notes for a full list. Enjoy!

We’re delighted to announce Screaming Frog SEO Spider version 20.0, codenamed internally as ‘cracker’. It’s incredible to think this is now our 20th major release of the software, after it started as a side project in a bedroom many years ago. Now is not the time for reflection though, as...

18/07/2023

Screaming Frog SEO Spider Version 19.0 is now available 😱🐸🕷️

This version contains a number of significant updates, new features and enhancements based upon user feedback and a little internal steer.

Full details: https://www.screamingfrog.co.uk/seo-spider-19/

New features include:

- 📊 Segments: You can now segment a crawl to better identify and monitor issues and opportunities from different templates, page types, or areas of priority.

- ✂️ Visual Custom Extraction: Open a web page in the inbuilt browser and select the elements you wish to extract from either the web page, raw HTML or rendered HTML.

- 🌌 3D Crawl Visualisations: 3D crawl diagrams, allowing you to move around nodes like you're in space...!

- ⚙️ Unified Config: The configuration has been unified into a single dialog, with links to each section. This makes adjusting the config more efficient than opening and closing each separately.

- 🖌️ Updated Design: A new, subtle GUI appearance with a refreshed look and feel.

- And more!

Head over to our blog post for a full list. Enjoy!

We’re delighted to announce Screaming Frog SEO Spider version 19.0, codenamed internally as ‘Peel’. This update contains a number of significant updates, new features and enhancements based upon user feedback and a little internal steer. Let’s take a look at what’s new. 1) Updated Design W...

14/12/2022

We've just released an update to version 18.1 of the SEO Spider, which includes small improvements and bug fixes:

https://www.screamingfrog.co.uk/seo-spider-18/ #18.1

If you're currently on version 18.0, you will be alerted in-app when there’s a new version available, which will have already silently downloaded in the background. You can then install it in a few clicks!

We’re excited to announce Screaming Frog SEO Spider version 18.0, codenamed internally as ‘Willow’. We’ve been busy working on one major feature we wanted to release pre to the Christmas holidays, and a variety of smaller, yet much-requested features and improvements. Let’s take a look at ...

05/12/2022

Screaming Frog SEO Spider v.18.0 is now available 😱🐸🕷️ !

We’ve been busy working on one major feature we wanted to release pre to the Christmas holidays, and a variety of smaller, yet much-requested features and improvements.

Blog post: https://www.screamingfrog.co.uk/seo-spider-18/

New features include:

1. GA4 Integration 📈 - It’s taken a little while, but like most SEOs, we’ve finally come to terms that we’ll have to actually switch to GA4. You’re now able to (begrudgingly) connect to GA4 and pull in analytics data in a crawl via their new API.

2. Parse PDFs 📃 - The SEO Spider will now crawl PDFs, discover links within them and show the document title as the page title. This means users can check to see whether links within PDFs are functioning as expected and issues like broken links will be reported in the usual way in the Response Codes tab.

You can also choose to ‘Extract PDF Properties’ and ‘Store PDF’ under ‘Config > Spider > Extraction’ and the PDF subject, author, created and modified dates, page count and word count will be stored.

3. Validation Tab🔍 - There’s a new Validation tab, which performs some basic best practice validations that can impact crawlers when crawling and indexing. This isn’t W3C HTML validation which is a little too strict, the aim of this tab is to identify issues that can impact search bots from being able to parse and understand a page reliably.

4. In-App Updates ⚙️ - You will now be alerted in-app when there’s a new version available, which will have already silently downloaded in the background. You can then install in a few clicks!

5. New Filters & Issues ⚠️ - There’s a variety of new filters and issues available across existing tabs that help better filter data, or communicate issues discovered. Many of these were already available either via another filter, or from an existing report like ‘Redirect Chains’. However, they now have their own dedicated filter and issue in the UI, to help raise awareness.

6. Flesch Readability Scores 📖 - Flesch readability scores are now calculated and included within the ‘Content‘ tab with new filters for ‘Readability Difficult’ and Readability Very Difficult’ (English only at the moment).

There's lots more, including auto-completing URL bar, crawl visualisation colour changes and authentication for scheduling / CLI.

Take a look at the blog post for a full list!

We’re excited to announce Screaming Frog SEO Spider version 18.0, codenamed internally as ‘Willow’. We’ve been busy working on one major feature we wanted to release pre to the Christmas holidays, and a variety of smaller, yet much-requested features and improvements. Let’s take a look at ...

14/09/2022

Our Copywriter Manager, Jack Prouse, has put together this great post on writing effective product copy.

Find out the best ways to format product descriptions, solve customer problems, write CTAs and more.

https://www.screamingfrog.co.uk/how-to-write-product-copy/

Find out how to make every word count when it comes to writing copy for ecommerce pages.

12/09/2022

We've just released a small update to the SEO Spider, which includes some minor improvements and bug fixes.

You can find more info about v17.2 here:

https://www.screamingfrog.co.uk/seo-spider-17/ #17.2

And download it here:

https://www.screamingfrog.co.uk/seo-spider/

Happy Monday!

We’re pleased to announce Screaming Frog SEO Spider version 17.0, codenamed internally as ‘Lionesses’.

07/09/2022

We've just released an update to the Log File Analyser, which includes some small improvements and bug fixes.

You can find more info here:

https://www.screamingfrog.co.uk/log-file-analyser-5-0/ #5.2

And download the latest version here:

https://www.screamingfrog.co.uk/log-file-analyser/

Enjoy!

The Screaming Frog SEO Log File Analyser allows you to upload your log files, verify search engine bots, identify crawled URLs and analyse search bot data and behaviour for invaluable SEO insight.

17/08/2022

Introducing Screaming Frog SEO Spider Version 17! 😱🐸🕷️

We've been busy working on the next round of exciting features, and we're pleased to announce that version 17 is now out in the wild.

https://www.screamingfrog.co.uk/seo-spider-17/

New features include:

1. ⚠️ Issues Tab – A new 'Issues' right-hand tab that details issues, warnings and opportunities. This makes it easier to identify potential problems and areas of improvement, with in-app explanations and an estimated 'priority' based on the potential impact.

For experienced users the Issues tab is useful to quickly spot top-level problems, and for those with less SEO expertise, it will help provide more much direction and guidance.

2. 🔗 Links Tab – A new 'Links' tab helps to hunt down link-based issues, such as pages with high crawl depth, pages without internal outlinks, pages using nofollow on internal links and more.

The filter thresholds are customisable, allowing you to adjust them to your preferred limits.

3. ⛔ New Limits – You're now able to adjust the number of URLs crawled by URL path, giving you increased control. For example, you can choose to crawl 500 URLs containing /products/, giving you enough of a sample to make informed decisions.

4. 🍏 Apple Silicon Version & RPM for Fedora – There's now a native Apple Silicon version available for users on M1/2 macs, and an RPM for Fedora Linux users. During internal testing, we found that crawling a locally hosted site was twice as quick, and loading a saved crawl was 4 times faster.

The native experience is much smoother overall in comparison to using the Rosetta2 emulation layer.

5. ✂️ Detachable Tabs – All tabs are now detachable! Simply right click on any tab and click 'detach' to pop it out and move it to where you see fit, such as onto a second monitor. To reattach, simply click the 'Reattach' button, or right click any tab and click 'Reset Tabs' to reattach all of them at once.

You're also able to 'Pin' detached tabs so that they always remain on top.

There's lots more, including scheduling UI improvements, multiple property configurations for URL inspection API, better tab re-ordering and configuration and more.

Take a look at the blog post for a full list!

We’re pleased to announce Screaming Frog SEO Spider version 17.0, codenamed internally as ‘Lionesses’.

16/06/2022

Running the Screaming Frog SEO Spider in the cloud is a game changer, and our newest guide helps you do just that.

By running the SEO Spider on Google’s Compute Engine, you can:

- Spin up Virtual Machines with specifications that you can easily scale up or down. An SEO Spider licence allows you to run as many instances of the SEO Spider in the cloud as you wish.

- Free up your local resources, negating the need to leave your machine running for large crawls.

- You or your team can access ongoing or finished crawls from anywhere.

- Using this method, we were able to crawl over 3million URLs, which cost less than £20/$25 in Google Cloud Platform charges.

- If you’re new to Google Cloud , you also get $300 free credit to play with.

Take a look:

https://screamingfrog.co.uk/seo-spider-cloud/