Fix Broken Links Using Screaming Frog: A Step-by-Step Guide

I recently spoke with a client who was struggling with broken link issues on his site. He was frustrated and didn’t know how to resolve them. This conversation inspired me to create a guide to help you fix broken links using Screaming Frog. Fixing broken links can be a time-consuming and challenging process, especially if you don’t have the skills or experience to manage them effectively. As a business owner, you cannot afford to lose traffic or conversions due to avoidable issues like broken links.  

What Are Broken Links?

Broken links are hyperlinks on your website that lead to non-existent or incorrect pages. These are usually marked by a 404 error, indicating that the requested page could not be found. Broken links can:

• Hurt your website’s credibility.

• Negatively impact your search engine rankings.

• Frustrate your users, leading to higher bounce rates.

The good news? They’re fixable! Let’s dive into how I audit broken links step by step using Screaming Frog.

Step-by-Step Guide to Auditing Broken Links with Screaming Frog

Step 1: Download and Install Screaming Frog

First, if you haven’t already, download Screaming Frog from their official website. The free version allows you to crawl up to 500 URLs, which is perfect for smaller websites. If your site has more pages, consider upgrading to the paid version.

Step 2: Configure Your Crawl Settings

Before starting the crawl, let’s configure Screaming Frog:

1. Open Screaming Frog.

2. Go to Configuration > Spider.

3. Ensure that you’re crawling HTML, JavaScript, CSS, and other resources.

4. Exclude query parameters or unnecessary URLs if needed.

This ensures you’re capturing all potential broken links on your site.

Step 3: Start the Crawl

Here’s how to crawl your website:

1. Enter your website URL in the search bar.

2. Click the “Start” button.

3. Wait for the crawl to complete. For larger websites, this may take some time.

Step 4: Analyze the Crawl Results

Once the crawl is complete, follow these steps:

1. Navigate to the Response Codes tab.

2. Filter by Client Error (4xx) to display all broken links.

3. Pay attention to the “Status Code” column to see specific errors, such as 404 or 410.

Step 5: Check Internal and External Links

Screaming Frog allows you to view broken links for both internal and external URLs. To analyze each:

• Internal links: These are links within your website. Broken internal links directly affect your site’s navigation.

• External links: These are links pointing to other websites. Broken external links can hurt user trust and your site’s authority.

You can find these under the Inlinks and Outlinks tabs, respectively.

Step 6: Export Your Data

To make fixing broken links easier, export the data:

1. Click on the Export button in the top right.

2. Save the file as a CSV.

3. Open the file in Excel or Google Sheets for easier filtering and tracking.

How to Fix Broken Links

Now that you’ve identified broken links, it’s time to fix them. Here’s how:

1. Update the Link

• Find the correct URL and replace the broken link.

• Use Screaming Frog’s “Inlinks” tab to locate all instances of the broken link on your site.

2. Set Up a 301 Redirect

• If the page has been moved, set up a 301 redirect to point users to the new page.

• This can be done via your website’s .htaccess file or through a plugin like Redirection (if using WordPress).

3. Remove the Link

• If the page is no longer relevant and cannot be replaced, remove the link altogether.

4. Contact External Website Owners

• For broken external links, reach out to the website owner and let them know their page is broken. Alternatively, replace the link with a working resource.

5. Conduct Regular Audits

• Make broken link audits a regular part of your website maintenance to catch issues early.

Pro Tips for Managing Broken Links

• Use Google Search Console: Combine Screaming Frog’s insights with Google Search Console’s crawl errors report for a comprehensive audit.

• Install a Broken Link Checker Plugin: If you’re using WordPress, plugins like Broken Link Checker can monitor and notify you of broken links in real time.

• Leverage Analytics: Use tools like Google Analytics to identify high-traffic pages. Fixing broken links on these pages should be a priority.

Other SEO Tasks Screaming Frog Can Help With

Screaming Frog isn’t just for auditing broken links. It’s a versatile tool that can help you with other crucial SEO tasks:

1. Finding Duplicate Content

• Use the Content tab to identify duplicate pages, titles, and meta descriptions.

• This helps improve your site’s content quality and avoid penalties.

2. Optimizing Page Titles and Meta Descriptions

• Navigate to the Page Titles and Meta Description tabs to review their lengths, duplicates, and missing metadata.

3. Checking for Missing or Broken Images

• Go to the Images tab to find missing or broken images and ensure all images have proper alt text.

4. Analyzing Internal Linking

• Use the Internal Links tab to review your site’s internal linking structure and ensure proper distribution of link equity.

5. Auditing XML Sitemaps

• Upload your XML sitemap and compare it to the crawled URLs to find discrepancies.

6. Identifying Slow Pages

• Review the Performance tab to find pages with slow load times and optimize them for better user experience.

7. Finding Orphan Pages

• Identify pages that aren’t linked from anywhere on your site, which might not be getting indexed properly.

Alternative Tools for Auditing Broken Links

While Screaming Frog is a fantastic tool, there are other options worth considering. Here are a few:

1. Google Search Console (Free)

• Helps identify crawl errors, including broken links.

• Provides insights directly from Google.

2. Ahrefs (Paid)

• Offers a robust site audit feature to find broken links.

• Tracks backlinks and highlights dead pages linked to your site.

3. SEMrush (Paid)

• Includes a comprehensive site audit tool.

• Detects broken internal and external links effectively.

4. Broken Link Checker (Free/Plugin for WordPress)

• Monitors your WordPress site for broken links.

• Provides real-time notifications.

5. Xenu Link Sleuth (Free)

• A lightweight tool for finding broken links.

• Suitable for smaller websites and quick audits.

Final Thoughts

Auditing and fixing broken links may seem tedious, but it’s essential for maintaining your website’s SEO health and user experience. With tools like Screaming Frog, the process becomes manageable and even straightforward.

If you’re too busy running your business and don’t have time to manage SEO tasks like these, I can help! As an experienced SEO professional, I specialize in helping businesses like yours achieve top rankings and maximize online visibility.

Let’s Work Together!

Contact me today for a free consultation, and let’s discuss how I can help your business thrive online.