How to Find and Remove Broken Links in Your Website

Knowing how to find and remove broken links from your website is very important knowledge as a website owner.

Broken links are basically links in your website that when clicked, land a user on a 404 error page. This means that the page in question does not exist on your website server.

This can be an inconvenience for search engine bots, but even more so to users on your website. Having broken links on your site will adversely affect the user experience on your website in large proportions.

For WordPress bloggers, there are plugins that can easily help you to find and remove these broken links. But relying on plugins for easy tasks like removing this, is not too good in the long run.

These WordPress plugins include:

1. Plugins that find broken links in your site like Broken Links Checker.
2. Plugins that manage 301 redirects automatically like Redirection.

Whether you use a plugin or not, it is highly recommended that you check your website regularly for broken links. So that they are found and removed. There are various of doing this, which includes:

Detecting and removing invalid URLs using Google WebMaster Tools

There are two reasons why pages are indexed in Google even though they don’t exist on your website:

1. You referenced an invalid internal link by mistake because of a typo. This is the time to recommend a very simple yet essential plugin for post authors – the Link to Post plugin for avoiding such mistakes).
2. You published a post and decided to change its permalink afterward (the post’s URL) after Google has already indexed the original link.

The best way to detect these errors is by using Google Webmaster Tools.

If you haven’t done so already, register your site there. It’s an essential tool to have for anyone running a website. One of the most important tools provided in GWT is the Remove URL tool, which allows you to remove invalid pages from Google search results.

Let’s see how to remove those bad URLs from Google’s index.

Detecting invalid pages that are indexed by Google

Once you have signed up for Google Webmaster Tools and have set it up and verified your site (see Google’s Getting Started guide for GWT), here is the process for finding invalid pages.

1. Click on Diagnostics from the left menu and select Crawl Errors.
2. Select the Not Found category.
3. If available, click to view which page contains the broken link.
4. To make sure that the URL is indeed indexed in Google, copy and paste the URL into Google’s search and see if any result comes up.

Removing URLs from Google’s search results

1. Click on Site Configuration, and select Crawl Access.
2. Select Remove URL.
3. Click on New removal request.

4. Select the first option to completely remove a page from Google search results. OR Select the fourth option in case you want to retain the page in search results but would like to remove the cached version of the page. This is useful in case Google displays an outdated version of the page in the “cached” link.

5. Click on Next.

6. Type the URL of the page you would like to remove from Google.

7. Make sure the first option is selected and then click on Add.

8. The URL to be removed should now appear in the list. If you want, you can add more pages for removal.

9. Click on Submit Removal Request.

Your request is now pending—in most cases, it only takes 2 to 3 days for Google to remove the URL.

10. Click on Site Configuration and select Crawl Access.

11. Select Remove URL.

12. Make sure the URL you requested to remove does not appear in the list of pending requests.

13. Click on Removed URLs to see that the URL is now listed there. To make sure that the URL was indeed removed from Google, copy and paste the URL into Google’s search box and see if any result comes up.

Detecting broken links using Xenu Link Sleuth

Another excellent tool I like using for hunting down broken links in my websites is Xenu Link Sleuth. You can download it here.

Unlike the WordPress plugins mentioned earlier in this article, Xenu is a standalone desktop application for Windows that outputs all your site links—whether they’re valid or invalid links—and groups them into a very readable fashion.

After you install Xenu, using it is really easy.

1. Click on File and select Check URL.
2. Type your website’s URL (e.g. https://www.webfx.com/blog/web-design/.
3. Wait for all links (site wide!) to be checked.
4. When Xenu asks whether you want a report, click on Yes.

You can enter your FTP server details, but I simply click on Cancel and Xenu generates an XHTML report locally (it opens a dialog window automatically). In the generated report, click on Broken links.

Sort it by link in the table of contents to see all the pages that have broken links in them (and the broken links of course).

Finally, once you’ve detected all broken links, what is left to do is to navigate to the posts and pages containing references to broken links. You should either fix or remove those links.

What are your own techniques and tools for finding broken links?

How often do you search your site for broken links? How important is it to eliminate invalid links, and why?