How to check URL for 404 using Selenium WebDriver?

We can check the URL for 404 using Selenium webdriver. A 404 check is actually done to verify if there are broken links in a page. On clicking such a link, we shall not be directed to the correct page.

A broken link can occur due to the following reasons −

• The landing page is no longer available.

• Some parts of the URL have been modified.

• An incorrect URL has been specified on the page.

• Firewall or geolocation limitations.

A URL can have the following status code −

• 5XX − Represents issue in server.

• 4XX − Represents resource cannot be determined.

• 3XX − Represents redirection.

• 2XX − Represents correct condition.

Thus we see that with only 2XX status code we can have a correct URL. We shall send a HTTP request and analyse its response code for all the links on the page.

Example

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import java.util.concurrent.TimeUnit;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLConnection;
import java.util.List;
public class BrokenURL{
   public static void main(String[] args) throws
   InterruptedException{
      System.setProperty("webdriver.gecko.driver",
         "C:\Users\ghs6kor\Desktop\Java\geckodriver.exe");
      WebDriver driver = new FirefoxDriver();
      // wait of 5 seconds
      driver.manage().timeouts().implicitlyWait(5, TimeUnit.SECONDS);
      driver.get("https://www.google.com/");
      //get list of elements with anchor tag
      List<WebElement> l = driver.findElements(By.tagName("a"));
      //iterate links
      for(int j=0; j<l.size(); j++) {
         WebElement e = l.get(i);
         //get URL of links with getAttribute()
         String u = e.getAttribute("href");
         // to catch MalFormedURLException
         try{
            //object of URL class
            URL link = new URL(u);
            // establish connection URL object
            HttpURLConnection c = (HttpURLConnection)link.openConnection();
            //have timeout
            c.setConnectTimeout(1000);
            // connection began
            c.connect();
            //getResponseCode() to obtain response code
            if(c.getResponseCode()== 200) {
               System.out.println(u+" − "+ c.getResponseMessage());
            }
            if(c.getResponseCode()== 404) {
               System.out.println(u+" − "+c.getResponseMessage());
            }
         }
         catch (Exception ex) {
         }
      }
   }
}

Output