How can BeautifulSoup be used to extract ‘href’ links from a website?

PythonServer Side ProgrammingProgramming

BeautifulSoup is a third party Python library that is used to parse data from web pages. It helps in web scraping, which is a process of extracting, using, and manipulating the data from different resources.

Web scraping can also be used to extract data for research purposes, understand/compare market trends, perform SEO monitoring, and so on.

The below line can be run to install BeautifulSoup on Windows −

pip install beautifulsoup4

Following is an example −


from bs4 import BeautifulSoup
import requests
url = ""
req = requests.get(url)
soup = BeautifulSoup(req.text, "html.parser")
print("The href links are :")
for link in soup.find_all('a'):


The href links are :


  • The required packages are imported, and aliased.

  • The website is defined.

  • The url is opened, and data is read from it.

  • The ‘BeautifulSoup’ function is used to extract text from the webpage.

  • The ‘find_all’ function is used to extract text from the webpage data.

  • The href links are printed on the console.

Published on 18-Jan-2021 17:23:53