- Data Structure
- Networking
- RDBMS
- Operating System
- Java
- MS Excel
- iOS
- HTML
- CSS
- Android
- Python
- C Programming
- C++
- C#
- MongoDB
- MySQL
- Javascript
- PHP
- Physics
- Chemistry
- Biology
- Mathematics
- English
- Economics
- Psychology
- Social Studies
- Fashion Studies
- Legal Studies
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
How can BeautifulSoup be used to extract ‘href’ links from a website?
BeautifulSoup is a third party Python library that is used to parse data from web pages. It helps in web scraping, which is a process of extracting, using, and manipulating the data from different resources.
Web scraping can also be used to extract data for research purposes, understand/compare market trends, perform SEO monitoring, and so on.
The below line can be run to install BeautifulSoup on Windows −
pip install beautifulsoup4
Following is an example −
Example
from bs4 import BeautifulSoup import requests url = "https://en.wikipedia.org/wiki/Algorithm" req = requests.get(url) soup = BeautifulSoup(req.text, "html.parser") print("The href links are :") for link in soup.find_all('a'): print(link.get('href'))
Output
The href links are : … https://stats.wikimedia.org/#/en.wikipedia.org https://foundation.wikimedia.org/wiki/Cookie_statement https://wikimediafoundation.org/ https://www.mediawiki.org/
Explanation
The required packages are imported, and aliased.
The website is defined.
The url is opened, and data is read from it.
The ‘BeautifulSoup’ function is used to extract text from the webpage.
The ‘find_all’ function is used to extract text from the webpage data.
The href links are printed on the console.
Advertisements
To Continue Learning Please Login
Login with Google