Data Structure
Networking
RDBMS
Operating System
Java
MS Excel
iOS
HTML
CSS
Android
Python
C Programming
C++
C#
MongoDB
MySQL
Javascript
PHP
Physics
Chemistry
Biology
Mathematics
English
Economics
Psychology
Social Studies
Fashion Studies
Legal Studies
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
Find the siblings of tags using BeautifulSoup
Data may be extracted from websites using the useful method known as web scraping and a popular Python package for web scraping is BeautifulSoup which offers a simple method for parsing HTML and XML documents, enabling us to extract certain data from web sites. Finding the siblings of a tag is a frequent task while scraping web pages and it can be defined as a tag's siblings are any additional tags that have the same parent as the primary tag. We will go through using BeautifulSoup to locate tags' siblings in this blog article.
Installation and Setup
To use BeautifulSoup, you must first install it using pip, a package manager for Python.
pip install beautifulsoup4
Once installed, you can import BeautifulSoup in your Python code −
from bs4 import BeautifulSoup
Syntax
The syntax for finding the siblings of tags using BeautifulSoup is as follows −
siblings = tag.findNextSiblings()
Here, tag is the tag whose siblings we want to find, and siblings is a list of all the siblings of the tag.
Algorithm
Use BeautifulSoup to first parse the HTML or XML content.
Passing the document to the BeautifulSoup function will get this done.
Use the find() function to locate the tag whose siblings you're looking for.
To locate every sibling of the tag, use the findNextSiblings() function.
Example 1
from bs4 import BeautifulSoup
html = """
<html>
<body>
<div>
<p>Tutorials Point Python Text 1</p>
<p>Tutorials Point Python Text 2</p>
<p>Tutorials Point Python Text 3</p>
</div>
</body>
</html>
"""
soup = BeautifulSoup(html, "html.parser")
tag = soup.find_all('p')[1]
siblings = tag.findNextSiblings()
print(siblings)
Output
[<p>Tutorials Point Python Text 3</p>]
Example 2
from bs4 import BeautifulSoup
html = """
<html>
<body>
<div>
<h1>Just A Simple Test Heading 1</h1>
<p>Tutorials Point Python Text 1</p>
<h2>Just A Simple Test Heading 2</h2>
<p>Tutorials Point Python Text 2</p>
<h3>Heading 3</h3>
<p>Tutorials Point Python Text 3</p>
</div>
</body>
</html>
"""
soup = BeautifulSoup(html, "html.parser")
tag = soup.find('h2')
siblings = tag.find_next_siblings()
print(siblings)
Output
[<p>Tutorials Point Python Text 2</p>, <h3>Heading 3</h3>, <p>Tutorials Point Python Text 3</p>]
Here, BeautifulSoup is used to extract the HTML content of a webpage and then locate the 'h2' tag within the HTML using the find() method. The find_next_siblings() method comes handy to locate all the siblings of the 'h2' tag.
Start by importing the necessary modules, BeautifulSoup and requests.
Use the requests module to submit a GET request to the URL of the website you wish to scrape. Use the response object's .text property to extract the page's HTML content.
When calling BeautifulSoup function, pass the HTML text and specify the "html.parser" parser.
Use the find() function to find the 'h2' tag, and then save the result in the tag variable.
Use the find_next_siblings() method to find all the siblings of the 'h2' tag and store them in the siblings variable.
Print the siblings
Example 3
from bs4 import BeautifulSoup
import requests
# Send a GET request to the URL
url = 'https://example.com'
response = requests.get(url)
# Extract the HTML content
html = response.text
# Parse the HTML content with BeautifulSoup
soup = BeautifulSoup(html, "html.parser")
# Find the 'h2' tag
tag = soup.find('h2')
# Find the siblings of the 'h2' tag
siblings = tag.find_next_siblings()
# Print the siblings
print(siblings)
Applications
Web scraping − You might wish to find particular tags on a webpage before extracting their siblings if you're attempting to extract information from it.
Analysis of data − If you have a sizable HTML file that contains data, you might wish to find specific tags and then remove their siblings for more investigation.
Automated testing − When evaluating online apps, it's possible to search for specific tags and then check if their siblings satisfy specific requirements.
Conclusion
The Python package BeautifulSoup makes it simple to extract data from HTML and XML files. We can quickly discover the siblings of a specific tag and gather important data by using the find next_siblings() function. This method has several uses, including automated testing, data analysis, and online scraping. Moreover, BeautifulSoup gives us several methods, like find_all(), find_parent(), and find_previous_sibling(), to browse the HTML or XML tree structure. These techniques enable us to automate tedious procedures and effectively retrieve the data we want.
611 Views
