Python - Building URLs


The requests module can help us build the URLS and manipulate the URL value dynamically. Any sub-directory of the URL can be fetched programmatically and then some part of it can be substituted with new values to build new URLs.


The below example uses urljoin to fetch the different subfolders in the URL path. The urljoin method is used to add new values to the base URL.

from requests.compat import urljoin
print urljoin(base,'.')
print urljoin(base,'..')
print urljoin(base,'...')
print urljoin(base,'/3764299/')
url_query = urljoin(base,'?vers=1.0')
print url_query
url_sec = urljoin(url_query,'#section-5.4')
print url_sec

When we run the above program, we get the following output −

Split the URLS

The URLs can also be split into many parts beyond the main address. The additional parameters which are used for a specific query or tags attached to the URL are separated by using the urlparse method as shown below.

from requests.compat import urlparse
url1 = ''
print urlparse(url1)
print urlparse(url2)

When we run the above program, we get the following output −

ParseResult(scheme='https', netloc='', path='/2/py-modindex.html', params='', query='', fragment='cap-f')
ParseResult(scheme='https', netloc='', path='/2/search.html', params='', query='q=urlparse', fragment='')

Useful Video Courses


Python Online Training

Most Popular

187 Lectures 17.5 hours

Malhar Lathkar


Python Essentials Online Training

55 Lectures 8 hours

Arnab Chakraborty


Learn Python Programming in 100 Easy Steps

136 Lectures 11 hours

In28Minutes Official


Python with Data Science

75 Lectures 13 hours

Eduonix Learning Solutions


Python 3 from scratch to become a developer in demand

70 Lectures 8.5 hours

Lets Kode It


Python Data Science basics with Numpy, Pandas and Matplotlib

63 Lectures 6 hours

Abhilash Nelson