Creating a Web Crawler in Python using Socket Programming

in this course we will  teach you how you could write a program acting as a robot to search through different websites to get useful information that you need . unlike most web crawling and web scraping courses which use high level modules and in fact you are just copy pasting code , in this course we will first teach you all the concepts behind networking required for this course. 

Networking section:

so if we want to write some code first we will explain the networking details behind that code and then we say why the code does what it does. after explaining everything in theory we will explore all of above concepts in action and in wireshark . so you will be familiar with concepts like tcp/ip , network address translation sockets and so on. after networking part we start programming section .

Programming section:
in this section first we use a high level module like requests . and we learn how to send http requests and receive the related http response . after that we go deeper by introducing you to socket module in python which is the most important module in networking in python . so we create a socket and we will learn about different methods in this module which with the help of them we start to send and receive data to webservers and vice versa . and we learn how to search through those data for our favorite and useful information . after that we create a website by making our kali linux acting as a webserver serving web pages and we learn how we could search through different pages looking for email addresses , links and so on .