Implement Concurrency with Threads in Python

Kiran P
Updated on 09-Nov-2020 10:20:23

288 Views

IntroductionPython has different approaches like using threads, subprocesses, generators and other tricks for concurrent programming.Before we go on and implement threads, let us understand what exactly is concurrency.Concurrency is a piece of logic Within a single program that allows to open up many distinct paths of execution, including separate streams of I/O, running SQL queries, so on in a way that the execution seems to be both simultaneous and independent of each other.How to do it..First we create a single thread to go through the site urls and later look at how to use threading concepts to speed up the ... Read More

Process Excel Files Data in Chunks with Python

Kiran P
Updated on 09-Nov-2020 10:18:17

3K+ Views

IntroductionIt seems that the world is ruled by Excel. I've been surprised in my data engineering work to see how many of my colleagues are using Excel as a critical tool for making decisions. While I'm not a big fan of MS Office and their excel spread sheets, i will still show you a neat trick to handle large excel spread sheets effectively.How to do it..Before we jump into the program directly, let us understand few basics on dealing excel spreadsheets with Pandas.1. Installation. Go ahead and install openpyxl and xlwt. If you are unsure if it is installed or ... Read More

Implement Immutable Data Structures in Python

Kiran P
Updated on 09-Nov-2020 10:16:00

326 Views

ProblemYou need to implement immutable data structures in Python.Introduction..Immutable data structures are very handy when you want to prevent multiple people modifying a piece of data in parallel programming at same time. Mutable data structures( e.g. Array) can be changed at any time while immutable data structures cannot be.How to do it..Let me show you step by step how to deal with immutable and mutable data structures.Example# STEP 01 - Create a Mutable array. # Define an array atp_players = ['Murray', 'Nadal', 'Djokovic'] print(f" *** Original Data in my array is - {atp_players}")*** Original Data in my array is ... Read More

Compress Files Using zipfile Module in Python

Kiran P
Updated on 09-Nov-2020 10:12:31

12K+ Views

ProblemYou want to create a compress files in python.IntroductionZIP files can hold the compressed contents of many other files. Compressing a file reduces its size on disk, which is useful when transferring it over the internet or between the systems using Control-m AFT or Connect direct or even scp.Python programs creates ZIP files using functions in the zipfile module.How to do it...1. We will be using zipfile and io packages. Install them with pip if any of the packages are missing on your system. If you are unsure, use pip freeze command to validate the packages.2. We will write a ... Read More

Append New Rows to DataFrame Using a Template in Python Pandas

Kiran P
Updated on 09-Nov-2020 10:07:27

696 Views

How to append new rows to DataFrame using a Template In Python Pandas.IntroductionBeing a data engineering specialist, i often end up creating more derived columns than rows as the role of creating and sending the data to me for analysis should be taken care of other database specialists. However, it is not true during all time.We have to create sample rows rather than waiting for data specialists team to send us the data. In this topic i will be showing the neat tricks for creating rows.How to do it..In this recipe, we will begin by appending rows to a small ... Read More

Parse HTML Pages to Fetch HTML Tables with Python

Kiran P
Updated on 09-Nov-2020 10:04:02

843 Views

ProblemYou need to extract the HTML tables from a web page.IntroductionThe internet, and the World Wide Web (WWW), is the most prominent source of information today. There is so much information out there, it is just very hard to choose the content from so many options. Most of that information is retrievable through HTTP.But we can also perform these operations programmatically to retrieve and process information automatically.Python allows us to do this using its standard library an HTTP client, but the requests module helps in obtaining web pages information very easy.In this post, we will see how to parse through ... Read More

String Formatting in PowerShell

Chirag Nagrekar
Updated on 09-Nov-2020 09:46:56

649 Views

To format string in PowerShell, you can use -F operator. When you use -F format, you need to provide the argument number in the curly brackets.ExamplePS C:\> $Str = "Hello PowerShell" PS C:\> "{0}" -f $str Hello PowerShellFor the multiple values, PS C:\> $Str = "Hello PowerShell" PS C:\> $str1 = "Rockstart" PS C:\> "{0} says {1}" -f $Str, $str1 Hello PowerShell says RockstartFrom the above example, we understood if we need to get the output for multiple variables using the -F operator then we can increment the number in curly brackets.To use the above output with the Write-Output command, ... Read More

Find MAC Address Using PowerShell

Chirag Nagrekar
Updated on 09-Nov-2020 09:45:59

4K+ Views

There are several ways to find the MAC address (Physical Address) of the system using PowerShell.Using the Get-NetAdapter commandUsing this command, we can retrieve the MAC address of the network adapter.Using GetMac commandIpconfig commandWe need to use Ipconfig /all to retrieve the mac address of all the adapters.ExampleIpconfig /all | Select-String -Pattern "Description","Physical"Output

Resolve DNS Address Using PowerShell

Chirag Nagrekar
Updated on 09-Nov-2020 09:42:22

18K+ Views

To resolve the DNS address using PowerShell, we need to use the Resolve-DNS address command. This command works similarly to Nslookup command.To resolve the A record (Name -> IP), you can directly provide the hostname and by default, it will retrieve all the records for the particular address.ExampleResolve-DnsName -Name Test1-win2k12OutputName                            Type  TTL Section IPAddress ----                            ----  --- ------- --------- Test1-Win2k12.labdomain.local    A   1200 Answer 192.168.0.107-Name parameter doesn’t accept the multiple-input. Only the single ... Read More

Install PowerShell Active Directory Module

Chirag Nagrekar
Updated on 09-Nov-2020 09:40:04

1K+ Views

To install the active directory module using PowerShell, you need Remote Server Administrator Tools (RSAT) on the server. It should be available in the Roles and Features section of the windows server operating system as shown below and you can enable it from the GUI as well.If you can’t find the RSAT then you can download it from the below location, appropriate to your OS version.https://www.microsoft.com/en-us/download/details.aspx?id=45520Once you have the RSAT tool available in your system, you can use the PowerShell command to enable this feature. To get the Active Directory tools features available in the system, use the below command.ExampleGet-WindowsFeature ... Read More

Advertisements