How to use ForEach-Object Parallel cmdlet in PowerShell?

PowerShellMicrosoft TechnologiesSoftware & Coding

Foreach-Object -Parallel command was introduced in PowerShell version 7, preview 3, and it is used for the parallel execution of the pipeline input and more details on this have been explained in this article.

Please Note: Foreach-Object and Foreach commands are the same but we can’t write Foreach -Parallel command because there is no such command, we need to use the full command, Foreach-Object -Parallel.

A simple example of running a parallel object.

Example

1..10 | ForEach-Object -Parallel{
    $_ * 1000  
    Sleep 1
}

Output

1000
2000
3000
4000
5000
6000
7000
8000
9000
10000

Let's compare the timings with and without -Parallel parameter.

Parallel Execution

$time1 = Measure-Command {
    1..10 | ForEach-Object -Parallel{
        $_ * 1000  
        Sleep 1
    }

}
    
Write-Output "Parallel Time : Minutes : $($time1.Minutes) , Seconds : $($time1.Seconds) , Milliseconds : $($time1.TotalMilliseconds) "

Output

Parallel Time : Minutes : 0 , Seconds : 2 , Milliseconds : 2500.9792 

Sequential Execution

$time1 = Measure-Command {
    1..10 | ForEach-Object {
        $_ * 1000  
        Sleep 1
    }

}
    
Write-Output "Sequential Time : Minutes : $($time1.Minutes) , Seconds : $($time1.Seconds) , Milliseconds : $($time1.TotalMilliseconds) "

Output

Sequential Time : Minutes : 0 , Seconds : 10 , Milliseconds : 10062.0786

You can see the significant difference between the above two executions that the parallel run is much faster than the sequential.

There is one more parameter that supports the parallelism of the foreach-Object loop is -ThrottleLimit means the number of threads for the parallel execution. By default, the throttle limit is 5. See the time difference when we increase the throttle limit.

Example

$time1 = Measure-Command {
    1..10 | ForEach-Object -Parallel{
        $_ * 1000  
        Sleep 1
    } -ThrottleLimit 10

}
    
Write-Output "Sequential Time : Minutes : $($time1.Minutes) , Seconds : $($time1.Seconds) , Milliseconds : $($time1.TotalMilliseconds) "

Output

Parallel Time : Minutes : 0 , Seconds : 1 , Milliseconds : 1615.4921 

If you compare the first parallel time example, the second is quite faster because we have increased the throttle limit. When you increase the throttle limit make sure which command is used for the parallel execution and the number of throttle limit have passed as an input otherwise the system will overwhelm with the script execution because that many of the PowerShell threads generated.

You can also run this command as a background job because it supports -AsJob parameter.

$fjob = 1..10 | ForEach-Object -Parallel{
        $_ * 1000  
        Sleep 1
} -ThrottleLimit 10 -AsJob 

$fjob | Receive-Job

The difference between the other remote command and foreach-Object -Parallel is that this command runs the jobs parallelly in the same context that is called the namespace while the other remote commands run sequentially on the remote computer using -ComputerName parameter or Invoke-Command and that is the reason parallel execution is much faster for the remote computers.

When you use the Parallel foreach loop in PS version 7 the only variable visible inside the parallel loop is the pipeline variable others outside variable or objects can be accessed with Using: keyword. For example,

Example

$computers = "Test1-Win2k12", "test1-Win2k16"
$command = "Win32_OperatingSystem"

$computers | ForEach-Object -Parallel {
   Get-CimInstance $using:command -ComputerName $_
}

You can see in the above script that only $computers can be accessible inside the Foreach loop via $_ but to access the $command we need to use $using:Command because it is outside the scope of the loop but this is not the case with a simple foreach loop.

raja
Published on 11-Nov-2020 12:45:15
Advertisements