'PowerShell - parallel threads all running on same CPU core
I'm writing a multithreaded PowerShell script to devide a rather long lasting task into several threads. My problem is, that all threads are running on the same CPU core (12 are available) when the Shell.Application-ComObject is involved. I need it to collect the titles of all documents in a specific folder (GetDetailsOf()-method). Sadly this particular method takes a little when applied to a couple of hundred or more files at once which is the actual motivation to parallelize this whole script.
Simplified code:
$NumCores = $env:NUMBER_OF_PROCESSORS
$URL = "C:\Windows\System32"
$ThreadCode = {
param([UInt32] $CoreAffinity, [Object[]] $FileInfos, [Object] $ShellFolder, [System.Collections.Hashtable] $CTXTable)
#Set CPU-Affinity first - NOT WORKING YET
$Signature = '[DllImport("kernel32.dll")]public static extern UIntPtr SetThreadAffinityMask(IntPtr hThread,UIntPtr dwThreadAffinityMask);'
$Affinator = Add-Type -MemberDefinition $Signature -Name "Win32SetThreadAffinityMask" -Namespace Win32Functions -PassThru
#$Affinator::SetThreadAffinityMask(<#HANDLE???#>, [System.UIntPtr]::new($CoreAffinity))
$Count = $FileInfos.Count
$DataArray = [System.Object[]]::new($Count)
#$Shell = New-Object -ComObject "Shell.Application"
#$ShellFolder = $Shell.NameSpace("C:\Windows\System32")
#The following loop only runs on one cpu core
for ($i = 0; $i -lt $Count; $i++)
{
$FI = $FileInfos[$i]
$ShellFile = $ShellFolder.ParseName($FI.Name)
$Title = $ShellFolder.GetDetailsOf($ShellFile, 21)
$DataArray[$i] = @{Fileinfo = $FI; Title = $Title}
#$DataArray[$i] = @{URL = $FI.FullName; ColumnData = $Columns; Text = $FI.Name; ImageKey = $ImgKey; Tag = @{URL = $FI.FullName; Extension = $Extension; Item = $FI; OpenAsLink = $OpenAsLink}}
}
#The following loop will run on all cores evenly when activated
<#
$j = 0
for ($i = 1; $i -lt [uint32]::MaxValue; $i++)
{
$j++
}#>
Write-Information $DataArray
#Finish Thread with some return values through the Information-Stream
Write-Information ("ManagedThreadId: " + [System.Threading.Thread]::CurrentThread.ManagedThreadId)
}
$RSP = [RunSpaceFactory]::CreateRunspacePool(1, $NumCores)
$RSP.ApartmentState = [System.Threading.ApartmentState]::MTA
$RSP.ThreadOptions = [System.Management.Automation.Runspaces.PSThreadOptions]::UseNewThread
$RSP.Open()
$PSArray = for ($i = 0; $i -lt $NumCores; $i++)
{
[PowerShell]::Create()
}
$Shell = New-Object -ComObject "Shell.Application"
$ShellFolder = $Shell.NameSpace($URL)
$DirectoryInfo = [System.IO.DirectoryInfo]::new($URL)
$InfoItems = $DirectoryInfo.GetFiles()
$ItemCount = $InfoItems.Count
$ItemsPerThread = [Math]::Floor($InfoItems.Count / $NumCores)
for ($i = 0; $i -lt $NumCores; $i++)
{
$PSArray[$i].RunSpacePool = $RSP
$Affinity = [Math]::Pow(2, $i)
$SubIndexLBound = $i * $ItemsPerThread
$SubIndexUBound = ($i + 1) * $ItemsPerThread -1
$SubItems = $InfoItems[$SubIndexLBound..$SubIndexUBound]
$PSArray[$i].AddScript($ThreadCode).AddArgument($Affinity).AddArgument($SubItems).AddArgument($ShellFolder).AddArgument($null) | Out-Null
}
$AsyncResults = for ($i = 0; $i -lt $NumCores; $i++)
{
$PSArray[$i].BeginInvoke()
}
for ($i = 0; $i -lt $NumCores; $i++)
{
$AsyncResults[$i].AsyncWaitHandle.WaitOne() | Out-Null
}
for ($i = 0; $i -lt $NumCores; $i++)
{
write-Host "i: $i`tThreadID: $($PSArray[$i].Streams.Information[1].MessageData)"
}
Finally I can see that all threads had a different ManagedThreadId but when I keep a look at the task manager all work was done by the same cpu core. But once I un-comment the second for-loop all CPU cores will be used evenly AFTER the first loop where I fetch all document titles.
So I want to force the threads to run on a specific core through "SetThreadAffinityMask" but so far I don't know how to get the handle for the current thread? Is this the correct approach at all or might there be better ways to read meta data from files?
Solution see comment by @zett42
$RSP.ApartmentState must remain as [System.Threading.ApartmentState]::STA. Then it works if you instantiate $Shell inside the thread.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
