代码之家  ›  专栏  ›  技术社区  ›  Alban

如何将多个目录复制到多个目标?(海量拷贝)

  •  -4
  • Alban  · 技术社区  · 7 年前

    我需要复制3个文件夹到多个目标(600+),我需要并行它。

    对于intance

    C:\Data1   →    D:\Data1
    C:\Data1   →    D:\Data2
    C:\Data1   →    D:\Data3
    D:\Data1   →    \\\VM1\c$\Data1
    D:\Data2   →    \\\VM1\c$\Data2
    D:\Data3   →    \\\VM1\c$\Data3
    …
    

    我为它构建了一个对象

    [pscustomobject][ordered]@{
        Source = 'C:\Data1'
        Dest = 'D:\Data1'
    },
    [pscustomobject][ordered]@{
        Source = 'C:\Data1'
        Dest = 'D:\Data2'
    },
    [pscustomobject][ordered]@{
        Source = 'C:\Data1'
        Dest = 'D:\Data3'
    },
    …
    
    1 回复  |  直到 7 年前
        1
  •  0
  •   Alban    7 年前

    我为每对夫妇调用一个Robocopy作业,但我限制了进程的名称和Robocopy线程的数量。

    function Copy-AsParallelJod {
        param (
            [Parameter(ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true,Mandatory = $true, Position = 0)]
                [ValidateScript({ $_ | Test-Path })]
                    [string]$Source,
            [Parameter(ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true,Mandatory = $true, Position = 1)]
                [Alias('Dest')]
                    [string]$Destination,
            $MaxRunningJob = 4,
            $MaxThreadByJob = 8
        )
        begin {
            $Start = Get-Date
            write-host (get-date $Start -format 'yyyy/MM/dd HH:mm:ss') -Width 150 -back DarkGreen
             # get-job | Remove-Job -Force
        }
        process {
            while ((Get-Job -State Running  | ?{$_.Name -like "Copy_*"}).count -ge $MaxRunningJob) {
    
                # Start-Sleep -Seconds 3
                wait-Job -Name 'Copy_*' -Any | Receive-Job  -AutoRemoveJob -Wait
            }
            $WorkName = (($Destination.split('\'))[-2..-1]) -join(' ')
    
            Write-host 'Copy-Job',' : ',$Source.padleft(55),'  To  ',$Destination.padright(55),'  @ ',(get-date -format 'yyyy/MM/dd HH:mm:ss') red
    
            Start-Job -ScriptBlock {
                param($Source, $Dest, $log="/LOG+:C:\MAJ\Robocopy.log.txt", $MaxThread)
                $Start = Get-Date
                New-Item -Path $dest -type Directory -Force | Out-Null
                [System.Threading.Thread]::CurrentThread.Priority = 'Lowest'
                $PathIdentity = ($Dest.split('\')[-2..-1]) -join('\')
    
                Robocopy $Source $Dest /E $log /TEE /MT:$MaxThread /S /DCOPY:DA /COPY:DAT /R:3 /W:2 | Out-Null
                Start-Sleep -Seconds 1
    
                $Time =  (Get-Date) - $start
                $Time = "$($time.Hours)h $("$($time.Minutes)".padleft(2,'0'))m $("$($time.Seconds)".padleft(2,'0'))s"
            } -ArgumentList $Source,$Destination,"/LOG+:$(split-path $Destination)\RoboCopy($(get-date -format 'yyyy.MM.dd HH.mm'))-$WorkName.log.txt",$MaxThreadByJob -name Copy_$WorkName | Out-Null
            Start-Sleep -Milliseconds 500
        }
        end {
            Get-Job -Name 'Copy_*' | Receive-Job -AutoRemoveJob -Wait
            $Time =  (Get-Date) - $start
            $Time = "$($time.Hours)h $("$($time.Minutes)".padleft(2,'0'))m $("$($time.Seconds)".padleft(2,'0'))s"
            write-host "$(get-date -format 'yyyy/MM/dd HH:mm:ss') [$Time]" -Width 150 -back DarkGreen
        }
    }
    

    我这样叫它

    [pscustomobject][ordered]@{
        Source = 'C:\Data1'
        Dest = 'D:\Data1'
    },
    [pscustomobject][ordered]@{
        Source = 'C:\Data1'
        Dest = 'D:\Data2'
    },
    [pscustomobject][ordered]@{
        Source = 'C:\Data1'
        Dest = 'D:\Data3'
    } | Copy-AsParallelJod -MaxRunningJob 4 -MaxThreadByJob 8