Perforce Public Knowledge Base - Parallel Sync and its benefits
× PRODUCTS SOLUTIONS CUSTOMERS LEARN SUPPORT
Downloads Company Partners Careers Contact Free Trials
Menu Search
Perforce
Reset Search
 

 

Article

Parallel Sync and its benefits

« Go Back

Information

 
Problem
Is there any way to speed up 'p4 sync' operation?
Solution

Overview

In release 2014.1, Perforce introduced "parallel sync" functionality that enabled Helix clients to transfer files from the Helix Server in parallel threads, as opposed to serial fashion (the current default).

To really see the benefits from parallel syncs, you need to have thousands of preferably small files to sync. The gains of parallel sync are most evident when the following conditions obtain:

  • On long-haul high latency networks, or any other network configuration where some combination limited network buffering, packet loss, or poor TCP Congestion Control behavior limit the available bandwidth with a single TCP flow is restricted.
  • With large compressed binary files, where the client must perform substantial work to decompress the files. Having multiple sync flows underway means that network I/O can be overlapped with file decompression, and that (on a multi-core client machine) multiple files can be decompressing simultaneously
See parallel processing in the Helix Versioning Engine Administrator Guide.
 

Implementing Parallel Sync

To run sync commands in parallel, use the "--parallel" option followed by a configurable number of threads. Additional, comma-separated arguments can be provided for the minimum number of threads (min), and the minimal file size (minsize) reported in kilobytes, and more.

For example:

p4 sync --parallel "threads=4,batch=8,batchsize=524828,min=9,minsize=589824" //depot/dir/...

In order for the parallel sync option to be enabled, the net.parallel.max variable must be set on the Helix Server to a number greater than zero.

p4 configure set net.parallel.max=8

as seen in "p4 help configurables":

net.parallel.max         0 Highest allowed degree of sync parallelism

The net.parallel.min and net.parallel.minsize server configurables:

net.parallel.min - Specifies the minimum number of files in a parallel sync. A sync that is too small does not initiate parallel file transfers (default value of 9)

net.parallel.minsizeSpecifies the minimum number of bytes in a parallel sync. A sync that is too small does not initiate parallel file transfers (default value of 576K)

affect whether a parallel sync is initiated or not. If the number of files or size of the files for the specified parallel sync don't meet the default or configured minimums, a parallel sync is not performed but a standards sync is.

For more details on syntax and options, please see "p4 help sync" or the Command Reference entry for p4 sync.

WE recommend testing parallel syncs to understand whether this feature suits your needs prior recommending it to your Helix users.

Debugging

Set dmc=3 to record in the Helix Server log each file synced.  This configurable is dynamic; there is no need to stop the server.

p4 configure set dmc=3

To stop this level of debugging, run

p4 configure unset dmc

Related Links

Feedback

 

Was this article helpful?


   

Feedback

Please tell us how we can make this article more useful.

Characters Remaining: 255