Perforce Public Knowledge Base - Parallel Sync and its benefits
Reset Search
 

 

Article

Parallel Sync and its benefits

« Go Back

Information

 
Problem
Is there any way to speed up 'p4 sync' operation?
Solution

Overview

In release 2014.1, Perforce introduced "parallel sync" functionality that enabled Perforce clients to transfer files from the Perforce Server in parallel threads, as opposed to serial fashion (the current default).

To really see the benefits from parallel syncs, you need to have thousands of preferably small files to sync. The gains of parallel sync are most evident when the following conditions obtain:

  • On long-haul high latency networks, or any other network configuration where some combination limited network buffering, packet loss, or poor TCP Congestion Control behavior limit the available bandwidth with a single TCP flow is restricted.
  • With large compressed binary files, where the client must perform substantial work to decompress the files. Having multiple sync flows underway means that network I/O can be overlapped with file decompression, and that (on a multi-core client machine) multiple files can be decompressing simultaneously
See parallel processing in the Helix Versioning Engine Administrator Guide.
 

Implementing Parallel Sync

To run sync commands in parallel, use the "--parallel" option followed by a configurable number of threads. Additional, comma-separated arguments can be provided for the minimum number of threads (min), and the minimal file size (minsize) reported in kilobytes, and more.

For example:

p4 sync --parallel "threads=4,batch=8,batchsize=524828,min=9,minsize=589824" //depot/dir/...

NOTE: In order for the parallel sync option to be enabled, the net.parallel.max variable must be set on the Perforce Server to a number greater than zero.

p4 configure set net.parallel.max=8

as seen in "p4 help configurables":

net.parallel.max         0 Highest allowed degree of sync parallelism

For more details on syntax and options, please see "p4 help sync" or the Command Reference entry for p4 sync.

Perforce recommends testing parallel syncs to understand whether this feature suits your needs prior recommending it to your Perforce users.
 

Debugging

Set dmc=3 to record in the Perforce log what files are being synced.

p4 configure set <server>#dmc=3


 

Related Links

Feedback

 

Was this article helpful?


   

Feedback

Please tell us how we can make this article more useful.

Characters Remaining: 255