Perforce Public Knowledge Base - Maximizing Perforce Performance
Reset Search
 

 

Article

Maximizing Perforce Performance

« Go Back

Information

 
Problem

How can I maximize performance using views, protections, maxresults, maxscanrows, and maxlocktime?

Solution

The Perforce Server's performance depends on the number of files you try to manipulate in a single command invocation, not on the size of the depot. For example, syncing a client view of 30 files from a 3,000,000-file depot is about as fast as syncing a client view of 30 files from a 30-file depot.

The number of files a single command affects is determined by:

 

  • p4 command line arguments (or selected folders, in the case of P4V or P4Win commands). Without arguments, a command operates on all files mapped by the client view.
  • Client views, branch views, label views, and protections. With unrestricted views and unlimited protections, a command operates on all files in the depot.

At sites where depots are very large, unrestricted views and unqualified commands make a Perforce Server work much harder than it needs to. When the Perforce Server answers a request, it locks the database for the duration of the computation phase. For normal operations, this is a successful strategy, as the Perforce Server can process requests quickly enough to avoid requests piling up. But abnormally large requests can take seconds, sometimes even minutes. If frustrated users hit CTRL-C and retry, the problem gets worse -- the Perforce Server starts using lots of memory and responds even more slowly.

Perforce users and administrators can prevent Perforce Server thrashing by:

  • Using "tight" views
  • Assigning protections
  • Setting maxresults to limit the amount of data affected
  • Setting maxscanrows to limit the number of revisions scanned
  • Setting maxlocktime to limit the amount of time spent during data scans

These options are described in more detail in the sections below.

Using "tight" views

Define client views (and branch and label views) so users access only the files they require. For example, the following "tight" client view is restricted to specific depot areas:

Example
//depot/main/svr/devA/...       //ws/main/svr/devA/...
//depot/main/dvr/lport/...      //ws/main/dvr/lport/...
//depot/rel2.0/svr/devA/bin/... //ws/rel2.0/svr/devA/bin/...
//depot/qa/s6test/dvr/...       //ws/qa/s6test/dvr/...

By contrast, the following unrestricted view is easier to set up but invites trouble when depots are very large:

//depot/...   //ws/...

Client views, branch views, and label views are defined using the p4 client, p4 branch, and p4 label commands, respectively, by the users who created them.

Note

Although using exclusionary mappings in views will reduce its scope, it will also introduce some significant performance implications as more data will need to be processed. This is discussed in depth in the following white paper:

http://www.perforce.com/sites/default/files/perforce-performance-shields-paper.pdf

Assigning protections

Protections are a type of view. Protections are set using p4 protect command, and they control which depot files can be affected by the commands that users run. Unlike views, however, protections can be set only by superusers. Protections also control read and write permission to depot files, but permission levels themselves have no impact on Perforce Server performance.

Protections can be assigned either to users or to groups.

Example
write user  sam        * //depot/admin/...
write group rocketdev  * //depot/rocket/main/...
write group rocketrel2 * //depot/rocket/rel2.0/...

Groups are created by superusers with the p4 group command. Groups make it easier to assign protections and enable you to set three performance-tuning options -- maxresults, maxscanrows, and maxlocktime -- described next.

Setting maxresults to limit the amount of data affected

Each group has a maxresults value associated with it. The default value is "unlimited", but a superuser can use p4 group to limit maxresults for a particular group. Users in the group cannot run any commands that affect more database rows than the maxresults limit. For most commands, the number of database rows affected is roughly equal to the number of files affected.

Example
Group: rocketdev
Maxresults: 10000
Users:
       bill
       ruth
       sandy

Ruth has an unrestricted client view. When she tries:

p4 sync 

her p4 sync command is rejected if the depot contains more than 10,000 files, and she sees an error message at her client such as:

Request too large (over 10000); see 'p4 help maxresults'.

She can get around this by syncing smaller sets of files at a time, for example:

p4 sync //depot/projA/...
p4 sync //depot/projB/...

She gets her files, without tying up the server to process a single, extremely large command.

Setting maxscanrows to limit the number of revisions scanned

Each group also has a maxscanrows value associated with it. The default value is "unlimited", but a superuser can use p4 group to limit maxscanrows for a particular group. Users in the group cannot run any commands that scan more revisions than the maxscanrows limit. This option is particularly useful for limiting wildcarded commands that would otherwise scan every revision, such as:

p4 files //...foo...
Example
Group: rocketdev
MaxScanRows: 50000
Users:
       bill
       ruth
       sandy

If there are more than 50,000 revisions in all depots and Bill issues the following command:

p4 files //...foo...

his command is rejected and the following message is seen at the client:

Too many rows scanned (over 50000); see 'p4 help maxscanrows'.

But if Bill narrows the scope of his command so it scans fewer than 50,000 revisions, it succeeds:

p4 files //depot/main/...foo...
//depot/main/foo#4986 - edit change 4994 (text)

With the scope narrowed, the impact on the server is far less.

The maxscanrows limit does not affect the following commands:

Setting maxlocktime to limit the amount of time spent during data scans

Each group also has a maxlocktime value associated with it. The default value is "unlimited", but a superuser can use p4 group to limit maxlocktime for a particular group. The maxlocktime limit is entered in milliseconds. Maxlocktime operates by starting a counter when the first table is (read) locked, and includes time waiting for any subsequent table readlocks. Because the number of users and commands run at any one time varies, this option varies in behavior due to the load on the server. If the time the readlocks are held/waited for a command exceeds the group maxlocktime, the commands fail.

Example
Group: rocketdev
MaxLockTime: 30000
Users:
       bill
       ruth
       sandy

Starting with the first readlock taken for the command, if the locks are held (including waiting) for a time exceeding maxlocktime, the command fails.

p4 opened ...

if the readlocks taken exceed 30000 (milliseconds = 30 seconds)

Operation took too long (over 30.00 seconds); see 'p4 help maxlocktime'.

The server log includes details about the command within the Trace output:

--- killed by MaxLockTime

Testing maxscanrows, maxresults, and maxlocktime:

Use the -z flag to immediately see the effect of maxscanrows, maxresults, or maxlocktime
$ p4 -zmaxScanRows=100000 sync


Too many rows scanned (over 100000); see 'p4 help maxscanrows'.
where maxScanRows is the appropriate max command, and 100000 is replaced by a number.
The -z flag can be used to estimate appropriate values for each max command.

For more information:

  • For information about how maxresults, maxscanrows, and maxlocktime are determined for users who belong to more than one group, see Article 49.
  • To display an informative comparison of commands and the amount of data they affect, use p4 help maxresults.
  • For a discussion of performance and tuning issues, and protections and access permissions, see the Perforce System Administrator's Guide.
  • Usage notes and syntax rules for the p4 commands mentioned above can be found in the Perforce Command Reference.

 

Related Links

Feedback

 

Was this article helpful?


   

Feedback

Please tell us how we can make this article more useful.

Characters Remaining: 255