Perforce Public Knowledge Base - Task Streams
Reset Search
 

 

Article

Task Streams

« Go Back

Information

 
Problem
Traditional branches create permanent metadata in the Perforce database, recording revision and integration history for the branch. When only a small number of files in the branch are changed, it becomes expensive to record permanent metadata for files in the branch that haven't changed. For example, if you create a branch of 100K files and only change three files in the branch, 100K files are recorded in the Perforce metadata for the purpose of versioning only three of them. Compound that with many developers creating many branches and you have the potential for excessive growth of permanent branch metadata recording unchanged files in branches. Task streams address this problem by:
  • Initially recording branch metadata in temporary storage
  • Only writing branch metadata to permanent storage when content changes occur in the task stream
  • Allowing for the deletion or unloading of the task stream when work on the task is complete, which removes or unloads branch metadata for unchanged files in the branch
 
Solution
OVERVIEW

What are task streams?

Task streams are branches dedicated to a single task. They are useful containers for managing tasks in progress and controlling how changes in those tasks are eventually promoted into more stable branches. They provide software developers with a method of quick branch creation and context switching, and also methods for limiting the growth of Perforce metadata related to branch creation. Task streams fit into any Perforce deployment as they can be used with streams and also with classic Perforce code lines.

When to use task streams?
Task streams are most ideal when work in the branch will only touch a small percentage of the overall files. For instance, a small to medium new feature or bug fix. If you've started work in a task stream and end up with a large percentage (> 50%) of files changed, it is possible to convert the task stream into a regular stream by changing the stream type. Note, this can only be done with a task stream that resides in the same depot as its parent, and the operation is not reversible.


WHAT'S NEW

New Stream Type
A new stream type of task has been introduced. Task streams can be used like any other stream with the following exceptions:
  • No re-parenting
  • No child streams
  • A task stream can reside in a different (stream) depot
Merging between task streams is allowed. As mentioned above, there is actually one chance to re-parent a task stream by changing the stream type. Note, this can only be done with a task stream that resides in the same depot as its parent and it's not reversible

Schema Changes
Two new database tables have been added, db.revtx and db.integtx, to hold branch metadata for task streams. The db.revtx table is blended into the db.rev namespace when using a client workspace of a task stream. The db.integtx table is used by p4 integrate when a task stream is the target for integration.
 
Public and Private Storage
With the introduction of task streams, we now have the notion of public and private storage for Perforce branch metadata. Public storage equates to permanent branch metadata in the db.rev and db.integed tables. Private storage equates to temporary branch metadata in db.revtx and db.integtx tables.


TASK STREAM BEHAVIORS

Branch Creation - Traditional vs. Task Stream
When not using task streams and branches are created:
  • Branch metadata is recorded in permanent public storage (db.rev and db.integed)
  • Branch metadata is universally accessible by anyone in Perforce with permissions to the files in the branch
When using task streams and branches are created:
  • Branch metadata is recorded in temporary private storage (db.revtx and db.integx)
  • The full task stream branch is visible only through a client workspace associated with the task stream

Promotion
Content changes in the task stream are automatically promoted from private to public storage. Promotion occurs for any change in the task stream where the file action is not a copy or branch. At the time of promotion, branch metadata is copied from temporary private storage (db.revtx,db.integtx) to permanent public storage (db.rev,db.integed). Once promoted, always promoted.

Command Behavior with Task Streams
When submitting to a task stream, p4 submit promotes branch metadata when the file action is not a copy or a branch. When submitting an integration from a task stream source (e.g. copy up), p4 submit duplicates integration history into private storage (db.integtx) otherwise subsequent merge-down requests will not get credit for the copy-up. 

When integrating to a task stream, p4 integ, p4 merge, and p4 copy consult private storage (db.integtx). When integrating from a task stream, p4 integ, p4 merge, and p4 copy consult public storage (db.integed, i.e. only promoted files).

Since p4 copy always tries to make the target stream look identical to the source stream, p4 copy is internally passed a flag when copying from a task stream that prevents it from deleting un-promoted files from the target.



CLEANING UP

Deleting or Unloading Task Streams
The ability to delete or unload a task stream after work is complete allows for removal or archive of the branch metadata recording files that were unchanged in the task stream:
  • Deleting a task stream permanently removes branch metadata from private storage 
  • Unloading a task stream moves branch metadata from private storage to the unload depot
If using the unload option, task stream data can later be restored using p4 reload. Note, failure to delete or unload a task stream simply shifts the problem of metadata growth from public to private storage and eliminates the benefit of task streams in terms of decreasing metadata growth.
 
 
Typical workflow
 
We have an existing stream depot (Ace) and a defined development stream (//Ace/dev). With the workspace associated with the development stream, we create a task stream and then p4 populate it from the development stream:
 
$ p4 stream -t task -P //Ace/dev //Ace/fixbug1
$ p4 populate -r -S //Ace/fixbug1
2 files branched (change 818).

Remember, only a workspace associated with a task stream can see the full branch in the task stream. Because the workspace is still associated with the development stream, we don't see any files in the newly created task stream:
 

 
$ p4 files //Ace/fixbug1/...
//Ace/fixbug1/... - no such file(s).

Once the workspace is associated with the task stream, we then see the full branch in the task stream:
 
$ p4 client -s -S //Ace/fixbug1
$ p4 files //Ace/fixbug1/...
//Ace/fixbug1/src.cpp#1 - branch change 818 (text)
//Ace/fixbug1/src.h#1 - branch change 818 (text)


We now commence work in the task stream and submit content changes:

$ p4 add //Ace/fixbug1/foo
//Ace/fixbug1/foo#1 - opened for add
$ p4 submit -d "Add new file to task stream"

$ p4 edit //Ace/fixbug1/src.cpp
//Ace/fixbug1/src.cpp#1 - opened for edit
$ p4 submit -d 'Update to src.cpp in task stream'

The workspace is associated with the task stream so we see the full branch:

$ p4 files //Ace/fixbug1/...
//Ace/fixbug1/foo#1 - add change 819 (text)
//Ace/fixbug1/src.cpp#2 - edit change 820 (text)
//Ace/fixbug1/src.h#1 - branch change 818 (text)

Content changes in a task stream are promoted when the file action is not a branch or copy. When we associate the workspace with the development stream, we see the changes have been promoted from the task stream:

$ p4 client -s -S //Ace/dev
$ p4 files //Ace/fixbug1/...
//Ace/fixbug1/foo#1 - add change 819 (text)
//Ace/fixbug1/src.cpp#2 - edit change 820 (text)

We're now ready to copy changes from the task stream to the development stream. We first merge down any changes from the development stream:
 
$ p4 client -s -S //Ace/fixbug1
$ p4 sync -q

$ p4 merge //Ace/dev/... //Ace/fixbug1/...
//Ace/fixbug1/src.cpp#2 - integrate from //Ace/dev/src.cpp#2
$ p4 resolve -am

$ p4 submit -d 'Merging from dev branch'

and then copy up changes from the task stream:
 
$ p4 client -s -S //Ace/dev
$ p4 sync -q

$ p4 copy //Ace/fixbug1/... //Ace/dev/...
//Ace/dev/foo#1 - branch/sync from //Ace/fixbug1/foo#1

$ p4 submit -d 'Copy changes from task stream to dev'

Finally we delete or unload the task stream.
 

 
$ p4 stream -d //Ace/fixbug1
Stream //Ace/fixbug1 deleted.
$ p4 unload -s //Ace/fixbug1
Stream //Ace/fixbug1 unloaded.

If the unload option was used, you can optionally reload the task stream at a later date:
 

 
$ p4 reload -s //Ace/fixbug1
Stream //Ace/fixbug1 reloaded.

 

Task Stream Reuse

When you submit files in your task stream, archives are created in the corresponding depot root folder. Once the work has been completed in the task stream, it is possible to delete this stream. However, as file content relating to this task stream remains in the archive, it is not possible to create a new task stream with this same name.

When you promote these changes to the task stream's parent stream, a lazy copy is created rather than a full file copy. 
The Perforce Administrator should ensure these archive files are incorporated into your backup processes and not removed (unless advised to do so by Perforce Support).

Related Links

Feedback

 

Was this article helpful?


   

Feedback

Please tell us how we can make this article more useful.

Characters Remaining: 255