Perforce Public Knowledge Base - Git-P4
Reset Search
 

 

« Go Back

Information

 
Problem

This document describes Git-p4. What does it do? How does a person use it? What are some of its limitations, and how does a person avoid or recover from those limitations?

Git-p4 is a Python script that connects Git to Perforce. It enables you to use Git for all your daily work, then copy your work to Perforce. It also enables you to import others' work from Perforce into Git.

 

Solution

To use Git-p4 with Perforce, you start by importing from Perforce to Git. Before following these steps, read the Recommendations section below for help with choosing values for these steps.

This article is further divided into the following sections for your convenience:



Importing Perforce to Git

To import a Perforce project into a Git repository, perform the following steps. Note: These steps come from the Git-p4 Usage wiki page at:

https://git.wiki.kernel.org/articles/g/i/t/Git-p4_Usage_e1b3.html.
 

  1. Create a directory for the client workspace root; for example:

    $ mkdir git-p4-area
    
    
  2. Create a P4CONFIG file that defines the Perforce connection settings P4PORT, P4USER, and P4CLIENT. For P4CLIENT, specify the name of the client workspace that you will create in step 3. For example, if your P4CONFIG environment variable was set to .p4config:

    $ cd git-p4-area
    $ vi .p4config
    P4PORT=perforce:1666
    P4USER=zig
    P4CLIENT=zig-git-p4
    

  • To create the client workspace, issue the p4 client command and configure the workspace. Example:

    $ p4 client
    
    Client: zig-git-p4
    
    Owner:  zig
    
    Root:   /Users/zig/git-p4-area
    
    Options: noallwrite noclobber nocompress unlocked nomodtime normdir
    
    View:
        //depot/main/cutie/... //zig-git-p4/main/cutie/...
    

  • Create a directory for the .git repository. This directory must not overlap the client workspace that you created in step 3. For example:
     
    ~/git-p4-area $ cd ..
    $ mkdir git-area
    

  • Copy the P4CONFIG file from Git-p4's directory to the repository directory. For example:

    $ cd git-area
    $ cp ../git-p4-area/.p4config ./
    

    The Git-p4 Usage wiki page recommends creating a .gitconfig file here, but the Perforce config file is sufficient.

Recommendations

Limitations When Importing

Avoid importing an entire depot unless that depot is small: fewer than 100,000 file revisions, less than 500MB. Git is designed to manage a single project per repository. Git performance degrades as its repository size increases. Git stores copies of every file, consuming disk space.

Avoid file types that Git does not support. Text, unicode, and other file types are fine, although Git will not translate line endings or character encoding within file contents.

file //depot/main/cutie/ext-merge/rsrc/base_deletion.png is a strange apple file that forks. Ignoring!

Note: Git does not support keyword expansion "+k" without custom scripting.

Prevent importation of unwanted files. Delete or move all such files out of the depot hierarchy, then restrict import to history starting after the offending files have moved away. Add offending files to .gitignore so that Git and git-p4 skip them. Note: The (untested) git p4 clone --use-client-spec command line option restricts Git-p4 to files that are visible within a Perforce client view. Unfortunately,  --use-client-spec is untested, and using it can result in missing files, or exclusion/minus-mapped lines incorrectly treated as inclusions.

Importing History

By default, Git-p4 imports head revisions from Perforce to Git. Git-p4 will copy any Perforce changelists submitted after this point, but none from before this point. If you require more history and context, you must copy all Perforce history to Git, which is a time- and disk-space-consuming process. To import all history, append @all to the depot path when importing.

git p4 clone //depot/path@all

Defining the Target Perforce Client Specification

Allocate a unique client specification for each Git-p4 import area (user + computer + Git-p4 import tree).

Keep the client view simple: specify a single line defining the subtree to be copied between Perforce and Git. Most Git-p4 operations operate on a single Perforce depot path. Do not use exclusions, overlay mappings or wildcards: complex client view mappings cause copy operations to fail.

Log In Before Issuing Any Git-p4 Commands

If the Perforce server is configured to require authentication, run p4 login with a ticket that will outlive the next Git-p4 operation. Git-p4 has some support for authentication, but Perforce has not tested it.

Start with a New Git Repository

  • Import Perforce files and history into Git. When specifying the depot path, do not append a trailing slash or the ... wildcard. The final "." argument to Git-p4 is the destination directory, that is, the current directory (and not a typo).

    To copy the head revision, omit @all. Example:

    $ git p4 clone //depot/main/cutie .
    
    Importing from //depot/main/cutie into .
    Initialized empty Git repository in /Users/zig/git-area/.git/
    Doing initial import of //depot/main/cutie/ from revision #head into 
    refs/remotes/p4/master
    //depot/main/cutie/BUILD.HOWTO                        | 0.0 MB (1/2925
    //depot/main/cutie/DOC_INDEX.html                     | 0.0 MB (2/2925
    //depot/main/cutie/Jamfile                            | 0.0 MB (3/2925
    ...
    
    //depot/main/cutie/windows/workspace/img/spinner.mng  | 47.9 MB (2925/2925 files)
    Checking out files: 100% (2925/2925), done.
    

    To import the entire history of the imported files, append @all to the depot path. The import will take a lot longer. Example:

    $ git p4 clone //depot/main/cutie@all .
    

    When the import operation finishes, all the requested files are copied from Perforce into Git. Copies of the files reside in the git workspace, ready for editing. For example:

    $ ls
    BUILD.HOWTO    core           api            util
    DOC_INDEX.html devtools       platform       vscript
    Jamfile        doc            plugins        vscript99.pdf
    Jamrules       gui            resources      widgets
    apps           cutie-vs8.sln  shell          windows
    cmd            cutie-vs9.sln  translations
    

    Note that Git-p4 does not copy or sync any files to its area. Git-p4 uses the p4 print command to acquire all files, bypassing the client workspace mechanism. For example:

    $ ls ../git-p4-area/
    $ p4 have
    File(s) not on client.
    • Do not import large binary files. Git performance degrades if it has to calculate MD5 hashes of large media files. "Large" is multi-hundred-megabytes or more.
    • Git handles small icon and image files for a typical GUI application just fine.
    • Do not import "strange" file types. Git does not support old Apple resource forks (Perforce filetype "apple" or "resource." Git will store only the data fork, silently stripping the resource fork the first time it recreates the file. Git-p4 will refuse to import the file:
"git p4 clone" does not work with an already-populated Git repository. The "git p4 sync" command does work with a populated repository, but this command is unidirectional (from Perforce to Git).

Do Not Commingle Perforce and Git

It is safe to submit changes to Perforce using a client workspace other than the one that Git-p4 uses, with a workspace root folder different from the ones that Git-p4 or Git use. This situation is essentially the same as a coworker submitting changes to Perforce, so bear in mind that it can cause conflicts during rebasing.

Do not submit changes to Perforce using the same client workspace that Git-p4 uses. This workspace is only sparsely populated. git p4 rebase does not sync files to this area: Git-p4 uses p4 print to acquire most file content. Files open in pending changelists in the Git-p4 client will interfere with git p4 submit.


Working in Git

Git-p4 workflow follows this pattern:

  1. Edit files
  2. Submit edits to Git
    ...repeat...
  3. git p4 rebase
  4. git p4 submit

Updating Your Git Repository: git p4 rebase

To obtain any recently checked-in changes, issue the "git p4 rebase" command. This command copies every Perforce change that was submitted since the previous git p4 rebase (or the initial git p4 clone).

This command is the Git-p4 equivalent of "merge down". It must be run before you attempt to copy your work from Git to Perforce.

Troubleshooting Merge Conflicts During Rebase

Merge conflicts that result from your changes must be resolved in Git, before you can complete a rebase operation. For example:

$ git p4 rebase
Performing incremental import into refs/remotes/p4/master git branch
Depot paths: //depot/
Import destination: refs/remotes/p4/master
Importing revision 2 (100%)//depot/f1.txt         | 0.0 MB (1/1 files)

Rebasing the current branch onto remotes/p4/master
First, rewinding head to replay your work on top of it...
Applying: Zig edits a file.
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
Auto-merging f1.txt
CONFLICT (content): Merge conflict in f1.txt
Failed to merge in the changes.
Patch failed at 0001 Zig edits a file.

When you have resolved this problem run "git rebase --continue".
If you would prefer to skip this patch, instead run "git rebase --skip".
To restore the original branch and stop rebasing run "git rebase --abort".

command failed: git rebase remotes/p4/master

To complete a rebase operation under such circumstances, perform the following steps:

  1. Resolve all conflicts using your preferred Git merge tool. The following example use vi:
    $ cat f1.txt
    <<<<<<< HEAD
    1. Bob edits a file, too.
    =======
    1. Zig edits a file.
    >>>>>>> Zig edits a file.
    2. --
    3. --
    ~/git-area $ vi f1.txt
    1. Zig edits a file. Bob edits a file, too.
    2. --
    3. --
    

  • Add all resolved files to Git's staging area. If you forget, "git rebase" will remind you; for example:
    $ git rebase --continue
    f1.txt: needs merge
    You must edit all merge conflicts and then
    mark them as resolved using git add
    ~/git-area $ git add f1.txt
    

Copying Changes to Perforce: git p4 submit

To copy changes from Git to Perforce, issue the "git p4 submit" command. This command creates and submits a new Perforce changelist for each git commit.

Always Rebase Before Submitting

...because somebody might have submitted a change to Perforce after you rebased.

If you omit rebasing before you submit, you might encounter merge conflicts, as illustrated in the following example.

$ git p4 submit
{'clientFile': '//zig-git-p4/...', 'code': 'stat', 'depotFile': '//depot/...',
'path': '/Users/zig/d/gp/...'}
depotPath://depot/...

found1
Perforce checkout for depot path //depot/ located at /Users/zig/d/gp/
Syncronizing p4 checkout...
//depot/f1.txt#2 - added as /Users/zig/d/gp/f1.txt
Applying 7b41bea7fc60b1917e033909843fe3a14260448d Zig edits a file

//depot/f1.txt#2 - opened for edit
error: patch failed: f1.txt:1
error: f1.txt: patch does not apply
Unfortunately applying the change failed!
What do you want to do?
[s]kip this patch / [a]pply the patch forcibly and with .rej files / [w]rite the
patch to a file (patch.txt) s
Skipping! Good luck with the next patches...
//depot/f1.txt#2 - was edit, reverted
Submit template unchanged. Submit anyway? [y]es, [n]o (skip this patch) n
f1.txt - file(s) not opened on this client.
All changes applied!
Performing incremental import into refs/remotes/p4/master git branch
Depot paths: //depot/
Import destination: refs/remotes/p4/master
Importing revision 2 (100%)//depot/f1.txt          | 0.0 MB (1/1 files)

Rebasing the current branch onto remotes/p4/master
First, rewinding head to replay your work on top of it...
Applying: Zig edits a file
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
Auto-merging f1.txt
CONFLICT (content): Merge conflict in f1.txt
Failed to merge in the changes.
Patch failed at 0001 Zig edits a file

When you have resolved this problem run "git rebase --continue".
If you would prefer to skip this patch, instead run "git rebase --skip".
To restore the original branch and stop rebasing run "git rebase --abort".

command failed: git rebase remotes/p4/master

In such a situation, it is best to discard the current pending/staged status, run git p4 rebase to pull all Perforce changes into Git, resolve any conflicts (see Troubleshooting Merge Conflicts During Rebase), and then try again, as follows:

  • Resume the rebase:
    $ git rebase --continue
    Applying: Zig edits a file.
    

    You have now rebased all of your Git changes upon the most recent Perforce change. Now git p4 rebase runs cleanly:

    $ git p4 rebase
    Performing incremental import into refs/remotes/p4/master git branch
    Depot paths: //depot/
    No changes to import!
    Rebasing the current branch onto remotes/p4/master
    Current branch master is up to date.
    
    • Discard current pending/staged status:
      $ git reset --hard HEAD
      HEAD is now at 5193345 Bob edits a file.
      ~/git-area $ rm -rf .git/rebase-apply
      ~/git-area $ git checkout master
      Previous HEAD position was 5193345... Bob edits a file. [git-p4: depot-paths
      = "//depot/": change = 2]
      Switched to branch 'master'
      

  • Rebase to pull remaining Perforce changes into Git. If merge conflicts occur, resolve as described above.
    $ git p4 rebase
    Performing incremental import into refs/remotes/p4/master git branch
    Depot paths: //depot/
    No changes to import!
    Rebasing the current branch onto remotes/p4/master
    
    Current branch HEAD is up to date.
    

Troubleshooting Merge Conflicts During Submission

Git history might contain a merge. Git-p4 cannot copy merge commits to Perforce. You must rebase or rewrite Git history to be linear before Git-p4 can copy it to Perforce.

  • After any conflicts are resolved and committed, resubmit.
    $ git p4 submit
    {'clientFile': '//zig-git-p4/...', 'code': 'stat', 'depotFile': '//depot/...',
    'path': '/Users/zig/d/gp/...'}
    depotPath://depot/...
    
    found1
    Perforce checkout for depot path //depot/ located at /Users/zig/d/gp/
    Synchronizing p4 checkout...
    ... - file(s) up-to-date.
    Applying 1debb3c8cb8a73961f4ab6eed3853fa66ca5095c Zig edits a file
    
    //depot/f1.txt#2 - opened for edit
    Submit template unchanged. Submit anyway? [y]es, [n]o (skip this patch) y
    Change 3 created with 1 open file(s).
    Submitting change 3.
    Locking 1 files ...
    edit //depot/f1.txt#3
    Change 3 submitted.
    All changes applied!
    Performing incremental import into refs/remotes/p4/master git branch
    Depot paths: //depot/
    Import destination: refs/remotes/p4/master
    Importing revision 3 (100%)//depot/f1.txt              | 0.0 MB (1/1 files)
    
    Rebasing the current branch onto remotes/p4/master
    First, rewinding head to replay your work on top of it...
    Nothing to do.
    

How to Avoid Trouble

Keep Git History Linear

Git-p4 cannot copy a branched or merged Git history to Perforce. Most Git users create small, short-lived branches to develop a feature.

branch created

When that feature is complete, they add this feature to the master branch not by merging, but by recreating the branch commits on top of the the master branch:

branch rebased

For details, read about "git rebase".

If branches cannot be rebased into a single linear history, delete them from history.

Look online for instructions on rewriting Git history, rewound HEAD pointers, "git cherry-pick", and "git revert".

branch created and merged

branch and master rebased


Recovering From a Failed Submit

Last Resort: Reconcile, then Start Over

If git p4 submit continues to fail, as a last resort it might be easier to start over than to fix git-p4. Git state can become tangled enough that untangling it is more work than any unsubmitted Git descriptions are worth. When this happens, use Reconcile Offline Work to copy changed files into Perforce, delete your Git repository, and start all over with a fresh git p4 clone, as follows.

  • Modify Git-p4's Perforce client to temporarily use your Git area as its workspace root
    $ p4 client
    ...
    Root:   /Users/zig/git-area
    ...
    

  • Use P4V or command line to reproduce your Perforce adds, edits and deletes.
    $ p4 diff -se | p4 -x- edit
    //depot/f1.txt#3 - opened for edit
    $ p4 submit -d "Reconciled Offline Work from git"
    Submitting change 4.
    Locking 1 files ...
    edit //depot/f1.txt#4
    Change 4 submitted.
    

  • Restore Git-p4's client root to the git-p4-area, away from your workspace.
    $ p4 client
    ...
    Root:   /Users/zig/git-p4-area
    ...
    

  • Delete the Git repository.
    ~/git-area $ rm -rf .git
    

  • Re-import Perforce history to Git
    $ git p4 clone //depot ./
    Importing from //depot into ./
    Initialized empty Git repository in /Users/zig/d/g/.git/
    Doing initial import of //depot/ from revision #head into refs/remotes/p4/master
    //depot/f1.txt                      | 0.0 MB (1/1 files)
    

Appendix: Installing Git-p4

Step 1: Download and Install Git

Download and install Git from the following location: http://git-scm.com/download

The platform-specific binary installers are fine for most Mac OS X and Windows users.

Note: Do not download and build Git from source. Building the help files for git help requires a long chain of tools that must download and build. Not all of these tools work on Mac OS X.

"Git via Git" works only on networks without HTTP proxies. It is possible to configure Git's HTTP proxy, but that's unnecessarily complex for most people. Only do this if you plan to use the git protocol over the internet, and your company network requires an http proxy.

To verify that Git is installed, issue the following command:

$ git --version

If properly installed, Git displays its version string; for example:

git version 1.7.3.3

Step 2: Download Git-p4

Git-p4 is part of the git distribution, which is available here:

http://git-scm.com

Download the latest ".tar.gz" file and unpack the source. It is not necessary to build it; the git-p4 script is currently separate from the rest of the git build process.

You can use the included git-p4 python script directly, or copy it elsewhere on your system, e.g.:

   tar xfz git-1.8.2.tar.gz
   cp git-1.8.2/git-p4.py /usr/local/bin/git-p4

To use the command as "git p4 clone", etc., add it as an alias to your $HOME/.gitconfig file, like this:

 

   git config --global alias.p4 /usr/local/bin/git-p4

Be sure to put in the same path there.

Or you can install git-p4 as described in Step 3.

Step 3: Add Git-p4 to git/libexec/git-core/

Copy or symlink Git-p4 to the git/libexec/git-core/ folder. On Mac OS X, the full path is usually /usr/local/git/libexec/git-core/. The names of the files in this directory indicate the command that the file implements; for example:

git                     git-gui                 git-rebase--interactive
git-add                 git-gui--askpass        git-receive-pack
git-add--interactive    git-hash-object         git-reflog
git-am                  git-help                git-relink
...                     ...                     ...
git-difftool--helper    git-p4                  git-tar-tree
...                     ...                     ...

Step 4: Verify that Git-p4 is installed

To verify that Git-p4 is properly installed, issue the following command:

$ git p4 help

Git-p4 displays the following output:

unknown command help
usage: /usr/local/git/libexec/git-core/git-p4 <command> [options]
valid commands: clone, rollback, shelve, debug, commit, rebase, branches,
sync, submit

Try /usr/local/git/libexec/git-core/git-p4 <command> --help for 
command specific help.

Related Links

Feedback

 

Was this article helpful?


   

Feedback

Please tell us how we can make this article more useful.

Characters Remaining: 255