Perforce Public Knowledge Base - Incremental Conversion from Subversion to Perforce
× PRODUCTS SOLUTIONS CUSTOMERS LEARN SUPPORT
Downloads Company Partners Careers Contact Free Trials
Menu Search
Perforce
Reset Search
 

 

Article

Incremental Conversion from Subversion to Perforce

« Go Back

Information

 
Problem
How do you mirror a Subversion repository in a Perforce depot?
How do you keep a Perforce depot automatically synchronized with a Subversion repository over time?
 
Solution

EXAMPLE USE CASE

This article explores the use of the Java-based Subversion to Perforce conversion utility, p4convert. The FreeBSD source code base is used in this incremental conversion example.  This conversion runs on an Ubuntu Linux machine; incremental conversions using other platforms will require customization of scripts and settings files.
 

Setting up the environment:

Identify remote depot and conversion directory

Identify a remote Subversion repository to synchronize with. The FreeBSD repository is at http://svn.freebsd.org/base. This example uses /Users/nmorse/freebsd for the conversion directory.

Run these commands to make sure Subversion is set up on your system:

$ svnadmin --version
$ svn --version

You can download and install Subversion from subversion.apache.org.

 

Copy the jar file and scripts into the conversion directory

Download the latest release of p4convert from the perforce workshop.

https://swarm.workshop.perforce.com/projects/perforce-software-p4convert

    You will also need some additional scripts which are available in the perforce public depot here:

    Incremental-Conversion Scripts


    Create and initialize a "clone" of the remote SVN repository

    You can use the "svnsync" utility to clone the remote SVN repository locally.

    $ svnadmin create svndepot
    $ cd svndepot/hooks
    $ cp pre-revprop-change.tmpl pre-revprop-change
    
    

    svnsync needs special permissions to run, edit pre-revprop-change , and change the last few lines from:

    if [ "$ACTION" = "M" -a "$PROPNAME" = "svn:log" ]; then exit 0; fi
    echo "Changing revision properties other than svn:log is prohibited" >&2
    exit 1
    
    
    to:
     
    if [ "$USER" = "svnsync" ]; then exit 0; fi
    echo "Only the svnsync user can change revprops" >&2
    exit 1
    
    
    Then set permissions to execute and initialize revision 0.
    
    $ chmod +x pre-revprop-change
    $ cd ../..
    $ svnsync init --username svnsync file:///Users/nmorse/freebsd/svndepot http://svn.freebsd.org/base
    

    You will see a response like:
    "Copied properties for revision 0"

    Set up a Perforce environment on localhost

    • Start a Perforce Server. Remember the port you specified.  The server must start empty.
    • Create a Perforce workspace in ./p4_client
    • In ./p4_client make sure that you can access the new Perforce Server from the command line by setting a $P4CONFIG environment variable and creating a P4CONFIG settings file, Here is an example:
    P4USER=perforce
    P4CLIENT=svn-client
    P4PORT=4446
    
    

    Get your scripts and utilities in order

    Attached to this article you will find incremental.tar which contains the following files:
     

    Configure your scripts

    • crontab.src This is a sample crontab entry for your use. Customize it for your own environment.
    • timeout3.sh This script allows the interruption of svnsync so you don't download the entire SVN repository the first time the converter is called. Place this script in your conversion directory.
    • lockNrun.sh This script is a wrapper around the "lockrun" binary which insures that the converter doesn't get launched from cron while a previous invocation is still running. Place this script in your conversion directory.
    • lockrun.c This is the 'C' source for the lockrun binary which needs to be compiled for your system and placed in the path.

    Edit cron-p4import.pl,  Look for my $cvtRoot = 'path' and make sure it contains the full path to your conversion directory, like this:
     
    my $cvtRoot        = '/Users/nmorse/freebsd';
    

    The $batchSize and $timeout variables may also be modified to change the desired "chunk size" for each time the cron job runs.  A bigger chunk size will import faster, but hog more system resources.
     

    Customize your converter configuration file

    Edit config.txt and make sure the following entries are correct for your Perforce Server port and conversion directory.  Notice the ':' in the .port option is escaped with a '\'.  The order of entries in the config file is not important except that the last two entries (com.p4convert.svn.start and com.p4convert.svn.end) need to remain at the end of the file.
     
    com.p4convert.p4.port=localhost\:4446
    com.p4convert.p4.clientRoot=/Users/nmorse/freebsd/p4_client/
    com.p4convert.log.out.filename=/Users/nmorse/freebsd/logger.txt
    com.p4convert.svn.avd.p4root=/Users/nmorse/freebsd/p4_client/
    
    
    Edit "crontab.src" to set the path and frequency with which the converter will be called.
     

    Controlling the conversion

    Manage this script using cron:
    • crontab -l - Show existing crontab entries
    • crontab crontab.src - Set up a cron job
    • crontab -r - Remove a crontab entry 


    Files generated by the conversion

    • logger.txt - verbose output from the cron job
    • convert.log - output from the converter
    • changeMap.txt - A table mapping Subversion revisions to Perforce change IDs
    • audit.log - A log of each file converted, used for validation
    • current.dump - A partial subversion dump file, generated for use by the converter
    • lock.lockrun - A lock file to prevent concurrent execution of the converter 


    Bugs and Problems

    Be consistent about which user runs the conversion (manually and from cron). If a bunch of files get written by root, you will not be able to run again as a non-privileged user.

    It is important to keep the changeMap.txt file in place between successive runs, as this file is used in the next run when doing an incremental conversion.
     
    Related Links

    Feedback

     

    Was this article helpful?


       

    Feedback

    Please tell us how we can make this article more useful.

    Characters Remaining: 255