Customers migrating to Perforce often manage their IP assets in diverse SCM systems. Very often, the customer is migrating from some SCM system(s) that we encounter regularly and for which we already have mature and well-understood tools and approaches (for example, p4convert).
But sometimes customers are migrating from SCM systems for which we have not (yet) created specific conversion tools. BBI provides a mechanism and methods for dealing with this situation. It avoids the need for vendor-specific programming details and doesn't require development time and effort.
BBI is a good solution for situations where "everything including the kitchen sink" isn't necessary or appropriate. The BBI approach is akin to a buffer where everything is set out so that the tool can pick and choose as much or as little as is required, and then combine the choices into exactly what is required.
The key feature of BBI is the ability to assemble a series of steps consisting of different intake operations and build these into a script representing a point in time. There can be as many sources as needed per step (snapshot), and these can be combined using the verbs provided in the import utility. Multiple snapshots can be applied to build up the content as needed. Converting to Perforce using the BBI approach is facilitated using the p4bbi utility; which parses the assets extracted from the legacy SCM and, using a series of commands in a conversion configuration file, saves and maps the files to Perforce.
The Perforce Workshop project contains all p4bbi source. https://swarm.workshop.perforce.com/projects/perforce_software-p4bbi
1) Define Your Branch Structure
All but the simplest conversions contain branched development paths that you will want to preserve in Perforce; it's essential to have a clear understanding of the source system's branching method, and how to map that method to Perforce.
The p4bbi branch command specifies the relationship between the source and target of branching operations, and can be thought of as an analog to the p4 integrate command.
As a best practice, one should carefully draw a branching diagram and visually confirm the relationship between the old and new branches, and gain a clear understanding the branches to be created in Perforce. This will go a long way toward ensuring completeness and a correct association of the assets.
2) Identify Which Snapshots Are Needed
As many or as few extractions as desired can be made from the source SCM. The extractions can then be stored for conversion purposes on the file system of the machine being used to run the conversion tools (p4bbi and p4bbi_validate). Each independent step, which is equivalent to a Perforce changelist, is specified in a single BBI configuration file. The whole process of converting using BBI can consist of multiple configuration files, each containing one or more snapshots.
The snapshot extracts can be formatted and held on the conversion machine in any one of the following formats:
- A directory tree of files of arbitrary depth
- A tar file or zipped tar file of the directory structure and files
- A zip file of the directory structure and files
- A command and its parameters that produces the files and directories needed
Each snapshot is mapped to a p4bbi baseline
command; these file source definitions are references by the other command steps in the BBI configuration file to indicate where content is coming from.
3) Set Up Perforce Destination Environment
As with any conversion, the location where the content is to be saved in Perforce needs to be defined. This information is provided in the P4 configuration file (not to be confused with the BBI configuration file) that specifies the connection, depot, and user details to be used. The information is specified as name-value pairs for the required P4 environment variables (P4PORT, P4USER, and P4CLIENT).
4) Define BBI Configuration Script Steps
In each of the BBI configuration files that are fed into the p4bbi utility the sources of information are defined as well as the operations (commands) to be performed on the content as it is applied to the depot. Commands available to the user are the following:
is the logical equivalent to a p4 add
or p4 edit
operation and is used to populate the depot with files or to add a revision to a file.
p4bbi safeupdate provides all of the capability of the update command with one additional feature: if the current head revision of the file that is being updated is deleted, the utility will report this and will not add a new file revision. This is to prevent a file that has been deleted in a mainline branch from being reincarnated inappropriately. By doing a safe update in this case, the code line practice of copying changes up to the mainline is supported.
p4bbi branch operation is equivalent to a p4 integ -v operation, which allows the definition of a new branch minimizing copying of files.
p4bbi copy_merge allows a specialized form of branch operation where the content from source branch takes precedence over the content in the target if it already exists or was deleted. The operation performs a p4 resolve -at to confirm this.
p4bbi record_as_merged provides a means to credit file content as being merged. Files are updated into the branch in a separate step, without having to perform the integration decide on resolution strategy. It is the inverse of the p4bbi copy_merge as it favours the target over the source and performs a p4 resolve -ay to confirm the changes.
Using a p4bbi rename allows assets in the depot to be "moved" from an old name to a new name while still preserving all of the file history. The process is handled by a p4 edit and then a p4 move.
5) Execute and Validate Conversion
Once the configuration script(s) and the detailed steps have been defined, executing them is accomplished by running the p4bbi convertor, implemented as p4bbi.py (a Python script).
The command line to execute p4bbi will be structured as:
python p4bbi.py <p4 configuration file> <bbi configuration file>
As the utility executes, it will provide status and diagnostic output on the console, allowing you to track the progress of the conversion.
Each of the BBI conversion steps can be validated, checking file and metadata consistency, using the p4bbi_validate utility, invoked using the same configuration files as the conversion but substituting the validate script in the conversion script's place, as specified below:
python p4bbi_validate.py <p4 configuration file> <bbi configuration file>