(This is the first in a three-part series of articles describing the migration of RTC SCM data to ClearCase.)
When I joined HCL, I was asked to provide tooling to help our internal customers migrate their SCM systems from Rational Team Concert ("RTC") to ClearCase. While there exist tools to help synchronize the two repositories in simplistic ways, there are no tools that will export the full history of SCM artifacts out of RTC and into ClearCase. Having been a developer of both the RTC and ClearCase products, I looked forward to the creative challenge of solving that problem.
A small number of requirements were expressed: export the full history of SCM artifacts out of multiple RTC repositories, retaining the appropriate SCM metadata; provide enough linkage between RTC changesets and their workitems so that such linkages can be replayed between ClearCase and some other bug tracking system (e.g. ClearQuest); perform the migration with a minimum of system downtime on either end.
Constraints and Approach
I was the primary developer assigned to the project, and was given only 3-5 months to complete the work such that it could be used in the production migration of our customers’ systems. Given that, I knew that I would have to constrain the solution and define pre-conditions to be successful (as well, I anticipated features that would be needed, but not yet asked for). The conditions:
Exporting SCM Metadata and Content - Design
I had earlier success at HCL writing a tool for exporting RTC workitem artifacts for import into ClearQuest, and settled on a similar design here: I would generate a JSON-formatted file that expressed the RTC object relationships in metadata and in parallel download the full content for versioned artifacts separate from that metadata, into the specified filesystem organized by element and version ids (RTC does not store deltas). The path to the versioned content would be referenced by their associated JSON metadata objects. A single process with multiple threads would coordinate the retrieval for both metadata and content.
Here's an example of how the resulting metadata looks for a single change within an exported changeset:
Here's what the usage statement looks like for the finished tool:
Exporting SCM Metadata and Content - Performance
I ran this tool on a Windows Server 2012 VM, configured as a 2-processor Xeon CPU running at 2.4G with 16GB of memory. Connected to a somewhat sluggish RTC server, I saw an average runtime of about 1.2 hours for the download of the metadata associated to 57k changesets and the content for 50k versioned files, running with ~100 threads. Optional content verification was enabled as was INFO level logging for diagnostics, which only adds to the runtime.
While I didn't have quantitative requirements for what defined acceptable performance for the export, I was satisfied with the result, exporting our customers’ repositories in better-than-expected time.
Part 2 – Importing SCM Metadata and Content
In Part 2 I'll describe the more interesting part of this exercise – importing the metadata and content into ClearCase/UCM.
Todd Lainhart is a senior technical contributor to the ClearCase product family. Todd has developed software for over 30 years, focused mainly on the CASE and configuration management product spaces.