Migrating From Rational Team Concert (RTC) to ClearCase: Part 2 - Import

(This is the second in a three-part series of articles describing the migration of RTC SCM data to ClearCase.)

Importing SCM Metadata and Content – Design
The importer would be designed to be coupled to the schema output by the exporter.  The expectation is that a description of the stream to import is described in metadata (JSON) as:

• (0 or more) Streams
  • (0 or more) Baseline Sets
    • (1 or more) Component Baselines
      • (1 or more) ChangeSets
        • (1 or more) Changes

...and the importer would then perform the following mapping:

• RTC Stream => ClearCase/UCM Project and Integration Stream
• RTC Baseline Set => ClearCase/UCM Composite Baseline
• RTC Baseline => ClearCase/UCM Baseline
• RTC ChangeSet => ClearCase/UCM Activity
• RTC Change => ClearCase SCM Operation (mkelem, checkin, rmname, mv, merge, etc.)

In the abstract, the object mapping is direct, except for the ChangeSet/Activity mapping: in RTC a ChangeSet is constrained to record changes to a single component while in ClearCase an Activity can record changes spanning components.  Unfortunately, this means that more Activities will be created in UCM as the result of the migration, than would be if you were operating directly in UCM.

The importer would create the names of the UCM objects based on the RTC names, converting characters that are illegal in UCM to “_”.  Because you can easily change these names in ClearCase after creation, I wasn’t too concerned if the automation didn’t convert the names exactly to the customers’ expectations – the customer could easily identify and change the names that didn’t fit their practice.

Importing SCM Metadata and Content – Operation
The importer was written in Java 8, dependent on the CMAPI libraries that ship with ClearCase, as well as some open-source third-party libraries.
 
When it starts, the importer reads and validates the JSON metadata file specified on the command line, and then creates a CMAPI ClearCase “provider” from which subsequent CMAPI operations issue, such as the creation of a temporary "bootstrap" dynamic view used for general housekeeping.  It then creates the attribute types that will be applied to ClearCase UCM metadata and versioned content that capture the original RTC identifiers from which these objects were created – for example, the RTC workitem id associated to the changeset from which the CC activity was created is stored on the activity as the RTC_WORKITEM_ID attribute type. You can see these various custom attributes in the Custom tab of the Properties view of the ClearTeam Explorer (Figure 3 below) or in the output of “cleartool describe”.
 
For each RTC stream identified in the metadata, a UCM project and stream are created if they don't already exist.  For each component referenced by the stream, a sub-vob component is created if it doesn't already exist as well as the component used to create composite baselines from.
 
Based on the value of the “threadPoolSize” argument, several temporary dynamic views are created.  The snapshots metadata are then processed serially, with each temporary view concurrently handling the responsibility of importing a baseline, its changesets, and its changes, including the processing and recording of merges.  Once all the baselines in the snapshot are imported, a composite baseline is created and the next snapshot is processed.
 
When the tool exits, all temporary views are removed and the connection to the ClearCase provider terminated.

​Figure 1 shows the output of the importer’s usage statement:

Figure 1 - Importer Usage Statement

​Figure 2 is a screenshot of ClearTeam Explorer showing the project, integration stream, and activities created as the result of a migration. The headline references the RTC work item number, description and changeset comment as they appear in RTC.

Figure 2 – Activities created from RTC changesets

​Figure 3 shows a screenshot displaying the custom attributes associated to one of the activities above:

Figure 3 - Custom Attributes

​Importing SCM Metadata and Content – Performance
You’ll recall from Part 1 of this article that the export for the metadata and content for an RTC stream of 57k changesets and 50k versioned files took about 1 hour.

The subsequent import of that specification into ClearCase running on the same machine took 9.5 hours, running 9 views in parallel, importing 20 component baselines per snapshot.  Considering the amount of data processed and created, I and the customer were satisfied with the final runtime – it met the “minimize downtime” requirement.  I did not test with a lesser or greater amount of concurrently running views to determine the right number for optimal performance (e.g. 20 views – one per component for this particular stream) – so there may be room for improvement here.

SCM Artifact Migration – Implementation Challenges
In the last and final part of this article I’ll discuss some of the implementation challenges I had to overcome when confronted with the limitations of RTC SCM relative to ClearCase/UCM.
​

Todd Lainhart
Senior Software Engineer
ClearCase

---

Todd Lainhart is a senior technical contributor to the ClearCase product family. Todd has developed software for over 30 years, focused mainly on the CASE and configuration management product spaces.