Performing a data migration with migrate module
Cyrve developed the Migrate module to support robust, repeatable data migrations into Drupal. There are other solutions to this problem, but I believe none are as robust as migrate. This article focuses on Migrate version 2.
Lets fast forward past the development process and focus on performing the import. Migrate provides Drush commands for performing the import. There is currently no user interface for performing the import. Drush is better suited to that task, since it is not subject to PHP timeout. Even Drupal's batch API is ill suited to import since it has to re-bootstrap Drupal for each batch.
I'm assuming that you are working with a snapshot of your Drupal site and thus it is safe to rollback as needed. On the source side, you can work with your live site or a snapshot. Migrate does not change anything in your source site so feel free to pull from live.
migrate-status command provides an overview of all your migrations. It tells you what migrations are enabled, and how far along they are in their import.
migrate-import command is your workhorse. It is responsible for fetching source records and saving them into Drupal. Further, it maps the ID of each source item to a Drupal ID. So, legacy accountId might be mapped to Drupal's userID. Here is migrate-import in action:
migrate-import supports lots of useful options such as idlist, itemlimit, and more. As with all drush commands, you append --help to see its documentation:
You can rollback a given migration or all migrations at any time with
migrate-rollback. Migrate uses the aforementioned map table to know exactly which items to delete during rollback.
If you get really flumoxxed, you always have the option of discarding your Drupal database and restoring from backup.
Migrate's other commands are more specialized, or useful only to migration developers. For the record, here is the complete list: