Why migrations take a year

Four habits account for most of the time.

  • Discovery by committee. Requirements are gathered in workshops, written into long documents and reviewed again. The result describes the old system in detail and the business only loosely.
  • Copying field by field. Each old field is matched to a field in the new system. Every custom field, workaround and duplicate is discussed and carried across, so the new system inherits the old system's problems.
  • Manual cleanup. Bad records are fixed by hand, often in spreadsheets, before or after the load. Nobody can repeat the work, so when the data is loaded again, the cleanup starts over.
  • Big-bang cutover. The new system is tested with sample data, then the business switches over on a chosen date. Problems that only real orders reveal turn up after the old system is gone.

Each habit makes the next one slower. A slow import is run rarely, so cleanup is done by hand, so the data can only be loaded once, so testing waits until the end.

Illustration titled Where the time goes. A typical project is a long run of workshops, field mapping, manual cleanup and cutover. A model-first project is much shorter: model, import, a long parallel run and cutover. A bar at the bottom reads: Spend the weeks testing with real work.
Figure 1. In a model-first migration, most of the calendar goes to the parallel run.

What can be fast: the model and the import

Two parts of a migration can be done far faster than most plans assume.

The data model can take days. Instead of documenting the old system, describe the business: its customers, products, orders, suppliers and invoices, and how they relate. That is a smaller job, and it produces the structure the new system should have. Don't copy your old database: design the data model first explains the approach. For the flooring wholesaler, this step took a couple of days.

The import can take minutes. When each source has its own mapping into the model and the load is automated, a full import is a program you run, not a project. The wholesaler's pipeline loaded 15 years of records, hundreds of thousands of them, in minutes. Because a full run is that quick, every correction is made in the mapping and followed by a complete rerun and a comparison with the old data. No one cleans records by hand. Why your data import should be rerunnable covers how that works.

The same applies to building what the business needs. Every feature and report the wholesaler requested was built, plus a B2B portal for its customers. The business now runs its own custom workflow, custom business rules and custom reporting, all supported in CommerceServer. It never had that in 15 years on Zoho.

A short timeline is not unique to this case. Intelligence Factory's published SafePass case reports that a small team designed, built, tested and deployed a vaccine and test compliance system in less than a month, while federal requirements were still changing. The case credits a model-driven method: when rules changed, the team updated the model and regenerated the affected parts of the system.

What should take weeks on purpose: the parallel run

The one phase that should not be rushed is the parallel run, the period when the old and new systems both operate. This is where the new system is tested with the business's real work instead of sample data.

Parallel runs usually fail because staff have to enter everything twice. The wholesaler's did not. Staff kept working in Zoho and emailed the PDF files it produced. Buffaly agents read each one, checked it and entered it in CommerceServer. When a file arrived in a layout the agents had not seen, Buffaly wrote a new reader for it and used it in the same session. We consider this key to how Buffaly works.

Every emailed file was a real transaction run through the new system. Problems surfaced while the old system was still in use and were fixed before cutover. After a month the wholesaler stopped using Zoho. Running old and new systems side by side without double entry explains the setup.

Illustration titled Fast on purpose, slow on purpose. A blue panel labeled Fast lists data model in days and import in minutes. A green panel labeled Slow lists parallel run for weeks and switching when ready. A bar at the bottom reads: Automation makes it fast. The parallel run makes it safe.
Figure 2. Automate the model and the import, and give the time to the parallel run.

A realistic timeline

PhaseYear-long planModel-first planFlooring wholesaler
Describe the businessMonths of workshopsDaysA couple of days
Load the dataField mapping and manual cleanup over monthsAutomated import, rerun after every fix15 years of records in minutes per run
Build what is missingOften deferred to a later phaseBuilt on the modelEvery requested feature and report, plus a B2B portal
Test with real workSample data near the endParallel run for weeksOne-month parallel run
Cut overOne fixed weekendWhen staff are readyOff Zoho after the month

The second migration is quicker again. A sister company of the wholesaler moved next, and what we had learned moving the first company became the tools for the second: the import method and the checks before and after each import. The second migration was the easy one describes it.

Questions to ask a vendor who quotes twelve months

  1. Which months are spent on what? Ask for the plan split into the phases in the table above.
  2. How long does one full import take, and can it be rerun from scratch after a fix?
  3. Will cleanup happen in the import or by hand? Hand cleanup has to be repeated every time the data is reloaded.
  4. When does the first real transaction run through the new system?
  5. Who does the data entry while both systems run? If the answer is your staff, entering everything twice, plan for the parallel run to slip.
  6. What has to be true before cutover, and who decides?

The business data migration checklist turns these into steps for before, during and after the move.

Split your own quote into phases

If you have a migration quote, mark each line as describing the business, loading data, building, testing with real work or cutting over. Then compare the time given to testing with real work against the time given to everything else. If testing is a few days at the end, the plan carries the risk of a year-long project and a big-bang switch.

Send us the timeline you were quoted

If you have been quoted a year to replace your system, send us the plan. We will tell you which phases we would automate, how long a parallel run would need to be for your business and what the move would look like on CommerceServer. Tell us about your current system, or read how full service works.