Before the move: a backup, a source list and a data model
Take a backup you can compare against
Before anything moves, take a full backup of the old system's data and leave it untouched. It has two jobs. It is your fallback if something goes wrong, and it is the reference that every later check compares against. Without it, the question "did everything come across?" can only be answered from memory.
List every source
The main system is rarely the only place business data lives. For the wholesaler, data also sat in XLS spreadsheets and CSV exports, database backups, PDFs, Dropbox folders and web pages. Each one is either a source to map or something the business decides to leave behind. Make that decision on purpose for each one. For every source, write down what it holds, who uses it and whether it is still being updated. How to combine spreadsheets, backups and PDFs into one system explains how to bring several formats together.
Design the data model before mapping fields
Decide what the data should look like in the new system before you match any old field to a new one. A field-to-field copy moves the old system's workarounds along with the data. Don't copy your old database: design the data model first explains the alternative. The model also sets the standard for the checks during the move. Each source is mapped to it, and a record that does not fit it becomes an exception instead of a bad record.
Before checklist
- A full backup of the old data, stored read-only, that someone has opened and checked
- A list of every source, with an owner and a keep-or-leave decision for each
- A data model the business has reviewed, naming what it deals with and how those things relate
- A mapping from each source to the model
- Cutover criteria agreed now, not at the end
During the move: import, compare, log and run in parallel
Make the import rerunnable
The import should run from start to finish with no hand edits, and finish fast enough to run again after every correction. The wholesaler's automated pipeline loaded 15 years of data, hundreds of thousands of records, in minutes. Each time a problem was found, the mapping was fixed and the whole import was run again from the start. Because nobody edited records by hand in the new system, every fix lived in the pipeline and survived the next run. Why your data import should be rerunnable explains the approach.
Compare every field with the backup
A record count shows that rows arrived. It does not show that they arrived correct. Compare the new data with the backup field by field.
On the wholesaler's project, every field brought across was compared with a backup of the old data. That was 293,986 fields, and the comparison finished with zero differences. Getting to zero meant tracing 43 exceptions back to source documents.
| What differed | Count | How it was resolved |
|---|---|---|
| Order dates | 35 | Recovered from the time the original order email was received |
| Products | 5 | Traced through vendor part numbers |
| Other | 3 | Traced to their source documents one by one |
Keep an exceptions log, and give every exception a decision
Every difference goes in the log with four entries: what differed, where the correct value came from, what was decided and what was changed. Fix the cause, in the mapping or the import, and then run the whole import again. Do not correct the record by hand, because the next run will bring the error back.
Some records cannot be proven against any source. Do not delete them to make the comparison pass. The wholesaler kept invoices that could not be proven against a source document and labeled them as historical source invoices. Every record was kept, and only records proven against a source are presented as verified.
Run the old and new systems side by side
The history is only half of the move. The business keeps working while it happens, and the new system needs testing with real transactions before the old one is switched off. A parallel run does both, as long as staff are not asked to enter everything twice.
The wholesaler kept Zoho as the system of record for a month. Staff kept working in Zoho and emailed the PDF Zoho produced. Buffaly agents read each file, checked it against the data model and entered it in CommerceServer, and anything that failed a check went to an exceptions list for a person to review. When a file arrived in a layout the agents had not seen, Buffaly wrote a new reader for it and used it in the same session. We consider this key to how Buffaly works. In Buffaly Builds Its Own Tools, an analysis of 1.2 million messages and over 380,000 tool calls from a real Buffaly instance found that 70% of newly created tools were used in the same session they were written.
Running old and new systems side by side without double entry covers how to set up the run, and the parallel run: email a PDF from Zoho, and Buffaly agents enter it in CommerceServer gives the wholesaler's full account.
During checklist
- One full import runs end to end without hand edits, quickly enough to rerun after each fix
- Every field is compared with the backup, not only the record counts
- Every difference is logged, traced to a source and given a decision
- Fixes are made in the mapping or the import, followed by a full rerun
- Records that cannot be proven are kept and labeled, not deleted
- A one-way parallel run handles real transactions, with a problem log and weekly totals compared in both systems
After the move: sign-off, cutover and archive
Sign off on evidence
Use the cutover criteria agreed before the move. The comparison shows no unexplained differences, every exception has a decision and the totals in the parallel run match. Then check that what the business asked for works on the new system, against the list of requests rather than a demo. For the wholesaler, every requested feature and report was built, plus a B2B portal for its customers. The business now runs its own custom workflow, custom business rules and custom reporting, all supported in CommerceServer. It never had that in 15 years on Zoho.
The people who do the work should sign off, not only the project lead. By the end of the wholesaler's month, the team did not want to go back to Zoho.
Cut over
Choose a cutover point and tell staff in advance. Process the last files from the old system, confirm the final totals match, and then stop entering work in the old system. Set it to read-only rather than deleting anything, so it stays available while staff get used to working without it.
Archive the old data
Keep a read-only archive of the old system's data before any subscription to it ends. Store the comparison backup, the mappings, the import and the exceptions log with it. Together they show where every record came from and why each exception was resolved the way it was. That answers later questions about old orders, and it gives the next migration a head start. The wholesaler is now off Zoho and no longer pays Zoho license fees.
After checklist
- Cutover criteria met and signed off by the people who do the work
- Every requested workflow and report checked on the new system
- Final files processed, totals confirmed and the old system set to read-only
- A read-only archive of the old data, plus the backup, mappings, import and exceptions log
- The archive taken before any subscription to the old system ends
Keep the checks for the next migration
The checks are worth more the second time. What was learned moving the wholesaler became the tools for moving its sister company: the import method and the checks run before and after it. The second move did not start from a blank page. The second migration was the easy one tells that story.
Plan your migration with these checks built in
If you are planning to move off Zoho or another system, tell us where your data lives today: the main system, the spreadsheets and exports, the backups and the documents. We will tell you how the import, the comparison and the parallel run would work on CommerceServer. Tell us about your current system, or read how full service works.