How to plan a systems integration

Before anyone writes integration code, you need answers to six questions. What business event starts the flow? Which system owns each piece of data? How do you get data in and out of each system? What happens when something fails? How will you know it is working? And who looks after it once it is live? When one of these is assumed rather than agreed, the problem usually shows up in production.

What should you work out before building an integration?

Planning an integration is mostly about decisions. The choice of API, message broker or file transfer comes later and is usually the easy part. The hard part is getting the owners of each system, and the people who run the business process, to agree how data should move and what should happen when it doesn't.

Plan each business flow on its own. "Integrate the CRM with finance" is too broad to plan. "When a deal is marked as won in the CRM, create a customer and a sales order in finance" is something you can design, build and test.

How do you define the business flow?

Start with the event that triggers the flow and the outcome the business expects. For an e-commerce platform sending orders to finance and warehouse systems, write it down plainly:

• Trigger: a customer pays for an order online.
• Outcome: finance has an invoice, the warehouse has a pick request, and stock levels are updated.
• Timing: the warehouse needs the order within minutes so it can ship the same day. Finance only needs it by the end of the day.
• Volume: typical orders per day, the peak rate during a sale or at month end, and the backlog to clear after an outage.

Then walk through what happens to an order after it is placed. Cancellations, refunds, part shipments and address changes are all part of the flow. Teams often plan the happy path in detail and discover the amendments only after go-live, when someone in finance asks why a refunded order is still showing as unpaid.

"Real time" and "overnight" lead to very different designs, and the right answer is often somewhere in between. Ask what actually goes wrong for the business if the data arrives an hour late.

Which system should own each piece of data?

For every entity the flow touches, such as customers, products, prices, orders and stock, decide which system is the master. The master is the system whose version wins. Every other system receives a copy, and any change made elsewhere is sent to the master rather than applied around it. Where ownership differs by field, record that too: the CRM might own a customer's contact details while finance owns their credit limit.

Without that decision, two systems end up editing the same record and the integration has to guess which change wins. That is where duplicate customers, prices that drift and stock figures nobody trusts come from.

A simple table settles it:

You also need a way to match records across systems. Decide which identifier links a customer in the CRM to the same customer in finance, and where that link is stored. Email addresses and names change, so they make poor keys.

How will you get data in and out of each system?

Check what each system actually offers before you design anything around it. The options usually include:

• a documented REST or SOAP API
• webhooks that tell you when something changes
• file import and export, often CSV or XML over SFTP
• direct database access, usually read-only and only where the supplier allows it
• nothing usable at all, which needs its own plan

For each one, find out the details that cause delays later. Is the API included in your licence, or a paid extra? Are there rate limits that will bite at peak volume? Can you get a test environment with realistic data? Who issues the credentials, and how long does that take? Will the supplier change the API without warning?

Access is one of the most common reasons integration work stalls. Request sandboxes, credentials and firewall changes early, while the design is still being agreed, so the build isn't left waiting for them.

Should the integration be direct calls, events or files?

Once you know the flow and what each system offers, the shape usually follows. A step that needs an immediate answer, such as a stock check at checkout, needs either a direct API call or a local copy of stock kept up to date by events. Either way, agree how out of date the figure can be and how stock is reserved so you don't oversell. A flow where several systems need to react to the same thing, such as a new order, suits events. Bulk data that only needs to arrive daily can stay as a file transfer, provided it is monitored.

Our guide to point-to-point or event-driven integration covers that choice in more detail. The point at the planning stage is to decide it for each flow on purpose, not to inherit whatever the first developer reached for.

What happens when something fails?

Every integration fails sometimes. A system is down for maintenance, a credential expires, a record arrives with a field missing. Plan for these before you build:

1. Retries: which failures should be retried automatically, how often, and for how long?
2. Duplicates: if a message is sent twice, will the receiving system create two invoices? Give each message a stable identifier so the receiver can record what it has already processed and skip a repeat.
3. Bad data: where do records that can't be processed go, and who looks at them?
4. Order: does it matter if a cancellation arrives before the order it cancels?
5. Catching up: after an outage, how do you replay what was missed without losing or doubling anything?
6. Rollback: if go-live goes badly, how do you switch back, and what happens to data created in the meantime?

Agree the answers with the system owners. "Finance will fix it manually" is a valid answer, as long as finance knows and agrees.

How will you know the integration is working?

A silent failure is worse than a loud one. An integration that stops sending orders without raising an error can go unnoticed for days. Decide up front:

• what gets logged for each message, including an identifier you can trace across systems
• which failures raise an alert, and who receives it
• how you spot data that has simply stopped arriving, not just errors
• how you check the two systems agree, for example a daily comparison of order numbers, amounts and statuses in each, not just a count

Who will support it once it is live?

Decide who owns the integration after go-live. Someone needs to receive the alerts, fix failed messages, renew credentials and certificates, and handle changes when one of the connected systems is upgraded. If that is your own team, they need documentation and a hand-over while the people who built it are still around. If it is a third party, agree what they cover and how quickly they respond.

Write down what was built: the flows, the data ownership table, which credentials it needs, where they are stored securely and who renews them, and how to restart and replay. An integration nobody understands becomes the next legacy problem.

What goes on an integration planning checklist?

• Each business flow is written down with its trigger, outcome, timing and volume.
• Amendments, cancellations and refunds are covered, not just new records.
• Every entity, or field where ownership is split, has one master system and an agreed matching key.
• Access to each system is confirmed: API, licence, limits, test environment and credentials.
• The integration style is chosen per flow.
• Retries, duplicates, bad data, ordering, catch-up and rollback have agreed answers.
• Logging, alerting and reconciliation are designed in.
• Ownership after go-live is agreed and documentation is planned.

If you are planning an integration and want a second pair of eyes on it, or someone to design and build it with your team, see our systems and API integration service.

Email [email protected] with your brief.