Before You Put AI on Top of Your ERP, Look at What's Underneath It
The operations director asks the new AI assistant a simple question: which orders are going to ship late this week?
The answer comes back in seconds. It's clear, confident and formatted beautifully. It's also wrong. The real answer depends on a spreadsheet the planner updates every afternoon, a custom field in the ERP that nobody has documented since 2019, and a status code that means "on hold" at one plant and "awaiting parts" at the other.
The AI didn't fail. It read exactly what the system told it. The problem was underneath.
Why Everyone Is Suddenly Talking About the ERP
For most mid-market companies, the ERP is where the business actually lives. Orders, inventory, suppliers, invoices, production schedules. So when AI moves from writing emails to making operational decisions, it runs straight into the ERP.
The big vendors have noticed. On October 7, IBM and SAP launched a joint offering aimed specifically at midsize and fast-growing companies, pitching cloud ERP modernization as the path to an "AI-ready digital core." One of their reference customers, a modular construction company, cut operational cycle times by 50% after a three-month SAP Cloud ERP project.
That's a real result. But the framing deserves a closer look, because it suggests the ERP itself is what's standing between your business and AI. Usually it isn't.
The Gap Between Using AI and Running on It
Mid-market companies have adopted AI quickly. Getting it to work across the business is a different story.
RSM's 2026 Middle Market AI Survey of 1,030 US and Canadian companies found that 86% have partially or fully integrated AI into their operations. But only 17% are pursuing enterprise-wide transformation. Among companies whose AI pilots had moderate or limited success, the two biggest barriers were data quality (53%) and integration challenges (47%).
Kaufman Rossin's 2026 research found the same pattern from a different angle. 94% of mid-market companies use generative AI, but only 2% have operationalized AI at scale, with legacy systems integration named as one of the main barriers. In manufacturing, the industry most tied to ERP data, 73% of mid-market companies are still testing AI and none have reached full company-wide deployment.
Those numbers describe the same wall. AI works in a pilot because the pilot uses a clean, narrow slice of data. It stalls when it has to work with the full, messy reality of how the business runs. And for most mid-market companies, that reality sits in and around the ERP.
Is My ERP Ready for AI?
Your ERP is probably ready for AI. The data in and around it often isn't.
Most mid-market ERPs, whether that's NetSuite, Microsoft Dynamics, Epicor, Sage or SAP, can expose their data to other systems. The trouble is what that data looks like after years of real-world use. A few signs it isn't ready:
- The real answer lives in a spreadsheet. If the planner, the controller or the warehouse lead keeps a sheet "because the system doesn't quite show it," the ERP is only telling AI part of the story.
- Custom fields nobody can explain. Fields added for a project five years ago, still populated by some teams and ignored by others.
- The same code means different things in different places. Status codes, product categories or customer types that each location or department uses its own way.
- Integrations that break quietly. Point-to-point connections between the ERP and other systems that fail without anyone noticing until a number looks off.
- No owner for the master data. Customers, items and suppliers get created by whoever needs them, so duplicates pile up and nobody's job is to fix them.
A person can work around all of these. They know which spreadsheet to check and which code to ignore. AI can't, unless someone writes that knowledge down and builds it into the data it reads.
Why Replacing the ERP Isn't Automatically the Fix
When the data is messy, replacing the ERP feels like a clean slate. It rarely works that way.
ERP projects are expensive and hard to land. Panorama Consulting's 2026 ERP Report found that more than a quarter of organizations went over their project budgets. And a new system doesn't clean anything on its own. Migrate inconsistent data into a modern ERP and you get a modernized version of the same mess, now with a bigger invoice attached.
That doesn't mean modernization is wrong. For many SAP customers, it isn't optional. Standard support for most SAP ECC 6.0 systems ends on December 31, 2027, a little over a year from now. Extended support runs to the end of 2030, at a higher maintenance cost. If you're in that position, the migration is coming either way. The question is whether you do the data work before it or pay for it twice.
The Layer Between Your ERP and Your AI
The fix most mid-market companies need sits between the ERP and the AI, not inside the ERP.
That means building a clean, governed data layer that pulls from the ERP and the systems around it, including the spreadsheets that hold the real answers. In practice, that layer does four things:
- Brings the scattered pieces together. ERP data, spreadsheet data and data from connected systems land in one place, through reliable pipelines instead of manual exports. We covered what modern pipelines look like in AI-Ready Data Pipelines: What Changed in 2026 and Why It Matters Now.
- Agrees on what things mean. One definition of "late order," "active customer" or "on hold," written down and applied everywhere.
- Gives master data an owner. Someone responsible for keeping customers, items and suppliers clean, with rules that stop duplicates at the source.
- Controls how AI acts, not just what it reads. When an AI agent needs to change something in the ERP, it should go through a defined, logged interface rather than writing directly to the database. That's what makes its actions traceable, which we wrote about in Half of Companies Had an AI Agent Go Off-Script.
This approach works with the ERP you already have. It also makes any future ERP migration easier, because the hard part, understanding and cleaning the data, is already done.
If You're Modernizing Anyway, Design for AI From Day One
If an ERP project is already on your roadmap, AI should be part of the brief, not something bolted on after go-live.
That means deciding before the project starts which decisions you want AI to help with, which data those decisions depend on, and where that data comes from today. It means cleaning master data before migration, not after. And it means building the integration layer as part of the project, so the new ERP isn't another system that AI can only partly see.
Most ERP projects are scoped around processes: order to cash, procure to pay. Adding one more question to that scoping, "what will AI need from this?", costs very little at the start and a lot later.
Where to Start This Quarter
Pick one operational question you'd like AI to answer. Late orders. Stock that won't move. Suppliers who keep missing dates.
Then trace every source the honest answer depends on. Every system, every spreadsheet, every person who "just knows." Count them.
If the answer lives entirely in the ERP, you're in better shape than most. If it lives in three systems, two spreadsheets and someone's head, that's not a reason to put AI on hold. It's a map of exactly what to fix first.
Common Questions About ERP and AI
Is my ERP ready for AI?
- Most modern ERPs can share their data with AI tools, so the system itself is rarely the blocker. The blocker is usually the data: undocumented custom fields, inconsistent codes across locations, duplicate master data, and critical information kept in spreadsheets outside the ERP. AI can only work with what it can see and understand.
Do I need to replace my ERP before using AI?
- No. Most mid-market companies can get their ERP data ready for AI by building a clean, governed data layer between the ERP and the AI tools that use it. Replacing the ERP without fixing the data usually moves the same problems into a more expensive system.
When does SAP ECC support end?
- Standard mainstream maintenance for SAP ECC 6.0 on enhancement packages 6 to 8 ends on December 31, 2027. SAP offers optional extended maintenance through December 31, 2030, at an additional cost. Older enhancement packages already reached the end of mainstream maintenance at the end of 2025.