Multi-supplier product data management system

Summary

A B2B distribution company worked with several external suppliers whose catalogs arrived in different formats and were largely loaded by hand. The work involved designing an ingestion, normalization and consolidation system that automatically compares prices across suppliers, selects the best offer based on business rules, and keeps both the ERP (Odoo) and a custom eCommerce platform in sync — eliminating manual reconciliation and pricing errors.

Context

The company operates a distribution model that relies on multiple external suppliers. Each supplier delivers their catalog in a proprietary format: CSV files with different encodings, XML via FTP, JSON feeds through API, spreadsheets sent by email. Products overlap between suppliers, but identifiers, attributes and units are inconsistent across sources.

The ERP — built on Odoo — acts as the core business system (purchasing, stock, invoicing). The custom eCommerce platform consumes ERP data to present the catalog to end customers. Any inconsistency between what the website shows and what the ERP considers current has a direct impact on margins, the buying experience and downstream administrative work.

Problem

The previous process depended on scattered scripts and manual loading:

Each supplier was treated as a separate case with no common pattern.
Prices were not compared automatically: supplier selection was based on habit or individual judgment.
Attributes arrived unnormalized: brands spelled in different ways, mixed units (g/kg, ml/l), categories incompatible across sources.
Loads were not idempotent; re-running an import could duplicate records or overwrite valid data.
The eCommerce platform reflected price and stock changes with a delay.
The team spent hours each week on manual reconciliation with no added value.

A quick fix would not solve the root cause: the problem was structural. Each new supplier made the system more fragile rather than more capable.

Goals

Unify catalog ingestion into a single pipeline.
Compare prices automatically and select the best offer based on explicit rules.
Guarantee idempotency in loads so they can be re-run without risk.
Normalize data before injecting it into the ERP.
Keep ERP and eCommerce in sync with full traceability.
Reduce the team’s operational burden and prepare the system to onboard new suppliers at low cost.

Technical approach

The system was designed in layers, explicitly separating what changes (each supplier) from what must remain stable (the business logic).

Ingestion layer (per-supplier adapters). Each supplier has a specific connector whose only responsibility is to fetch the catalog in its native format (FTP, REST API, scheduled download, file parsing) and dump it into a shared intermediate schema. This layer is the only one that knows each source’s specifics. Adding a new supplier means adding an adapter, not touching the core.

Normalization layer. Raw catalogs go through cleaning and validation: unit unification, brand and category normalization via editable mapping tables, detection of equivalent codes (EAN, internal reference, manufacturer SKU) and mandatory field validation. Records that fail validation are isolated in an error queue for review without blocking the rest of the batch.

Consolidation and best-price selection layer. Once normalized, equivalent products from different suppliers are grouped by canonical identifier. A configurable policy is applied to that group, considering net price, availability, lead time and historical supplier reliability. The result is a single “winning” record per product, with full traceability of which supplier was chosen and why.

Odoo integration. Synchronization with the ERP uses idempotent operations: each consolidated product is identified by a stable external key, and the sync distinguishes between create, update and deactivate. Custom modules were implemented to extend the product model with alternative supplier information and price selection history, without breaking the ERP’s standard logic.

eCommerce output. The platform consumes an enriched, frontend-ready view rather than raw ERP data. This allows adding calculated fields (price tiers, labels, variant groupings) without polluting the ERP model and keeps website performance independent of the ERP’s current state.

Observability. Every pipeline run produces structured metrics and logs: records received per supplier, discarded records, products that changed winning supplier compared to the previous batch. Turns an opaque process into an auditable one.

Decisions and trade-offs

Building a “complete” system covering all edge cases from the start was ruled out. The priority was to cover the highest-volume suppliers first and leave the rest for later iterations, with the architecture already prepared to absorb them.

Real-time price comparison against suppliers was avoided: the operational cost and dependency on external APIs did not justify it for this case. Comparison runs in scheduled batches, which provides stability and decouples website response times from the state of each supplier.

Normalization is kept declarative, in mapping tables editable by the business team, rather than embedded in code. This adds some initial configuration but means common changes (a new brand, a renamed category) don’t require technical intervention.

Finally, the decision was made not to rewrite Odoo’s standard modules but to extend them. The integration is more maintainable in the medium term and reduces risk during future ERP upgrades.

Result

Catalog updates moved from a recurring manual task to unattended execution.
Best-price selection stopped depending on individual judgment and became a traceable, reproducible policy.
Propagation times for supplier changes to the eCommerce platform were significantly reduced.
The team recovered hours each week previously spent on manual reconciliation.
Onboarding a new supplier stopped being a project and became a bounded task of adding an adapter.
Discrepancies between ERP and website were significantly reduced, with traceability to investigate any remaining ones.

Tech stack

Java platform
Spring Framework / Spring Batch / Spring Data / Spring Web
PostgreSQL
FTP connectors, REST clients, CSV/XML/JSON parsers
Batch processing with idempotency control
Custom eCommerce platform integrated via API
Structured logging and execution metrics

What this case demonstrates

This case demonstrates the ability to approach a heterogeneous integration problem from an architectural perspective: separating what changes (each supplier) from what must remain stable (business logic), turning implicit rules into explicit and traceable policies, and leaving the system prepared to grow without each new addition adding technical debt. It combines deep Odoo knowledge, data pipeline design and business judgment to decide where to invest effort and where not to.

If you work with multiple suppliers and catalog management is becoming a bottleneck — inconsistent data, outdated prices, recurring manual loads — I can help you analyze the current flow and design a realistic architecture that scales with your business.