Cleaning up 30,000 duplicate Shopify products without a week of manual work
A product sync issue created roughly 30,000 duplicate products in the B2B Shopify store. Instead of working through them manually, we built a controlled cleanup workflow.
30,000duplicate products cleaned upAt a glance
~30,000
duplicate products processed
~6 hours
for the automated deletion run
Up to 1 week
of manual catalogue work avoided
What we did
- Duplicate detection
- Review workflow
- Catalogue cleanup
- Shopify Admin API automation
The challenge
The B2B Shopify store had accumulated around 30,000 duplicate products.
Cleaning that catalogue manually would have meant working through hundreds of Shopify Admin batches, while deleting everything automatically without validation would have introduced unnecessary risk.
How we helped
- 01
Automated the repetitive part
We built a workflow that analysed the catalogue and classified products into three groups: clear duplicates, products requiring review and products that should remain untouched.
- 02
Kept human review where it mattered
Instead of asking the team to inspect tens of thousands of products, they only needed to review the uncertain cases once before deletion started.
- 03
Let Shopify handle the cleanup automatically
After approval, the confirmed duplicates were removed through the Shopify Admin API. The workflow itself was built within a day, while the full deletion process ran automatically for a little over six hours.
The outcome
A catalogue cleanup that could realistically have consumed close to a week of repetitive manual work was completed safely with a single review step and an automated deletion process.
All confirmed duplicates were removed successfully without legitimate products being deleted.
Insights
Automate the repetitive part.
The workflow sorted 30,000 products into clear duplicates, cases to review and products to leave alone.
Keep people on the uncertain cases.
The team reviewed only the unclear products, once, instead of inspecting tens of thousands.
Validate before you delete.
One review step made a fully automated deletion safe: no legitimate product was removed.
“Would highly recommend Milos. Fast learner, can operate in non structured/growing environment.”
Amy KleinChocolate Executive, De Bonte Koe