Swiss Chemical Company

Creating automated web scraping tool to augment data assets with external web data in near real-time leading to best-in-class data analysis and AI models

Challenge

• Client’s data assets contained, amongst other, external web data obtained on competitor’s websites
• Needed to figure out a better way to gather new data and managed existing data assets
• Goal to automate the collection and scraping process of new data

Solution

• Combined cloud services, public libraries and proxy servers to create a tool that continuously crawls relevant web resources automatically
• The tool crawls resources, retrieves key data, and structures those data to facilitate analysis

Business impact

• Information is always up-to-date thanks to automatic crawling
• Replaces unnecessary manual work and reduces potential for errors
• Best-in-class data assets to maximise quality of analysis and AI models