Cloud
Enhancing operations with near-real-time analytics
The Business Challenge
As the world’s leading integrated producer of concentrated phosphates and potash (essential for fertilizers and nutrition in agriculture), our client’s mining and production facility ecosystem stretches across multiple countries and three main verticals: operations and mining, corporate and regional services, and global strategic marketing and supply chains.
Business verticals
Segment
This ecosystem generates a vast amount of data. However, like many large-scale organizations, its legacy analytics system struggled to leverage all the data collected to extract meaningful insights and analytics. A substantial amount of time was required to construct and deconstruct models in SAP HANA. This led to restricted usage of data by other business applications that could only leverage specific data models. A dependency on inefficient, middleware systems for data model and ingestion incurred high latency issues from source to reports.
Mired further by the inability to scale and leverage distributed computing for data processing, the client embarked on creating a bespoke solution.
The goal was a Central Data Analytics system to empower its business leaders and users to find information quickly and in line with their specific business needs and support effective business decisions. It would provide a central warehouse for all the reporting and analytics requirements — crossing functions including corporate and finance data across regions for both B2B and B2C lines of business. The system would also connect supply chain logistics and asset data integration with mining operations to ensure seamless end-to-end visibility for proactive and timely corrective actions. As well as capturing employee safety-related data and generating associated reports at the CXO level.
The system architecture was constructed understanding that it was not a one-and-done solution — it would need to continuously evolve with analytics initiatives and the business’ needs over time. The first stage was built on SAP, with other data sources added according to need. It evolved the clients’ system from a legacy data warehouse architecture to a modern lake house, catering to all three business verticals. However, this 1.0 system had a traditional architecture with a SQL server and leveraged Microsoft Azure Data Factory to cater to business analytics.
Looking to the future and how it could derive even more from its data, the client sought to reconfigure its system architecture to create a 2.0.
The producer was seeking a “one source of truth” solution that would enable it to:
- Expedite data model development
- Leverage data from across business applications
- Remove latency issues
- Scale data processing capabilities
- Reduce operational costs
- Enhance trust on data
The AI Transformation
With empowering the business with analytics-ready data and self-service analytics as key objectives, the client selected Microsoft Azure as its preferred cloud choice for its 2.0 system. This aligned with its existing investments — but also was driven by the Data & Analytical services maturity delivered through Microsoft Azure Cloud Scale Analytics services.
This system has the capability to process structured, semi-structured and unstructured data at varying speeds to batch in near real-time through Microsoft Azure Synapse (SQL, Pipelines and Spark). Batches are stored in Microsoft Azure Data Lakes Gen 2 to be leveraged by Synapse Analytics, which can then be used by Power BI to produce visualizations in a seamless workstream. Power BI uses a data quality and governance process that increases trust of data for self-service decision making.
In all, this empowers users to quickly find specific information from one cloud-based and centralized location to provide streamlined access to global data across operations.
To help the system run smoothly, serverless data integration and transformation is supported by the implementation of Microsoft Azure Data Factory and the client’s Azure IR on-premises data integration. This further enables the use of Logic Apps and Azure API to capture event/pattern-driven data. And, to ensure the ability for the system to continuously evolve at speed when needed, the system also employed Microsoft Azure DevOps — enabling the system to improve at a faster pace than with traditional development approaches.
Beyond technology, the 2.0 system leverages Modern Data Platform practices with a focus on designing for self-service analytics. Business-centric standardized and reusable data models are organized by business data domains across Supply Chain, Sales & Marketing, Operations, Purchasing and Logistics. Reusable framework-based data ingestion by archetype enhances data discovery and visibility of business data. Bronze, Silver and Gold data zones are leveraged for capture, curation and consumption of data. And it is designed to handle global data from far-reaching countries — with the necessary data security compliance considerations at each point.
The Business Results
Enabling the client to break down data silos and enable the business to analyze data across business lines to support decisions, the new system architecture is already providing several key benefits, including:
Offloading analytics workloads from other platforms, reducing overall operating costs by up to 20%
Reduction in delivery cycle time, thus accelerating time to value
The client’s solution puts near-real-time data analytics into the hands of those that need it most, when they need it, supporting:
Operational efficiency
Accuracy and trust
IntegrationBy enabling analytics capabilities across data sources that were previously siloed — and offloading workloads from SAP, Salesforce, Oracle/SQL server, APIs, SFTPs and more — the client has enabled widespread data analytics capabilities across the organization. The increased trust and accuracy Microsoft Power BI provides is also having an impact, reducing the delivery cycle significantly.
With “one source of truth” that is primed to scale and evolve into the future, this integrated producer of concentrated phosphates and potash is realizing the power of self-service analytics nourished by AI.
At HCLTech, we continuously look to expand analytics capabilities into new markets, and we’ve applied the reusable framework at the foundation of this client’s system to position at least five more customers in the manufacturing, mining and logistics domain.
