Our client is an American multinational biotechnology company focusing on innovative treatments for rare diseases. They initially struggled with data silos and disjointed analytics, especially during COVID-19, due to increased demand and lack of a dedicated data operations team. HCLTech started a Snowflake data migration project to improve data management and analytics, allowing for better scalability, speed and capacity to meet their business needs.The platform aimed to provide a complete data solution covering all aspects of data ingestion, data processing, data quality and curation and delivering data to end customers in a timely and efficient manner. Currently, Snowflake provides a global solution using data sharing across markets, facilitating real-time data consumption for accurate reporting, executing effective marketing campaigns, timely decision-making, improved sales performance and enhanced field-force efforts. Serving as a single source of truth, we helped our client to harness the full power of the data platform.
The challenge
Navigating data complexity with ease using Snowflake
Our client’s data was scattered across multiple data sources and data entrepreneurs struggled with the siloed data in different formats. Furthermore, their existing data platform struggled to support curated data sets globally for multiple Business Intelligence (BI) solutions and relied on manual data loading, complicating the handling of historical data. The other challenges included:
- Scattered data across multiple sources causing data silos
- Skilled technical specialists (data analysts, data scientists) working on multiple and complex source systems leading to high support costs
- Generating accurate reports from different data sources using multiple file formats and inconsistent data types, resulting in data discrepancy
- Lack of integration and migration capabilities, reducing efficiency
- An unscalable platform that couldn’t meet business user data needs
- Limited incremental data processing in the reporting layer
- The field forces (medical representatives) couldn’t perform at optimal levels and work proactively on campaigns andevents when the legacy systems were used
- The existing platform had limited data security features like end-to-end data encryption, account and user-level security, data masking etc.
- Lack of data modelling and nonavailability of a semantic layer which are essential for performing BI activities
- The data sharing and time travel features were missing altogether
Snowflake platform was a platform of choice for our client as it provided an integrated data landscape for performing seamless data operations, data analytics and reporting tasks.
The Objective
Nurturing a data-driven culture and business ownership with Snowflake
Our clients to utilize Snowflake's robust data solution platform to integrate their data landscape for handling diverse data from internal and external sources, thereby enabling seamless data operations, analytics and business reporting. HCLTech collaborated with Snowflake to implement strategies aligned with client goals. The key objectives included:
- Setting up a unified cloud-based data warehouse (as a single source of truth) to support incremental data processing and enhance efficiency
- Ensuring seamless handling of current and historical data stored in the Snowflake data warehouse and AWS S3
- Creating a domain-agnostic semantic layer, facilitating data sharing across global markets, for affiliates and different personas, improving scalability
- Enhancing decision-making using Snowflake’s inbuilt applications like Snowflake dashboards and Snowflake Streamlit for reporting and intuitive forecasting with transparency and accuracy
- Improving data management capabilities by integrating BI solutions within Snowflake
- Empowering business users to use agile-driven tools like Jira and ServiceNow for maintaining data and solving any technical issues swiftly.
Various data types (unstructured, semi-structured, structured, batch) from source systems were ingested into Snowflake’s single pool for data analytics.
The Solution
Integrating data and enhancing analytics with Snowflake for a global Biotech leader
HCLTech leveraged the Snowflake platform to create an integrated for seamless operations, analytics, and reporting across the client’s markets. Snowflake aggregated data from various sources, processed it through batch job scheduling and facilitated report generation via Snowflake dashboard, Snowflake Streamlit. The platform design supported sales processes, field force optimisation and data accessibility in real-time or from AWS S3 for historical data. Multiple data marts within Snowflake data warehouse reduced business complexity, enabling better team collaboration. Snowflake’s tools improved data transparency and decision-making, while agile tools like Jira and ServiceNow enhanced data maintenance and business value
The Impact
Streamlined data integration and enabled advanced analytics
The Snowflake platform served as a unified source of truth, efficiently handling data storage, structure and attributes with minimal user dependency. It enabled easy aggregation of segmental data for analytics by business users and supported predictive data models without requiring users to manage operational aspects. Internal teams used it to monitor customer engagement, manage drug release readiness, track pharmaceutical distribution, field force effectiveness, regional sales, finance, patient details and doctor availability. The marketing team leveraged Snowflake to drive new consumer acquisition, redesign targeting strategies and improve retention through Business Intelligence (BI) and Customer Relationship Management (CRM) tools, resulting in significant campaign performance improvements. Snowflake cloud platform adoption provided below benefits.
- Improved scalability: Snowflake enhanced scalability by doubling efficiency through the use of its unified analytics platform for predictive modeling and Snowpipe for ingesting over a billion rows daily, while simultaneously providing three times faster real-time data access for global users.
- Cloud Analytics drives robust performance: Cloud analytics significantly boosted performance by providing enhanced data insights through comprehensive reporting for leadership and business stakeholders, while accelerating delivery through optimized data models for commercial, sales, medical, and digital data consumed within business intelligence solutions.
- Snowflake optimized cost: Snowflake significantly optimized costs by achieving approximately 25% savings through its fully automated dynamic optimizer and reducing resource costs by 40% due to decreased reliance on technical personnel for data platform management.
- Advanced Analytics: Snowflake provided advanced analytics capabilities, including data-sharing and time travel features not available in legacy systems, benefiting our clients. Its platform's ML and AI-driven data science app support, with seamless integrations to Spark, R, and Python, offered essential scalability for various models and algorithms.
- Embracing agile software delivery and DevOps: By leveraging Snowflake's Data Cloud, HCLTech accelerated agile software delivery and DevOps practices. This enabled developers to focus on innovation, while benefiting from unlimited performance, concurrency, and scale. Consequently, feedback to the marketing team was significantly expedited, near real-time data empowered field forces and robust built-in security features surpassed traditional RDBMS offerings.
