Overview
While more solutions are getting hosted on the cloud and for most businesses, the journey is still in transit from on-premises to cloud-hosted solutions; there is a constant requirement to share voluminous data between applications hosted on-premises to cloud, taking into consideration the latency, infrastructure, performance, security and cost which comes as a symbiosis between two different applications which are hosted on different infrastructure setups.
Considering this scenario, HCLTech has developed a platform to be reused across SaaS solutions as a future NexGen transformation solution. HCLTech Advantage Luminate Connect platform helps manage large volumes of data and reduces the processing time.
Our Offering
Shift to open-source solutions
Transitioning from Oracle Staging Tables to PostGres DB open source as the staging tables
Seamless file syntax validation at source
Avro Files have been used as the standard data format to transfer data from source to target
Delta services
Identify full load VS any delta changes on the existing data sets and only push the delta changes to the target systems, significantly cutting down the data volume and integration latency
Microservices
Blue Yonder validation, token generation, data archival and other services
Spark Jobs
For transformation and delta processes
Airflow
For overall orchestrating the workflow and checking the job status at each stage during trigger
Key Benefit
Cost reduction
Switching from licensed solutions to open-source solutions brings down the overall cost of purchase and ownership
Efficient handling of high volume of data
Switching from CSV to Avro Files brings down significant and drastic reduction in the number of files and the file sizes
Shorter processing time
This platform helps in drastic reduction in the overall processing time of the files and integration from source to target systems