Job description:

evelop scalable data pipelines using GCP tools such as Dataflow, Dataproc, BigQuery, Composer, and Dataplex.Write efficient Python code and complex SQL queries for data processing, extraction, transformation, and loading tasks.Strong analytical and problem-solving skills, with the ability to derive insights from complex datasets.Build and maintain data lakeslakehouses on GCP, integrating structured and unstructured data sources while adhering to governance and security standards.Implement robust auditing mechanisms and exception handling processes to ensure data quality and reliability.Apply Slowly Changing Dimension methodologies to manage historical data and track changes over time effectively.Possess expertise in Apache Spark and its optimizations for processing large-scale data efficiently.Design and implement Directed Acyclic Graphs (DAGs) for orchestrating complex data workflows.Work closely with cross-functional teams, including data scientists and business stakeholders.Must have working experience with various storage formats like Parquet, Avro, JSON, and XML, understanding their characteristics and best practices

• To oversee quality assurance processes, ensuring adherence to coding standards , implementation of best practices and perform Value creation and KM activities.
• To ensure process improvement and compliance, and participate in technical design discussion and to review technical documents.
• Responsible for shaping the overall project strategy working closely with stakeholders to define project scope, objectives, deliverables and keeping track of schedule to ensure on time delivery as per the defined quality standards.
• To work closely with the development team, On-site Engineers to understand technical requirements and work with them to address and resolve technical issues.
• Identify and flag potential risks and issues that may impact project timelines or quality, develop mitigation strategies or contingency plans to address risks and provide regular project updates to key stakeholders.