Prudential's purpose is to be partners for every life and protectors for every future.
Our purpose encourages everything we do by creating a culture in which diversity is celebrated and inclusion assured, for our people, customers, and partners.
We provide a platform for our people to do their best work and make an impact to the business, and we support our people's career ambitions.
We pledge to make Prudential a place where you can Connect, Grow, and Succeed.The Data Engineer is responsible for Extracting, Loading, and Transforming (ELT) as well as managing the Change Data Capture (CDC) using Qlik Replicate to ingest data into Data Lake, Data Models, Data Marts, and the 7-Sisters Master Data Management platform that helps establish data-driven decision-making across the organization.This is a new Data Team (Tribe) under the IT Department that will drive the Data Strategy for the organization.
They will work with respective Data Owners to develop the data models, data marts, and data analytics for the whole PruBSN organization, following the regional Data COE approach to maintain good data governance and develop the 7-Sisters Master Data Management.
The Data Strategy for PruBSN will be executed via the 'Hub & Spoke' approach, the 'Hub' being this core Data Team under IT, collaborating and working with the respective department Data Stewards and PruBSN Data Heroes (Power BI superusers), i.e., the 'Spokes'.Key Accountabilities:Data Integration and Orchestration:Designs and implements data pipelines using Azure Data Factory to orchestrate data movement from various sources (databases, APIs, etc.)
into Azure storage solutions (e.g., Azure Data Lake Storage, Azure Blob Storage).ETL Processing and Transformation:Develops and manages ETL (Extract, Transform, Load) processes within Azure Databricks clusters, utilizing Spark (PySpark/Scala) for efficient data transformation.Performance Optimization:Continuously optimizes data pipelines for efficiency and scalability using Spark techniques within Databricks and optimizes storage solutions for optimal performance.Data Pipeline CI/CD:Designs, implements, maintains, and enhances CI/CD pipelines for data pipelines using Azure DevOps services (e.g., Azure Pipelines, Azure Repos).
This includes automating testing, deployment, and configuration management for data infrastructure on Azure.Azure Infrastructure Management:Provisions, configures, and manages infrastructure for data pipelines using Azure services (e.g., Azure Data Factory, Azure Databricks VMs, Azure Functions).Version Control and Collaboration:Utilizes GitHub, Azure DevOps services (e.g., Azure Repos) to manage code changes for data pipelines and collaborates with other engineers on infrastructure development.Security and Compliance:Ensures secure deployment of data pipelines on Azure, adhering to data privacy and compliance regulations using Azure Security Center and other security tools.
Implements data quality and governance processes to ensure the accuracy, consistency, and reliability of data.Monitoring and Troubleshooting:Sets up Azure Monitor to track data pipeline health and performance and implements alerting systems for potential issues through Azure Alerts.
Performs troubleshooting to identify and resolve issues with data pipelines and infrastructure.Scripting:Writes scripts using Azure Functions/ Azure Services CLI to automate tasks related to data pipeline deployment and management.Additional Skills:Experience with Azure DevOps, GitHub tools, and methodologies.Understanding of cloud security, governance best practices in Azure.Qualification:Bachelor's degree in IT, computer science, or a relevant field.2-5 years of experience in the data engineering/analyst domain.Able to work independently in a fast-paced and complex environment.Good communication skills (both verbal and written) – English is mandatory.Expert knowledge in data, data engineering, and other related subdomains.Background in cloud, cloud-native applications, and data flows.Working knowledge of MS SQL, PostgreSQL, CDC, data Streaming Application (e.g., Apache Kafka, Qlik Replicate) and other related systems.Prudential is an equal opportunity employer.We provide equality of opportunity of benefits for all who apply and who perform work for our organisation irrespective of sex, race, age, ethnic origin, educational, social and cultural background, marital status, pregnancy and maternity, religion or belief, disability or part-time/fixed-term work, or any other status protected by applicable law.
We encourage the same standards from our recruitment and third-party suppliers taking into account the context of grade, job, and location.
We also allow for reasonable adjustments to support people with individual physical or mental health requirements.About Us
Prudentialplc provides life and health insurance and asset management, with a focus on Asia and Africa.
We help people get the most out of life, by making healthcare affordable and accessible and by promoting financial inclusion.
We protect people's wealth, help them grow their assets, and empower them to save for their goals.
The business has more than 18 million life customers in Asia and Africa.
Prudential has been providing trusted financial security for 95 years and is listed on stock exchanges in London, Hong Kong, Singapore, and New York.We are proud to be included in the 2023 Bloomberg Gender Equality Index.
The index measures gender equality across five pillars: female leadership and talent pipeline, equal pay and gender pay parity, inclusive culture, sexual harassment policies, and pro-women brand.
Our inclusion in this global index is testament to our commitment to nurturing diverse talent.
#BloombergGEI.Prudential plc is not affiliated in any manner with Prudential Financial, Inc., a company whose principal place of business is in the United States of America or with the Prudential Assurance Company, a subsidiary of M&G plc, a company incorporated in the United Kingdom.#J-18808-Ljbffr