Cyberjaya, Malaysia| Posted on 30/09/2024
Deriv, a leading player in the online trading industry, is on a mission to make trading accessible to anyone, anywhere. For 25 years, our innovative products and services have attracted a dedicated customer base of more than 2.5 million people worldwide.A career at Deriv will give you the opportunity to join a Platinum-accredited Investors in People (IIP) organisation and a certified Great Place to Work (GPTW). You'll be part of a global team of more than 1,400 exceptional professionals across 25 offices. Join us on our mission to shape the future of online trading and be part of a dynamic team driving innovation and excellence in the financial markets.Your roleAt Deriv, we are seeking an experienced and detail-oriented Senior Data Engineer who will play a pivotal role in shaping the future of our data architecture. Utilizing your expertise in technologies such as Google Cloud Platform, Airflow, Python, Docker, and PostgreSQL, you will help provide the company with reliable business intelligence solutions. In this role, you will develop, test, and maintain data processing architectures and build robust Extract, Transform, and Load (ETL) pipelines. Your contributions will be essential in making our data warehouse accurate and trustworthy.Your key responsibilitiesData Integrity & Security: Ensure data integrity while extracting data from both in-house and third-party complex sources, managing systematic storage while guaranteeing data security, accuracy, and accessibility.
Data Solutions Development: Leverage your expertise in data engineering to provide tangible business solutions and insights that drive informed decision-making.
Data Warehouse Architecture: Design and build a high-performance, secure, and scalable data warehouse and pipeline to support data science projects while following industry best practices.
Debugging & Improvement: Troubleshoot complex issues, recommend improvements, and ensure the efficient functioning of the ETL pipelined architecture.
Data Transformation: Transform raw data into easily digestible tables for Data Analysts, facilitating effective analysis and reporting.
Continuous Learning: Stay updated on company products and new releases to efficiently plan changes in our data warehouse or pipelines as needed.
RequirementsWhat is neededEducation:Bachelor's degree in Computer Science, Data Engineering, or a related field.Experience:Minimum of 7 years in the data engineering field, with a proven track record of successful data pipeline architecture.Data Modeling Techniques: Expertise in data modeling techniques such as Kimball star schema, Anchor modeling, and Data vault methodologies.Programming Skills: Competence in object-oriented or functional scripting languages, particularly Python.Database Proficiency: Strong proficiency in both relational SQL and NoSQL databases, particularly PostgreSQL, with knowledge of PITR, Pg_basebackup, WAL archival, and Replication strategies.Data Warehousing: Familiarity with column-oriented storage systems or data warehouses such as Parquet, Redshift, and BigQuery.ETL/ELT Development: In-depth knowledge of developing and maintaining ETL/ELT data pipelines and utilizing workflow management tools like Airflow.Hands-On GCP Experience: Practical experience with Google Cloud Platform (GCP) services, including BigQuery, scheduled queries, Cloud Storage, and Cloud Functions.Data Accuracy: Familiarity with alerting and self-recovery methods to maintain data accuracy.Analytical Skills: Strong analytical skills with an ability to translate complex data into optimal business decisions.Peer Review Proficiency: Experience in peer reviewing pipeline codes and suggesting improvements.Communication Skills: Strong communication and presentation skills with fluency in spoken and written English.Why Join UsAt Deriv, we value innovation and collaboration. As a Senior Data Engineer, you will find an environment that supports professional growth and empowers you to make impactful contributions to our data infrastructure. Join us as we continue to leverage data to drive our business forward and meet the evolving needs of our clients. If you're passionate about data engineering and ready to take on exciting challenges in a dynamic environment, we encourage you to apply!A chance to work with top talent from across the globe (70+ nationalities)Ample team-building and bonding activitiesGreat overseas travel opportunitiesCompetitive salary and annual performance bonus#J-18808-Ljbffr