Data Engineer Specialist

Details of the offer

Database Development & Administration (Information & Communication Technology)
Key ResponsibilitiesData Infrastructure and Architecture:Designing and maintaining the architecture used to store and process large sets of data, such as data warehouses, data lakes, and distributed systems. Creating data pipelines that extract, transform, and load (ETL) data from multiple sources into centralized systems for analysis.
Data Integration:Working with various data sources, including APIs, databases, and flat files, to integrate and merge datasets from multiple systems. Ensuring data is available in real-time or batch processing modes as per the requirements of the business or data analysts.
Data Quality and Validation:Ensuring data integrity, cleanliness, and accuracy throughout its lifecycle. Identifying and resolving any issues with missing, inconsistent, or erroneous data.
Collaboration with Data Scientists and Analysts:Collaborating with Data Scientists to ensure the infrastructure supports machine learning models and data analysis. Ensuring that analysts have the data they need in the correct format for reporting and analysis.
Automation and Optimization:Automating manual data collection and transformation processes. Streamlining data workflows to ensure faster processing and real-time analytics.
Position RequirementsEducation:Diploma or Bachelor's degree in Computer Science, Information Technology, or related field, or equivalent work experience.Fresh Graduates are welcome!
Experience:More than 2 years of relevant work experience, diploma, bachelor degree or above, computer and related majors;Fresh Graduates are welcome!
Skills and Knowledge:Proficient in Linux operating system, with in-depth understanding of the system;
Proficient in at least one scripting language and static language, with experience in large-scale system design preferred;
Familiar with TCP/IP and HTTP protocols, with in-depth understanding of the protocols and practical troubleshooting experience preferred;
Familiar with container technology and container orchestration technology, with experience in k8s production operation and maintenance preferred;
Familiar with database principles, with in-depth understanding of common database engines preferred;
Have in-depth understanding of distributed systems, familiar with common open source basic components of the Internet (nginx, redis, kafka, mysql, hbase, zookeeper, hadoop, etc.);
Strong sense of responsibility, proactive, love of learning, and focus on teamwork;Flexibility:Outstanding candidates may be considered with more flexible requirements.
Application ProcessTo help fast track investigation, please include here any other relevant details that prompted you to report this job ad as fraudulent / misleading / discriminatory.#J-18808-Ljbffr


Nominal Salary: To be agreed

Source: Whatjobs_Ppc

Requirements

My - System Support Specialist (Risk Management)

Key Responsibilities: Liaise with various departments on all aspects of client trading Performing settings in MT4 Administrator and relevant systems Preparin...


Zeal Group - Kuala Lumpur

Published 22 days ago

Filenet Developer

Job Description: Job Title: FileNet Developer Job Mode: Contract Key Responsibilities: Design, develop, and implement custom FileNet applications and soluti...


Vdart Software Services Pvt. Ltd. - Kuala Lumpur

Published 22 days ago

Master Data Management Specialist

We never ask for payment as part of our selection process, and we always contact candidates via our corporate accounts and platforms. If you are approached f...


Tmf Group - Kuala Lumpur

Published 22 days ago

Ai Filmmaker (100% Remote - Apac)

Join Tether and Shape the Future of Digital FinanceAt Tether, we're not just building products, we're pioneering a global financial revolution. Our cutting-e...


Tether Operations Limited - Kuala Lumpur

Published 22 days ago

Built at: 2024-12-25T01:14:53.345Z