Job Summary
We are seeking a Lead / Senior Data Engineer to design, build, and optimize our core data infrastructure. In this role, you will bridge the gap between raw data and production-ready environments, ensuring our analytics systems are scalable, reliable, and high-performing. You will collaborate closely with Data Scientists to support advanced modeling and lead data engineering initiatives.
Key Responsibilities
Data Pipelines: Build and orchestrate automated ETL/ELT pipelines for batch and real-time streaming data.
Architecture: Design and maintain scalable data lake and data warehouse structures.
Data Modeling Support: Partner with Data Scientists to optimize data pipelines, feature stores, and datasets required for statistical modeling.
Performance: Tune and optimize complex SQL queries and data workflows for large-scale datasets.
Leadership: Establish technical best practices, ensure data quality, and mentor junior engineers.
Requirements
Experience: Minimum 4+ years in Data Engineering, Big Data, or Data Architecture.
Education: Degree in Computer Science, Software Engineering, or a related quantitative field.
Technical Proficiency:
Languages & Core Libraries: Mastery of Python (including advanced Pandas and NumPy for data manipulation) and expert-level SQL.
Data Modeling & ML Support: Strong familiarity with preparing data for statistical modeling and predictive frameworks (e.g., XGBoost, Regression, Decision Trees).
Infrastructure & Orchestration: Experience with modern data stack tools (e.g., Spark, Airflow, dbt) and cloud data platforms (AWS, GCP, or Azure).
Benefits & Work Environment
Flexible Hybrid Arrangement: Balance of remote work autonomy and in-office collaboration.
Professional Growth: Lead high-impact architectural decisions alongside a talented AI and analytics team.
Competitive Compensation: Monthly salary up to RM 15,000, commensurate with experience and technical assessment.
