- Excellent Benefits
- Exciting Projects
- Open Communication Culture
about the company
Randstad has recently partnered with a growing company within the software solutions industry. Your future employers work with clients on a global scale, offering innovative and secure IT solutions.
key responsibilities:
- Pipeline Engineering: Design, build, and maintain highly scalable data pipelines for both batch and real-time streaming data sources.
- Orchestration: Develop, schedule, and monitor complex data workflows using Apache Airflow, ensuring high availability and fault tolerance.
- Cloud Infrastructure: Architect and manage robust data storage and processing solutions utilizing cloud data warehouses in either AWS (e.g., Redshift, Snowflake on AWS) or Azure (e.g., Synapse Analytics, Snowflake on Azure).
- Streaming Architecture: Implement and manage real-time data streaming architectures (e.g., Kafka, Kinesis, Event Hubs) to support low-latency data applications.
- Data Quality & Governance: Establish automated data quality checks, anomaly detection, and robust error-handling mechanisms to ensure high data fidelity.
- Mentorship & Collaboration: Work closely with data scientists, analysts, and software engineers to understand data needs, and provide technical guidance and code reviews for junior engineers.
requirements:
- Experience: 4+ years of dedicated professional experience in data engineering, data infrastructure, or a closely related field.
- Programming Languages: Advanced proficiency in Python (object-oriented programming, data structures, scripting) and complex SQL (window functions, query optimization, CTEs).
- Workflow Orchestration: Proven hands-on experience building and managing DAGs in Apache Airflow.
- Cloud Ecosystems: Deep experience with cloud-native data warehousing and computing in either AWS or Azure.
- Real-Time Data: Demonstrable experience working with streaming data pipelines and technologies (such as Apache Kafka, AWS Kinesis, Azure Event Hubs, or Apache Flink).
- Engineering Best Practices: Strong understanding of CI/CD, version control (Git), and containerization (Docker/Kubernetes).
nice to haves:
- Big Data Processing: Experience building and optimizing distributed data processing applications using Apache Spark (PySpark or Scala).
- Data Architecture: Proven background in enterprise Data Architecture, including dimensional modeling (Kimball), data vault, or implementing modern paradigms like Data Mesh or Medallion architectures.
- Infrastructure as Code: Familiarity with deploying infrastructure using Terraform, CloudFormation, or ARM templates.
how to apply
Kindly click on the applicable link to apply if you are interested and suitable for this role. Alternatively, you can reach out to me via LinkedIn for a confidential discussion.
Sundar Ravindran | Randstad