lead data engineer [hadoop & informatica].

you will drive the development of scalable data architectures and engineering solutions. You will manage and optimize data pipelines, oversee ETL processes, and ensure data quality and governance. Utilizing tools such as Hadoop, SQL, and Informatica, you will collaborate with cross-functional teams to deliver impactful data solutions and guide junior engineers.

about the job

• design and oversee ETL and ELT processes to prepare data for analysis
• build and sustain automated data pipeline architectures for optimal efficiency
• ensure data quality through thorough validation, cleansing, and profiling
• consolidate data from various sources including Teradata, Oracle, and Hadoop
• create scalable data architectures that facilitate integration, transformation, and analysis across multiple data sources
• develop and execute complex SQL queries and stored procedures
• enhance data integration processes for improved performance and reliability while addressing and resolving any related challenges

knowledge, skills and experience

• minimum 6 years of experience in Data Engineering exposed to Data Lakes & Data warehouse
• data modelling and architecting skills including data normalization, dimensional data modelling
• experience in building automated data pipelines on big data platform - Hive, Spark, Hbase, Kafka etc
• experience on working with ETL tool - Informatica Powercenter
• experience on working on relational databases like Teradata, Oracle, MSSQL etc.
• experience in ingesting Semi-structured/un-structured files into Hadoop platform
• good knowledge on Shell scripting & Python scripting

how to apply