Assistant Manager

Data Engineers design and build data systems and pipelines. Responsibilities include developing data processing workflows, optimizing data storage, and ensuring data accuracy. You will collaborate with data scientists and analysts to meet data requirements and resolve data issues. Strong experience in data engineering and problem-solving skills are required.

• Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.

• Create the data integration and data diagram documentation.

• Lead the data validation, UAT and regression test for new data asset creation.

• Create and maintain data models, including schema design and optimization.

• Create and manage data pipelines that automate the flow of data, ensuring data quality and consistency.

• Strong knowledge on Python and Pyspark

• Expectation is to have ability to write Pyspark scripts for developing data workflows.

• Strong knowledge on SQL, Hadoop, Hive, Azure, Databricks and Greenplum

• Expectation is to write SQL to query metadata and tables from different data management system such as, Oracle, Hive, Databricks and Greenplum.