HCLTech—Senior Technical Support Specialist

Job description

Description: Job Summary · Design of data solutions on Databricks including delta lake, data warehouse, data marts and other data solutions to support the analytics needs of the organization. · Apply best practices during design in data modeling (logical, physical) and ETL pipelines (streaming and batch) using cloud-based services especially Python & Pyspark · Design, develop and manage the pipelining (collection, storage, access), data engineering (data quality, ETL, Data Modelling) and understanding (documentation, exploration) of the data. · Design Analytics / Gold Layer using DBT models, macros and manage it following best practice. Implement DQ and unit test in dbt, Experience in dbt clould is preferred and working with Databricks sql · Interact with stakeholders regarding data landscape understanding, conducting discovery exercises, developing proof of concepts, and demonstrating it to stakeholders. · Experience to work on Collibra for DQ and data governance is plus · Knowledge on dbt to model and build out layers in dbx is plus · AWS experience on S3, Redshit, Glue etc is also required Key Responsibilities · Design of data solutions on Databricks including delta lake, data warehouse, data marts and other data solutions to support the analytics needs of the organization. · Apply best practices during design in data modeling (logical, physical) and ETL pipelines (streaming and batch) using cloud-based services especially Python & Pyspark · Design, develop and manage the pipelining (collection, storage, access), data engineering (data quality, ETL, Data Modelling) and understanding (documentation, exploration) of the data. · Design Analytics / Gold Layer using DBT models, macros and manage it following best practice. Implement DQ and unit test in dbt, Experience in dbt clould is preferred and working with Databricks sql · Interact with stakeholders regarding data landscape understanding, conducting discovery exercises, developing proof of concepts, and demonstrating it to stakeholders. · Experience to work on Collibra for DQ and data governance is plus · Knowledge on dbt to model and build out layers in dbx is plus · AWS experience on S3, Redshit, Glue etc is also required Skill Requirements Other Requirements