Birlasoft—Technical Lead-Data Engg

Job description

Description: Area(s) of responsibility Key Responsibilities • - Design and develop scalable ETL/ELT pipelines using PySpark, Spark SQL, Python, and Databricks. - Develop complex transformation logic across Bronze, Silver, and Gold layers. - Implement reusable transformation frameworks and common data processing utilities. - Optimize transformation workloads for performance, scalability, and cost. - Apply data quality, validation, and business-rule checks throughout the pipeline. • Enterprise Data Modelling - Design scalable data models using: - Dimensional modelling - Star/Snowflake schemas - Data Vault 2.0 - Canonical data models - Develop curated data products and analytical datasets using Databricks SQL and Delta Lake. - Support enterprise-wide data harmonization across clinical, regulatory, safety, and commercial domains. • Experience supporting data platforms across: - Clinical trials – EDC, CDMS, CTMS - Regulatory and submission systems - Pharmacovigilance and safety - Commercial analytics - Real-World Evidence (RWE) • Ensure adherence to relevant regulatory and data governance requirements including GxP, 21 CFR Part 11, HIPAA/GDPR, and ALCOA+ principles. Required Qualifications • 10–15 years of experience in Data Engineering and enterprise data platforms. • Strong hands-on experience with Databricks, Apache Spark, PySpark, Python, and SQL. • 4+ years of experience with Databricks/Spark-based data engineering preferred. • Strong experience in: • Data migration and modernization • ETL/ELT development • Batch and streaming data pipelines • Delta Lake • Databricks Workflows • Unity Catalog • Data modelling • Data quality and reconciliation • Experience with cloud platforms – AWS/Azure/GCP. • Strong understanding of CI/CD, Git, Jenkins, and DevOps practices. • Experience working in Life Sciences/Pharma environments preferred.