LatentView Analytics—Assistant Manager - Data Engineering

Job description

Description: Role summary: We are looking for a Lead Fabric Data Engineer with 7+ years of experience to lead the design, development, and implementation of enterprise data solutions using Microsoft Fabric. The role will focus on building scalable data pipelines, implementing Lakehouse and Medallion Architecture, migrating legacy workloads, optimizing data platforms, and providing technical leadership to data engineering teams. Roles & Responsibilities: Lead the design, development, and deployment of end-to-end data pipelines using Microsoft Fabric Data Factory, Dataflows Gen2, and Spark Notebooks. Implement and optimize Medallion Architecture (Bronze/Silver/Gold) using Delta Lake, OneLake, schema evolution, partitioning, and performance optimization techniques. Drive data migration and integration from Azure Synapse, SQL Server, Databricks, and Snowflake into Fabric Lakehouse and Warehouse environments. Develop batch, streaming, and event-driven data solutions using Real-Time Intelligence, Eventstreams, KQL Database/Eventhouse, and OneLake capabilities. Design and optimize semantic models and Power BI integration, including Direct Lake, Import, and DirectQuery modes for high-performance analytics. Implement data governance, security, CI/CD, Git integration, automated testing, deployment pipelines, and Microsoft Purview for secure and governed data platforms. Provide technical leadership and mentorship through code reviews, architectural guidance, documentation, engineering standards, and collaboration with architects and business stakeholders. Required Skills: 7+ years of experience in Data Engineering, Data Platform Development, or ETL Architecture, with 1.5+ years of hands-on Microsoft Fabric experience. Strong expertise in Microsoft Fabric, OneLake, Lakehouse, Warehouse, Data Factory, and Real-Time Intelligence. Strong hands-on experience with Python/PySpark, Apache Spark, SQL/T-SQL, Delta Lake, and DAX. Strong understanding of Kimball dimensional modeling, star schemas, Data Lakehouse architecture, partitioning, and performance optimization. Experience with Power BI semantic models, including Direct Lake, Import, and DirectQuery. Hands-on experience with Azure DevOps/GitHub Actions, Fabric Deployment Pipelines, Git, CI/CD, and Microsoft Purview. Knowledge of M, PowerShell, or Azure CLI and strong problem-solving, communication, and technical leadership skills. Preferred Certifications: - Microsoft Certified: Fabric Data Engineer Associate (DP-700) - Microsoft Certified: Fabric Analytics Engineer Associate (DP-600) - Microsoft Certified: Azure Data Engineer Associate (DP-203)