Lead Data Engineer

Role Overview: We are seeking an experienced Lead Data Engineer to join our billing transformation program. The engineer will be responsible for sourcing telemetry data, building scalable data pipelines, supporting rating logic, and preparing billing-ready outputs for SAP invoicing. The role requires strong experience with AWS Glue, Snowflake, Python, SQL, and CI/CD practices. The candidate will also be responsible for implementing data quality checks, reconciliation controls, audit trails, and operational monitoring to ensure billing accuracy. This is a critical role supporting a transformation from raw telemetry usage data to rated, validated, and invoice-ready billing data.

Key Responsibilities: · Data ingestion pipelines from telemetry systems. · Curated usage datasets in Snowflake. · Rating logic implementation and billing-ready datasets. · SAP invoice feed generation. · Data quality and reconciliation framework. · Exception and error-handling process. · CI/CD pipelines for data engineering deployments. · Monitoring, alerting, and operational dashboards. · Technical documentation and runbooks.

Required Skills: · Strong hands-on experience in data engineering, ETL/ELT development, and large-scale data pipeline design. · Experience with AWS Glue, PySpark, Python, SQL, and cloud-based data processing. · Strong experience working with Snowflake as a data warehouse or enterprise data platform. · Good understanding of usage-based billing, rating engines, invoice feeds, or financial data processing. · Experience building data quality checks, reconciliation frameworks, audit controls, and exception handling processes. · Hands-on experience with CI/CD tools such as GitHub Actions, GitLab CI, Jenkins, AWS Code Pipeline, or similar. · Strong SQL skills for data analysis, validation, troubleshooting, and performance optimization. · Experience working with batch and event-driven data ingestion patterns. · Understanding of data modelling, partitioning, performance tuning, and cost optimization.