Amazon—Data Engineer II, Prime Video Advertising

Job description

Description: Prime Video is a first-stop entertainment destination offering customers a vast collection of premium programming in one app available across thousands of devices. Prime members can customize their viewing experience and find their favorite movies, series, documentaries, and live sports – including Amazon MGM Studios-produced series and movies; licensed fan favorites; and programming from Prime Video add-on subscriptions such as Apple TV+, Max, Crunchyroll and MGM+. All customers, regardless of whether they have a Prime membership or not, can rent or buy titles via the Prime Video Store, and can enjoy even more content for free with ads. Are you interested in shaping the future of entertainment? Do you want to invent, disrupt and redefine what entertainment means for fans, creators, and communities all around the globe? As part of the Prime Video team, you’ll get to work on projects that are fast-paced, challenging, and varied. Also, you’ll get to experiment with new possibilities, take risks, and collaborate with remarkable people. We’ll look for you to bring your diverse perspectives, ideas, and skill-sets to make Prime Video even better for our customers. With global opportunities for talented creatives, technologists, marketing professionals and so much more, you can decide where a career with Prime Video takes you! We are seeking a highly skilled and technically adept Data Engineer to join our PVA Data Engineering team. In this role, you will design, build, and maintain scalable, fault-tolerant data pipelines and infrastructure to support data-driven insights. Leveraging native AWS technologies and Amazon’s internal tools, you will handle data ingestion, transformation, integration, and ensure high standards of data quality through validation, cleansing, and deduplication. You will also be responsible for architecting high-performance, scalable, and cost-efficient big data processing and storage solutions, automating infrastructure with CI/CD principles, and optimizing data models to drive efficient ETL processes. You will be interfacing with several key services and APIs, write code and build applications in a scalable manner to extract, process, ingest unstructured data in a streaming, batch and asynchronous fashion. Your role will also focus on data governance, ensuring compliance with policies, implementing access controls, encryption, retention, and audit mechanisms. Additionally, you will work on continuous improvement by automating processes and adopting the latest data engineering technologies. Collaboration with Data Scientists, Business Intelligence Engineers (BIEs), Software Development Engineers (SDEs), Product Managers (PMs), and functional managers will be key to delivering tailored data solutions. You'll also manage diverse technologies such as Python, EMR, Spark, Iceberg, Airflow, and many AWS data services like Glue, Athena, Redshift to build a scalable platform that supports analytics and experimentation. The ideal candidate must be customer-obsessed with a passion for advertising, able to synthesize a variety of technologies, and effectively lead complex projects. In this position, your attention to detail and dedication to high-quality, well-documented data products will empower stakeholders to make better, data-driven decisions. Key job responsibilities - Design and implement scalable, fault-tolerant data pipelines using AWS technologies and internal Amazon tools to extract, transform, and load data from multiple sources. - Collaborate cross-functionally with BIEs, Data Scientists, PMs, and SDEs to understand data requirements and deliver customized data solutions. - Automate infrastructure deployment with CI/CD pipelines and ensure streamlined processes for deployment and maintenance. - Ensure data quality through robust validation, cleansing, and deduplication techniques. - Implement data governance standards, including access control, encryption, data retention, deletion policies, and audit mechanisms to ensure compliance and security. - Continuously improve and optimize data pipelines and infrastructure, staying up to date with emerging technologies and implementing automation and monitoring tools. - Build a scalable and reliable data platform supporting analytics and experimentation for intuitive, self-service data products. - Write high quality code and build scalable applications that interface with critical services and APIs to extract and process unstructured data - Work with a range of data technologies, including Python, EMR, Spark, Iceberg, Airflow, and many AWS data services like Glue, Athena, Redshift to create end-to-end pipelines that consolidate data from disparate systems. About the team Prime Video Advertising - Data, Measurement and Insights(“PVA-DMI”) team is a central data and analytics team within the PV-Ads org that plays a pivotal role in fueling the worldwide growth of Amazon’s video advertising business. We own data platform, reporting, dashboards, measurement, and analytical solutions for PV Ads. Requirements: - Bachelor's degree - 3+ years of data engineering experience - Experience with data modeling, warehousing and building ETL pipelines - Experience with SQL - Knowledge of professional software engineering & best practices for full software development life cycle, including coding standards, software architectures, code reviews, source control management, continuous deployments, testing, and operational excellence - Knowledge of distributed systems as it pertains to data storage and computing - Knowledge of batch and streaming data architectures like Kafka, Kinesis, Flink, Storm, Beam - Experience as a data engineer or related specialty (e.g., software engineer, business intelligence engineer, data scientist) with a track record of manipulating, processing, and extracting value from large datasets - Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS - Experience with Apache Spark / Elastic Map Reduce - Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions - Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases) - Master's degree in computer science, engineering, analytics, mathematics, statistics, IT or equivalent - Experience programming with at least one modern language such as C++, C#, Java, Python, Golang, PowerShell, Ruby - Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.