Senior Data Engineer

This role is for one of the Weekday's clients

Min Experience: 5 years

Location: Bengaluru

JobType: full-time

We are looking for a seasoned Senior Data Engineer to design, build, and maintain scalable data pipelines that support growing data volumes and complex processing needs. This role involves close collaboration with cross-functional teams to enhance data models that power business intelligence tools, with the goal of improving data accessibility and enabling data-driven decision-making across the organization.

Requirements

Key Responsibilities:

Data Pipeline Development and Maintenance:

Design, develop, and manage robust data pipelines and ETL workflows on AWS.
Utilize AWS services including S3, Glue, Lambda, and RDS to handle and process large-scale datasets.
Support analytics and machine learning use cases with efficient and scalable data solutions.

DevOps and Automation:

Apply DevOps best practices in data engineering workflows, with a focus on automation, CI/CD, and infrastructure as code (IaC).
Develop and manage IaC templates using tools like Terraform or CloudFormation to provision AWS infrastructure.
Implement monitoring, logging, and alerting systems to ensure the reliability, performance, and quality of data pipelines and models.

Qualifications:

Minimum of 5 years of hands-on experience in building and managing ETL/ELT processes.

Key Skills & Competencies:

Cloud Technologies: Proficient with AWS services such as S3, Glue, Lambda, RDS, and SageMaker.
Programming: Advanced knowledge of Python and experience with libraries like Pandas for data processing.
DevOps: Familiarity with CI/CD pipelines, Terraform/CloudFormation for infrastructure automation, and monitoring tools like CloudWatch.
Data Engineering: Strong grasp of data warehousing concepts and ETL/ELT pipelines.
Collaboration: Excellent communication and interpersonal skills; ability to work effectively in cross-functional Agile teams.
Problem Solving: Strong analytical thinking and a solution-oriented mindset.
Soft Skills: Effective planning, time management, organizational abilities, and the capacity to work under tight deadlines.