Technical Lead Big Data Engineer (AWS)
- Remote (anywhere)
- Full-time
- Software engineering
- Posted recently
You’ll apply on Softserve’s own site.
About the role
SoftServe is a global digital services and consulting company with over 30 years of experience and more than 11,000 professionals across 16 countries, helping businesses worldwide solve complex challenges through technology.
Our Data & Analytics Center of Excellence (CoE) brings together data specialists to drive excellence in data, analytics, and business intelligence. We deliver solutions across Data Strategy & Governance, Data Platform Engineering, Data Migration, and Reporting & BI.
In this role, you’ll support and enhance a modern AWS-based data platform, focusing on data ingestion, processing, and platform reliability. You’ll use Python, PySpark, and Apache Spark to maintain and optimize existing data pipelines while collaborating with a distributed engineering team. You’ll work with healthcare data in a regulated environment, and the role includes a three-month business trip to the United States in Q1 2027.
Responsibilities
- Support, maintain, and enhance an existing AWS-based data platform and data ingestion pipelines
- Develop and optimize data processing workflows using Python, PySpark, Apache Spark, and AWS Glue
- Monitor and troubleshoot data pipelines, ensuring reliability, performance, and data quality
- Work with AWS services such as S3, Aurora PostgreSQL, Athena, Lambda, SQS, SNS, and EventBridge
- Maintain and improve workflow orchestration using Apache Airflow
- Support infrastructure and deployment processes using Docker, Terraform, and Amazon EKS
- Apply software engineering best practices, including automated testing, code quality checks, and CI/CD processes
- Utilize AI-assisted development tools such as Claude Code to support implementation, testing, and code reviews
- Collaborate with remote engineering teams to resolve technical issues, implement improvements, and ensure platform stability
- Support data security, governance, and compliance practices within a regulated healthcare environment
Requirements
- 5+ years of experience in Big Data or Data Engineering, including supporting production data platforms
- Strong proficiency in Python, PySpark, and Apache Spark for large-scale data processing
- Hands-on experience with AWS data services such as Glue, S3, Athena, Aurora PostgreSQL, and Lambda
- Experience with Apache Airflow for data pipeline orchestration and scheduling
- Strong knowledge of SQL, data ingestion, ETL processes, and data quality practices
- Experience with Docker, Terraform, and CI/CD pipelines using GitHub Actions or similar tools
- Familiarity with Python development and testing tools such as pytest, Pydantic, ruff, or mypy
- Understanding of healthcare data regulations such as HIPAA/HITRUST, multi-tenant architectures, or data retention requirements is a plus
- Readiness to travel to the United States for a three-month business trip in Q1 2027
- Upper-intermediate or higher level of English, with strong communication and problem-solving skills
Some of our benefits
- SoftServe University: Access continuous learning opportunities, certification support, and leadership development programs to grow your skills and advance your career
- Medical insurance: Health coverage for you and your family
- Food vouchers: Additional support for your everyday expenses
- Extra days off: Enjoy additional time off to rest, recharge, and maintain a healthy work-life balance
- Dare to Care: Our holistic well-being program, designed to support your overall wellness
- Anniversary Program: Celebrating your milestones and recognizing your journey with SoftServe
