Hi,
My name is Rohit Chauhan, and I am a Staffing Specialist at Novia Infotech LLC. I am reaching out to you on an exciting job opportunity with one of our clients.
Job Title: Senior Data Engineer – AWS & Python
Location: Malvern, PA
Job Summary
We are seeking a highly skilled Senior Data Engineer with strong expertise in AWS, Python, and PySpark to design, build, and optimize scalable, event-driven data pipelines. The ideal candidate will have hands-on experience with modern cloud data platforms, streaming architectures, and distributed data processing frameworks.
Key Responsibilities
- Design, develop, and maintain event-driven data pipelines using AWS services such as Kinesis, MSK (Kafka), Lambda, Step Functions, SQS/SNS, Glue, and EMR
- Build robust ETL/ELT workflows using Python and PySpark for large-scale data processing
- Implement and optimize Spark-based transformations, partitioning strategies, and performance tuning
- Design and manage data lakes and data warehouses using Amazon S3, Glue Catalog, Athena, and Redshift
- Develop real-time streaming solutions with features like checkpointing, idempotency, schema evolution, and stateful processing
- Ensure data quality, monitoring, and observability using tools like CloudWatch and Datadog
- Implement data security and governance best practices including IAM roles, encryption (KMS), and networking
- Build reusable frameworks, libraries, and implement CI/CD pipelines for automated deployments
- Collaborate with data scientists, analysts, and business stakeholders to deliver reliable datasets
- Lead design reviews, mentor junior engineers, and enforce engineering best practices
Required Qualifications
- 6–8+ years of overall IT experience
- 5+ years of hands-on experience in Data Engineering
- Strong expertise in Python and PySpark
- Advanced experience with AWS Cloud Services including:
- S3, Glue, EMR, Lambda
- Step Functions, Kinesis/MSK (Kafka)
- DynamoDB, Athena, Redshift
- Strong experience in building event-driven and streaming data pipelines
- Proficiency in SQL for ETL and analytical workloads
- Experience with workflow orchestration tools such as Apache Airflow or AWS Step Functions
- Hands-on experience with CI/CD tools, Git, and Infrastructure-as-Code (Terraform or CloudFormation)
- Strong understanding of distributed systems, data modeling, and performance tuning
- Knowledge of cloud security, encryption, networking, and compliance
Preferred Qualifications
- Experience with Core Java
- Exposure to real-time data streaming frameworks
- Experience working in Agile environments
|
Rohit Chauhan IT Recruiter A: 4421 Avenida Ln, McKinney, TX, 75070
|
You received this message because you are subscribed to the Google Groups "NoviaJobs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to noviajobs+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/noviajobs/CAJ0-OE8BGgkFoSyh29zdEccGKFowjCguthg2_Ky%3D6o%3DwJ_sBXQ%40mail.gmail.com.
No comments:
Post a Comment