Hi,
This is Diksha Chaudhary working with Novia Infotech. We have the below contract job opportunity with one of our direct clients and would like to check if you have any resources available. Please send across the resume of your consultants along with the contact information at the earliest to diksha.c@noviainfotech.com.
Role: Senior Data Engineer – Azure (Python / Distributed Data Processing)
Location: Cincinnati, OH
Duration: 12+ Months
Interview type: Telephonic / video
Description:
We are seeking a highly experienced Senior Data Engineer with strong expertise in Microsoft Azure, Python, and distributed data processing. The ideal candidate will take ownership of stories and drive them to completion across the full SDLC, delivering scalable and high-quality data solutions.
This role involves building and orchestrating enterprise data pipelines, enhancing CI/CD processes, and ensuring data quality and observability in cloud-based environments.
Key Responsibilities
Data Engineering & Pipeline Development
- Develop distributed data processing pipelines (PySpark and/or Snowpark).
- Orchestrate multi-step data transformation workflows.
- Build transformation logic using Object-Oriented Programming (OOP) principles.
- Implement data quality checks for ingested and post-processed data.
- Ensure data observability via monitoring and alerting solutions.
- Maintain and enhance existing applications and data pipelines.
Cloud & DevOps Enablement
- Build and manage cloud resources using Infrastructure as Code (IaC).
- Enhance and maintain CI/CD pipelines for production deployments.
- Manage dependencies (Conda, venv, etc.).
- Debug and performance tune enterprise applications.
Testing & Code Quality
- Perform unit, integration, and regression testing.
- Implement automated testing frameworks (PyTest or similar).
- Follow software engineering best practices and SOLID principles.
- Use Git/GitHub for version control and collaboration.
Agile & Collaboration
- Participate in sprint planning, estimation, and retrospective sessions.
- Mentor junior engineers and contribute to knowledge sharing.
- Collaborate with cross-functional teams on internal and external-facing applications.
- Continuously improve engineering processes and delivery standards.
Required Skills & Experience
- 10+ years of overall IT experience.
- Bachelor’s degree in Computer Science, MIS, Mathematics, Business Analytics, or related technical field.
- 2+ years of professional data development experience.
2+ years of SQL development experience.
- Strong proficiency in:
- Python and Python frameworks
- Object-Oriented Programming (OOP)
- Distributed data processing (PySpark / Snowpark)
- CI/CD pipelines
- Git/GitHub
- Automated testing (PyTest)
- Experience with:
- Data Quality processes
- Data Observability tools
- Infrastructure as Code
- Performance tuning
- Experience with cloud technologies (Azure preferred; AWS/GCP acceptable).
- Understanding of Agile/Scrum principles.
Preferred Skills
- Experience with Java and Spring Framework.
- Experience working in large-scale enterprise data environments.
- Strong analytical and problem-solving capabilities.
|
' |
Diksha Chaudhary |
E: diksha.c@noviainfotech.com |
|
You received this message because you are subscribed to the Google Groups "NoviaJobs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to noviajobs+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/noviajobs/CAEm%3D8YXZu4pKnmOgwDBW-XjCnkS2a2mneBCBeyA4QSYXXph-fw%40mail.gmail.com.
No comments:
Post a Comment