Hi,
Job Title: Senior Data Engineer – Azure / Python
Location: Blue Ash, OH (In-Person – Onsite Required)
Duration: 12+ Months
Experience Required: 10+ Years
Competencies: Microsoft Azure | Python | Agile Way of Working
Job Summary:
We are seeking a highly experienced Senior Data Engineer to design, build, and optimize distributed data processing solutions in a cloud-native environment. The ideal candidate will take ownership of stories across the full SDLC, develop scalable data pipelines, enhance CI/CD processes, and ensure data quality and observability within Azure-based ecosystems.
This role requires strong hands-on expertise in Python, Azure cloud services, distributed processing (PySpark/Snowpark), CI/CD, and Agile delivery practices.
Key Responsibilities:
Data Engineering & Pipeline Development
· Develop distributed data processing pipelines using PySpark and/or Snowpark.
· Orchestrate multi-step data transformation pipelines.
· Implement transformation logic using Object-Oriented Programming (OOP) principles.
· Create and maintain data quality checks for ingested and processed data.
· Ensure data observability via monitoring and alerting mechanisms.
Cloud & Infrastructure
· Build and manage Azure cloud resources using Infrastructure as Code (IaC).
· Support and enhance CI/CD pipelines for data applications.
· Work with containerization and cloud-native services.
Testing & Quality Assurance
· Perform unit, integration, and regression testing (PyTest or similar).
· Debug and tune enterprise applications for performance and scalability.
· Ensure adherence to software engineering best practices.
Application Maintenance & Enhancement
· Maintain and enhance existing enterprise applications.
· Improve internal and external data solutions and processes.
· Participate in estimation, sprint planning, and retrospective reviews.
Leadership & Collaboration
· Mentor junior engineers and contribute to team growth.
· Bring innovative perspectives to solve complex technical challenges.
· Work in an Agile/Scrum environment with cross-functional teams.
Required Skills & Qualifications:
· Bachelor’s degree in Computer Science, MIS, Mathematics, Business Analytics, or related field.
· 10+ years of overall experience in data engineering and software development.
· Strong hands-on experience with:
o Python frameworks
o SQL
o Azure Cloud Services (preferred)
o Distributed processing (PySpark / Snowpark)
o CI/CD pipelines
o Infrastructure as Code
· Proficiency in:
o Object-Oriented Programming (OOP)
o SOLID principles
o Automated testing (PyTest)
o Version control systems (Git, GitHub)
o Dependency management tools (Conda, venv)
· Experience with:
o Data observability and monitoring
o Data quality frameworks
o Performance tuning enterprise systems
o Debugging distributed systems
· Understanding of Agile principles (Scrum).
Nice to Have:
· Experience with Java and Spring Framework.
· Knowledge of Retail domain.
· Experience with GCP or AWS in addition to Azure.
Shrey Sharma
Technical Recruiter
LinkedIn: https://www.linkedin.com/in/shrey-sharma-322992370/
Novia Infotech, 4421 Avenida Ln, McKinney, TX, 75070, www.noviainfotech.com
You received this message because you are subscribed to the Google Groups "NoviaJobs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to noviajobs+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/noviajobs/CA%2BRQAPOnZOGyHCxzx1h8T6ON8ANFXZPGrkcRqDcbSkE4tvKadA%40mail.gmail.com.
No comments:
Post a Comment