As a Data Engineering Intern, you will play a crucial role in refining and updating our current Azure data pipeline and CI/CD pipeline to enhance real-time data flow. This position offers a unique opportunity to work with advanced cloud technologies and contribute to meaningful projects.

Key Responsibilities:

·       Collaborate with the data engineering team to refine and optimize Azure data pipelines for real-time data processing.

·       Assist in the development and maintenance of CI/CD pipelines, ensuring efficient deployment of data models and applications.

·       Contribute to the design and implementation of scalable and robust data solutions.

·       Participate in code reviews and adhere to best practices in data engineering.

·       Engage in troubleshooting and resolving technical issues related to data pipelines and workflows.

·       Document technical processes and contribute to the continuous improvement of our data engineering practices.

Qualifications:

·       Preferred enrollment in a Master's or PhD program in Computer Science, Data Science, Engineering, or a related field, but candidates with a Bachelor’s degree and relevant experience are also encouraged to apply.

·       Demonstrated experience in data engineering, preferably with Azure cloud services.

·       Familiarity with CI/CD tools (Azure DevOps and GitLab).

·       Previous internship experience in a similar role is highly desirable.

·       Strong programming skills in Python, SQL, or other relevant languages.

·       Ability to work in a fast-paced, team-oriented environment.

·       Excellent problem-solving and analytical skills.

·       Effective communication and interpersonal abilities.

Project Deliverables:

1. Refined and Optimized Azure Data Pipelines:

• Documentation detailing refinements made to Azure data pipelines, highlighting optimizations for real-time data processing.
• Improved pipeline efficiency metrics or performance reports.

1. CI/CD Pipeline Development and Maintenance:

• Codebase demonstrating contributions to CI/CD pipeline development.
• Maintenance records showcasing efficient deployment of data models and applications.

1. Design and Implementation of Scalable Data Solutions:

• Design documents or diagrams illustrating contributions to scalable and robust data solutions.
• Implemented solutions demonstrating scalability and robustness.

1. Code Reviews and Best Practices Adherence:

• Reports or summaries from code review participation, indicating adherence to best practices in data engineering.
• Contributions to establishing or enhancing best practices documentation.

1. Troubleshooting and Issue Resolution:

• Records or reports outlining troubleshooting processes and resolutions for technical issues related to data pipelines and workflows.

1. Technical Process Documentation:

• Documented technical processes related to data engineering, contributing to continuous improvement efforts.
• Updated or new documentation reflecting improved data engineering practices.

Employer Commitment:

Our company is committed to providing a minimum of 5 hours of mentorship per team. This mentorship will be tailored to address specific project challenges, offer career advice, and foster a supportive learning environment. The mentorship sessions aim to complement the technical guidance and ensure that students not only complete the project successfully but also gain valuable insights into real-world applications of their skills.

This combination of technical support, collaborative tools, and dedicated mentorship hours aims to empower students, fostering an environment conducive to their growth and success in completing the project.