Big Data Analytics
General
- Graduate
- 150 learners; teams of 4
- 90 hours per learner
- Dates set by experience
- Learners self-assign
Preferred companies
- 5/5 project matches
- Montreal, Quebec, Canada
- Academic experience
- Any
- Any
Categories
Project timeline
-
January 18, 2023Experience start
-
April 15, 2023Experience end
Overview
- Details
-
A team of 3-5 students will implement a data-science project using Big Data technologies Apache Spark, Dask or scikit-learn.
- Learner skills
- Python, Data analytics, Research
- Deliverables
-
- Project summary: The project summary will be a 400-word abstract available as a Markdown (.md) document in a public or private GitHub repository. The summary will report on project definition and model design. It will describe the dataset used in the project and its main characteristics (number and type of features), the research questions to be addressed in the project, the class of models to be applied to the dataset, and the algorithms that will be used. At least two algorithms must be used and compared.
- Project data model: The project data model will be delivered as a Jupyter notebook containing code and explanations to implement data preparation, model training and preliminary model evaluation.
- Final project presentation: The final project presentation will go through the final Jupyter notebook implemented for the project, putting special emphasis on model evaluation and summarizing the other project milestones.
Project Examples
In this assignment, students will work on a dataset to answer specific exploratory questions by applying one or more techniques seen in class: supervised learning, recommender systems, unsupervised clustering, frequent itemset mining, data stream analytics, graph analysis, and similarity search. Students will implement the project in Python, using Jupyter notebooks and a data analytics library among Apache Spark, Dask or scikit-learn.
As a participating organization, you’ll be asked to provide a particular dataset and a first set of related questions to be answered by the team using the dataset.
The expected project milestones are as follows:
- Project definition: students will summarize the project, including: (1) the dataset of interest, (2) the set of exploratory questions to be answered with the dataset, using techniques studied in class.
- Model design: students will choose a class of models in {supervised learning, recommender systems, unsupervised clustering, frequent itemset mining, data stream analytics, graph analysis, similarity search}. They will outline how the data model could be applied to the dataset to answer the exploratory question(s). They will research algorithms and techniques to implement this class of model.
- Data preparation: students will inspect the dataset, identify missing data, outliers, data types (categorical data in particular), and write Apache Spark or Dask programs to correct for potential issues.
- Model implementation: students will implement the model with Apache Spark, Dask or scikit-learn.
- Model evaluation: students will identify evaluation metrics for the model, implement, and discuss them.
Additional company criteria
Companies must answer the following questions to submit a match request to this experience:
The proposed project includes a dataset that the students will be able to access and analyze.