Note that there are separate sets of assignments for CS 451/651 and CS 431/631. Make sure you work on the correct asssignments!

CS 431/631 Assignments

Assignment 2 due 4:00 pm Feb. 714

This assignment requires you to compute statistics over tokens in the text of Shakespeare's plays, as you did for Assignment 1. However, instead of performing the analysis purely in Python, you will use Spark (with a Python driver program).

You will need to upload this file to Google Colab:

• A2.ipynb: this is the starter notebook for A2.

As usual, open the starter notebook on the hub to get started working on the assignment.

Submitting Assignment 2

When you have finished, submit A2 using the same procedure you used for A1:

1. Download your A2.ipynb file from the Jupyter hub to your computer.
2. Submit your A2.ipynb file to your Git repository using the web interface like you did in assignment 1.

That's it! Note that your A2.ipynb should be committed and pushed to origin before the deadline.

Hints for Q5 + Q6:

• Use broadcast variables.
• Spark accumulators are helpful for counting things.
• You can learn broadcast variables and accumulators here.

Back to top