Assignment 1
ITCS-6010/8010: Cloud Computing for Data Analysis, Spring 2012

Assignment 1:

• The due date for Assignment 1 is 11:59:59pm on Thursday, February 2 .

• Assignment 1 description: assign1.pdf .

• WordCount Example , including code.
  See Word Count for more details.

• Sample data files: small.txt , medium.txt , big.txt. (Actually, these are all relatively small files.)

• Source code for wordcount program .

• These two lines can help you get the filename.
  FileSplit fileSplit = (FileSplit) reporter.getInputSplit();
  String fileName = fileSplit.getPath().getName();

• Example output format for wordcount program: wordcount_small.txt when run on small.txt .

• Example output format for wordcount program: wordcount_all.txt and document wordcount program: docwordcount_all.txt when run on all input files. (Note that these output files have been truncated.)

  So the output <<word, filename>, count> pairs are formatted one per line as <word> :<filename> <wordcount in file> .

• Please use the following naming convention for the output files you submit. If output is generated by the word count program on a single input file, name the file "wordcount_<inputfilename>.txt". For example, wordcount output for input file "small.txt" should be "wordcount_small.txt".If output is generated by the word count program on all input files, name the file "wordcount_all.txt". Name the output generated by the document word count program on all input files "docwordcount_all.txt".

• Assignment 1 submission: Please submit using Moodle. You should zip your output files (following naming convention above), your source code for the document word count program, and any README files so we can run your code.
• IMPORTANT: Please name the zip file with your userid (eg, sakella.zip). If you have multiple submissions, you can add a version number extension to the filename (eg, sakella.v3.zip).

• You can submit your assignment multiple times. However only the most recent submission will be graded. The timestamp on your most recent submission will also be used to compute late days used.

  Also, note that the assignment is due by Thursday, February 2, 11:59:59pm, that is midnight of Feb 2.

Hadoop Resources

• For a Hadoop overview, see the MapReduce tutorial.

• For an Apache Hadoop install, see the Hadoop Quick Start guide and Mohammad Hasan's notes from the help session.

• See the discussion of the WordCount example, including information on compiling WordCount.java and creating a jar file.

• For a Cloudera Hadoop install on VMWare Player, see Namrata Kaushik's instructions from the help session.

• For a Cloudera Hadoop install on Linux, see the Cloudera download site. You should install Cloudera Manager Free Edition.