Assignment 3
ITCS-6190/8190: Cloud Computing for Data Analysis, Fall 2014

Assignment 3:

• The due date for Assignment 2 is 11:59:59pm on Friday, October 3, 2014 .

• Assignment 3 description: assign3.pdf .

• Sample data files: roadnot.txt by Robert Frost, daffodils.txt by William Wordsworth, cagedbird.txt by Maya Angelou.

• These two lines can help you get the filename.
  FileSplit fileSplit = (FileSplit) reporter.getInputSplit();
  String fileName = fileSplit.getPath().getName();

• Example output format for inverted index program: index_all.txt and full inverted index program: fullindex_all.txt when run on all input files. (Note that these output files have been truncated.)

• We would like your query program to be able to read in queries from a file containing query terms, one query per line. Here is an example query file, query.txt , and the format of the output of the query program query_output.txt .

• Please use the following naming convention for the source code you submit.
  Simple inverted index: Index.java
  Query on simple inverted index: QueryIndex.java
  Full inverted index: FullIndex.java
  Query on full inverted index: QueryFullIndex.java

• New clarifications: Updated 11:00am, Sep 29, 2014: Please submit the index file for "roadnot.txt", index file for all three example input files above, full index file for "roadnot.txt", and index file for all three example input files above.

  Please use the following naming convention for the output files you submit. If output is generated by the inverted index program on a single input file, name the file "index_<inputfilename>.txt". For example, inverted index output for input file "roadnot.txt" should be "index_roadnot.txt". If output is generated by the inverted index program on all input files, name the file "index_all.txt". Name the output generated by the full inverted index program on all input files "fullindex_all.txt".

• New: Updated 11:50pm, Sep 28, 2014: Here is an example query file, query2.txt , and the format of the resulting output of the full index query program query2_output_fullindex.txt.

• New and Important: Updated 11:50pm, Sep 28, 2014: The Index.java and FullIndex.java programs should take as input arguments the input folder and output folder. The QueryIndex.java and QueryFullIndex.java programs should take three input arguments in the following order: QueryFileName IndexFileName OutputFileName. Please DO NOT hardcode any filenames in your programs.

  IMPORTANT: In your assignment submission, please comment your code and include a README file indication usage on an example. This will help the TA run your program.

• New : Updated 11:00am, Sep 29, 2014: You may find it helpful to use the RandomAccessFile class in Java for your query program.

• Assignment 3 submission: Please submit using Moodle. You should zip your output files (following naming convention above), your source code for the inverted index program and full inverted index program and the corresponding query programs, and any README files so we can run your code.

  IMPORTANT: Please name the zip file with your userid (eg, sakella.zip). If you have multiple submissions, you can add a version number extension to the filename (eg, sakella.v3.zip).

  Note: The assignment is due by Friday, October 3, 11:59:59pm. You can submit your assignment multiple times. However only the most recent submission will be graded. The timestamp on your most recent submission will also be used to compute late days used.