Fall 2017 ITCS6157/8157: Visual Databases

Course Outline:

  • Unsupervised machine learning tools: clustering
  • Supervised machine learning tools: SVM and GMM
  • Database indexing tools for CBIR/CBVR
  • Image and video analysis techniques;
  • Unsupervised Machine Learning Tools;
  • Supervised Machine Learning Tools;
  • Text Document Analysis, Clustering, Indexing & Classification
  • Semantic Image/Video Classification
  • Deep Learning for Large-scale visual recognition
  • Big Data Analysis
  • Applications of Visual Database Techniques: image-based product search
  • Image and video coding standards, such as JPEG, JPEG2000, MPEG-1, 2, 4;
  • Image and Video description standard: MPEG-7 and XML;
  • Future Trends of Visual Search
  • Suggested Text Books (optional):

  • A. Rosenfeld, D. Doermann, D. DeMenthon, ``Video Mining", Kluwer Academic Publishers, 2003.
  • Yihong Gong, W. Xu, ``Machine Learning for Multimedia Content Analysis", Springer, 2007.
  • others
  • Articles and journal papers.
  • Grading Format:

  • Project Implementation & Bonus 25%-45%
  • Middle and Final test 75%
  • Classroom: Woodward Hall 130

    Class Time: Wednesday 6:30PM-9:15PM

    Instructor Office Time:

  • Wednesday 1:30PM-5:00PM or make appointment
  • Mid-Term Test: Oct. 11, 2017 (one page note is allowed)

    Papers for Motivated Students:

  • Informedia Project at CMU: paper 1, Paper 2, Paper 3, Paper 4. presented by presentation slides
  • Project at Columbia University: Paper 1 , Paper 2 , Paper 3 , Paper 4 , Paper 5 . presented by presentation slides
  • Project at University of Amsterdam: Paper 1 , Paper 2 , Paper 3 , Paper 4 . presented by Presentation slides
  • Concept Ontology for Text Classification: Paper 1 , Paper 2 , Paper 3 , Paper 4 , presented by Presentation slides
  • Concept Ontology for Multimedia Classification: Paper 1 , Paper 2 , Paper 3 , Paper 4 , presented by Presentation slides
  • Volume-Based Video Representation: Paper 1, Paper 2 , Paper 3 , Paper 4, presented by Presentation slides
  • Video/Image Visualization: Paper 1 , Paper 2 , Paper 3 , Paper 4 . presented by
  • Multiple Instance Learning for Image Classification: ``paper 1", paper 2, paper 3, paper 4, paper 5, paper 6, paper 7, paper 8, paper 9, presented by
  • Web Image Analysis and Indexing: ``paper 1", paper 2, paper 3, paper 4, paper 5, paper 6, paper 7, presented by
  • Multi-Label Learning for Image Classification: ``paper 1", paper 2, paper 3, paper 4, paper 5, paper 6, paper 7, presented by
  • Entity Extraction: ``paper 1", paper 2, paper 2, paper 3, paper 4, paper 5, presented by
  • ``Sharing visual features for multiclass and multiview object detection", presented by presentation slides
  • S. Tong, E. Chang, ``Support Vector Machine Active Learning for Image Retrieval",, ACM Multimedia, presented by Presentation slides
  • K. Barnard, D. Forth, ``Learning the Semantics of Words and Pictures", ICCV, presented by Presentation sides
  • A. Mojsilovic, et al., ``Matching and Retrieval Based on the Vocabulary and Grammar of Color Patterns", , IEEE Trans. on Image Processing, presented by
  • C. Carson, et al., ``Blobworld: Image Segmentation using EM and its Application to Image Querying", , IEEE Trans. on PAMI,
  • A. Oliva, et al., ``Modeling the shape of the scene: A Holistic Representation of the Spatial Envelop", Int. J. of Computer Vision,
  • W.H. Adams, et al., ``Semantic Indexing of Multimedia Content using Visual, Audio, and Text Cue", EURASIP, presented by Presentation sides
  • M. Naphade et al., ``A Factor Graph Framework for Semantic Video Indexing", IEEE Trans. on CSVT, presented by
  • Z. Ghahramani et al, ``Factorial Hidden Markov Models", Machine Learning.
  • Course Projects:

  • Project Description
  • Every student should select one for implementation.
  • Detials of Course Schedule:

  • Introduction 8/23/2017
  • Unsupervised Machine Learning: Data Clustering Tools: 8/30/2017

  • Data Clustering: Density-Based Approach
  • Metric Learning
  • Spectral Clustering: Relation-Based Approach
  • AP Clustering
  • No.1 Homework Assignment: due day 9/21/2017

  • No. 1 Homework Assignment
  • Supervised Machine Learning Tools 1: 9/6/2017

  • Machine Learning: SVM Classifier
  • Machine Learning: GMM Classifier
  • No.2 Homework Assignment: due day 10/18/2017

  • No. 2 Homework Assignment
  • Supervised Machine Learning Tools 2: 9/13/2017

  • Tree Classifiers: Decision Tree
  • Ensemble Learning: AdaBoost
  • Image Analysis and Representation Tools: 9/20/2017

  • Content-Based Image Analysis
  • Bag of Visual Words and Dictionary Learning
  • Image Classification and Annotation: 10/4/2017

  • Image Classification over Ontology
  • Image Classification over Visual Tree
  • Brief Review for Mid-term Test: 10/4/2017

    Mid-Term Test: 10/11/2017

  • mid-test (one page note is permitted)
  • Content-Based Image Retrieval: 10/18/2017

  • Content-Based Image Retrieval
  • Relevance Feedback for Image Retrieval
  • Project Introduction: 10/18/2017

  • Projection Introduction: Course Project Lists
  • what you should do: (a) select a project implementation; (b) send your choice to TA at tzhao4@uncc.edu; (c) prepare yourself to finish your project on time!
  • Project due day: 11/29/2017-12/6/2017 (make appointment with TA tzhao4@uncc.edu for assessment!)

    Video Analysis for Feature Extraction: 10/25/2016

  • JPEG Standards for Image Coding
  • MPEG Standards for Video Coding
  • Video Shot Detection From MPEG Bit Stream
  • Video Classification and Cross-Media Analytics and Mining: 11/1/2017

  • Video Object Detection for Video Database Indexing
  • Semantic Video Classification Techniques
  • Personalized Recommendation: 11/8/2017

  • Harvest Social Images and Web Image Indexing
  • Personalized News Recommendation
  • Personalized Image Recommendation
  • Image Privacy and Security: 11/15/2017

  • Database and System Security Issues
  • Other Topics for Self-Study: 11/7/2017-12/3/2017

  • Multimedia Analytics
  • Content for Self-Study
  • MPEG-4 Object-Based Video Coding Standard
  • Traditional Techniques for Video Database Indexing
  • Bayesian Learning for Classification
  • Multimedia System Design
  • Network Protocols for Video Transmission
  • QoS Control for Video Transmission
  • Video Transcoding for Adaptive Streaming
  • Existing Systems Introduction: IBM QBIC
  • 11/22/2017: Thanksgiving holiday, no class

    Text Document Analysis and Indexing Tools: 11/29/2017

  • Text Document Representation and Indexing
  • Information Extraction from Text Documents
  • Text Classification
  • Text Clustering
  • Information Retrieval: introduction
  • Information Retrieval: Inverse File Indexing
  • Database Indexing Tools:

  • High-Dimensional Database Indexing: KD Tree
  • High-Dimensional Database Indexing: R-Tree
  • Challenges for Visual Database: Open Discussion
  • Deep Learning and Big Data Analytics: 12/6/2017

  • Deep Learning and Big Data Analytics
  • Review for Final Test: 12/6/2017

  • Final Test: Dec. 13, 8:00PM-10:30PM!

    (one page note is allowed, but no Internet access)
  • Do what you can, with what you have, where you are! ---Theodore Roosevelt---