ITCS5152/4152: Computer Vision (Spring 2020)

We start to use Webex for our class from 3/19/2020

Course Description

This course provides an introduction to computer vision, which includes fundamentals of image formation, camera imaging geometry, feature detection and matching, stereo, motion estimation and tracking, image/video classification, image/video segmentation and scene understanding. We'll develop basic methods for applications that include finding known models in images/videos, depth recovery from stereo, automated image alignment, boundary detection, and recognition. The course focus is to develop the intuitions and mathematics of the methods in lecture, and then to learn about the difference between theory and practice in the projects. Deep learning methods are introduced because of recent attentions from both industry and academy.

Learning Objectives:

Students should be able to:
  • Recognize and describe both the theoretical and practical aspects of computing with images.
  • Connect issues from Computer Vision to Human Vision.
  • Describe the foundation of image formation and image analysis. Understand the basics of 2D and 3D Computer Vision.
  • Become familiar with the major technical approaches involved in computer vision.
  • Describe various methods used for registration, alignment, and matching in images.
  • Get an exposure to advanced concepts leading to object and scene categorization from images.
  • Know deep learning methods for image classification and object detection, segmentation and recognition.
  • Build computer vision applications.
  • Prerequisites:

    No prior experience with computer vision is assumed, although previous knowledge of visual computing or signal processing will be helpful. The following skills are necessary for this class:
  • Data structures: You'll be writing code that builds representations of images, features, and geometric constructions.
  • Programming: Projects are to be completed and graded in Python. All project starter code will be in Python. TA (if we will have one) will support questions about Python. If you've never used Python that is OK, as long as you have programming experience.
  • Math: Linear algebra, vector calculus, and probability. Linear algebra is the most important and students who have not taken a linear algebra course have struggled in the past.
  • Grading:

    Your final grade will be made up from:
  • 40% from at least 2 programming projects
  • 60% from 2 written exams (mid-term and final)
  • You will lose 5% each time for missing class. You will lose 5% for three times for your late comings.

    Suggested Text Books (optional):

  • Richard Szeliski: "Computer Vision: Algorithms and Applications"
  • Classroom:

    Woodward Hall 135

    Class Time:

    Thursday 4:00PM-6:45PM

    Instructor Office Hours:

  • Thursday 11:00AM-13:30PM; 6:45PM-8:45PM or make appointment
  • Class TA:

  • Name: Changlin Li, Email: cli33@uncc.edu
  • Office Location: Woodward Hall TA's Office (second floor)
  • Office Hours: Thursday 10:00AM-15:30PM
  • Exam Schedules:

  • Mid-Term Test: March 12, 2020, 4:00PM-6:45PM, (one page note is allowed)
  • Final Test: May 7, 2020, 2:00PM-4:30PM, (one page note is allowed)
  • Course Projects:

  • Project Description (you can start as early as possible).
  • Every student should select at least three projects for implementation and demonstration.
  • Project 2 is requred for every student in this class!
  • The deadline of project due day is May 7, 2020, but you are encouraged to make early schedule to demonstrate to TA ( Changlin Li, cli33@uncc.edu ).
  • Bonus on final grade will be given to top project implementations, up to 20%-40%.
  • We will not offer group project, every student must do own three projects individually!
  • Course Schedule (Weekly Topics & Assignments):

    Course Introduction and Class Policy: 1/9/2020

  • Course Introduction & Class Policy 1/9/2020
  • Cameras and ISP (Image Signal Processing): 1/16/2020

  • Cameras and Optics
  • Distortion-Free Wide-Angle Portraits on Camera Phones
  • Image Filtering: Part I
  • Image Filtering: Part II: Project 1 is assigned!
  • ISP (Image Signal Processing) Pipeline: 1/23/2020

    The future of digital camera is about computational photography but not optics
  • ISP pipeline
  • Traditional Approaches for Image Denoising
  • Interest points and corners and Local image features: 1/30/2020

  • Edge detectors
  • Interest points and corners
  • Local image features
  • Bag of Visual Words
  • RANSAC and transformations: 2/6/2020

  • Hough transformation & image fitting
  • RANSAC for image alignment Project 2 is assigned
  • Application of Image Alignment: Burst for image HDR
  • Image Classification: Traditional Approaches 2/13/2020

  • Machine Learning Tools for Image Classification: Part I: Traditional Approaches Project 3 is assigned
  • Machine Learning Tools for Image Classification: Part II: Ensemble Approach
  • 2/20/2020: Today's Class is canceled by UNCC because of bad weather; Students should exercise good judgment and monitor local weather forecasts when determining travel plans

    Deep Networks for Image Classification: 2/27/2020

  • Deep Networks for Image Classification: Part V
  • Light-Weight Deep Networks for Mobile Applications: Part VI
  • Review for Mid-term Test 2/27/2020

    No Class: 3/5/2020: Spring Break

    Mid-term Test: 3/12/2020

    Tree-Based Image Classification: 3/19/2020: We start to use Webex for our class from today

  • Machine Learning Tools for Image Classification: Part III: Concept Ontology
  • Machine Learning Tools for Image Classification: Part IV: Visual Trees
  • Object Detection with sliding windows: 3/26/2020

  • Object and Face Detection with Sliding Windows: Part I Project 4 is assigned
  • Object detection with sliding windows: Part II (Deep Learning) Project 5 is assigned
  • Object Detection with sliding windows: 3/26/2020

  • Object detection with sliding windows: Part III: YOLO & SSD (Deep Learning)
  • Semantic Image Segmentation: 4/2/2020

  • Traditional Approaches for Semantic Image Segmentation
  • Deep Learning for Semantic Image Segmentation: Deeplabs
  • Autoencoder Networks for Semantic Image Segmentation: 4/2/2020

  • Deep Learning for Semantic Image Segmentation: SegNets
  • Deep Learning for Semantic Image Segmentation: U-Nets
  • Image Instance Segmentation: 4/9/2020

  • Deep Learning for Image Instance Segmentation: Mask-RCNN
  • Deep Learning for Image Instance Segmentation: RefineNets
  • Deep Learning for Image Instance Segmentation: YOLACT
  • Automatic ISP Tuning: 4/9/2020

  • AI-ISP: Joint Image Denoising & Demosaicking
  • Automatic ISP Tuning
  • Depth Estimation and RGB-D Image Segmentation: 4/16/2020

  • Depth Estimation from Stereo
  • Deep Learning for RGB-D Image Segmentation
  • Monocular Depth Estimation
  • Camera Calibration
  • Stereo Matching
  • Optical Flow Estimation and Semantic Video Segmentation: 4/23/2020

  • Optical Flow Estimation
  • Deep Learning for Optical Flow Estimation
  • Deep Learning for Semantic Video Segmentation
  • Deep Learning for Video Instance Segmentation
  • Applications: Large-Scale Visual Recognition and Retrieval: 4/23/2020

  • Large-Scale Image Retrieval
  • Large-Scale Visual Recognition
  • Deep Learning for Large-Scale Visual Recognition
  • Applications: Low Power Mobile Computer Vision: 4/23/2020

  • Low Power Mobile Computer Vision Project 6 is assigned
  • Lifelong Learning for Mobile Computer Vision
  • Introduction of AutoML: 4/23/2020

  • Automatic Machine Learning
  • Automatic Search of Neural Network Architectures
  • Scale Issue for Computer Vision Systems: Data & Quality Matters 4/23/2020

  • Crowdsourcing for Image Labeling
  • Learning from Online Images: Part I
  • Learning from Online Images: Part II
  • Learning from Noisy Images (Crowdsourcing & Online Images)
  • Theory of Generative Adversarial Nets
  • Synthetic Image Generation via GANs
  • Deep Learning with Synthetic Images
  • Schedules for Project Demonstration and Assessment: 2/3/2020-5/6/2020

  • You can make appointment with TA for individual project demonstration and grading.
  • Final Test: 5/7/2020, 2:00PM-4:30PM

    Acknowledgements

    The materials from this class rely significantly on slides prepared by other instructors, especially James Hays, Derek Hoiem and Svetlana Lazebnik. Each slide set and assignment contains acknowledgements. Feel free to use these slides for academic or research purposes, but please maintain all acknowledgements.

    Do what you can, with what you have, where you are! ---Theodore Roosevelt---