(BME 580.464) Advanced Topics in Computer Vision: Segmentation, Reconstruction and Recognition of Dynamic Scenes
Instructor: Rene Vidal web e-mail

Class Hours: T 4:45-6:05pm 202 Shafer, W 4:45-6:05pm 313 Hodson

Office Hours: M 5-6, T 6-7, 308B Clark Hall

Course Description
This class will cover state-of-the-art methods in dynamic vision, with an emphasis on segmentation, reconstruction and recognition of static and dynamic scenes. Topics include: reconstruction of static scenes (tracking and correspondences, multiple view geometry, self calibration), reconstruction of dynamic scenes (2-D and 3-D motion segmentation, non rigid motion analysis), recognition of visual dynamics (dynamic textures, face and hand gestures, human gaits, crowd motion analysis), as well as geometric and statistical methods for clustering and unsupervised learning, such as K-means, Expectation Maximization, and Generalized Principal Component Analysis. For a more detailed schedule see the course syllabus.
Announcements
Class cancelled on 03/01/2005. Class will be made up on Monday (03/07/2005) Venue : Hodson 301 4:45-6:00pm
Lectures
  1. Feature Extraction, Matching and Correspondences
  2. Reconstruction of Static Scenes
  3. Central Clustering, GPCA and Reconstruction of Dynamic Scenes.
  4. Recognition of Dynamic Textures and Human Gaits.
    • Read papers from www.vision.ucla.edu
Homeworks
Please submit the code of your homework at Submit HW Please Submit the projects using the homework submission link above. Submit it in the Project proposal as final_report.pdf
  1. Homework 1: Due Wednesday, February 16th, 2005, beginning of class. Solution.
  2. Homework 2: Due Wednesday, March 2nd, 2005, beginning of class. Solution.
  3. Homework 3: Due Wednesday, April 6th, 2005, beginning of class.
  4. Homework 4:Due Tuesday, May 2nd, 2005.

    5. Resource for Homework 4

    Problem

    Code

    Sequences/Images

    Problem 2

     

    Spinal Cord
    Penguin
    Elephants
    Night Sky

    Problem 3

     

    Tiger
    Zebra
    Sand

    Problem 4

    GPCA

    Kanatani1
    Kanatani2
    Kanatani3
    three-cars
    can-book

    Problem 5

    synth.m
    dytex.m

    Ocean

    Heart

Midterms
Midterm 2 reading material: the midterm will cover all topics discussed in class from first midterm onwards. This includes but is not limited to the following chapters from GPCA book: 1,2 (PCA), 3 (Kmeans and EM), 5 (GPCA), 6 (Robust GPCA), 8 (Image Segmentation), 9 (2-D Motion Segmetation). Chapter 10 of GPCA book is not yet finished. Instead, you can read Chapter 8 of MASKS book for 3-D motion segmentation. Slides covering all the material are on the web. As per the material for the last lecture, please read papers on Dynamic textures (IJCV) and gait recognition (CVPR01) from Soatto's group available at www.vision.ucla.edu.
Midterm 2 questions: Question 1 will be about Kmeans, Ksubspace and GPCA, particularly, please study the derivation (cost function, Lagrange multipliers, etc.) of the Kmeans and Ksubspaces algorithms in great detail. Question 2 will be on motion segmentation using GPCA, particularly cases involving subspaces of different dimensions. Question 3: TBS (to be surprised). Notice that the level of difficulty of the material for midterm 2 is >= than that of the material for midterm 1.
Projects
Presentations will be on Wednesday May 11th at Clark 314

  • Reconstruction, Segmentation and Recognition of Dynamic Scenes
    • 1.00-1.12 Jim Taylor: Hand Symbol Sequence Recognition
    • 1.15-1.27 Landon Unninayar: Gesture Recognition
    • 1.30-1.54 Alvina Goh and Dheeraj Singaraju: Gait Recognition Using Hybrid Systems
    • 2.00-2.12 Vinutha Kallem: Multi-body Motion Estimation using More than Three Perspective Views
    • 2.15-2.27 Daniel Abretske: Reconstruction of Nonrigid Motions from Perspective Views
    • 2.30-2.42 Camille Izard: Crowd Motion Segmentation
  • Tracking, Registration and Segmentation of Biomedical Images
    • 3.00-3.12 Avinash Ravichandran: Segmentation of MR Images of a Beating Heart
    • 3.15-3.27 Tara Johnson: Registration of MR Images of a Beating Heart
    • 3.30-3.42 Sharmishtaa Seshamani: Endometrial Tracking and Mosaicing
    • 3.45-3.57 Georgios Kaloutsakis: Toward Segmentation of Human Organs in Laparoscopic Surgery using Robotic Surgical System

 

Suggested Projects

  • How to incorporate common calibration constrains in motion segmentation from two perspective views?
  • How so segment motion models of different type, such as multiple fundamental matrices and multiple homographies?
  • How to segment rigid-body motions from multiple (four or more) perspective views?
  • Segmentation of nonrigid motions from multiple affine views
  • Segmentation of nonrigid motions from multiple perspective views
  • Modeling, segmentation and recognition of crowd motion
  • Recognition of human gaits
  • Recognition of hand gestures