CSE 678 Face and Gesture Recognition

Spring 2011


Instructor: Dr. Yun (Raymond) Fu

Course Webpage: http://www.cse.buffalo.edu/~yunfu/course/CSE678_Spring2011.htm

Course Syllabus:

Times: Tuesday and Thursday 5:00pm-6:20pm

Location:  220 Natural Sciences Complex (NSC), North Campus

Office Hours: Wednesday 2pm¡ª4:30pm by appointment

Office Hours Location: Bell Hall 241


Course Overview

Face and gesture recognition is an advanced technology that utilizes the intrinsic physiological or behavioral traits of individual for machine-based automatic and reliable identification. It attracts much attention due the increasing demand for the security, privacy, and health care related human-centered applications. This course covers the state-of-the-art face and gesture recognition technologies, including face/human detection, face/body tracking, face recognition, head/body pose estimation, expression recognition, body language recognition, gait analysis, hand/body/eye gesture, action/activity analysis, and so forth. Multimodal, multimodality, and soft-biometric frameworks will also be discussed. Fundamental knowledge covered by the course include pattern recognition, feature extraction, classifier, probabilistic models, image processing, and machine learning. Tools and techniques for practical face and gesture recognition system design as well as hands-on exercises and projects will be provided.



CSE 555 or CSE 574, and CSE 573; or permission by instructor.



Students will be graded on participation (at most two times absence), homework, a mid-term examination, and a final project and presentation. The final grade will be composed as follows:

Class Participation................. 10%                   Homework............................. 30%
Mid-Term Exam.................... 20%                   Final Project........................... 40%


Reference Books (Not required to purchase)

Class lecture slides will be provided by the instructor for each student before each class, either printout or electronic file. Students will be asked to find more self-learning content from Internet resource. Recommended textbooks are:

1.     Pattern Classification (2nd Edition) Editors: Richard O. Duda, Peter E. Hart , David G. Stork, ISBN: 978-0-471-05669-0, Wiley, 2000, 680 pages

2.     Recent Advances in Face Recognition, Editors: Kresimir Delac, Mislav Grgic and Marian Stewart Bartlett, IN-TECH, Vienna, Austria, 2008, 236 pages

3.     Face Processing: Advanced Modeling and Methods, Editors: Wenyi Zhao and Rama Chellappa Elsevier/Academic Press, 2005, 768 pages

4.     Sushmita Mitra, Tinku Acharya, ¡°Gesture Recognition: A Survey¡±, IEEE Transactions on systems, man, and cybernetics - part c: applications and reviews, vol. 37, no. 3, May 2007.

5.     Intelligent Video Surveillance: Systems and Technology, Y. Ma and G. Qian (Eds.), Taylor and Francis Group, 2009


Course Topics and Schedules








Jan 18 & 20

Course Information and Introduction

No class



Jan 25 & 27

Image/Video Processing

Math/Stat Basics



Feb 1 & 3

  Pattern Recognition

Feature and Classifier



Feb 8 & 10

Face Detection/Tracking

Face Recognition in Images(TA)

HW 1



Feb 15 & 17

Face Recognition in Video

Expression Recognition



Feb 22 & 24

Human Detection/Tracking

Pose Estimation



Mar 1 & 3

Proposal Presentations




Mar 8 & 10

Eye Gesture Recognition


HW 2



Mar 15 & 17

Spring Recess--No class

Spring Recess--No class



Mar 22 & 24

Action/Activities Recognition(TA)

Hand Gesture



Mar 29 & 31

Paper Discussion

Paper Discussion



Apr 5 & 7

Soft Biometrics I

Soft Biometrics II

HW 3



Apr 12 & 14

Face and Gesture in Multimedia




Apr 19 & 21

Multimodality System Design

Project Presentations



Apr 26 & 28

Project Presentations

Final Proposal



May 3 & 5

Reading Day--No class

Final Exams--No class


* One or two guest lecturers will be invited to present some topics if funding is available for honoraria or expenses.


Final Project

The final project has two options: demo design or computing tool design. The demo design mainly focuses on the programming of a real-world vision system. The computing tool design is mainly to implement or invent an (new) algorithm, test on some face/gesture database and provide comparisons and discussions. The data should be real data, which can be either collected by the individual or borrowed from somewhere (with permission and acknowledgement). Students can use any API or programming language they like. Students can work on the project by themselves or team up with other students in the class. The team members cannot be more than two.


To grade the final project, three aspects will be considered. 1) proposal presentation (20%); 2) final project presentation (30%); 3) final project report and software package (50%). Late submission without instructor¡¯s permission may not be considered. Typically, we do not anticipate that the grades for each team member will be different. However, we reserve the right to assign different grades to each team member if the efforts or contributions they make are apparently different and unbalanced. Bonus points may be earned if the project shows significant novelty and large potential for real-world applications. Those projects may get our guidance for further paper publications.


Proposals and Reports

Please consider following contents when you prepare for your proposals and final reports:

  • Project title
  • Team members¡¯ names, affiliations and emails (one or two members)
  • The project option you choose (demo or tool)
  • Motivations of the project
  • Real-world applications
  • Data source and background (in detail)
  • Tools and programming languages used in the project
  • Contributions of the work (the work by the authors)
  • Novelty of the work (optional)
  • Techniques (need to present details)
  • Division of work for each team member
  • Challenges and solutions
  • Future work, extensions, improvements
  • Additional comments
  • References (including all papers, links, source codes, etc.)



The presentation slides, the final report and software package should be submitted to yunfu@buffalo.edu on time.


Last Update: 08-17-2010, Copyright 2004~2010, Yun Fu, All Rights Reserved