Real time shape recognition from video feed

Just to keep myself occupied during some "empty" days between courses, I decided to implement a real time shape recognition system in Matlab. Most of the ideas are based on "Shape Recognition and Pose Estimation for Mobile Augmented Reality" by Nate Hagbi et al. However I didn't implement the pose estimation part since I run out of time. Moreover I encountered some difficulties regarding easier parts of the system (oh yeah image retrieval is not my field per se).

   So the idea behind it is simple: locate the most prominent shape (through filtering such as adaptive thresholding), compute its convex hull and perform matching via a bag-of-words approach. Therefore the system firstly needs to be trained. I use SIFT features on a series of projective transformations of the original shape, similar to "Visual Similarity based 3D Shape Retrieval Using Bag-of-Features" by Zhouhui Lian et al. A demonstration is presented below:

Real time shape recognition using Matlab from Jim B. on Vimeo.

You can easily observe some errors; for example whenever the system identifies that a shape exists but doesn't correspond to any of the stored figures, then it matches it to shape "3". Moreover, in order to make the system faster, I forced it to perform matching only when the image difference between frames is higher than a threshold.This leads to no processing whenever quick movements occur. Finally since shape extraction is based on the contrast of an image, then shadows are often misclassified as shapes.
    Anyway, I will upload the code for this sometime in the near future. It's mostly a concatenation of other people's ideas and code but it does the trick. It's also a good starting point for further development.