Human body parsing has a tremendous number of applications such as surveillance, human-machine interaction, and motion capture. It is also particularly challenging due to the large variations in shape and appearance of clothing, high degree of freedom from body articulations, and self-occlusions that are common in scenes containing people.

Our approach models the appearance of the human body as a rich grammar describing both syntactic and semantic relations between parts and their appearance. Due to the rich compositional nature of the grammar, our model can represent and detect fine-level detail of the body despite the large amount of articulation and variable part appearance due to clothing.

Ambiguity is present in most natural images and caused by phenomenon such as projection, occlusion, and background clutter, as well as imperfections in our models. Human observers tend to have little difficulty coping with ambiguity, however, and can often reason about multiple interpretations of a scene simultaneously. Motivated by this behavior, we expand traditional hierarchical part-based graphical models to explicitly assign interpretations to each part. Ambiguity can then be expressed as constraints on part interpretations. Inference is computed using a data-driven Markov chain Monte Carlo strategy using an extension of the Swendsen-Wang cluster sampling algorithm called C4. The advantage of this algorithm is that it can explore multiple solutions by jumping directly between local maxima in a single transition, much like how humans are able to switch between multiple ambiguous interpretations.

The light dome at UCLA was built as a generic platform to rapidly capture images of objects under multiple lighting conditions, inspired by the LightStage project from USC. The dome consists of 75 high intensity LED lighting units producing 240 lumens each. A custom designed controller board drives the intensity of each light independently, allowing arbitrary lighting and intensity patterns to be specified for each frame up to a maximum of 4000 fps. The current software for the dome synchronizes the lighting controller with the shutter control of any compliant IEEE1394/DV camera. The light dome is currently disassembled and in storage. If you are interested in using the dome for a project, please email me.