The SFB588 Basic Kitchen Tasks Dataset


grate mash pour chop stir
sweep mill slice roll saw


The Basic Kitchen Tasks dataset consists of 10 action sequences with a total of 48 action units. Each action sequence has been recorded 20-30 times resulting in an overall of 250 action sequence samples and over 6000 action unit samples.The video data is captured with 30 fps and a resolution of 640x480 px with a Prosilica GE680C camera.

Parallel to the video data acquisition, five action sequences were recorded with a marker based motion capture system (Vicon). Each sequence was repeated 20 times, 100 samples with over 2400 action units were recorded. Reflective markers were attached to the test persons upper body and mapped onto a kinematic model to calculated the related joint angle trajectories of the test persons motions. The system output s a feature vector of 24 joint angles, the pose of an upper body model. For the recognition deltas of joint angles are used as input vectors. Further description of the data as well as evaluation details can be found in our papers on Visapp 2012 [1], KI 2010 [2] and Humanoids 2009 [3]. The upper body joint angel estimation is based on the Master Motor Map - Human Model. Details about the model can be found in [4], see pp. 11-14 for the complete model description. For any questions or comments please contact Hilde Kuehne ( and Dirk Gehrig (



If you use this data set in your publications, please refer to our Visapp 2012 paper [1] or to our Humanoids paper [3].

  • Video data (~ 216 MB, Actions: chop, grate, mash, mill, pour, roll_out, saw, slice, stir, sweep)
  • Vicon data (~ 73 MB, Actions: chop, grate, mash, pour, stir)
  • Joint angle data (~ 9 MB, Actions: chop, grate, mash, pour, stir)



H. Kuehne, D. Gehrig, T. Schultz, R. Stiefelhagen.
On-line Action Recognition from sparse Feature Flow
International Conference on Computer Vision Theory and Applications 2012 (pdf) (bibtex)


Dirk Gehrig, Thorsten Stein, Andreas Fischer, Hermann Schwameder, Tanja Schultz
Towards Semantic Segmentation of Human Motion Sequences
33rd Annual German Conference on Artificial Intelligence 2010, KI 2010, Karlsruhe, Germany (pdf) (bibtex)


D. Gehrig, H. Kuehne, A. Woerner, T. Schultz.
HMM-based Human Motion Recognition with Optical Flow Data.
Humanoids 2009, Humanoids 2009, Paris, France, 09. December 2009 (pdf)(bibtex)


C. Simonidis S. Gärtner, M. Do.
Spezifikationen zu den Ganzkörpermenschmodellen im SFB 588
Technical Report, KIT 2009 (pdf)