Introduction

This page introduces Toyota Smarthome dataset. Smarthome has been recorded in an apartment equipped with 7 Kinect v1 cameras. It contains common daily living activities of 18 subjects. The subjects are senior people in the age range 60-80 years old. The dataset has a resolution of 640×480 and offers 3 modalities: RGB + Depth + 3D Skeleton. The 3D skeleton joints were extracted from RGB. For privacy-preserving reasons, the face of the subjects is blurred. Currently,  two versions of the dataset are provided: Toyota Smarthome Trimmed and Toyota Smarthome Untrimmed.

Toyota Smarthome Trimmed has been designed for the activity classification task of 31 activities. The videos were clipped per activity, resulting in a total of 16,115 short RGB+D video samples.  activities were performed in a natural manner. As a result, the dataset poses a unique combination of challenges: high intra-class variation, high-class imbalance, and activities with similar motion and high duration variance. Activities were annotated with both coarse and fine-grained labels. These characteristics differentiate Toyota Smarthome Trimmed from other datasets for activity classification. [Paper Link][Supp]

Toyota Smarthome Untrimmed (TSU) is targeting the activity detection task in long untrimmed videos. Therefore, in TSU, we kept the entire recording when the person is visible. The dataset contains 536 videos with an average duration of 21 mins. Since this dataset is based on the same footage video as Toyota Smarthome Trimmed version, it features the same challenges and introduces additional ones. To better tackle the real-world challenges in the untrimmed video, we densely annotate the dataset with 51 activities. [Paper Link]

Both datasets are available on request. Please fill the form below. For more details mail us at toyotasmarthome@inria.fr

News

  • 2019/11/01 The Toyota Smarthome Trimmed is accepted in ICCV’19. The data is available for request.
  • 2020/12/01 The Toyota Smarthome Untrimmed (TSU) dataset is released. The data is available for request.
  • 2021/01/05 We update the skeleton data (V1.2) for the Toyota Smarthome Trimmed. The new skeleton is based on our Pose Refinement method.

Samples of Activities in Toyota Smarthome Trimmed

Sample of Activities and Videos in Toyota Smarthome Untrimmed

License

The dataset is provided for academic research only. The full license can be found here. Please read carefully the terms and conditions of the license and any accompanying documentation before you download and/or use the Toyota Smarthome dataset. By downloading and/or using the Data, you acknowledge that you have read these terms and conditions, understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not download and/or use the Software. Any infringement of the terms of this agreement will automatically terminate your rights under this License.

Toyota Smathome Data Request

Toyota Smathome Data Request Form

Please fill-up the form below to download the dataset. By clicking the submit button you acknowledge that you have read the license (link), understand it, and agree to be bound by it. Before your submission, please double check your email address !!

Agreement *

*This dataset complies with GDPR European Regulation.

Comments are closed.

  • Bibtex of Toyota Smarthome Trimmed Dataset:

    @InProceedings{Das_2019_ICCV,
        author = {Das, Srijan and Dai, Rui and Koperski, Michal and Minciullo, Luca and Garattoni, Lorenzo and Bremond, Francois and Francesca, Gianpiero},
        title = {Toyota Smarthome: Real-World Activities of Daily Living},
        booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
        month = {October},
        year = {2019}
    }
  • Bibtex of Toyota Smarthome Untrimmed Dataset:

    @misc{dai2020toyota,
        author = {Dai, Rui and Das, Srijan and Sharma, Saurav and Minciullo, Luca and Garattoni, Lorenzo and Bremond, Francois and Francesca, Gianpiero},
        title = {Toyota Smarthome Untrimmed: Real-World Untrimmed Videos for Activity Detection}, 
        year = {2020}, 
        eprint = {2010.14982}, 
        archivePrefix = {arXiv}, 
        primaryClass = {cs.CV}
    }