

{"id":4,"date":"2011-12-08T11:55:34","date_gmt":"2011-12-08T11:55:34","guid":{"rendered":"http:\/\/project.inria.fr\/template1\/?page_id=4"},"modified":"2022-09-19T13:31:01","modified_gmt":"2022-09-19T11:31:01","slug":"home","status":"publish","type":"page","link":"https:\/\/project.inria.fr\/toyotasmarthome\/","title":{"rendered":"Introduction"},"content":{"rendered":"<p><a href=\"https:\/\/project.inria.fr\/toyotasmarthome\/files\/2020\/12\/logo-2-e1608646898649.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-214 aligncenter\" src=\"https:\/\/project.inria.fr\/toyotasmarthome\/files\/2020\/12\/logo-2-1024x256.png\" alt=\"\" width=\"900\" height=\"225\" \/><\/a><\/p>\n<p>This page introduces\u00a0<strong>Toyota Smarthome<\/strong>\u00a0dataset. Smarthome has been recorded in an apartment equipped with 7 Kinect v1 cameras.\u00a0It contains the common daily living activities of 18 subjects. The subjects are senior people in the age range 60-80 years old.\u00a0The dataset has a resolution of 640\u00d7480 and offers <strong>3 modalities<\/strong>: <strong>RGB<\/strong> + <strong>Depth<\/strong> + <strong>3D Skeleton<\/strong>. The 3D skeleton joints were extracted from RGB.\u00a0For privacy-preserving reasons, the face of the subjects is blurred. Currently,\u00a0 two versions of the dataset are provided: Toyota Smarthome Trimmed and Toyota Smarthome Untrimmed.<\/p>\n<p><strong>Toyota Smarthome Trimmed<\/strong> has been designed for the activity classification task of 31 activities. The videos were clipped per activity, resulting in a total of 16,115 short RGB+D video samples. \u00a0activities were performed in a natural manner. As a result, the dataset poses a unique combination of challenges: high intra-class variation, high-class imbalance, and activities with similar motion and high duration variance. Activities were annotated with both coarse and fine-grained labels. These characteristics differentiate Toyota Smarthome Trimmed from other datasets for activity classification. [<a href=\"https:\/\/openaccess.thecvf.com\/content_ICCV_2019\/papers\/Das_Toyota_Smarthome_Real-World_Activities_of_Daily_Living_ICCV_2019_paper.pdf\">Paper Link<\/a>][<a href=\"https:\/\/openaccess.thecvf.com\/content_ICCV_2019\/supplemental\/Das_Toyota_Smarthome_Real-World_ICCV_2019_supplemental.pdf\">Supp<\/a>]<\/p>\n<p>The data loader and evaluation codes for <strong>Toyota Smarthome Trimmed<\/strong> can be found here:<\/p>\n<ul>\n<li><a href=\"https:\/\/github.com\/srijandas07\/i3d_smarthome\">RGB version<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/YangDi666\/2s-AGCN-For-Daily-Living\">Skeleton version<\/a>.<\/li>\n<\/ul>\n<p><strong>Toyota Smarthome Untrimmed (TSU)<\/strong> is targeting the activity detection task in long untrimmed videos.\u00a0Therefore, in TSU, we kept the entire recording when the person is visible. The dataset contains 536 videos with an average duration of 21 mins. Since this dataset is based on the same footage video as Toyota Smarthome Trimmed version, it features the same challenges and introduces additional ones. To better tackle the real-world challenges in the untrimmed video, we densely annotate the dataset with 51 activities.\u00a0[<a href=\"https:\/\/arxiv.org\/pdf\/2010.14982.pdf\">Paper Link<\/a>], [<a href=\"https:\/\/ieeexplore.ieee.org\/document\/9762531\">TPAMI version<\/a>]<\/p>\n<p>A <strong>demo<\/strong> of the proposed action detection method on the TSU dataset is available at this [<a href=\"https:\/\/drive.google.com\/file\/d\/115AXsub8L4Lbzt_DPfMILeHNFjFGK3Gl\/view?usp=sharing\">link<\/a>].<\/p>\n<p>The pre-extracted feature and working pipeline for <strong>Toyota Smarthome Untrimmed<\/strong> can be found below. The features are pre-trained on the training set of the data:<\/p>\n<ul>\n<li><a href=\"https:\/\/mybox.inria.fr\/f\/63c80895c71643f0abde\/\">RGB (I3D) feature<\/a><\/li>\n<li><a href=\"https:\/\/mybox.inria.fr\/f\/a46938b7995c46f28855\/\">3D Pose (AGCN) feature<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/dairui01\/Toyota_Smarthome\/tree\/main\/pipline\">Pipeline code (including the feature extractor)<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/dairui01\/TSU_evaluation\">Evaluation code<\/a><\/li>\n<\/ul>\n<p>The <strong>visualization<\/strong> for the\u00a0<strong>skeleton data<\/strong> on Smarthome is provided in this [<a href=\"https:\/\/github.com\/dairui01\/Toyota_Smarthome\/tree\/main\/Tools\/Pose_Visualization\">repo<\/a>].<\/p>\n<p>Both datasets are available on request. Please <em><strong>fill out the form below to obtain the<\/strong><\/em> <strong><em>original videos<\/em><\/strong>. For more details mail us at <em><strong>toyotasmarthome@inria.fr<\/strong><\/em><\/p>\n<h3>News<\/h3>\n<ul>\n<li>2019\/11\/01\u00a0 The <strong>Toyota Smarthome Trimmed\u00a0<\/strong>is accepted to ICCV&#8217;19. The data is available for request.<\/li>\n<li>2020\/12\/01\u00a0 The\u00a0<strong>Toyota Smarthome Untrimmed<\/strong> (TSU) dataset is released.\u00a0The data is available for request.<\/li>\n<li>2021\/01\/05\u00a0 We update the <strong>skeleton data (V1.2)<\/strong> for the <strong>Toyota\u00a0Smarthome Trimmed<\/strong>. The new skeleton is based on <a href=\"https:\/\/openaccess.thecvf.com\/content\/WACV2021\/papers\/Yang_Selective_Spatio-Temporal_Aggregation_Based_Pose_Refinement_System_Towards_Understanding_Human_WACV_2021_paper.pdf\">our Pose Refinement method<\/a>.<\/li>\n<li>2022\/04\/11\u00a0 The <strong>Toyota Smarthome Untrimmed<\/strong> (TSU) has been accepted to T-PAMI. Some new features for this dataset will be updated soon. Stay tuned!<\/li>\n<li>2022\/04\/24\u00a0 We released the pipeline codes including the\u00a0<strong>data loader<\/strong> and <strong>evaluation metrics\u00a0<\/strong>for Toyota Smarthome Trimmed: for RGB version (I3D):\u00a0[<a href=\"https:\/\/github.com\/srijandas07\/i3d_smarthome\">link<\/a>], for Skeleton version (2S-AGCN) [<a href=\"https:\/\/github.com\/YangDi666\/2s-AGCN-For-Daily-Living\">link<\/a>].<\/li>\n<li>2022\/05\/16\u00a0 We released the Toyota Smarthome Untrimmed&#8217;s\u00a0<strong>pre-extracted feature<\/strong> and the <strong>running <a href=\"https:\/\/github.com\/dairui01\/Toyota_Smarthome\/tree\/main\/pipline\">pipeline<\/a><\/strong>.<\/li>\n<li>2022\/07\/18\u00a0 To address the different research interests for the action detection task,\u00a0we released the <strong>Balanced<\/strong>\u00a0and <strong>Joint-view<\/strong> TSU in this <a href=\"https:\/\/github.com\/dairui01\/Toyota_Smarthome\/tree\/main\/Other_version\">repo<\/a>. By default, the TSU dataset is unbalanced and unsynchronized.<\/li>\n<li>2022\/09\/01 We have released a <a href=\"https:\/\/github.com\/dairui01\/Toyota_Smarthome\/tree\/main\/Tools\/Pose_Visualization\">repo<\/a> for the <strong>pose visualization<\/strong> on the Smarthome dataset.<\/li>\n<li>2022\/09\/19 We have released the I3D <strong>feature extractor<\/strong> along with the pre-trained model for TSU (<a href=\"https:\/\/github.com\/dairui01\/Toyota_Smarthome\/tree\/main\/pipline\/I3D\">link<\/a>)<\/li>\n<\/ul>\n<h3>Samples of Activities in Toyota Smarthome Trimmed<\/h3>\n<p><iframe loading=\"lazy\" title=\"smarthome activities\" width=\"900\" height=\"506\" src=\"https:\/\/www.youtube.com\/embed\/tLHzf8f30QM?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen><\/iframe><\/p>\n<h3>Sample of Activities and Videos in Toyota Smarthome Untrimmed<\/h3>\n<p><iframe loading=\"lazy\" title=\"[TPAMI&#039;22] Toyota Smarthome Untrimmed (TSU) dataset\" width=\"900\" height=\"675\" src=\"https:\/\/www.youtube.com\/embed\/ptJ67dgMxN0?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen><\/iframe><\/p>\n<h3><\/h3>\n<h3>License<\/h3>\n<p>The dataset is provided for academic research only. The full license can be found <a href=\"https:\/\/project.inria.fr\/toyotasmarthome\/files\/2020\/12\/License_v2.pdf\" target=\"_blank\" rel=\"noopener\">here<\/a>. Please read carefully the terms and conditions of the license and any accompanying documentation before you download and\/or use the Toyota Smarthome dataset. By downloading and\/or using the Data, you acknowledge that you have read these terms and conditions, understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not download and\/or use the Software. Any infringement of the terms of this agreement will automatically terminate your rights under this License.<\/p>\n<h3>Toyota Smathome Data Request<\/h3>\n<div class=\"frm_forms  with_frm_style frm_style_formidable-style\" id=\"frm_form_8_container\" >\n<form enctype=\"multipart\/form-data\" method=\"post\" class=\"frm-show-form \" id=\"form_requesttoyotasmarthomedataset\" >\n<div class=\"frm_form_fields \">\n<fieldset>\n<legend class=\"frm_screen_reader\">Toyota Smathome Data Request Form<\/legend>\r\n\r\n<div class=\"frm_description\"><p>Please fill-up the form below to download the dataset. By clicking the submit button you acknowledge that you have read the license (<a href=\"https:\/\/project.inria.fr\/toyotasmarthome\/files\/2020\/12\/License_v2.pdf\">link<\/a>), understand it and agree to be bound by it. To ensure academic usage, please fill in your <strong>work email address<\/strong>, the request from the personal email address will be ignored (e.g. gmail). If your work\/school email address is using Gmail, please provide your supervisor's name and email in the project description. Before your submission, <strong> please double check your email address<\/strong> !!  If you do not receive the data download link or feedback in 3 days, please contact <a href=\"mailto:toyotasmarthome@inria.fr\">toyotasmarthome@inria.fr<\/a> or fill in this <a href=\"https:\/\/docs.google.com\/forms\/d\/e\/1FAIpQLSdl-wH5xMitp6Nh88Ieu3gRcpuFSB_Ja_IgCZ2gGdf8Bn0Kjw\/viewform?usp=sf_link\">google form<\/a>. <\/p>\n<\/div><div class=\"frm_fields_container\">\n<input type=\"hidden\" name=\"frm_action\" value=\"create\" \/>\n<input type=\"hidden\" name=\"form_id\" value=\"8\" \/>\n<input type=\"hidden\" name=\"frm_hide_fields_8\" id=\"frm_hide_fields_8\" value=\"\" \/>\n<input type=\"hidden\" name=\"form_key\" value=\"requesttoyotasmarthomedataset\" \/>\n<input type=\"hidden\" name=\"item_meta[0]\" value=\"\" \/>\n<input type=\"hidden\" id=\"frm_submit_entry_8\" name=\"frm_submit_entry_8\" value=\"eaefe811dd\" \/><input type=\"hidden\" name=\"_wp_http_referer\" value=\"\/toyotasmarthome\/wp-json\/wp\/v2\/pages\/4\" \/><div id=\"frm_field_32_container\" class=\"frm_form_field form-field  frm_required_field frm_top_container frm_first frm_half\">\r\n    <label for=\"field_jcny8\" id=\"field_jcny8_label\" class=\"frm_primary_label\">First Name\r\n        <span class=\"frm_required\">*<\/span>\r\n    <\/label>\r\n    <input type=\"text\" id=\"field_jcny8\" name=\"item_meta[32]\" value=\"\"  data-reqmsg=\"First Name cannot be blank.\" aria-required=\"true\" data-invmsg=\"Text is invalid\" aria-invalid=\"false\"  \/>\r\n    \r\n    \r\n<\/div>\n<div id=\"frm_field_22_container\" class=\"frm_form_field form-field  frm_required_field frm_top_container frm_alignright frm_half\">\r\n    <label for=\"field_3zj0p\" id=\"field_3zj0p_label\" class=\"frm_primary_label\">Last Name\r\n        <span class=\"frm_required\">*<\/span>\r\n    <\/label>\r\n    <input type=\"text\" id=\"field_3zj0p\" name=\"item_meta[22]\" value=\"\"  data-reqmsg=\"Last Name cannot be blank.\" aria-required=\"true\" data-invmsg=\"Text is invalid\" aria-invalid=\"false\"  \/>\r\n    \r\n    \r\n<\/div>\n<div id=\"frm_field_25_container\" class=\"frm_form_field form-field  frm_required_field frm_top_container\">\r\n    <label for=\"field_ojs5z\" id=\"field_ojs5z_label\" class=\"frm_primary_label\">Email address (Please use your work email address, e.g., inria.fr)\r\n        <span class=\"frm_required\">*<\/span>\r\n    <\/label>\r\n    <input type=\"email\" id=\"field_ojs5z\" name=\"item_meta[25]\" value=\"\"  data-reqmsg=\"Email address (Please use your work email address, e.g., inria.fr) cannot be blank.\" aria-required=\"true\" data-invmsg=\"Please enter a valid email address.\" aria-invalid=\"false\"  \/>\r\n    \r\n    \r\n<\/div>\n<div id=\"frm_field_23_container\" class=\"frm_form_field form-field  frm_required_field frm_top_container frm_full\">\r\n    <label for=\"field_gmuil\" id=\"field_gmuil_label\" class=\"frm_primary_label\">Affiliation (Your Institute\/Company\/University Name)\r\n        <span class=\"frm_required\">*<\/span>\r\n    <\/label>\r\n    <input type=\"text\" id=\"field_gmuil\" name=\"item_meta[23]\" value=\"\"  data-reqmsg=\"Affiliation cannot be blank.\" aria-required=\"true\" data-invmsg=\"Text is invalid\" aria-invalid=\"false\"  \/>\r\n    \r\n    \r\n<\/div>\n<div id=\"frm_field_21_container\" class=\"frm_form_field form-field  frm_required_field frm_top_container frm_full\">\r\n    <label for=\"field_yvzmz\" id=\"field_yvzmz_label\" class=\"frm_primary_label\">Request dataset type\r\n        <span class=\"frm_required\">*<\/span>\r\n    <\/label>\r\n    \t\t<select name=\"item_meta[21]\" id=\"field_yvzmz\"  data-reqmsg=\"Request dataset type cannot be blank.\" aria-required=\"true\" data-invmsg=\"Request dataset type is invalid\" class=\"auto_width\" aria-invalid=\"false\"  >\n\t\t<option  value=\"\" selected='selected'> <\/option><option  value=\"Toyota Smarthome Trimmed\">Toyota Smarthome Trimmed<\/option><option  value=\"Toyota Smarthome Untrimmed\">Toyota Smarthome Untrimmed<\/option><option  value=\"Both\">Both<\/option>\t<\/select>\n\t\r\n    \r\n    \r\n<\/div>\n<div id=\"frm_field_34_container\" class=\"frm_form_field form-field  frm_required_field frm_top_container\">\r\n    <label for=\"field_jgj0r\" id=\"field_jgj0r_label\" class=\"frm_primary_label\">Short description of your Project\r\n        <span class=\"frm_required\">*<\/span>\r\n    <\/label>\r\n    <textarea name=\"item_meta[34]\" id=\"field_jgj0r\" rows=\"5\"  data-reqmsg=\"Short description of your Project cannot be blank.\" aria-required=\"true\" data-invmsg=\"Short description of your Project is invalid\" aria-invalid=\"false\"  ><\/textarea>\r\n    \r\n    \r\n<\/div>\n<div id=\"frm_field_26_container\" class=\"frm_form_field form-field  frm_required_field frm_top_container vertical_radio\">\r\n    <div  id=\"field_tlr5t_label\" class=\"frm_primary_label\">Agreement\r\n        <span class=\"frm_required\">*<\/span>\r\n    <\/div>\r\n    <div class=\"frm_opt_container\" aria-labelledby=\"field_tlr5t_label\" role=\"group\">\t\t<div class=\"frm_checkbox\" id=\"frm_checkbox_26-0\">\t\t\t<label  for=\"field_tlr5t-0\">\n\t\t\t<input type=\"checkbox\" name=\"item_meta[26][]\" id=\"field_tlr5t-0\" value=\"I have read the license, understand it, and agree to be bound by it.\"  data-reqmsg=\"Agreement cannot be blank.\" data-invmsg=\"Agreement is invalid\"   aria-required=\"true\"  \/> I have read the license, understand it, and agree to be bound by it.<\/label><\/div>\n<\/div>\r\n    \r\n    \r\n<\/div>\n<div id=\"frm_field_37_container\" class=\"frm_form_field form-field  frm_none_container\"><div  id=\"field_624si\" class=\"frm-g-recaptcha\" data-sitekey=\"6LcRPE8aAAAAABupcfCKlOb6K2UcfEMu9IM_Iicf\" data-size=\"normal\" data-theme=\"light\"><\/div>\r\n    \r\n    \r\n<\/div>\n\t<input type=\"hidden\" name=\"item_key\" value=\"\" \/>\n\t\t\t<div id=\"frm_field_38_container\">\n\t\t\t<label for=\"field_xbq6r\" >\n\t\t\t\tIf you are human, leave this field blank.\t\t\t<\/label>\n\t\t\t<input  id=\"field_xbq6r\" type=\"text\" class=\"frm_form_field form-field frm_verify\" name=\"item_meta[38]\" value=\"\"  \/>\n\t\t<\/div>\n\t\t<input name=\"frm_state\" type=\"hidden\" value=\"eX6rx8JhhnKEX8f96UlvOjh81LiPy3PRaWQdcjVQBDk=\" \/><div class=\"frm_submit\">\r\n\r\n<button class=\"frm_button_submit\" type=\"submit\"  >Submit<\/button>\r\n\r\n<\/div><\/div>\n<\/fieldset>\n<\/div>\n\n<p style=\"display: none !important;\" class=\"akismet-fields-container\" data-prefix=\"ak_\"><label>&#916;<textarea name=\"ak_hp_textarea\" cols=\"45\" rows=\"8\" maxlength=\"100\"><\/textarea><\/label><input type=\"hidden\" id=\"ak_js_1\" name=\"ak_js\" value=\"247\"\/><script>\ndocument.getElementById( \"ak_js_1\" ).setAttribute( \"value\", ( new Date() ).getTime() );\n<\/script>\n<\/p><\/form>\n<\/div>\n\n<p>*This dataset complies with <strong>GDPR European Regulation<\/strong>.<\/p>","protected":false},"excerpt":{"rendered":"<p>This page introduces\u00a0Toyota Smarthome\u00a0dataset. Smarthome has been recorded in an apartment equipped with 7 Kinect v1 cameras.\u00a0It contains the common daily living activities of 18 subjects. The subjects are senior people in the age range 60-80 years old.\u00a0The dataset has a resolution of 640\u00d7480 and offers 3 modalities: RGB +\u2026<\/p>\n<p> <a class=\"continue-reading-link\" href=\"https:\/\/project.inria.fr\/toyotasmarthome\/\"><span>Continue reading<\/span><i class=\"crycon-right-dir\"><\/i><\/a> <\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"open","template":"","meta":{"footnotes":"","_members_access_role":[],"_members_access_error":""},"class_list":["post-4","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/project.inria.fr\/toyotasmarthome\/wp-json\/wp\/v2\/pages\/4","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/project.inria.fr\/toyotasmarthome\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/project.inria.fr\/toyotasmarthome\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/project.inria.fr\/toyotasmarthome\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/project.inria.fr\/toyotasmarthome\/wp-json\/wp\/v2\/comments?post=4"}],"version-history":[{"count":50,"href":"https:\/\/project.inria.fr\/toyotasmarthome\/wp-json\/wp\/v2\/pages\/4\/revisions"}],"predecessor-version":[{"id":327,"href":"https:\/\/project.inria.fr\/toyotasmarthome\/wp-json\/wp\/v2\/pages\/4\/revisions\/327"}],"wp:attachment":[{"href":"https:\/\/project.inria.fr\/toyotasmarthome\/wp-json\/wp\/v2\/media?parent=4"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}