Presentation of IM-TWIN project
1) presentation of the project (year 1)
The following promotional video gives an overview of the IM-TWIN project. Meant for a general audience, it describes the main objectives and scope of the project.
2) project final technological outcomes (year 3)
The following promotional video gives an overview of the technological tools developed during the IM-TWIN project. In details the video describes 3 devices: (1) the Sensorised T-Shirt, (2) the Eye Contact Detector and (3) the Transitional Wearable Companion toys. The devices proved to be reliable tools to support the early intervention in Autism Spectrum Disorders and comparable Neurodevelopmental Disorders.
PlusMe: a Transitional Wearable Companion (TWC)
This video gives an overview of the functional features of the experimental PlusMe interactive toy. The device is currently in using and improving during the PlusMe and IM-TWIN projects. PlusMe was used for the first time as a potential therapy support tool to encourage social interaction behaviour (e.g., eye contact and imitation) in children diagnosed with high-functioning ASD.
The first results of the pilot are available at CHI Conference on Human Factors in Computing Systems (here); in addition, here is possible to see selected clips from the pilot.
Experimental sessions using Panda Plusme
The following video shows selected clips from a pilot experiment run on 3 children (age 36, 42, 50 months), diagnosed with high-functioning ASD. The test was run in May-June 2021, within the IM-TWIN related European project "PlusMe" (www.plusme-h2020.eu).
The playful activities involve a therapist and an experimenter (who is in the same room but never in the camera field of view). The therapist, exploiting the PlusMe toy, tries to stimulate the child's engagement in the social interaction. The PlusMe outputs (lights and sounds) are triggered both by the child (who touches the paws) and by the experimenter (who set the toy rewarding patterns) according to the child's requests. The experiment planned 4 sessions, one per week, lasting about 10 minutes.
Preliminary results, obtained with 9 participants, are described in the project deliverables D4.2 "Empirical validation: IM-TWIN".
Octopus X-8: a TWC to train turn-taking competence
The following 3 videos give a general overview of the interactive smart toy octopus X-8, a novel TWC designed to train turn-taking competence during shared play with children diagnosed with neurodevelopmental disorders.
1) Octopus X-8: functional features
The video describes the toy features, how it works and the 3 games currently implemented.
2) Octopus X-8: first pilot
The video presents selected clips from the pilot where X-8 was used with three children diagnosed with neurodevelopmental disorders. The pilot aims to analyse the effectiveness of the TWC in training turn-taking competence, thanks to the ability of the toy to distinguish between two users, producing rewarding sensory-motor feedback, when the turn-taking rule is respected.
3) Octopus X-8: data collection capabilities
The video presents a feasibility test, which shows how X-8 can collect a set of data, potentially useful to describe the social interaction between child and therapist. The device sensors activities, the type of game, the timing of rewards, etc. are saved in a log file, which can be analysed, to obtain a standardised and objective description of the experimental session.
4) Octopus X-8: platform feasibility test
The video presents a second feasibility test, where data collected by Octopus X-8 and the Eye Contact Detector tool are combined in a log which -- once analysed -- can provide an objective evaluation of social interaction between child and therapist. Purpose of the test is to check the consistency between the observed behaviour and the data collected by the tools. The test, lasting about 5 minutes and 30 seconds, involves a neurodevelopmental therapist and a child, aged 41 months, undergoing neuropsychological assessment.
(The video is part of the supporting material of the paper "A tangible, toy-based platform to evaluate the child's social interaction in turn-taking game: a prospective on monitoring neurodevelopmental disorders", to appear in the proceeding of the international conference Interaction Design and Children, IDC 2024).
Eye contact detector
The following video shows the "eye contact detector", a tool implemented through camera glasses (i.e. glasses embedding a micro camera) to detect the eye contact between two people. When there is eye contact, this event is indicated by a green box and the probability that the event has happened; When eye contact doesn't happen, the event is indicated by a red box.
The camera glasses are currently being tested on children diagnosed with neurodevelopmental disorders and with typical development.
The tool uses a computer vision algorithm, based on deep neural network, described in "Chong, E., Clark-Whitney, E., Southerland, A. et al. Detection of eye contact with deep neural networks is as accurate as human experts. Nat Commun 11, 6386 (2020), DOI: https://doi.org/10.1038/s41467-020-19712-x"
Eye Contact Detector: pilot test
This video shows a pilot test, where the Eye Contact Detector tool was used with a 38-month-old child, undergoing neuropsychological assessment, during play activity with a therapist and Panda PlusMe.
Linux App "Eye contact detector"
The following video shows the "eye contact detector" GUI. This tool let the researcher to upload a video and process it, to obtain the log of eye contact events.
Sensorised t-shirt
The following video shows an early test where the sensorised t-shirt, developed by the partner PLUX, was tested on a typically developed child, aged 30 months. Purpose of the test was to assess the quality of collected signals, during play activities. The t-shirt collects the following data: ECG, EDA, Body Temperature and Body Movements (through 3 accellerometers).
Sensorised T-Shirt, test with an ASD child
This video shows a reliability test where the Sensorised T-Shirt was used with a ASD Developed 36-month-old child, while playing with Panda PlusMe. As shown, the collected physiological data are stable even when the child moves a lot. An initial analysis showed how about 94% of data (over a period of 12 minutes) is characterised by high quality.
Echo: an AI-based toy for symbolic play
The video shows the features of Echo, a novel smart toy designed to support symbolic play. The video is part of the supplementary material for the paper "Echo: an AI-based toy to encourage symbolic play in children with Autism Spectrum Conditions", to appear in the proceedings of the ACM International Conference on Tangible, Embedded and Embodied Interaction - TEI '25 (04-07 March 2025, Bordeaux, France).