Overview

IM-TWIN in a nutshell

IM-TWIN is an interdisciplinary project (involving computer scientists, experimental psychologists, designers, neurodevelopmental psychiatrists and therapists), which aims to develop new interactive technologies to support the early diagnosis and the early therapy of children with Autism Spectrum Disorders (or ASD) and other neurodevelopmental conditions, characterised by impairments in the social, emotional and communicative areas.

IM-TWIN genesis

IM-TWIN acronym stands for "from Intrinsic Motivations to Transitional Wearable INtelligent companions for autism spectrum disorder". What is a Transitional Wearable Companion - TWC? This is a new concept of soft interactive toy (Özcan et al. 2016), designed for young children, with three important features:

  • interactivity: it has to provide several, different, rewarding outputs (as coloured lights, sounds, vibrations) in response to child's behaviour, so to stimulate the curiosity and the engagement;
  • sociality: its functioning has to promote an easy set-up of social games and activities involving other people (e.g. turn-taking games, imitation activities);
  • transitional properties: its design has to arouse emotional attachment and reassuring feelings, as a security blanket.

Figure 1: PlusMe first prototype and the control application.

  The first experimental prototype of TWC is called PlusMe: this is a soft pillow looking like a panda, which can be worn around the neck (Fig. 1). Thanks to inner electronics, it can emit coloured lights and amusing sounds when touched on the paws. Such outputs are very gratifying for small children, as they stimulate curiosity and drive to spontaneous exploration (i.e. they leverage on Intrinsic Motivations - IM). Hues and type of sounds are established by a wirelessly-connected control tablet, so that the same toy can produce several, different rewarding sensory patterns. Thanks to these features, the toy could be used to set-up playful social activities, where the child (who handles the panda and triggers the sensorial outcomes), has to cooperate with the adult caregiver (who handles the control tablet and mediates the toy responses), to obtain the desired PlusMe output.

Figure 2: an example of "imitation game" where a typical developed children mimics the researcher's behaviour.

PlusMe was designed in collaboration with neurodevelopmental therapists as a potential tool to support the early treatment of children diagnosed with ASD. PlusMe functional features could in fact be exploited to ecouncorage the development and reinforcement of the basic social competences (e.g. imitation, joint attention, eye-contact) which are critical for the cognitive development of children. The prototype has been used in two promising pilot experiments with ASD and Typical Developed (TD) children (Fig. 2), aged between 24-48 months (Sperati et al. 2019, 2020).

 

IM-TWIN main goals

IM-TWIN project aims to build, around the TWC original concept, a technological system to support the work of neurodevelopmental therapists and neuropsychiatrists, involved in the diagnosis and treatments of all neurodevelopmental conditions characterised by impairments of emotional, social, communicative areas, with particular reference to ASD. Such system has an ambitious main goal: to "understand" (through Artificial Intelligence, Machine Learning algorithms, innovative devices and wearable sensors) the affective state of children during the therapeutic activity (Fig. 3). Affective information is often unclear or "hidden" in ASD children; this knowledge would then be extremely helpful for therapists, to adapt and tune the activities and make them more effective in improving the social competence of children.

Figure 3: the physiological signals, recorded through wearable sensors, will be categorised by an Artificial Intelligence into affective states.

In more detail, the IM-TWIN system will be formed by the following integrated components:

  • wearable sensors, embedded in sensorised garments suitable for small children (e.g. t-shirts or bracelets), to detect and record physiological signals, as heart rate and electrodermal activity, related to the affective states;
  • the latest algorithms, based on Artificial Intelligence and Machine Learning, to decode the physiological data into clear affective states as positive and negative engagement;
  • interactive “intelligent” TWC toys, to be used as support tools during the therapeutic activities, to stimulate the children curiosity and motivate them to naturally engage in social interactions.

The figure 4 shows how these components could be used in a possible scenario, where the therapist and the child play together: through the information on the child's affective state provided by the tablet, the therapist could adapt the PlusMe sensory feedback, and choose the most rewarding and gratifying outputs to maintain a positive engagement.

Figure 4: an overview of the IM-TWIN integrated system. The components are: an improved version of PlusMe interactive toy; physiological sensors integrated in a wearable garment (as a bracelet or a t-shirt); ambient cameras to detect salient social events. All incoming data will be analised through Artificial Intelligence algorithms to "disclose" the child's affective state during the therapeutic activity. 

An Interdisciplinary project

IM-TWIN project is carried out by an highly interdisciplinary consortium, combining different expertise in distinct research fields:

  • the National Research Council of Italy, through the Institute of Cognitive Sciences and Technologies, expert in autonomous robotics and Artificial Intelligence;
  • the Center for Research and Interdisciplinarity, expert in developmental psychology and in the processes of learning in children;
  • the University of Rome Sapienza, through the dep. of Human Neuroscience, expert in the diagnosis and treatment of neurodevelopmental disorders;
  • Plux Wireless Biosignals S.A., a company specialising in the development of innovative products for healthcare and research, as advanced biosignal monitoring platforms and wearable body sensors;
  • the Utrecht University, through the dep. of Information and Computing Sciences, expert in the processing of physiological signals into affective states by Machine Learning;
  • the Quantum Leap IP, a third party company providing support in both the technology trasfer and the management of intellectual property rights.

Related projects

Goal-Robots: Goal-based Open-ended Autonomous Learning Robots
IM-TWIN originates as a technological extension of GOAL-Robots project (www.goal-robots.eu). This project investigated the importance of Intrinsic Motivations (IM) in the cognitive development of both human and artificial agents. In brief, IMs are mechanisms related to the interest for novel objects, surprising events, and success for achieving desired outcomes that trigger behaviours and support learning processes in the absence of direct biological pressure. IMs are maximally apparent in children engaged in playful activities. PlusMe first prototype was developed within GOAL-Robot, to study the learning of sensory-motor contingencies driven by IMs in typical developed toddlers (further information on the relationship between the project and the device are available here).

PlusMe: Transitional Wearable Companions for the therapy of children with Autism Spectrum Disorders
This project focuses on the technological improvement of PlusMe device. In particular it aims to develop the current prototype, so to make it closer to a ready-for sale product, available for therapists and researchers (further information can be found on the dedicated webpage www.plusme-h2020.eu).