title: Developing an eBook-Integrated High-Fidelity Mobile App Prototype for Promoting Child Motor Skills and Taxonomically Assessing Children's Emotional Responses Using Face and Sound Topology.
abstract: Developing gross and fine motor skills and expressing complex emotion is critical for child development. We introduce "StorySense", an eBook-integrated mobile app prototype that can sense face and sound topologies and identify movement and expression to promote children's motor skills and emotional developmental. Currently, most interactive eBooks on mobile devices only leverage "low-motor" interaction (i.e. tapping or swiping). Our app senses a greater breath of motion (e.g. clapping, snapping, and face tracking), and dynamically alters the storyline according to physical responses in ways that encourage the performance of predetermined motor skills ideal for a child's gross and fine motor development. In addition, our app can capture changes in facial topology, which can later be mapped using the Facial Action Coding System (FACS) for later interpretation of emotion. StorySense expands the human computer interaction vocabulary for mobile devices. Potential clinical applications include child development, physical therapy, and autism.
authors.affiliation: Department of Biomedical Informatics, Columbia University, New York, NY ; HIV Center for Clinical and Behavioral Studies, New York Psychiatric Institute and Columbia University, New York, NY.
authors.forename: William
authors.initials: W
authors.lastname: Brown
authors.affiliation: Department of Biomedical Informatics, Columbia University, New York, NY.
authors.forename: Connie
authors.initials: C
authors.lastname: Liu
authors.affiliation: School of Nursing, Columbia University, New York, NY.
authors.forename: Rita Marie
authors.initials: RM
authors.lastname: John
authors.affiliation: School of International and Public Affairs, Columbia University, New York, NY.
authors.forename: Phoebe
authors.initials: P
authors.lastname: Ford
mesh_terms: D056667:Biometric Identification; D002648:Child; D002657:Child Development; D004644:Emotions; D005145:Face; D006801:Humans; D063731:Mobile Applications; D009048:Motor Skills; D011381:Programming Languages