]Everyday experience is multi-sensory, and user experience (UX) design aims to extend this to interactions with products, services, and designed worlds. However, tools and pedagogies for UX are overwhelmingly visual, whereas human-rights-based accessibility legislation mandates the inclusion of diverse peoples, including blind and partially sighted individuals. Coupling auditory and haptic UX techniques from human-computer interaction with industrial design’s (ID) cross-modal tradition of prototyping physical products fostered our novel cross-modal UX course for second-year ID undergraduates.
Projects
Our Research
Explore this selection of recent projects from the past decade.
A cross-modal UX design pedagogy for industrial design
- Zainab Husain, University of Toronto
- Lucas Temor, University of Toronto
- Peter Coppin, OCAD U
Narcissus and Echo: Reflections on an Art-Science Collaboration
Narcissus and Echo: Reflections on an Art-Science Collaboration
- David Steinman, University of Toronto
- Peter Coppin, OCAD U
More than a decade ago, the authors proposed establishing a basis for scientific exploration of blood-flow dynamics intertwined with the visual arts. Here they present a case study showing how paradigms they codeveloped for visually abstracting cerebral aneurysm blood flows were extrapolated to sonification and bimodal representations, and how a close interdisciplinary partnership was effected by guiding engineering students versed in the arts and artists adept with digital technology toward final outcomes greater than the sum of their parts.
Tangible
Construction Kit for Blind and Partially Sighted Drawers
Tangible Construction Kit for Blind and Partially Sighted Drawers
- Mitali Kamat, MDes, OCAD U
- Alvaro Quevedo, PhD, Ontario Tech
- Peter Coppin, MFA, PhD, OCAD U
Drawing as an activity aids problem solving, collaboration, and presentation in design, science, and engineering and artistic creativity as well as expression in the arts. Unfortunately, blind, and partially sighted learners still lack an inclusive and effective drawing tool, even in the digital age. In response, this research aims to explore what an effective drawing tool for blind and partially sighted individuals (BPSI) would be.
A
Diagram Must Never Be Ten Thousand Words
A Diagram Must Never Be Ten Thousand Words
- David Barter, Mdes, OCAD U
- Peter Coppin, MFA, PhD OCAD U
Web Content Accessibility Guidelines (WCAG) require digital diagrams to be tagged with screen readable text descriptions for access by blind and partially sighted individuals (BPSI). The aim of these guidelines is to comply with human rights-based accessibility legislation, which aims to preserve the normative agency of BPSI (the ability to reflect on, evaluate and act upon a conception of what constitutes a worthwhile life for themselves). However, theories from the Diagrams community suggest that text and diagrams offer distinctly different constraints. For example, Shimojima’s Constraint Hypothesis (the relationship of structural constraints to target problem constraints in a mode of representation establish the variance of inferential potential) and the interrelated free ride phenomenon (additional inferences can be made in a representation if the relationship of constraints is a good match). Therefore, a guideline that requires the text description of a diagram (such as via the WCAG) might limit the normative agency diagram users who are BPSI. Despite the apparent necessity of providing non-visual alternatives of diagrammatic properties for accessibility, they are rarely explored or developed sufficiently to be consistently provided to BPSI. Thus, we argue that the affordances of diagrammatic representations provide possibilities for normative agency that are lost if not represented non-visually in diagrams designed for accessibility.
Shared Intentionality in Hyflex Education
- Tamara Crasto, Mdes, OCAD U
- Peter Coppin, MFA, PhD OCAD U
Inclusive education usually focuses on including a diverse range of students while neglecting to focus on an equally important stakeholder: instructors with disabilities. Additionally, instructors with disabilities are rarely represented in inclusive education research. This longitudinal participatory study documents diverse instructors’ lived experiences in remote and hyflex education, during, transitioning and “after” the Covid-19 pandemic. Hyflex education provides the flexibility to choose between virtual or face-to-face experiences or remote and collated interactions. This approach grew during the transition out of the COVID-19 pandemic lockdown to harness positive affordances of different modalities to increase inclusivity and accessibility. Current practices and three models of hyflex execution are documented in this report. Hyflex interaction comes with challenges communicating, coordinating and collaborating across environments. Codesigned interventions addressing these challenges are presented in this report. The effectiveness of coordination and collaboration can be understood through Tomesello’s concept of shared intentionality, which is when people have joint attention and intention during interactions. A developed model mapping shared intentionality, through (inter)action and information flow in hyflex environments, is presented. The implications of an abundance or scarcity of information and action within this model is discussed as (the coined term) shared intentionality black holes.
Fostering shared intentionality for diverse learners through cross-sensory interaction design
- David Barter, MDes, OCAD U
- Tamara Crasto, Mdes, OCAD U
- Erin Lee, Mdes, OCAD U
- Robert Ingino, Mdes, OCAD U
- Peter Coppin, MFA, PhD OCAD U
As the theme of this year’s conference suggests, cognitive diversity among learners and educators is increasingly acknowledged. However, in our societies that increasingly require advanced education, training, and technical skills, the pressure to standardize learning objectives, delivery techniques and delivery tools, especially online, is high. In these situations, learners and educators of diverse cognitive phenotypes and abilities experience learning environments that are a poor match for their abilities, making effective delivery of educational content challenging. However, with such vast human variation, many learners and educators are experiencing benefits as well as challenges in online settings, accelerated by the COVID-19 pandemic. For example, working remotely provides neurodiverse individuals with greater control over their environments, in terms of noise, light, potential distractions and comfortable seating. (Das et al., 2021). In contrast, structured routines (e.g., commuting to class) that aid executive functioning are often lost (St. Amour, 2020). Neurodiverse learners may benefit from this new paradigm if their accessibility challenges can be met.
Design and evaluation of an audio game-inspired auditory map interface
Design and evaluation of an audio game-inspired auditory map interface
- Brandon Biggs, MDes, OCAD U
- James Coughlin, PhD, Smith-Kettlewell Eye Research Institute
- Peter Coppin, MFA, PhD OCAD U
Access to non-visual maps has long required special equipment and training to use; Google Maps, ESRI, and other commonly used digital maps are completely visual and thus inaccessible to people with visual impairments. This project presents the design and evaluation of an easy to use digital auditory map and 3D model interactive map. A co-design was also undertaken to discover tools for an ideal nonvisual navigational experience. Baseline results of both studies are presented so future work can improve on the designs. The user evaluation revealed that both prototypes were moderately easy to use. An ideal nonvisual navigational experience, according to these participants, consists of both an accurate turn by turn navigational system, and an interactive map. Future work needs to focus on the development of appropriate tools to enable this ideal experience.
Perceptually Motivated Sonification of Spatiotemporally-Dynamic CFD Data
Everyday perception and action are fundamentally mult-isensory. Despite this, the sole reliance on visualization for the representation of complex 3D spatiotemporal data is still widespread. In the past we have proposed various prototypes for the sonification of dense data from computational fluid dynamics (CFD) simulations of turbulent-like blood flow, but did not robustly consider the perception and associated meaning making of the resultant sounds. To reduce some of the complexities of these data for sonification, in this work we present a feature-based approach, applying ideas from auditory scene analysis to sonify different data features along perceptually-separable auditory streams. As there are many possible features in these dense data, we followed the analogy of “caricature” to guide our definition and subsequent amplification of unique spectral and fluctuating features, while effectively minimizing the features common between simulations. This approach may allow for better insight into the behavior of flow instabilities when compared to our previous sonifications and/or visualizations, and additionally we observed benefits when some redundancy was maintained between modalities.
Cross-Sensory Globe
- Uttara Ghodke, MDes, OCAD U
- Sowmya Somanath, PhD, OCAD U (now at the University of Victoria)
- Peter Coppin, MFA, PhD, OCAD U
We present our participatively and iteratively designed 3D audio-tactile globe that enables blind and low-vision users to perceive geo-spatial information. Blind and low-vision users rely on learning aids such as 2D-tactile graphics, braille maps and 3D models to learn about geography. We employed participatory design as an approach to prototyping and evaluating four different iterations of a cross-sensory globe that uses 3D detachable continents to provide geo-spatial haptic information in combination with audio labels. Informed by our participatory design and evaluation, we discuss cross-sensory educational aids as an alternative to visually-oriented globes. Our findings reveal affordances of 3D-tactile models for conveying concrete features of the Earth (such as varying elevations of landforms) and audio labels for conveying abstract categories about the Earth (such as continent names). We highlight the advantages of longitudinal participatory design that includes the lived experiences and DIY innovations of blind and low-vision users and makers.
Decision Aid for Cross-Sensory STEM Learning
- Rachel Han, MDes, OCADU
- Marta Wnuczko. PhD, OCAD U
- Peter Coppin, MFA, PhD, OCAD U
Students with visual impairments (VI) miss out on science because of inaccessible visual graphics (such as pictures and diagrams) of the phenomena that are the focus of curricula. My project examines how efforts to translate these into non-visual representations, such as raised line graphics, tend to be less effective than expected because they are perceived using “rules” of haptic perception by VI learners but developed using “rules”' of visual perception by sighted designers. In response, I introduce my recommendations, in the form of a decision aid, informed by a series of interlinked concatenated studies consisting of user testing, workshops, and co-design sessions composed of multi-disciplinary teams that included VI educators, learners, inclusive designers, musicians, and domain experts from engineering and the cognitive neuroscience.
Role of Diagrammatic Properties of Virtual Work During Covid for Blind and Partially Sighted Employees
- Erin Lee, MDes, OCADU
- Mahadeo Sukhai, PhD, Canadian National Institute for the Blind
- Peter Coppin, MFA, PhD, OCAD U
The Covid-19 pandemic altered workplaces. For those with ‘office jobs,’ this meant working ‘virtually,’ or remotely, from home. This transition forced organizations and workplaces to exercise flexibility, adapt workflows and rely on Information and Communication Technologies (ICTs) to work remotely. However, Blind and Partially Sighted Individuals (BPSI) face challenges accessing work digitally, setting up their home offices, financing assistive devices, equipment and software, remote communications and employer support (Ginley, 2020). In response, with the Canadian National Institute for the Blind (CNIB), this Major Research Project (MRP) reports on the results of a longitudinal participatory design study investigating the impact of working and training over a distance for BPSI. This study found evidence of assumptions about BPSI, stigma, how effectively ICTs transmit perceptual cues and a physical environment bias where accessibility practices defaulted to the brick-and-mortar workspace. What emerged is a model to assist in understanding how ICTs synchronize experiences for the construction of shared intentionality in virtual work environments. Shared intentionality, the capacity to engage with others in cooperative activities with joint goals and intentions (Tomasello, 2005; Schweikard and Schmid, 2020), was a powerful way to interpret the disparities that BPSI faced as a result of the physical-to-virtual work environment transition.
Non Visual Drawing Tool for blind and Partially Sighted Drawers
- Mitali Kamat, MDes, OCAD U
- Alvaro Uribe Quevedo, OntarioTech University
- Peter Coppin, MFA, PhD, OCAD U
Drawing as an activity aids problem solving, collaboration, and presentation in design, science, and engineering and artistic creativity as well as expression in the arts. Unfortunately, blind, and partially sighted learners still lack an inclusive and effective drawing tool, even in the digital age. In response, this research aims to explore what an effective drawing tool for blind and partially sighted individuals (BPSI) would be. Raised-line drawing kits aim to provide this, but in prior work, our usability tests of raised line graphics with blind and partially sighted participants rated the raised line graphics that we tested as barely comprehensible relative to 3D models, which they rated as highly comprehensible. Semi-structured interviews with our participants afterward suggest that they found 3D models to be more comprehensible because these are consistent with haptic principles of perception whereas conventions of raised line graphics, such as a line representing a surface edge, replicate visual cues of source images and thereby violate haptic principles of perception. Therefore, we hypothesize that a drawing tool for blind and partially sighted drawers could be effective by recruiting affordances of 3D models. Through co-design sessions conducted during the Covid-19 pandemic with blind and partially sighted drawers (BPSD), we prototyped a tangible 3D model construction kit for non-visual haptic drawing with a digital interface to a 3D virtual environment. Our current investigation of user needs is informing us of our ongoing iterative development of an accessible 3D scanning application that is enabling blind and partially sighted individuals to build and scan in 3D models constructed from a more flexible range of materials beyond what was possible with our previous prototype.
say hello
We would love to hear from you.
COPYRIGHT Ⓒ 2021 PERCEPTUAL ARTFACTS LAB at OCAD UNIVERSITY