Exploring and Transforming Spaces Through High-Dimensional Gestural Interactions

Abstract. Almost every map or globe we come into contact with is distorted in some way, be it through cartographic projection, vertical exaggeration or data-driven morphing of distances in cartograms. And yet, once we utilize Virtual Reality technologies to position ourselves in a virtual reconstruction of a real or planned space, we usually default to a strict adherence to its real-world proportions and spatial relations. In search of an alternative conception of how such environments can be explored, this paper investigates a novel way of using the embodiment and high interactivity afforded by current VR technology to let users apply a wide range of transformations to their surroundings. Instead of utilizing a large number of predefined gestures that need to be learned before use, the full state of a user’s hand (including rotation, position, and joint angles) is tracked, directly mapped to a transformation matrix, and then selectively applied to the 3D environment. This is a complex and high-dimensional form of interaction, but through its embodied nature users can develop familiarity with it by unguided trial and error. Once accustomed, they can bend, shear, and manipulate the space around them with a variety of self-discovered gestural interactions. In the course of this paper, we discuss technical considerations, physiological limitations, possible use cases, as well as a number of recognizable gestures that emerged from the space of possible interactions after prolonged use.