Exploration of the potential benefits of virtual and augmented reality to foster the production and sharing of knowledge: that’s one of the tenets of the “4D Space”, a collaborative space for experimentation open to students who are pursuing the ESILV’s “Innovation, Research and Manufacturing” and Innovation programmes at the De Vinci Innovation Center.
These projects are carried out in real knowledge production and sharing spaces. They are developed by the students of the Pole Leonard de Vinci’s cross-disciplinary centre of expertise (DVIC) during their year of the Innovation, Research and Manufacturing Master’s degree or as part of the Innovation programme, offered in the 4th and 5th years of the engineering cycle.
The purpose of these forums is both to make choices in terms of equipment and cutting-edge technologies, to generate educational content, such as training tutorials, and to carry out collaborative projects in various fields: human-machine interfaces, robotics, flexible and active materials, wearables (connected devices), 4D Space, high-performance computing, and so on.
4D Space, a new experimentation space to reinvent telepresence
The “4D Space” enables students to test the latest practices and advances in virtual and augmented reality while experimenting with different telepresence systems. In a room designed for 4D experiences, the students try, using 3D headsets and acoustic equipment that allows spatial sound, to reproduce part of the sensory perceptions of the human being, namely sight, hearing and touch. A fan network connected to these systems makes it possible to recreate the notion of wind in a virtual environment as if we were there.
“There’s a lot of brainstorming going on to explore technological development: How do we create combinations to restore sensory perception? How do we develop olfactory artifices? How do we operate a 3D construction of the acoustic, of the visual? The conceptual dimension focuses on the notion of telepresence. For what uses? In which perspective? “explains Clément Duhart, head of the De Vinci Innovation Center.
For example, one of the telepresence implementations on which the 4D Space students are currently working is the treatment of phobias associated with vertigo, using virtual reality. With the help of headsets that allow a virtual experience similar of walking on a board, with the feeling of emptiness on the top, phobic people will be able to work on their capacity to cross this board virtually, trying to overcome the fear, which may help to treat the phobia. This sensation reconstruction project may be useful in the therapeutic evaluation of vertigo.
Tidmarsh, from Massachusetts to 4D Space, via the MIT Media Lab
Tidmarsh, a 600-acre former cranberry bog in Massachusetts converting back to natural wetlands, is the centrepiece of a research project that began five years ago at the MIT Media Lab.
“How do we help nature to recreate tides, knowing that it takes several centuries to rebuild itself naturally?” is the question that this project aims to answer.
Tidmash’s sensor installations create an online virtual reality world. At 4DSpace, we are working on the notion of telepresence, allowing students who climb into the machine to feel the same sensations as if they were in the middle of a tide: odours, wind, temperature, humidity”. (Clément Duhart, head of the De Vinci Innovation Center)
Students test telepresence in this protected environment using a Doppelmarsh, a cross-reality sensor data browser which fully recreates the Tidmarsh site in real-time.
Using a KAT VR personal VR Omni-Directional Treadmill for walking, running, bending, rising, 360° rotation and Hololens goggles, the students of Lab 4DSpace can observe the changing nature of this ecosystem as it evolves over time and space. This new type of documentary is also used to encourage public engagement and raise awareness of ecological restoration.
Learn maths while “making music with the universe”
The 4D Space students also work on sonification experiments: creating music that is controlled by external parameters, such as the room temperature or the rhythm of cardiac respiration. The students use an analogue synthesizer, like the ones used to create electronic music. This synth allows the students to split the input signal in two by trying to control electronic circuits, which, at the output, will let them test acoustic distortion, by producing notes of which the rhythm, speed and timbre can be controlled according to external parameters.
“The exciting thing about this work is that the students learn electronics through the sense of acoustics: they will be able to apply the different electronic transformations that they have seen in class to the analogue synthesizer signals, listen to the result and experience what they have learned.
How do we use the senses to gain a better understanding of maths?” (Clément Duhart, head of the De Vinci Innovation Center)
In this way, students build their module that can be incorporated within the various interconnected modules that are included in the synthesizer. “It’s a bit like, as MIT professor Joseph Paradiso used to say, ‘making music with the Universe’,” explains Clément Duhart.
Unlike digital, which encodes a physical signal in the digital world, in sound, the physical signal remains in the physical world. This application enables students to understand better the mathematical principles that govern the mechanisms that can be seen in electronic signals: phase shifting, changing amplitudes, etc.
More than a tool for learning mathematics, this project can also be a useful starting point for anyone wishing to get involved in in-depth studies around electro-acoustics, a relatively young field which is now becoming an increasingly important part of the musicological research.
