JOURNEYS INTO VIRTUAL REALITY

Creating sound to include on my footage, was another fascinating curve of synchronicities. The April snowfall that was accidentally recorded was a never previously experienced weather anomaly that created an apocalyptic scene. Trees were already green, snow was falling heavily, road accidents were escalating, and the sirens of ambulances were echoing through the city non-stop. I wanted to use the original audio, the sound that was recorded on the spot. There is a transition halfway through the footage from this sound to a 432Hz Earth-meditation tune.

Looking at the video, now with sound, made me wonder ‘how beautiful and serene apocalypse could be’. There was a strange contradiction between the calm, peaceful performance that recalled childhood and safety and the collapsing outside world. I sensed I was searching for the integration of these two seemingly contradictory aspects.
This made me realise there was more to this scene than one can perceive with simply the senses. Another more transcendent layer that does not bother with the concepts of global warming, the struggles of everyday life or everyday death. A layer that is not concerned with choosing in what way experiences should manifest, what is good or bad, if the human race exists or not, a layer, where there is no duality of life and death; that is the pure existence that we all are. I could best represent this layer with the sound of breathing.

The sound of breathing stretches across the entire video aligning with the rhythm of the swing.

I recorded some sound-bites of breathing in the studio by night when it was silent as the birds in the nearby woods were asleep. I recorded a few versions of ocean-breathing and open-mouth-breathing. Though for different purpose it was the same action as doing breathing exercises after yoga. And unexpectedly, it also had the same effect. My yoga teacher had warned us about open-mouth-breathing, as it can be quite powerful, bringing up repressed emotions and past baggage. When it comes to breathing, I just realised, gosh, there is so much past baggage.

When I finished recording, I listened back to the sound-bites a few times. I found all the flows and little nuances that made every breath sound so different interesting, even though I had tried to be consistent. Listening them brought up a stream of associations. First, they sounded like someone breathing deeply but slowly whilst sleeping. Listening again, I thought it sounds like someone breathing through a machine, the digitalised qualities of it. After yoga, I find ocean breathing calming and pleasant. It sounds like the waves of the ocean hitting the shore. Listening to it from the speaker, it was different. I thought it sounds like a hospital scene, with someone dying.

As usual, when I work with VR, there is something synchronistic about this video clip, its timing, the ways it brings up all these intense emotions in so many different ways. Having snowfall in April, meeting and re-meeting my tutor from Virtual Awakening Creative Studios, receiving this bursary and then the pandemic. There is something about artworks, how they bring about or signpost significant shifts in the fabric of the universe. At least the universe I am experiencing.

Now that meditation, breathing and contemplation have allowed me to release some painful childhood traumas safely, I was ready to come back to the duality of those everyday struggles differently, with a clearer sound. Now, when I look at the footage as it is ready (find it here: Wings of Reality) it speaks to me about motherhood. It tells me about the struggles of implementing creative solutions to stop global warming, to turn human life on ‘Spaceship Earth’ sustainable, the difficulties and resistance involved in implementing such solutions, whilst knowing that all this struggle might be a waste long term. The message is that no solution can work with an infinite number of human beings living on a finite piece of land. Today’s tendency points toward overpopulation. There is an extinction rate of species never experienced before, because of loss of habitat. Pandemics are also the result of the same issue. As long as humanity is unable to expand motherhood from ‘owning a child’ to ‘mothering every living being’ modern civilisation will not be sustainable. Creativity without compassion is empty.

This especially applies to Europe. Owning a child is no longer a necessity for survival in Western societies. When do we, who consider ourselves educated and conscious, stop pointing fingers and blame those societies who are not in this privileged situation? When do we take responsibility for our own actions and stop multiplying? When do we stop taking the moral high-ground and make conscious decisions ourselves?

Population density by countries:

https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population_density#/media/File:Population_density_of_countries_2018_world_map,_people_per_sq_km.svg

World population counter:

https://www.worldometers.info/world-population/

I made a few shoots in my studio to practice on and allow some distancing from the original project. It is so much easier to get things right in such a controlled environment with consistent lighting and a steady camera. At the same time, I kept on returning to the balcony footage without expectations. Just pondering, allowing myself the chance to see what arises. My tutor’s guidance also helped, he would ask the right questions. Sometimes I misunderstood these questions in a way that made me consider aspects I hadn’t thought of before. He kept on asking, “what is it exactly that you would like as an end result?”; “what are the elements that you think are the most important and you certainly want to keep?”; “would there be anything you would like to add or change, just maybe you don’t yet know how to?”. It was so difficult to conjure up confused replies on these questions on the spot, but I really wanted to say something other than “I don’t know”, so I just spontaneously uttered out any ideas that came up right there and then. I like this type of inquiry so much. It overrides thinking and comes up with suggestions intuitively, from the heart. I heard myself say: Can I put wings on the viewer? Animated ones? That the viewers would see on either side, flapping as they swing back and forth? …and other weird ideas regardless of technology and possibilities.

We completed the course, having learnt how to stitch 360 video using Mistika software and to edit sound and image (add effects, change colour, speed and mask the tripod) in Adobe Premiere Pro and Photoshop. It was a great course, very informative, concise and clearly structured. I could not be more grateful for this learning opportunity. After the final session, I was still pondering the balcony project.

‘What is it that I certainly want to keep and what is it that I don’t need and I am willing to trade?’ … I want to keep everything from the visual sphere.

‘How can I include the wings?’ That would take another course or two to learn.

‘How to avoid viewers getting nauseated by the erratic camera movement in VR?’ No clue…

I was playing with polishing the quality of the stitching and trying to get rid of stitch-lines in the image when it suddenly dawned on me. I am willing to trade the VR format. I was looking at the equirectangular format of the VR video on the computer screen. This is that elongated, distorted landscape image that is used to edit VR. It is the format in which the sphere of 360-degree reality around the camera is stretched out into 2D (see image attached). I realised this 2D format is the right medium to communicate the content best; this video does not want to be experienced as VR. It is willing to show another perspective of reality. A perspective that is exploring the edges of VR and 2D video. It made sense, as so far, all my major projects were discovering the boundaries of one medium with another, incorporating their contrasting aspects in the content. This video was recorded and edited as VR, but it does not intend to show itself as such. Whilst I was playing it, it also dawned on me; before I stabilised the horizon with the software, the image had a strange effect. It was divided into two halves by the vertical axis of the camera with the balcony and the character in the centre, while the visual sphere on both sides were moving like two wings. The right-wing the ‘home’ and left-wing the ‘outside world’ from the position of the mother. This orientation is reversed from the position of the child. (Some association here to politics: The mother whose life was crippled by communism and the child whose life is crippled by far-right capitalism.)

The swing’s movement created an undulating movement of 360-degree reality in relation to the camera on it. In other words:

From your centre, it is totally normal that reality is flapping about you like two giant wings, as long as there is a swing constant in your visual sphere :).

I find it fascinating how reality can be observed from so many different perspectives. Still, the perspective never changes the fact that the centre is always constant because we can only experience duality from singularity. From our own centre. Just like the VR camera. This idea recalls the well-known meditation technique called ‘The Headless Way’. It teaches to differentiate one’s direct experience from the narrative the brain creates about that experience. The method’s primary facilitator, Richard Lang, calls it the science of the 1st person.

With his words: “Consider this – what you are depends on the range of the observer. At several feet you appear human. At closer ranges you are cells, molecules, particles… At greater ranges you are a city perhaps, a country, the planet, the star, the galaxy…

But what are you at zero distance? In other words, what are you really?

Others cannot tell you because they always remain distant from you. But you are at your own centre so you are well-placed to look and see what you are there.”

I just learnt about this technique this summer during a super-intensive meditation course—no wonder it is bubbling up from the subconscious as another piece of the balcony-project puzzle.

Here I am having this super intense VR editing ‘crash course’ online, straight from some beautiful mountains in Mexico. A bit surreal. Based on indications I had received from UAL, I thought we could jam the basics into five 90 minutes sessions. That proved to be overly optimistic. Some sessions were two hours long and loaded with new information. Thankfully the tutor was very generous with his time and kept everything well-organised. He was also patient in moments when I was overwhelmed or tired and started to sink into chaos. We were passing screen share and mouse control back and forth, and, considering the limited ability of my Mac handling such data traffic, things went surprisingly fluently. I received a detailed presentation with easy-to-remember graphics for each session plus the recording of each session, so I can remember every step.

This here is my homework (attached photo). Now I’m getting a taste for it, the framework of stitching and editing VR. It is soooo exciting! And so much to learn, it seems to be an endless flow of new information. The footage provided quite a challenge. Because the camera was fixed on a swing, that was constantly moving, the software does not synchronise the footage correctly. I then have to do this frame by frame manually. It takes hours and hours of adjusting the image to create a two minutes video. I have an appreciation for the work that went into my past short films even more now! Finally, I get a coherent image of the 360-degree environment stretched out into what is called equirectangular format. This distorted landscape image that is on my screen in the attached image.

During the process, I realised a few things about the video that entirely changed my idea about what I want to do with it. ‘Changed’ at this stage meant giving up my expectations about it and focusing on learning. What happened was that I realised, some camera movement is ok, and was ok in the past when I used a larger swing which moved slower. This time, however, I pushed it too far. Literally. This smaller swing moves faster, and it has a much wider amplitude, which would make the viewer seasick in VR. Perhaps not so if one is watching it on a swing as I was planning to present it, being able to perfectly align the physical movement of the body with the movement of the virtual environment. That way, one’s haptic sense delivers matching information to visual perception. But it is quite tricky if possible, at all. The sensation of the body swinging forward while seeing it moving backwards is crazy. I myself love it, it makes me scream like being on a rollercoaster, but it’s not everyone’s cup of tea. Some people are more susceptible to motion sickness or can get a fright from such a destabilising experience. It generates a sense of chaos. Embodying that chaos is a challenge that is not my intention to create, as I don’t have information regarding its possible neurological effects.

This is my brother in the swing about 50 years ago. How technology changed since then! 2017 I had the raw footages recorded on this swing with 6 GoPro cameras ready to be stitched together to a 360 video, and some initial ideas of what I wanted to do with it.

Stitching/editing 4 minutes of 360° video costs about £2-3000 if outsourced, so learning it became the only way I could continue this project. Hence the search for a teacher and funds begin. I wrote an endless amount of emails to companies and individuals involved in VR. It is hard to believe but not a single reply was received. Not even a refusal. I called one studio that organises VR workshops in the UK under the umbrella of Raindance. They were extremely rude on the phone; still, I was considering putting up with them. Their workshop got cancelled due to the pandemic. I asked all my friends and every human being I met during the last two years if they have any idea where I could learn these techniques. Nobody had any. I emailed Gobelins in Paris, no English course. Talked to Hungarian film producers, no one knew VR in depth. Emailed UAL and LCC, I received quotes from both, that only Elon Musk could afford (I wasn’t going to put up with this one), later taking back their quote, deciding not to arrange it anyway due to the pandemic. It was so disheartening, if the project is not so close to my heart I would have indeed given up.

I am fortunate I met Jose from Virtual Awakening XR Creative Studio on a Movement Medicine workshop. I actually re-met him, as I have seen his work at a Breaking Convention event years before, an incredibly moving and well-researched VR tour called “Death Is Only The Beginning”. From the studio’s website:

“Death Is Only The Beginning” is based on a conceptual representation of a near death experience (NDE).

This is not “edutainment”. It is an experience best completed with a strong personal intention.

This mindfully guided VR experience facilitates an empathic resonance with the experience and process of dying. 

Through immersive storytelling, participants explore various stages of an NDE, as detailed in the medical literature by those given a second chance at life.

They are invited to a life review, raising awareness of critical global, social and relational issues facing all of humanity today, and invited to consider how they might proceed after they come back to their bodies and make the most this precious “second chance”.

Preparatory and follow-up work is essential to make the most out of this experience.”

Come 2020 and all its turmoil, lo and behold, with the financial support of a-n The Artists Information Company and Arts Council England and with the professional presence of Virtual Awakening this project received the green light to go ahead. I was now set up to learn what I needed for it. It felt like that magical moment when all stars align, and a gate to a new path suddenly opens.

In 2017 I had the idea of another short film that had to be shot in Budapest, where I was born. Literally. Filmed on the balcony of the apartment where I was born. Well, I wasn’t born on the balcony, but there was a small swing I used from the moment I could sit. I sold the apartment in 2017, and before handing it over I wanted to go back and hang the swing where it had been when I was a child, place the camera on it and push. Me performing as my mother, as she had done exactly the same forty years before with me. This way, I thought, when I watch the film through the VR headset, sitting on a swing in my studio in London, I could transport myself back to that early experience and would be able to connect to that phase of my childhood, this time in the role of my own mother. A little tweak of the past, a memory reconsolidation.

Memory reconsolidation is “the brain’s natural, neural process that can produce transformational change: the full, permanent elimination of an acquired behavior or emotional response. It is the brain’s innate process for fundamentally revising an existing learning and the acquired behavioral responses and/or state of mind maintained by that learning. (Ecker, 2015)” https://www.thescienceofpsychotherapy.com/glossary/memory-reconsolidation/

As I couldn’t bring Surroundvision to Budapest, I had to invest in the recording equipment and learn how to use it. I intended to record the scene in snow. I had been waiting the entire winter for the snow, but it was that rare year, maybe the first one ever in Budapest, without a single flake. I thought ok, then I will shoot it in the sunshine in the spring, shortly before the hand over. I arranged my journey for April. A week before travelling the weather forecast said ‘rainy and cloudy’… I gave up on my plans regarding the weather for the scene. The balcony is on the first floor, but the building is on a hillside, high enough to see the entire valley and a great chunk of the city below. Will my memories be reconsolidated to this gloomy weather?

I arrived in Budapest at night and went to straight the apartment. Next morning when I woke up and looked out the window, I saw a white blanket of snow covering the surrounding hills, trees and houses, still falling in large chunks. Hungary has never seen this much snow in April. It was surreal. I jumped out the bed, set up the camera and shot the scene a few times, experimenting with different ways to place myself in relation to the swing, until my bare feet got frozen to the concrete. By the end, the snow stopped falling, and it turned into rain. I thought I make another few shots, it looked quite like autumn with this sad weather, now that the snow melted. I took another few recordings over the next two hours. By the end of the day the weather had cleared up, and it started to look as it really was, April, with fluffy clouds swimming by, some sun, everything green. I made another few recordings, the last few in bright sunny weather. At the end of the day, I had the entire four season recorded on the balcony to reconsolidate. Talk about synchronicities!