Category: Augmented Reality

Numen

Numen:
“The spirit or divine power presiding over a thing or place.
A spiritual force or influence often identified with a natural object, phenomenon, or place.”

numens-3

I have returned to some 360 video pieces I was doing in November 2016 of places that held a significance for me, but abandoned as I was frustrated at the quality of the final video. I was using a Ricoh theta S 360 camera and the still images are excellent, but the video footage is very soft. In retrospect it may be just when added to Youtube with the 360 meta tag the quality deteriorates because of the added YouTube compression, but even in the Homido player app I have been using in my iPhone the quality looks muddy. In a nutshell I need a 4k 360 camera, I have been looking at Vuze Camera due to be shipped next month, but even the examples shown online look kind of soft also. Sigh. Another route would be to use Unity gaming engine, but I’m after realistic footage, not a computer game type aesthetic.

“The most beautiful thing we can experience is the mysterious. It is the source of all true art and science” – Albert Einstein

numens-1

“There is no such thing as an environment: wherever we look for it we find all kinds of objects – biomes, ecosystems, hedges, gutters and human flesh. In a similar sense there is no such thing as Nature . I’ve seen penguins, plutonium, pollution and pollen but I’ve never seen Nature.” – Timothy Morton: Realistic Magic Objects, Ontology, causality.

How did numen, a Latin term Numen meaning nod of the head come to be associated with spiritual power? The answer lies in the fact that the ancient Romans saw divine force and power operating in the inanimate objects and nonhuman phenomena around them. They believed that the gods had the power to command events and to consent to actions, and the idea of a god nodding suggested his or her awesome abilities-divine power.

Eventually, Latin speakers began using numen to describe the special divine force of any object, place, or phenomenon that inspired awe (a mystical-seeming wooded grove, for example, or the movement of the sun), and numen made the semantic leap from “nod” to “divine will or power.

Jumping from the Romans to the Greeks…the Platonic solids. The five special polyhedra are the tetrahedron, the cube, the octahedron, the icosahedron, and the dodecahedron, but I opted to use some low-poly 3D object shapes instead, namely a Cone, Sphere and Torus. Objects to thrust into the landscape.

numens-2

This also brings me back to the main problem of storytelling and narrative in an immersive environment, you need some kind of device to try and direct the viewer where to go/look. The piece ‘Lost’ from 2015 uses a firefly as a device to follow, but you can at any moment look away, hence two peoples experience of the piece will be different.

green-screen-360-cyc

To capture green screen footage for 360, also some kind of 360 cyc as above would need to be built.

Note to Self:

  • Back to the camera/drawing board. All these disparate elements really need to start coming together, too many tangents. Time wasted.
  • The Cone, Sphere and Torus may be narrative devices to compel viewers to look/jump to a certain section of the 360 round.
  • Gaming narrative the way forward?…leading players/viewers down a set path and explaining their surroundings. But what instead of the gameplay? that is the journey through a games levels. For 360?
  • Does a viewer want to have agency (control) or not?…or be a passive viewer, this is 360 NOT full VR remember.
  • Immersive, immersive, immersive…

360 Multi-verse

Firstly lets differentiate between Full VR and 360/VR video.
With Full VR:

  • Each moment is generated live
  • The user interacts to control what happens

With 360 Video:

  • Its a single experience rendered in advance
  • The viewer chooses where to look
  • In traditional film editing we think in terms of frames.

    “In a 360 environment a ‘frame’ is a relative window of experience derived from the visitor’s field of vision. This makes everything a potential frame, but also makes a premeditated frame based on my own interests presumptuous and, well, wrong most of the time.” [1]

    These visuals below are more reflective of the spatial reality of the medium, more apt to its multi-verse tendencies where every path exists simultaneously. Worlds of experience extending from one another, much like ripples in a pond or rings in a trunk of a tree.

    points-1

    We need to identify the potential experiences in each world, evaluate the probability that they will occur, and then take into account how a visitor might engage with them, I could then identify possible paths. I could rotate these worlds around each other, using the most probable potential experiences to guide someone through.

    points-2

    Then perhaps I could work backwards through these layers of experience, take what insights editing these worlds provide and use them to help shape the creation of these worlds from the start.

    “…there needs to be the existence of a unique link between the mind of the creator and the mind of the visitor. It appears to be very specific to this medium and something that could have never existed until presence became a factor.” [2]

    Note to Self:

    • Work out a DIY method of writing a 3D treatment/screenplay. current methods will not suffice.
    • To capture proper 360 green screen film footage of actors, a curved circular cyc green screen would have to be constructed, with maybe spot lighting from above?

    Mixed Quasar

    An experiment/Trial and Error investigation to make a video more immersive. Heres the original regular mp4 video below;

    and here the 360 version with the ‘Spatial Media Metadata Injector’ code to make it 360 added below.

    Drag the video above to see it 360 (view in Chrome browser). It appears very messy, low resolution and nausea inducing, as it was rotating already, a more regular video may be a better example. I haven’t bother to edit/stitch the join also.

    Drag the video above to see it 360 (view in Chrome browser). This is a 360 still .jpg taken from Ricoh Theta S, and a blender 3D rotating object, both imported into Final Cut Pro X, exported as an MP4 and spatial meta injector code applied, no audio.

    Drag the video above to see it 360 (view in Chrome browser). This is taken by my Ricoh Theta S 360 camera walking in a circle, and a blender 3D rotating object, both imported into Final Cut Pro X, exported as an MP4 and spatial meta injector code applied, audio. This video was sourced from my iPhone.

    Drag the video above to see it 360 (view in Chrome browser). This is a 360 video taken by my Ricoh Theta S 360 camera walking in a straight line , and a blender 3D rotating object, both imported into Final Cut Pro X, exported as an MP4 and spatial meta injector code applied, audio. Screen is split into x2 spheres. Be interesting to see which ones play as 360 in app on cardboard viewer. This video was sourced from the camera.

    Drag the video above to see it 360 (view in Chrome browser). This is a 360 video taken by my Ricoh Theta S 360 camera walking in a straight line , and a blender 3D rotating object, both imported into Final Cut Pro X, exported as an MP4 and spatial meta injector code applied, audio. This video was sourced from my iPhone.

    Note to Self:

    • I will now try to view the various files in Google cardboard via a few VR apps to see which work and to see how immersive they are.
    • The aesthetic is not important here, nor is the fact the footage isnt stitched properly, its just a series of quick tests and reference for myself to see how the imported source material and exported files work best, so as to create a quick working methodology.

    VR/MR Maker

    I was in London for a few days so popped down to a UAL affiliated VR/MR maker day in Chelsea College of Art.

    maker-1

    It was fairly informal and there were various desks/sections featuring Arduino, Leap Motion, Virtual Reality etc..I was interested in getting an overview into how one sets up a Virtual reality environment in 3D gaming software Unity, and create a Terrain, work with collidors, prefabs etc..

    maker-2

    I also got a demonstration of a 360 3D/Spherical camera the Ricoh Theta S, this is for grabbing 360 live video or still footage. Overall a worthwhile and enlightening day.

    Buggy G

    I revisited some 90s video and cover imagery from FSOL’s (Future sound of London) ‘Lifefroms’ album. The artist was Buggy G Riphead and in a decade that was dominated by some really bad fractal and rave imagery for album artwork and flyers the FSOL work seems strangely contemporary.

    buggy-1

    Maybe its all the current interest in VR, AR and all things 3D?

    buggy-2

    Note to Self:

    • Don’t Mention: The ‘Lawnmower Man’ or ‘Strange Days’ movies.

    Hail Fellow Well Met

    After a chance remark on the Low Residency I decided to revisit a short film I was exploring, a kind of Rural/Tech Sci-fi Wicker Man. It may be more feasible to explore now via different technology/approach as a very short piece.

    “If any sufficiently advanced technology is indistinguishable from magic…then many of us live in an age of almost Mystic Wonder”.

    hail-5

    “Come with us West, to the very edge of Europe, there we will build another kingdom of Zeros and ones”.

    hail-6

    “The Fates lead those who will, those who won’t…they drag”.

    hail-8

    “You must awaken the the dark wonder that lies within your savage Pagan heart”.

    Cardboard Reality

    I finally got around to getting my hands on a Google Cardboard VR headset. Cardboard is a low-cost, easy-to-get virtual reality viewer that transforms a phone into a basic VR headset.

    cr-1

    Film Auteur Werner Herzog had been ranting on about VR [1],

    “What reality is the cockroach at my feet in the kitchen experiencing? It is not my reality, we only share the same space.”

    so I wanted to see what all the fuss was about.

    Is Virtual Reality and Augmented Reality really a new 21st century art form? With not too much bother I assembled the Cardboard viewer, its a cheap alternative to the myriad of other viewers out there…HTC Vive VR headset, Durovis Dive, Homido, Samsung Gear VR, Carl Zeiss VR One, Cmoar, OSVR, Fibrum, HTC Vive, Sony Morpheus, Oculus Rift DK1 etc. The Oculus Rift and Sony Morpheus (which is more game orienteted) looked the most promising to me.

    Anyway back to my poor man’s version, the cardboard…I downloaded a few apps that looked interesting.

    cr-2

    First up was Bjorks Stonemilker, which was good, especially the way she seemed to jump out from her own body and into a new position…but I managed to ‘break it’ by trying to take a screenshot whilst it was playing, but this was interesting in itself as it threw up some code and ‘inner workings’ as to how it might be made. Stonemilker was directed by Andrew Thoamas Huang and produced by VRSE.com. They seem to be ahead of the pack in the VR game, Chris Milk is a former video artist who runs it.

    The next x3 pieces I watched were all produced by VRSE, ‘Take Flight’, a marvellous short excursion into the heavens above New York City. ‘Evolution of Verse’ that takes you face to face with a foetus in the womb, quite amazing, and finally ‘Catatonic’ a creepy wheelchair ride through an insane asylum. All three were fairly amazing, you the viewer being immersed directly into their environment. Catatonic was the most unsettling, as when you swivel 360 degrees and look up directly at the orderly who is pushing you in the wheelchair right above you…you realise its not an orderly anymore!

    cr-3

    So how does film making for VR differ from traditional film making? some main points below;

    • You can’t frame a 360 shot
    • There are no cuts
    • Death of the Close-up?
    • The character can know you are there, and be right beside you
    • The main protagonist sacrilegiously cuts through the fourth wall, and makes a direct connection with you via eye contact
    • You must try and draw the viewers eyes to the different places they can look at and explore
    • Scale. Object sizes aren’t always in real-world ratios. Sometimes certain scale ratios are based on what feels right, rather than what would be mathematically correct
    • Focus on movement that matters, so that movements are computed in real time to adjust to the viewers perspective
    • Sound. Directional binaural sound
    • Don’t overload your rendering machines, Oculus headsets show images at 90 frames per second which is a huge computional burden. Reduce the load in a CG production, mathematical bounding boxes are calculated around objects, if the viewer isn’t look at something in particular, then it doesn’t render

    The Cardboard was the best $10 I ever spent.

    Note to Self:

    • Not sure how this impacts on my making, if at all, as the technology seems to be customised or else priced out of reach.
    • There is a great 360 degree camera called ‘Neo’ by a company by Jaunt, but they aren’t even for sale, they envisage leasing them out to interested parties in the future.