【Case Study】Lumens & MoonShine Animation-Taipei Music Center
By Kieron Seth & Isaac Chen, Lumens
March 25, 2022 24656
.jpg)
▶ MoonShine Animation
MoonShine Animation is an artist-centered VFX company based in Taiwan. MoonShine brings together directors, designers, artists, and technologists to collaborate on projects that involve video effects, animation, advertising, film, and virtual reality. The collective has developed ground-breaking somatosensory games, 4D immersive experiences, real-time animation and mixed reality virtual production studios. In 2020, MoonShine built the world’s first Mixed-Reality (MR) Theater in Taiwan.
▶ The Project
MoonShine's most recent project was driven by a spirit of experimentation and a commitment to equity. The concept was to install a bullet time rig with free view-point live production system in Taipei Music Center, Taiwan, which would empower viewers to produce their own viewing experience on their own consumer devices.
Highlight
◾Bullet Time Video Production
◾ Free View-Point video recording System
◾ 16 NDI PTZ Camera Array Rig
◾ NDI Video-over-IP Technology
◾ Highly Flexible and Scalable IP-based Structure
◾ MoonShine AI Complementary Frame Reconstruction
▶ The Challenge
Traditionally, a broadcast places all editorial decisions with the producer and director. The MoonShine team was inspired by the film The Matrix and its celebrated bullet time scenes to enable audiences to control and select their preferred viewing angles and dynamic camera moves in real-time.
In the 1999 movie, actor Keanu Reeves seems to slow time and acrobatically evade bullets as the camera moves dynamically and fluidly around the actor. This 'bullet time' effect is composed from a series of individual images composited into a video stream. MoonShine took the concept of the mobile camera and applied it to live production.
▶ The Solution
Rather than using still images which require lengthy post production, Moonshine employed 16 4K PTZ units with live video output. The system merges multiple IP streams into one free-viewpoint multi-angle live video, to create a true bullet-time live video. By using multiple NDI-output Lumens VC-A71PN cameras, the production crew was able to design a semi-circular rig that could simultaneously shoot 16 different angles.
With each NDI stream feeding into a multi-view system, each camera angle is streamed independently to viewers. The audience is then able to select any viewing angle and switch their preferred view with ease. This makes the experience immersive and engaging, and eliminates the risk of blind spots within the venue. MoonShine's engineering team employed an AI-based complementary frame insertion tool between each camera to make streaming smoother when switching from one camera position to another.
"We installed 16 NDI PTZ cameras on the second floor of the auditorium at the Taipei Music Center. Using video-over-IP transmission technology, the feeds from these 16 cameras are sent to the view-point video recording system and processed for transmission. We chose Lumens PTZ cameras which work well with our free view-point video recording system. The software also controls the cameras' focus, de-focus, iris control, and so on."
Commented Jelly, MoonShine Animation's Project Manager
▶ The IP Advantage
Lumens PN series cameras use NDI|HX technology. NDI is an advanced IP protocol which transports high quality, low latency video signals over Ethernet-based networks. Cost-effective, easy to install and simple to configure, NDI networks are easily scalable allowing more cameras, recorders, encoders and video mixers to be added to the workflow, as required. Bandwidth-efficient, NDI can be run over 10G or 1G networks, delivering video signals, power to the cameras and control data at the same time.
▶ Hybrid Approach
The NDI rig gives the production team enormous flexibility. By installing robotic, remotely controlled PTZ cameras, the producers can move each camera independently if required to capture any position on the stage. This means that as well as positioning the 16 cameras as a synchronized arc for bullet time production, the crew can take full control of the rig to direct a traditional multi-cam shoot. The VC-A71PN cameras can be controlled via MoonShine's software, or using the Lumens VS-KB30 hardware unit, an intuitive and tactile camera interface which many camera operators prefer for live production.
"Lumens VC-A71PN PTZ camera supports NDI and PoE. It can be powered, imaged, and controlled through a network connection, which is very convenient for the installers. We positioned the camera 30 meters away from the stage. Featuring a 30x optical zoom, we can capture all the performers on stage. The VC-A71PN provides 4K image quality, and it performs well even in a low-light environment. VC-A71PN can effortlessly integrate into the free view-point video recording system with NDI transmission technology. Lumens also provides timely technical support, allowing us to undertake multiple shoots without issue."
Noted Ted, MoonShine Animation's System Integration Engineer
▶ Video
▶ Related Products
Click here Back to Top ↑