From digital visual effects to emerging in-camera visual effects: investigating the change of workflow, occupational roles and common challenges in Southeast Asian and East Asian countries

Abstract

In-camera Visual Effects (ICVFX) occupy an intermediate position between film and game production. It is an emerging filmmaking technique that combines traditional cinematography and motion tracking technologies with a real-time game engine and computer-generated imagery (CGI). This allows actors to perform against a real-time generated background displayed by an LED wall, enabling the film production crew to view the scene and deliver a close-to-final product on set. ICVFX has been widely embraced not only by Hollywood but also by the East Asian (EA) and Southeast Asian (SEA) production industries. This research aims to investigate the impact of ICVFX on workflow, occupational roles, and shared challenges occurring in EA and SEA-based production companies.

Compared to green screen VFX production, the ICVFX workflow offers an iterative and interactive process supported by real-time technology. It seamlessly integrates filmmaking, game development, and LED wall-related methodologies. The workflow begins with film pre-production methods and progresses through two phases of game engine-oriented creation, followed by two LED wall-related review and rehearsal phases before principal photography. As a result, the demand for match-move and roto departments, prevalent in green screen VFX production, is significantly reduced. Additionally, instead of replacing the green screen, ICVFX is considered a distinct method with its own workflow features. With its advantages, limitations, and applications, ICVFX production serves as an alternative tool instead of fully replacing the green screen.

Concomitantly, ICVFX transforms the departmental landscape by introducing new departments and roles responsible for the entire production. These include the Virtual Production (VP) supervisor, the virtual art denepartment (VAD), and the "Brain Bar." This study investigates occupational changes by examining four key roles from each production technique. To adapt to ICVFX, the VFX supervisor must work closely with the VP crew led by the VP supervisor; the environment artist transforms into the VAD artist; the VFX compositor and technical director become part of the "Brain Bar." This transformation creates a demand for technical artists (TA) and requires the artists to be versatile and able to work “live” with on-set communication styles.

With an understanding of the new workflow and occupational roles, this study pays equal attention to the ICVFX industry in EA and SEA countries. Five main challenges are observed and analysed from sampled interviews, highlighting a different industry landscape compared to Hollywood. Furthermore, industry and academic-related suggestions and recommendations are provided to address these challenges.