Home » Canon and iPairs conduct demonstration experiment of multi-camera shooting that automatically tracks 3D characters.


VTuber in Japan 2026.11.16

Canon and iPairs conduct demonstration experiment of multi-camera shooting that automatically tracks 3D characters.

On November 5th, Canon Inc. and i-Pairs Inc. conducted a proof-of-concept experiment for live content production. The goal was to create an environment where real people and virtual characters can be easily filmed together using multiple cameras. In the experiment, the two played rock-paper-scissors and were filmed from multiple angles.

Traditional multi-camera virtual shooting required setting up tracking equipment for each camera and calibrating the real and virtual spaces to match. Creating compositions that match the movements of non-existent characters was also a difficult task.

The solution used to address this challenge was Canon’s multi-camera shooting solution, “Multi-Camera Orchestration.” Based on video analysis from an overhead camera, it automatically recognizes and tracks the subject and adjusts the composition of multiple cameras. This includes the relative positions of the cameras and the stage, and the FreeD protocol output by the cameras (a communication standard for sending information such as camera pan, tilt, and zoom).
This system uses that information to synchronize the shooting conditions of the virtual camera and the real camera.


This eliminates the need for external tracking devices and complex alignment. For character movement, we used i-Pairs’ motion capture correction software, “n-Links Retarget.” This software seamlessly applies the performer’s skeletal structure to the CG character, with a processing speed of 120 frames per second. It also resolves the instability during real-time operation that was a problem with conventional software, providing stability to withstand long recording sessions.

Ito Mamoru, President and CEO of iPairs, commented that the way the camera automatically tracks the characters created a sense of realism, as if the characters were actually there. He also stated that the reduced burden of setting up and operating the shooting environment is a major advantage for production teams.

This article is an English translation of an original Japanese article, translated by the Mogura VR editorial team.

Reference

https://prtimes.jp/main/html/rd/p/000001259.000013980.html

Related Articles