_
=

XR - Extended and augmented reality

Final VFX processor for compositing the camera with the set extension and AR

XR projects needs a specific XR/AR license. If you'd like to try out these features to decide whether you want to purchase an XR/AR license, please contact us.

Explanation

The set-extension consists of sending the final stream, composed of the camera stream to which we add the virtual layout above. This is achieved with a dedicated Processor This section describes how to configure Smode processors in order to configure your output(s) and also make some nice and smooth stage pre-visualisations Read More in which the Video Input Extract the image from a Video Input Device Read More must be placed first. To avoid delays between the camera stream and the extension, follow the technique described below.

Extended Reality

The purpose of the extended part is to add content outside the boundaries of the LED screens, creating a seamlessly blended image.
Create a Processor This section describes how to configure Smode processors in order to configure your output(s) and also make some nice and smooth stage pre-visualisations Read More in the Pipeline Define how video content is processed in a project Read More and select your Physical Camera A camera that simulate a real-one Read More as its Current Camera.
In this Processor This section describes how to configure Smode processors in order to configure your output(s) and also make some nice and smooth stage pre-visualisations Read More , create a Video Input Extract the image from a Video Input Device Read More that corresponds to the tracked camera video stream.
If you are not on-site and don't have a real camera at the moment, you can create a Compo Composite 2D Layer, Shape layer and 3D Layer together. [Ctrl] + [Shift] + [N] to create a new composition after opening Smode Read More with a Stage Preview Renders the stage in Preview mode Read More to simulate the video input.

Then, create a new Compo Composite 2D Layer, Shape layer and 3D Layer together. [Ctrl] + [Shift] + [N] to create a new composition after opening Smode Read More , drag and drop the 'back' Content Map This section describes how to set up your content maps and use Content Area Read More inside it and set it's renderer to Unproject 3D Project the Content Map into a 3D plane according to a perspective point of view Read More . It will virtually project the Content Map This section describes how to set up your content maps and use Content Area Read More onto a 3D plane according to the Physical Camera A camera that simulate a real-one Read More 's perspective.
You need to apply a mask on this new Compo Composite 2D Layer, Shape layer and 3D Layer together. [Ctrl] + [Shift] + [N] to create a new composition after opening Smode Read More to display the camera stream only within the LED screen boundaries. To do so, create a Layer mask Turn any 2D generator into a mask Read More and inverse it. By default the Layer mask Turn any 2D generator into a mask Read More will create a Uniform Generate a solid, uniform image, filled with a unique color Read More . Select a Compo Composite 2D Layer, Shape layer and 3D Layer together. [Ctrl] + [Shift] + [N] to create a new composition after opening Smode Read More instead.
In this Compo Composite 2D Layer, Shape layer and 3D Layer together. [Ctrl] + [Shift] + [N] to create a new composition after opening Smode Read More , delete the Uniform Generate a solid, uniform image, filled with a unique color Read More and create a 3D layer Stage Display A 3D generator that refers to a stage element Read More for each LED screens. Select the corresponding LED screen in the Stage Display A 3D generator that refers to a stage element Read More parameters.

Smooth the Extension's mask

Change the renderer of the Stage Display A 3D generator that refers to a stage element Read More to a AutoIlluminate Surface Display geometry's triangles as a AutoIlluminate surface Read More . Apply a Placement mask Solid mask that has a 2D Placement Read More on the Uniform Generate a solid, uniform image, filled with a unique color Read More . At the moment, the edges of the Placement mask Solid mask that has a 2D Placement Read More are very sharp but you can soften them by using the 'Feather' parameter. By default, the link is set to 'All,' so if you change one of the values, they will all be modified as well.
Change the link to 'None' and change the values to smooth the edges you want.
It's advised to expose the Feather values in a Parameter Bank Group a bunch of Exposed Parameter together Read More by right clicking on the value and selecting 'Expose as clamped vertical canvas value' or 'Expose as percentage' between 0% and 100%. This way, you can easily modify all the feather valuesy you exposed at once directly from the Parameter Bank Group a bunch of Exposed Parameter together Read More without having to go back to the parameters of each mask each time. You can also change and expose the 'Exponent' value.

Augmented Reality

If you want to use AR elements, start by creating a Content Map This section describes how to set up your content maps and use Content Area Read More specifically for AR content if you don't already have one. Make sure to check 'Alpha layer'. Then create a new Compo Composite 2D Layer, Shape layer and 3D Layer together. [Ctrl] + [Shift] + [N] to create a new composition after opening Smode Read More in the final VFX Processor This section describes how to configure Smode processors in order to configure your output(s) and also make some nice and smooth stage pre-visualisations Read More and drag and drop your Content Map This section describes how to set up your content maps and use Content Area Read More inside. Set it's renderer to Unproject 3D Project the Content Map into a 3D plane according to a perspective point of view Read More .
If you don't have transparency in your AR content, check if your Content Map This section describes how to set up your content maps and use Content Area Read More and Compo Composite 2D Layer, Shape layer and 3D Layer together. [Ctrl] + [Shift] + [N] to create a new composition after opening Smode Read More have the 'Alpha layer' activated. If you use Unreal check the topic How-To Unreal In Smode How to integrate your Unreal content inside Smode Read More .

Delaying the Extension Content

Delaying the Extension and Ar content for syncing the Show animation
For this simply add a Frame Delay Travel an x number of frames into the past Read More onto the Content Map This section describes how to set up your content maps and use Content Area Read More . The delay you need to input into the Frame Delay Travel an x number of frames into the past Read More is the delay of the close Loop calculated by the XR - Latency Calibration Automatically evaluate the amount of frame delay between sending a picture to the stage and receiving it back through the camera. Read More

Sync the Video input and Traking Data

if your VFX look like this when rotating the camera, you need to to adjust the Camera Tracking Data Latency Sync the Camera Video Input with the Tracking Data Read More

Match the Extended and AR Color to the video input

To make the color calibration you can follow this guide: XR - Color Calibration Blend perfectly the walls colors with the virtual surrounding. Read More . It will go throw the step form recording the color to applying the Smart Lut Modify colors using a LUT from a XR Display information Read More to your VFX processor
At the end your VFX processor will look like this:
That's it! You are now ready to go on stage to proceed to the XR - Calibration procedures Calibrate your XR Studio Read More .

See Also: