Panorama180 Render : Demos for Runtime

Developer : ft-lab (Yutaka Yoshisaka).
06/13/2021 - 06/16/2021.

Back

Demos for Runtime

This is a description of the demo scene for the runtime feature added in Panorama180 Render ver.2.0.0.
In the "Panorama180Render/Demo/Scenes/Runtime" folder, you will find the Runtime demo scene.

I used "Oculus Integration v.29.0" to check VR operation.

"PC-VR" indicates PC-VR execution on Unity Editor, or PC-VR execution after build.
I have confirmed that it works with Oculus Quest 2 + Oculus Link.
”Oculus Quest2(Native)" was built as Android and tested on Oculus Quest2 natively.

In all demos, Panorama180Render is added as a component of the main camera.

This can be added from [Scripts]-[Panorama180Render]-[Panorama180Render] by pressing the [Add Component] button in the Inspector window when the GameObject is selected.

In VR(When using Oculus Integration), add Panorama180Render to the "OVRPlayerController/OVRCameraRig/TrackingSpace/CenterEyeAnchor" GameObject.


See also "Use API from script (Runtime)" for information on how to use the API for runtime.

blocks

Unity EditorRuntimePC-VROculus Quest2(Native)
oo--

Overview

When executed, a panoramic expanded RenderTexture will be displayed in the upper right corner.
This demo shows how to capture panorama 180-3D and panorama 360-3D by switching between them.


This sample starts the Panorama180 Render's panorama creation capture with "Panorama180Render.BeginCapture()".
Always shows the panorama for the specified RenderTexture.

Operation method

Use [A][S][D][W] on the keyboard or sticks on the gamepad to move.
Press [C] on the keyboard or [X] on the gamepad to toggle the capture on/off.
When Capture is Off, it hides the camera for panorama and does not do the capture itself.
By pressing [V] on the keyboard or [Y] on the gamepad, the XZ of the camera's rotational elements to be captured will be taken into account (Synchronize Gaze Direction : On)。
If this is Off, only the Y rotation of the camera will be considered. In this case, the XZ rotation will always be zero.


Press [B] on the keyboard or [B] on the gamepad to switch between Panorama 180-3D and Panorama 360-3D.


Program Description

Assign the script "Panorama180Render/Demo/Scripts/Runtime/Sample_Capture.cs" to any GameObject.
From the component, you can add it by [Scripts]-[Panorama180RenderSample]-[Sample_Capture].
Gets the Panorama180Render component assigned to the main camera.
using Panorama180Render = Panorama180Render.Panorama180Render; ... if (Camera.main != null) { GameObject g = Camera.main.gameObject; if (g != null) { m_panorama180Render = g.GetComponent<Panorama180Render>(); } }
"using Panorama180Render = Panorama180Render.Panorama180Render;" makes it easier to access Panorama180Render.
In the following, various parameters of Panorama180 Render are specified in Start().
// Set GameViewMode (Default view). m_panorama180Render.SetGameViewMode(Panorama180Render.GameViewMode.Default); // Set IPD(Interpupillary distance. unit m). m_panorama180Render.SetIPD(0.064f); // Set texture size as Cubemap (512/1024/2048/4096). m_panorama180Render.SetRenderTextureSize(1024); // Post Processing On / Off. m_panorama180Render.SetPostProcessing(true); // Output resolution at Panorama180. m_panorama180Render.SetOutputTextureSizeType(Panorama180Render.OutputTextureSizeType._1024); // Output resolution at Panorama360. m_panorama180Render.SetOutputTextureSizeType360(Panorama180Render.OutputTextureSizeType360._2048);
For details on how to use each API, please refer to "Use API from script (Runtime)".

Activate the cameras for panorama expansion by calling "Panorama180Render.BeginCapture();".
Disable the cameras for panorama expansion with "Panorama180Render.EndCapture();".
Between "BeginCapture()" and "EndCapture()", the panorama is always expanded from the camera.

The panorama expanded RenderTexture is obtained by "Panorama180Render.GetRenderTexture()" in OnGUI.
// Get RenderTexture while capturing. if (m_panorama180Render.IsCapture()) { // Get RenderTexture. RenderTexture rc = m_panorama180Render.GetRenderTexture(); }
The RenderTexture retrieved here will always be the same between "BeginCapture()" and "EndCapture()".
It is heavy because it creates panoramic expansion textures from multiple cameras and updates the RenderTexture for each frame.

blocks_assignRenderTexture

Unity EditorRuntimePC-VROculus Quest2(Native)
ooo-

Overview

When executed, it will assign the RenderTexture of the acquired panorama on the block.
Always view the panoramic 360-3D from the camera.


For the "blocks" demo scene, a panoramic RenderTexture was displayed in OnGUI.
In this demo scene "blocks_assignRenderTexture", a RenderTexture is assigned to the material of Unlit for the Mesh in the scene.

Operation method

Use [A][S][D][W] on the keyboard or sticks on the gamepad to move.

Program Description

Assign the script "Panorama180Render/Demo/Scripts/Runtime/Sample_CaptureRenderTexture.cs" to any GameObject.
From the component, add [Scripts]-[Panorama180RenderSample]-[Sample_CaptureRenderTexture].
The flow is almost identical to the demo scene "blocks".
In Update(), RenderTexture is retrieved and reflected in the material's texture.
RenderTexture rt = m_panorama180Render.GetRenderTexture(); if (rt != null) { if (m_mat.mainTexture == null) { #if !UNITY_2019_OR_NEWER m_mat.SetTexture("_UnlitColorMap", rt); // HDRP/Unlit m_mat.SetTexture("_BaseColorMap", rt); // HDRP/Lit #endif m_mat.mainTexture = rt; } }
It is heavy because it creates panoramic expansion textures from multiple cameras and updates the RenderTexture for each frame.

I have also confirmed that it works with PC-VR.
Also, specifying the texture size in "Panorama180Render.SetRenderTextureSize()", "Panorama180Render.SetOutputTextureSizeType()", and "Panorama180Render.SetOutputTextureSizeType360()" will take longer if the resolution is increased too much.
Native execution in Oculus Quest2 was too heavy to be practical.

blocks_sync_RenderTexture

Unity EditorRuntimePC-VROculus Quest2(Native)
oooo

Overview

When executed, it will display two empty white surfaces on the block.


Press the [C] button on the keyboard or the [X] button on the gamepad to display the Panorama 180-3D capture.
For VR, press the [X] button on the Oculus Quest to display a Panorama 180-3D capture.


This demo shows how to get the color/depth texture of a panorama as a RenderTexture.
Panorama180 Render will not process the image except at the moment of capture. The load will be applied during the few frames of the capture.

Operation method

Use [A][S][D][W] on the keyboard or sticks on the gamepad to move.
Press the [C] button on the keyboard or the [X] button on the gamepad to display the Panorama 180-3D capture.
For VR, press the [X] button on the Oculus Quest to display a Panorama 180-3D capture.

Program Description

Assign the script "Panorama180Render/Demo/Scripts/Runtime/Sample_CaptureSyncRenderTexture.cs" to any GameObject.
From the component, add [Scripts]-[Panorama180RenderSample]-[Sample_CaptureSyncRenderTexture].

Specifies to get the Depth texture in Start().
// Set depth texture type. m_panorama180Render.SetDepthTextureType(Panorama180Render.OutputDepthTextureType.DepthDefault); // Set depth texture linear. m_panorama180Render.SetDepthLinearType(Panorama180Render.OutputDepthLinearType.Linear);
In addition, the depth range of Depth is narrowed to make the shading of Depth explicit.
m_panorama180Render.SetCameraNearFarDistance(0.1f, 10.0f);
The distance to the Near clip is set to 0.1 and the distance to the Far clip is set to 10.0.

This sample does not call "Panorama180Render.BeginCapture()".
After this, a coroutine is used to get the panoramic expanded RenderTexture one time while keeping synchronization.

Coroutines are described as follows.
IEnumerator GetPanoramaRenderTextureCo () { if (m_panorama180Render == null) yield return null; // Call coroutine. var fc = m_panorama180Render.GetRenderTextureCo(); var curCoroutine = StartCoroutine(fc); // Wait coroutine. yield return curCoroutine; // Get RenderTexture. // The RenderTexture that can be obtained is temporary. Panorama180Render.PanoramaTextureData rt = (Panorama180Render.PanoramaTextureData)fc.Current; // Assign RenderTexture to Material. if (rt != null) { if (m_mat != null && rt.colorTexture != null) { if (m_mat.mainTexture == null) { // In the case of HDRP before Unity 2018, // even if RenderTexture is added to the mainTexture of Material, it will not be reflected. // Therefore, the texture is forcibly specified. #if !UNITY_2019_OR_NEWER m_mat.SetTexture("_UnlitColorMap", rt.colorTexture); // HDRP/Unlit m_mat.SetTexture("_BaseColorMap", rt.colorTexture); // HDRP/Lit #endif m_mat.mainTexture = rt.colorTexture; } } if (m_depthMat != null && rt.depthTexture != null) { if (m_depthMat.mainTexture == null) { #if !UNITY_2019_OR_NEWER m_depthMat.SetTexture("_UnlitColorMap", rt.depthTexture); // HDRP/Unlit m_depthMat.SetTexture("_BaseColorMap", rt.depthTexture); // HDRP/Lit #endif m_depthMat.mainTexture = rt.depthTexture; } } } }
This coroutine retrieves the panoramic expanded color and depth textures from the Panorama180 Render and reflects them as textures for the specified material.
At this point, wait for the cameras to capture and generate a panoramic image for panorama expansion.
After the panoramic image is generated, hide the cameras for capture and stop the capture process.

If the Panorama180 Render fails to get the component, it will exit the coroutine without doing anything.
if (m_panorama180Render == null) yield return null;
"Panorama180Render.GetRenderTextureCo()" is the coroutine that performs a single capture.
"StartCoroutine(fc)" calls the coroutine.
var fc = m_panorama180Render.GetRenderTextureCo(); var curCoroutine = StartCoroutine(fc);
Wait for the end of the coroutine by specifying "yield return curCoroutine;".
Since panoramic image are generated from captures from multiple cameras, this process may take several frames.
yield return curCoroutine;
The following process returns the resulting RenderTexture in the "Panorama180Render.PanoramaTextureData" structure.
Panorama180Render.PanoramaTextureData rt = (Panorama180Render.PanoramaTextureData)fc.Current;
"rt.colorTexture" is the color texture, and "rt.depthTexture" is the depth texture.
Both are RenderTexture.
If the RenderTexture does not exist, null will be entered.
This is specified as a texture for each material, and the result is reflected in the Mesh on the scene.
The "GetPanoramaRenderTextureCo" coroutine described above is called as follows when the specified button is pressed.
StartCoroutine(GetPanoramaRenderTextureCo());
This call does not wait on the main thread. Be aware that the resulting RenderTexture is returned asynchronously.

I have also confirmed that it works with PC-VR.
I have also confirmed that it works natively running on Oculus Quest2.
However, specifying the texture size in "Panorama180Render.SetRenderTextureSize()", "Panorama180Render.SetOutputTextureSizeType()", and "Panorama180Render.SetOutputTextureSizeType360()" can be time-consuming if the resolution is increased too much.
When under load, VR will stop for a moment, during which time you will see the camera shake.

blocks_sync_RenderTextureSave

Unity EditorRuntimePC-VROculus Quest2(Native)
ooo-

Overview

Save the panorama texture to a file after the capture process in the demo scene "blocks_sync_RenderTexture".

Operation method

Use [A][S][D][W] on the keyboard or sticks on the gamepad to move.
Press the [C] button on the keyboard or the [X] button on the gamepad to display the Panorama 180-3D capture.
For VR, press the [X] button on the Oculus Quest to display a Panorama 180-3D capture.
This will output the "output.png" file at the location of the executable file.

Program Description

Assign the script "Panorama180Render/Demo/Scripts/Runtime/Sample_CaptureSyncRenderTextureSave.cs" to any GameObject.
From the component, add [Scripts]-[Panorama180RenderSample]-[Sample_CaptureSyncRenderTextureSave].

Most of the scene is the same as the "blocks_sync_RenderTexture" demo scene, with the file saving process at the end.
using Panorama180Render = Panorama180Render.Panorama180Render; using Panorama180RenderFileUtil = Panorama180Render.PanoramaUtils.FileUtil; .... // Save file. RenderTexture colorTex = rt.colorTexture; if (colorTex != null) { // When using Unity Editor, the "output.png" file is output under the project directory. // At runtime after build, the "output.png" file is output in the same location as the executable file directory. m_saveRenderTexture(colorTex, "output.png"); } ... private void m_saveRenderTexture (RenderTexture rt, string filePathName) { if (rt == null) return; Panorama180RenderFileUtil.SaveRenderTextureToFile(rt, filePathName); }
The "SaveRenderTextureToFile" in "Panorama180Render.PanoramaUtils.FileUtil" saves the RenderTexture of the first argument to the file of the second argument.
This process takes time because it is done in the main thread.

Back