Implementation of Generalized Perspective Projection on the Unity

This article shows an implementation of Generalized Perspective Projection on the Unity. That means that makes a generalized perspective projection matrix for arbitrary view.

This implementation refer the paper below link.

I’ll show you demos implementated it on the Unity. You can see the result at below video.

First, the video will show just demo that rotation and projection to the perpendicular plane from the camera.

Second, the video will show what the plane (such as window) display from arbitrary view point.

I’ve hosted this example on the GitHub. Please see the example below link if you want.

edom18/GeneralizedPerspectiveProjectionTest

Idea

This idea based on project to perpendicular plane from the view point. So we need to calculate several points on the plane and orthornormal vectors.

Define the corner points

First, we’ll define the corner points on the screen plane (it means near clip plane).

Left image shows you what points located on the plane.

We define these point, Pa, Pb and Pc.

Calculate the orthornormal vectors on the plane

Next, we’ll calculate the orthornormal vectors on the plane.

It can calculate easily, because we already have corner points. We use that to calculate the vectors like below.

Vu = (pc - pa).normalized;
Vr = (pb - pa).normalized;
Vn = Vector3.Cross(vu, vr).normalized;

Calculate off-axis point on the plane

Let’s see the first figure, it shows you where the view point is located.

So, regular frustum’s view point is on the middle of the plane. But we want the point when view point (Pe) is moved.

Let’s see the second figure, it shows you where the view point is located when the view moved.

As you can see that point is not locate the middle. Let’s calculate the point.

Fortunately, we can calculate it easily. See the third figure, it shows you vectors what how far and direction from view to each corner point.

It can be calculated as below.

Vector3 va = pa - pe;        Vector3 vb = pb - pe;        Vector3 vc = pc - pe;

As you can see that so easy!

Let’s calculate values for the frustum function

Then we can calculate the parameters, l,r,t,b by the vectors above.

// Find the distance from the eye to screen plane.
float d = -Vector3.Dot(va, vn);
// Find the extent of the perpendicular projection.
float nd = n / d;
float l = Vector3.Dot(vr, va) * nd;
float r = Vector3.Dot(vr, vb) * nd;
float b = Vector3.Dot(vu, va) * nd;
float t = Vector3.Dot(vu, vc) * nd;

You’ll notice what is d? d means scale factor how difference near-plane parameters from n.

See a figure below.

As you can see two triangles in the figure. They are similarity triangles. We want to get parameters for frustum function. The function takes 6 parameters, l,r,t,b and n, f. n means how far near-plane from view point, f means how far far-plane from view point. It will be taken same values of camera setting.

But l,r,t,b are multiplied scale that calculated by d. Because l,r,t,b are same ratio in near-plane’s l,r,t,b parameters. So if they are multiplied by n/d then that converted to correct values.

It’s to time to implement these code on the Unity. I will show all of code first.

All code

private void UpdateFrustrum()
{
    float n = _camera.nearClipPlane;
    float f = _camera.farClipPlane;
    Vector3 pa = _pa.position;
    Vector3 pb = _pb.position;
    Vector3 pc = _pc.position;
    Vector3 pe = _pe.position;
    // Compute an orthonormal basis for the screen.
    Vector3 vr = (pb - pa).normalized;
    Vector3 vu = (pc - pa).normalized;
    Vector3 vn = Vector3.Cross(vu, vr).normalized;
    // Compute the screen corner vectors.
    Vector3 va = pa - pe;
    Vector3 vb = pb - pe;
    Vector3 vc = pc - pe;
    // Find the distance from the eye to screen plane.
    float d = -Vector3.Dot(va, vn);
    // Find the extent of the perpendicular projection.
    float nd = n / d;
    float l = Vector3.Dot(vr, va) * nd;
    float r = Vector3.Dot(vr, vb) * nd;
    float b = Vector3.Dot(vu, va) * nd;
    float t = Vector3.Dot(vu, vc) * nd;
    // Load the perpendicular projection.
    _camera.projectionMatrix = Matrix4x4.Frustum(l, r, b, t, n, f);
    _camera.transform.rotation = Quaternion.LookRotation(-vn, vu);
}

Note that my implementation changed to calculate cross product because the Unity use left hand coordinate. Thus we should exchange arguments for cross product function.

Modification for this example

Notice the code above to use only frustum and rotation. The paper tells us how to calculate final matrix for perspective matrix, such as P, M and T.

T matrix means where view is. Think of that, Unity has a system to render the CG. The camera represent a view system, so T matrix means just where camera is located. Thus, we don’t need to calculate T matrix. The camera position means T matrix.

Next, M means to make move all of vertices to front of view. In other words, all of vertices are located in front of view. This means that matrix rotate the camera. so we can rotate the camera via Unity API, just use Quaternion.LookRotation.

Quaternion.LookRotationtakes 2 arguments. First one is the vector forward the camera (-vn). Second one is up vector ( vu). As result, the quaternion means how to rotate the camera to be perpendicular to the plane.

To use objects detection with the frustum

You can use objects detection with this custom frustum easily.
Unity provides an API for object detection like below.

private void CheckTargets()
{
    Plane[] planes = GeometryUtility.CalculateFrustumPlanes(_camera);
    foreach (var col in _colliders)
    {
        col.GetComponent<Renderer>().material.color = Color.white;
        
        if (GeometryUtility.TestPlanesAABB(planes, col.bounds))
        {
            col.GetComponent<Renderer>().material.color =  Color.blue;
        }
    }
}

This result is below.

The important things are GeometryUtility.CalculateFrustumPlanes and GeometryUtility.TestPlanesAABB methods.

These are used for object detection in the frustum. As you can see above video, if you give a custom values for Matrix4x4.Frustum method, the API is still correctly.

Also it can be used zooming

I said above that to multiplied by n/d to generalize values. If we multiply scale to the n/d value then the camera frustum is scaled. I’ll show you it at below video.

It’s useful functionality.