{"id":249,"date":"2016-04-22T22:37:21","date_gmt":"2016-04-22T22:37:21","guid":{"rendered":"https:\/\/projects-old.etc.cmu.edu\/vr-rehearsal\/?p=249"},"modified":"2016-04-25T21:26:28","modified_gmt":"2016-04-25T21:26:28","slug":"graphics-optimization-1-lighting","status":"publish","type":"post","link":"https:\/\/projects-old.etc.cmu.edu\/vr-rehearsal\/index.php\/2016\/04\/22\/graphics-optimization-1-lighting\/","title":{"rendered":"Graphics & Optimization #1: Lighting"},"content":{"rendered":"

It is always necessary to squeeze out every drop of performance in order to run a 3D app on an Android phone. Running the app in VR even doubles the necessity and difficulty.\u00a0Fortunately, for VR Rehearsal, the fact that the position of presenter and virtual audiences in the environment remains still makes it possible to remarkably optimize the graphics performance in several aspects. We managed to render more than 50 animated humanoid models in a realistic environment with decent framerate by applying a number of tricks.<\/p>\n

Because our environments are completely static, lightmap is the nice solution which pre-computes the lighting data in a texture even with GI information. Then in real-time, the shader can get the lighting by simply querying the lightmap texture. The following code is the diffuse shader used for rendering the environment that reads the lightmap data to apply lighting:<\/p>\n

 <\/p>\n

Shader “VR_Rehearsal_app\/LightmapDiffuse”
\n{
\nProperties
\n{
\n_MainTex (“Texture”, 2D) = “white” {}
\n}
\nSubShader
\n{
\nTags
\n{
\n“RenderType”=”Opaque”
\n“Queue” = “Geometry”
\n“IgnoreProjector” = “True”<\/p>\n

}
\nLOD 100<\/p>\n

Pass
\n{
\nName “Lightmap”
\nLighting Off<\/p>\n

CGPROGRAM
\n#pragma vertex vert
\n#pragma fragment frag
\n#pragma fragmentoption ARB_precision_hint_fastest
\n\/\/ make fog work
\n\/\/#pragma multi_compile_fog<\/p>\n

#include “UnityCG.cginc”<\/p>\n

struct appdata
\n{
\nhalf4 vertex : POSITION;
\nhalf2 uv : TEXCOORD0;
\nhalf2 uv_lm : TEXCOORD1;
\n};<\/p>\n

struct v2f
\n{
\nhalf4 vertex : SV_POSITION;
\nhalf4 uv : TEXCOORD0; \/\/xy: texture uv; zw: lightmap uv
\n\/\/UNITY_FOG_COORDS(1)
\n};<\/p>\n

uniform sampler2D _MainTex;
\nuniform half4 _MainTex_ST;<\/p>\n

v2f vert (appdata v)
\n{
\nv2f o;
\nUNITY_INITIALIZE_OUTPUT(v2f, o);
\no.vertex = mul(UNITY_MATRIX_MVP, v.vertex);
\no.uv.xy = TRANSFORM_TEX(v.uv, _MainTex);
\no.uv.zw = v.uv_lm *unity_LightmapST.xy + unity_LightmapST.zw;
\n\/\/UNITY_TRANSFER_FOG(o,o.vertex);
\nreturn o;
\n}<\/p>\n

fixed4 frag (v2f i) : SV_Target
\n{
\n\/\/ sample the texture
\nfixed4 col = tex2D(_MainTex, i.uv);
\n\/\/ apply fog
\ncol.rgb *= DecodeLightmap(UNITY_SAMPLE_TEX2D(unity_Lightmap, i.uv.zw));
\n\/\/UNITY_APPLY_FOG(i.fogCoord, col);
\nreturn col;
\n}
\nENDCG
\n}
\n}
\n}<\/p>\n

 <\/p>\n

The fragment shader here is extremely simple: one line for reading diffuse texture, another line for reading lightmap, and multiple them together (We can apply the fog as well without too much pain). However, the resulting quality is surprisingly good:<\/p>\n

\"rpis\"

Large auditorium environment<\/p><\/div>\n

\"conf\"

Small conference room environment<\/p><\/div>\n

However the reality isn\u2019t always so easy. It took us long time to tweak the parameters of Unity\u2019s lightmap system in order to generate good lightmaps for our scenes. At first we were troubled by the problem that some triangles receives light bleeding in from the lightmap for another triangle. This was solved by closing all the edges of the model, and increasing the margin between lightmap patches. Then we experienced strange stripe artifacts on the walls of the conference room environment. It turned out that the problem was from inappropriate shadow parameters, and was solved by increasing the baked shadow radius of all the scene lights.<\/p>\n

 <\/p>\n

We\u2019ve removed real-time light calculation for static environment, but what about characters with animations that are instantiated in runtime? It is impossible to pre-compute lightmap for moving objects in real time. But fortunately, Unity also provides light probe which is based on the idea that the lighting at any position can then be approximated by interpolating between discrete samples. The interpolation is far cheaper than real lighting, yet still produces a reasonably good quality. All we need to do is to setup our sampling points in the scene:<\/p>\n

\"lp_conf\"

Setting up light probes in conference room<\/p><\/div>\n

\"lp_rpis\"

Setting up light probes in auditorium<\/p><\/div>\n

and use those information to interpolate lighting in our shader. The interpolation can be either done in vertex shader or fragment shader. Obviously, we choose to do that in vertex shader:<\/p>\n


\nShader \"VR_Rehearsal_app\/LightProbeDiffuse\"
\n{
\nProperties
\n{
\n_MainTex (\"Albedo (RGB)\", 2D) = \"white\" {}
\n}
\nSubShader
\n{
\nTags
\n{
\n\"RenderType\"=\"Opaque\"
\n\"Queue\" = \"Geometry\"
\n\"IgnoreProjector\" = \"True\"
\n\"BW\"=\"TrueProbes\"
\n}
\nLOD 100<\/code><\/p>\n

Pass
\n{
\nName “LightProbe”
\nTags
\n{
\n“LightMode” = “ForwardBase”
\n}
\n\/\/Fog
\n\/\/{ Mode Global }
\nCGPROGRAM<\/p>\n

#pragma vertex vert
\n#pragma fragment frag
\n#pragma fragmentoption ARB_precision_hint_fastest
\n#include “UnityCG.cginc”
\n#include “Lighting.cginc”<\/p>\n

struct appdata
\n{
\nhalf4 vertex : POSITION;
\nfixed3 normal : NORMAL;
\nhalf2 texcoord : TEXCOORD0;
\n};<\/p>\n

struct v2f
\n{
\nhalf4 pos : SV_POSITION;
\nhalf2 uv : TEXCOORD0;
\n\/\/fixed4 vLightingFog : TEXCOORD1; \/\/xyz: vertex light color; (w: vertex fog data)
\nfixed3 vLightingFog : TEXCOORD1; \/\/xyz: vertex light color; (w: vertex fog data)
\n};<\/p>\n

uniform sampler2D _MainTex;
\nuniform half4 _MainTex_ST;<\/p>\n

v2f vert(appdata v)
\n{
\nv2f output;
\nUNITY_INITIALIZE_OUTPUT(v2f, output);<\/p>\n

output.pos = mul(UNITY_MATRIX_MVP, v.vertex);
\noutput.uv = TRANSFORM_TEX(v.texcoord, _MainTex);<\/p>\n

half3 worldPos = mul((float3x3)_Object2World, v.vertex.xyz);
\nfixed3 worldN = mul((float3x3)_Object2World, v.normal);<\/p>\n

\/\/light probe
\noutput.vLightingFog.xyz = ShadeSH9 (float4(worldN,1.0));<\/p>\n

\/\/output.vLightingFog.w = exp(-length(_WorldSpaceCameraPos – worldPos) * unity_FogParams.x);
\nreturn output;
\n}<\/p>\n

fixed4 frag(v2f input) : SV_TARGET
\n{
\nfixed4 col = tex2D(_MainTex, input.uv) * fixed4(input.vLightingFog.xyz, 1.0);
\nreturn col;
\n\/\/return lerp(unity_FogColor, col, input.vLightingFog.w);
\n}
\nENDCG<\/p>\n

}<\/p>\n

}<\/p>\n

FallBack Off
\n\/\/FallBack “Mobile\/VertexLit”
\n}<\/p>\n

 <\/p>\n

Notice the function ShadeSH9 in vertex shader is the interpolation function. And the fragment shader is still simple: read the diffuse texture and multiple the color by per-vertex interpolated lighting. The following pictures show the shaded character using light probe and normal diffuse shader.<\/p>\n

\"Left:

Left: Light probed diffuse shader. Right: Unity standard shader.<\/p><\/div>\n

Now we have precomputed lighting data (either lightmap or lightprobe) for different environments, the last thing to do is to load them correctly. We need to deal with dynamically loading those data because we generate anything in the same scene (which is better to manage than multiple scenes). However, it turns out that unity by default cannot bake lightmaps into prefabs, and it cannot automatically load multiple sets of lightprobe. We end up writing our own helper scripts to bake lightmaps into prefabs based on this thread: http:\/\/forum.unity3d.com\/threads\/problems-with-instantiating-baked-prefabs.324514\/<\/a>, and exporting all the lightprobe information as binary files.<\/p>\n","protected":false},"excerpt":{"rendered":"

It is always necessary to squeeze out every drop of performance in order to run a 3D app on an Android phone. Running the app in VR even doubles the necessity and difficulty.\u00a0Fortunately, for VR Rehearsal, the fact that the position of presenter and virtual audiences in the environment remains still makes it possible to Read more about Graphics & Optimization #1: Lighting<\/span>[…]<\/a><\/p>\n","protected":false},"author":6,"featured_media":273,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"pgc_meta":"","footnotes":""},"categories":[1],"tags":[],"class_list":["post-249","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-vr-rehearsal"],"_links":{"self":[{"href":"https:\/\/projects-old.etc.cmu.edu\/vr-rehearsal\/index.php\/wp-json\/wp\/v2\/posts\/249"}],"collection":[{"href":"https:\/\/projects-old.etc.cmu.edu\/vr-rehearsal\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/projects-old.etc.cmu.edu\/vr-rehearsal\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/projects-old.etc.cmu.edu\/vr-rehearsal\/index.php\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/projects-old.etc.cmu.edu\/vr-rehearsal\/index.php\/wp-json\/wp\/v2\/comments?post=249"}],"version-history":[{"count":20,"href":"https:\/\/projects-old.etc.cmu.edu\/vr-rehearsal\/index.php\/wp-json\/wp\/v2\/posts\/249\/revisions"}],"predecessor-version":[{"id":297,"href":"https:\/\/projects-old.etc.cmu.edu\/vr-rehearsal\/index.php\/wp-json\/wp\/v2\/posts\/249\/revisions\/297"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/projects-old.etc.cmu.edu\/vr-rehearsal\/index.php\/wp-json\/wp\/v2\/media\/273"}],"wp:attachment":[{"href":"https:\/\/projects-old.etc.cmu.edu\/vr-rehearsal\/index.php\/wp-json\/wp\/v2\/media?parent=249"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/projects-old.etc.cmu.edu\/vr-rehearsal\/index.php\/wp-json\/wp\/v2\/categories?post=249"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/projects-old.etc.cmu.edu\/vr-rehearsal\/index.php\/wp-json\/wp\/v2\/tags?post=249"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}