Category: HPL3

Originally posted by Luis.

Hey there! Vacation time is really close now and there’s so much to do before it arrives, but I still wanted to drop by and keep you posted on new editor stuff, so I’m gonna keep it short this time.

We have a fly mode for the perspective camera now! Yes, and it works nicely enough. No video today, but enjoy this animated GIF instead:

Originally posted by Luis.

Hey! Today I’m sharing the results of the last two weeks of work on the ParticleEditor with you. I’ve had loads of time for additions and improvements — I’m just gonna go over the most notable ones briefly:

• Live update of particles: No more resetting the whole particle system to see the effect that little parameter you changed actually has!
• Control of the update speed: Something looks weird but you can’t spot what it is? Just slow the whole thing down. Works everytime.
• Easing functions for fading values: We don’t have tweakable curve controls just yet, but these work like a charm in the meantime. 
• Helper graphs: A really nice addition so you can preview how fades are going to work. Together with easing functions, a slider and the live update, this is fun just to play around with.

Of course, a little video works way better than words for showing these off, so here it is:

Originally posted by Peter.

HPL3 is our first engine to support both PC and consoles. To make it easy to support multiple platforms and multiple shading languages we have decided to use our own shading language called HPSL. Shader code written in HPSL goes through a shader parser to translate it to the language used by the hardware.

The shader written in HPSL is loaded into the engine at runtime, the code is then run through a preprocess parser that strips away any code that is not needed by the effect or material. After that the stripped code is translated to the language used by the hardware (GLSL #330 on PC and PSSL on the PS4) and then compiled.

HPSL uses the same syntax as the scripting or engine code. HPSL is based on GLSL #330 but some of the declarations are closer to HLSL.

// Example code

@ifdef UseTexture

uniform cTexture2D aColorMap : 0;

@endif



void main(in cVector4f px_vPosition,

                in cVector4f px_vColor,

                in cVector4f px_vTexCoord0,

                out cVector4f out_vColor : 0)

{

          cVector4f vColor = px_vColor;





@ifdef UseTexture

                    vColor *= sample(aColorMap, px_vTexCoord0.xy);

          @endif



          out_vColor = vColor;

}



//Preproccess step

void main(in cVector4f px_vPosition,

                in cVector4f px_vColor,

                in cVector4f px_vTexCoord0,

                out cVector4f out_vColor : 0)

{

          cVector4f vColor = px_vColor;



          out_vColor = vColor;

}




// Translation step

#version 330

#extension GL_ARB_explicit_attrib_location : enable



in vec4 px_vColor;

in vec4 px_vTexCoord0;

layout(location = 0) out vec4 out_vColor;



void main()

{

          vec4 px_vPosition = gl_FragCoord;

          bool px_bFrontFacing = gl_FrontFacing;

          int px_lPrimitiveID = gl_PrimitiveID;



          vec4 vColor = px_vColor;



          out_vColor = vColor;

}

Preprocessing

All the shader code used in SOMA is handwritten. In order to keep all the relevant code at the same place and to be able to quickly optimize shaders HPL3 uses a preprocessing step. This has been used for our previous games as well. A preprocessor goes thorugh the code and removes large chunks that are not needed or used by the effect or material. The lighting shader used in SOMA contains code used by all the different light types. Changing a preprocess variable can change a light from a point light to a spotlight or can be used to enable shadow mapping. The preprocessor strips blocks of code that are not used, this increases performance since code that has no visual effects is removed completely. Another feature of the preprocess parser is the ability to change the value of a constant variable, this can be used to change the quality of an effect.

// SSAO code

for(float d = 0.0; d < $kNumSamples; d+=4.0)
{
          // perform SSAO…
}

The preprocessor makes it easy to do complex materials with multiple textures and shading properties while only performing the heavy computations for the materials that need it.

Translation

After the preprocess strips the code it is ready to get translated. In the first step all the variable types and special functions are converted to the new language. Then the main entry function is created and all the input and output is bound to the correct semantics. In the last step the translated code is scanned for texture and buffers that get bound to the correct slot. 

Compilation

The translated code is then compiled. If a compilation error occurred the translated code is printed to the log file along with the error message and corresponding row for easy debugging.

Summary

In order to deliver the same visual experience to all platforms and to make development faster we decided on using our own shading language. The code is translated to the language used by the hardware and compiled at runtime. Supporting other shading languages in the future will be very easy since we only need to add another converter. 

HPSL translates to GLSL #330 which requires OpenGL 3.3 (DirectX 10 feature set). This means that SOMA will require a DirectX 10 or newer graphic card.

Modders will still be able to write shader code directly in GLSL if they chose to.

HPSL Reference

Syntax

HPSL uses the same syntax used by the scripting language.

A texture contains both the image and information about what happens when it is sampled. If you are used to OpenGL/GLSL then this is nothing new. DirectX uses a different system for storing this information. It uses a texture for storing the data and a separate sampler_state that controls filtering and clamping. Using the combined format makes it easy to convert to either GLSL or HLSL.

Textures need to be bound to a slot at compilation time. Binding is done by using the “:” semantic after the texture name.

//bind diffuse map to slot 0
uniform cTexture2D aDiffuseMap : 0;

Entry Point and Semantics

The entry point of a shader program is the “void main” function. Input and output of the shader is defined as arguments to this function. The input to the vertex shader comes from the mesh that is rendered. This might be information like the position, color and uv mapping of a vertex. What the vertex shader outputs is user defined, it can be any kind of information that the pixel shader needs. The output of the vertex shader is what gets sent to the pixel shader as input. The variables are interpolated between the vertices of the triangle. The input of the pixel shader and the output of the vertex shader must be the same or else the shaders won’t work together. Finally the output of the pixel shader is what is shown on the screen. The pixel shader can output to a of maximum 4 different render targets at the same time.

Some of the input and output are System defined semantics. System Semantics are set or used by the hardware. 

Input to the vertex shader is user defined. HPL3 has a few user defined semantics that work with our mesh format.

It is possible to add more user defined semantics if needed:

//vertex shader

uniform cMatrixf a_mtxModelViewProjection;



void main(in cVector4f vtx_vPosition,

               in cVector4f vtx_vColor,

               in cVector4f vtx_vTexCoord0,

               out cVector4f px_vColor,

               out cVector4f px_vTexCoord0,

              out cVector4f px_vPosition)

{                          

          px_vPosition = mul(a_mtxModelViewProjection, vtx_vPosition);

          px_vColor = vtx_vColor;

                             px_vTexCoord0 = vtx_vTexCoord0;

}



//pixel shader

uniform cTexture2D aColorMap : 0;



void main(in cVector4f px_vPosition,

               in cVector4f px_vColor,

               in cVector4f px_vTexCoord0,

               out cVector4f out_vColor : 0)

{

         out_vColor = px_vColor * sample(aColorMap, px_vTexCoord0.xy);

}

Functions

HPSL is based on OpenGL 3.3 and GLSL version 330 and supports almost all of the GLSL arithmetic functions.

There are some functions that are different from GLSL. This is to make it easier to support HLSL and PSSL.

Texture sampling use functions specific to the HPSL language.

It is also possible to use language specific code directly. Some languages and graphic cards might have functions that are more optimized for those systems and then it might be a good idea to write code specific for that language.

@ifdef Lang_GLSL
                  vec4 vModifier = vec4(lessThan(vValue, vLimit));
@else
                  cVector4f vModifier = step(vValue, vLimit);
@endif

Originally posted by Luis.

So I’m back with another HPL LevelEditor feature update, the Poser EditMode.

Back in the Amnesia days, whenever we wanted to add details which implied a unique skeletal model with different poses, we had to go all the way back to the modelling tool, set them up there and save them as a different mesh files. So we pretty much ended up with lots of replicated and redundant data (the corpse piles come to mind as I’m writing this), plus the burden of having to prepare everything outside the editor.

So, how to fix this? This is where the poser mode comes in. In a nutshell, it takes a skeletal mesh and exposes its bones to be translated and rotated in whatever way you fancy. This is useful not only for organic and creature geometry, but to create details like cables, piping and anything you can add a skeleton to.

There’s not much to be added, so see it in action in this little video.

Some links in this article have expired and have been removed.

Screen space ambient occlusion (SSAO) is the standard solution for approximating ambient occlusion in video games. Ambient occlusion is used to represent how exposed each point is to the indirect lighting from the scene. Direct lighting is light emitted from a light source, such as a lamp or a fire. The direct light then illuminates objects in the scene. These illuminated objects make up the indirect lighting. Making each object in the scene cast indirect lighting is very expensive. Ambient occlusion is a way to approximate this by using a light source with constant color and information from nearby geometry to determine how dark a part of an object should be. The idea behind SSAO is to get geometry information from the depth buffer.

There are many publicised algorithms for high quality SSAO. This tech feature will instead focus on improvements that can be made after the SSAO has been generated.

SSAO Algorithm

SOMA uses a fast and straightforward algorithm for generating medium frequency AO. The algorithm runs at half resolution which greatly increases the performance. Running at half resolution doesn’t reduce the quality by much, since the final result is blurred.

For each pixel on the screen, the shader calculates the position of the pixel in view space and then compares that position with the view space position of nearby pixels. How occluded the pixel gets is based on how close the points are to each other and if the nearby point is in front of the surface normal. The occlusion for each nearby pixel is then added together for the final result. 

SOMA uses a radius of 1.5m to look for nearby points that might occlude. Sampling points that are outside of the 1.5m range is a waste of resources, since they will not contribute to the AO. Our algorithm samples 16 points in a growing circle around the main pixel. The size of the circle is determined by how close the main pixel is to the camera and how large the search radius is. For pixels that are far away from the camera, a radius of just a few pixels can be used. The closer the point gets to the camera the more the circle grows – it can grow up to half a screen. Using only 16 samples to select from half a screen of pixels results in a grainy result that flickers when the camera is moving.

Bilateral Blur

Blurring can be used to remove the grainy look of the SSAO. Blur combines the value of a large number of neighboring pixels. The further away a neighboring pixel is, the less the impact it will have on the final result. Blur is run in two passes, first in the horizontal direction and then in the vertical direction.

The issue with blurring SSAO this way quickly becomes apparent. AO from different geometry leaks between boundaries causing a bright halo around objects. Bilateral weighting can be used to fix the leaks between objects. It works by comparing the depth of the main pixel to the depth of the neighboring pixel. If the distance between the depth of the main and the neighbor is outside of a limit the pixel will be skipped. In SOMA this limit is set to 2cm.

To get good-looking blur the number of neighboring pixels to sample needs to be large. Getting rid of the grainy artifacts requires over 17×17 pixels to be sampled at full resolution.

Temporal Filtering 

Temporal Filtering is a method for reducing the flickering caused by the low number of samples. The result from the previous frame is blended with the current frame to create smooth transitions. Blending the images directly would lead to a motion-blur-like effect. Temporal Filtering removes the motion blur effect by reverse reprojecting the view space position of a pixel to the view space position it had the previous frame and then using that to sample the result. The SSAO algorithm runs on screen space data but AO is applied on world geometry. An object that is visible in one frame may not be seen in the next frame, either because it has moved or because the view has been blocked by another object. When this happens the result from the previous frame has to be discarded. The distance between the points in world space determines how much of the result from the previous frame should be used.

Temporal Filtering introduces a new artifact. When dynamic objects move close to static objects they leave a trail of AO behind. Frostbite 2’s implementation of Temporal Filtering solves this by disabling the Temporal Filter for stable surfaces that don’t get flickering artifacts. I found another way to remove the trailing while keeping Temporal Filter for all pixels.

Temporal Blur

I came up with a new way to use Temporal Filtering when trying to remove the trailing artifacts. By combining two passes of cheap blur with Temporal Filtering all flickering and grainy artifacts can be removed without leaving any trailing. 

When the SSAO has been rendered, a cheap 5×5 bilateral blur pass is run on the result. Then the blurred result from the previous frame is applied using Temporal Filtering. A 5×5 bilateral blur is then applied to the image. In addition to using geometry data to calculate the blending amount for the Temporal Filtering the difference in SSAO between the frames is used, removing all trailing artifacts. 

Applying a blur before and after the Temporal Filtering and using the blurred image from the previous frame results in a very smooth image that becomes more blurred for each frame, it also removes any flickering. Even a 5×5 blur will cause the resulting image to look as smooth as a 64×64 blur after a few frames.

Because the image gets so smooth the upsampling can be moved to after the blur. This leads to Temporal Blur being faster, since running four 5×5 blur passes in half resolution is faster than running two 17×17 passes in full resolution. 

Upsampling

All of the previous steps are performed in half resolution. To get the final result it has to be scaled up to full resolution. Stretching the half resolution image to twice its size will not look good. Near the edges of geometry there will be visible bleeding; non-occluded objects will have a bright pixel halo around them. This can be solved using the same idea as the bilateral blurring. Normal linear filtering is combined with a weight calculated by comparing the distance in depth between the main pixel and the depth value of the four closest half resolution pixels.

Summary 

Combining SSAO with the Temporal Blur algorithm produces high quality results for a large search radius at a low cost. The total cost of the algoritm is 1.1ms (1920×1080 AMD 5870). This is more than twice as fast as a normal SSAO implementation.

SOMA uses high frequency AO baked into the diffuse texture in addition to the medium frequency AO generated by the SSAO.

Temporal Blur could be used to improve many other post effects that need to produce smooth-looking results.

Ambient Occlusion is only one part of the rendering pipeline, and it should be combined with other lighting techniques to give the final look.

References

1. http://gfx.cs.princeton.edu/pubs/Nehab_2007_ARS/NehEtAl07.pdf
2. http://dice.se/wp-content/uploads/GDC12_Stable_SSAO_In_BF3_With_STF.pdf

Code Appendix

 // SSAO Main loop



//Scale the radius based on how close to the camera it is

 float fStepSize = afStepSizeMax * afRadius / vPos.z; 
 float fStepSizePart = 0.5 * fStepSize / ((2 + 16.0));    



 for(float d = 0.0; d < 16.0; d+=4.0)
 {
        //////////////
        // Sample four points at the same time
        vec4 vOffset = (d + vec4(2, 3, 4, 5))* fStepSizePart;
        

        //////////////////////
        // Rotate the samples
        vec2 vUV1 = mtxRot * vUV0;
        vUV0 = mtxRot * vUV1;

        vec3 vDelta0 = GetViewPosition(gl_FragCoord.xy + vUV1 * vOffset.x) - vPos;
        vec3 vDelta1 = GetViewPosition(gl_FragCoord.xy - vUV1 * vOffset.y) - vPos;
        vec3 vDelta2 = GetViewPosition(gl_FragCoord.xy + vUV0 * vOffset.z) - vPos;
        vec3 vDelta3 = GetViewPosition(gl_FragCoord.xy - vUV0 * vOffset.w) - vPos;

        vec4 vDistanceSqr = vec4(dot(vDelta0, vDelta0),
                                 dot(vDelta1, vDelta1),
                                 dot(vDelta2, vDelta2),
                                 dot(vDelta3, vDelta3));

        vec4 vInvertedLength = inversesqrt(vDistanceSqr);

        vec4 vFalloff = vec4(1.0) + vDistanceSqr * vInvertedLength * fNegInvRadius;

        vec4 vAngle = vec4(dot(vNormal, vDelta0),
                            dot(vNormal, vDelta1),
                            dot(vNormal, vDelta2),
                            dot(vNormal, vDelta3)) * vInvertedLength;


        ////////////////////
        // Calculates the sum based on the angle to the normal and distance from point
        fAO += dot(max(vec4(0.0), vAngle), max(vec4(0.0), vFalloff));

}



//////////////////////////////////
// Get the final AO by multiplying by number of samples

fAO = max(0, 1.0 - fAO / 16.0);



------------------------------------------------------------------------------ 



// Upsample Code

 
vec2 vClosest = floor(gl_FragCoord.xy / 2.0);
vec2 vBilinearWeight = vec2(1.0) - fract(gl_FragCoord.xy / 2.0);

float fTotalAO = 0.0;
float fTotalWeight = 0.0;

for(float x = 0.0; x < 2.0; ++x)
for(float y = 0.0; y < 2.0; ++y)
{
       // Sample depth (stored in meters) and AO for the half resolution  
       float fSampleDepth = textureRect(aHalfResDepth, vClosest + vec2(x,y)); 
       float fSampleAO = textureRect(aHalfResAO, vClosest + vec2(x,y));

       // Calculate bilinear weight 
       float fBilinearWeight = (x-vBilinearWeight .x) * (y-vBilinearWeight .y);
       // Calculate upsample weight based on how close the depth is to the main depth
       float fUpsampleWeight = max(0.00001, 0.1 - abs(fSampleDepth – fMainDepth)) * 30.0;

       // Apply weight and add to total sum 
       fTotalAO += (fBilinearWeight + fUpsampleWeight) * fSampleAO;
       fTotalWeight += (fBilinearWeight + fUpsampleWeight);
}

// Divide by total sum to get final AO
float fAO = fTotalAO / fTotalWeight;



-------------------------------------------------------------------------------------



// Temporal Blur Code



//////////////////
// Get current frame depth and AO
vec2 vScreenPos = floor(gl_FragCoord.xy) + vec2(0.5);
float fAO = textureRect(aHalfResAO, vScreenPos.xy);

float fMainDepth = textureRect(aHalfResDepth, vScreenPos.xy);   



//////////////////
// Convert to view space position

vec3 vPos = ScreenCoordToViewPos(vScreenPos, fMainDepth);



/////////////////////////
// Convert the current view position to the view position it 

// would represent the last frame and get the screen coords
vPos = (a_mtxPrevFrameView * (a_mtxViewInv * vec4(vPos, 1.0))).xyz;

vec2 vTemporalCoords = ViewPosToScreenCoord(vPos);
        
//////////////
// Get the AO from the last frame
float fPrevFrameAO = textureRect(aPrevFrameAO, vTemporalCoords.xy);
float fPrevFrameDepth = textureRect(aPrevFrameDepth, vTemporalCoords.xy);



/////////////////
// Get to view space position of temporal coords
vec3 vTemporalPos = ScreenCoordToViewPos(vTemporalCoords.xy, fPrevFrameDepth);
        
///////
// Get weight based on distance to last frame position (removes ghosting artifact)
float fWeight = distance(vTemporalPos, vPos) * 9.0;



////////////////////////////////




// And weight based on how different the amount of AO is (removes trailing artifact)

// Only works if both fAO and fPrevFrameAO is blurred 
fWeight += abs(fPrevFrameAO - fAO ) * 5.0;

////////////////
// Clamp to make sure atleast 1.0 / FPS of a frame is blended
fWeight = clamp(fWeight, afFrameTime, 1.0);       


fAO = mix(fPrevFrameAO , fAO , fWeight);
   

------------------------------------------------------------------------------

Originally posted by Luis.

Hey there! So I finally dropped by to write a quick post on our stuff after quite a long hiatus (exactly 3 years, 3 months and 8 days long according to my last post) and Thomas pretty much took over the blog. Right now most of you might be wondering who’s typing even . The name is Luis and I mainly code the tools we use here at Frictional (also the ones you might have been using for making your own Amnesia custom stories) After the quick introduction, here comes a little editor feature showoff.

If you ever used the Amnesia tools yourself, it’s more than probable that you already know about this little buddy here:

One could say it is functional. I mean, you actually can pick a color by using it. But it had a little problem. It sucked, and real bad I must add.

When you are editing a level and need to pick a color for anything, be it the diffuse color for a light or the color multiplier for some illumination texture, you are probably going to need a nice system that allows for quick tweaking with not much sweat. This is definitely not the case of our old color picker, mainly because of two reasons: the RGB color space and lack of immediate preview of changes.

Selecting a color in the RGB color space is pretty straightforward you might think. And it is indeed, you only need to set the values for the Red, Green and Blue components and you are all set. That’s it. Okay, but what if you need to choose from a range for a particular color tonality? How are you supposed to know which RGB values you need to set for that? Summing up, RGB is a pretty uninintuitive system when it comes to edition, so we pretty much had to update the color picker to use the HSB color model as well.

This model describes colors as follows:

• the H value, ranging from 0 to 360, controls the hue for the resulting color; 
• S, which stands for saturation and ranges from 0 to 100, indicates, in layman terms, how washed the color will look, with 0 being white and 100 equalling the original hue color (for a B value of 100); 
• and finally B, standing for brightness and ranging from 0 to 100 as well, is quite self explanatory and sets how bright the final color will be, with 0 meaning black and 100 meaning the color that is set by the other 2 parameters.

As for the immediate preview, anyone who used the old picker knows that you had to exit the picker in order to apply and see the changes. When you need to tweak things, which involves constantly changing values and checking out how it looks, this setup is bound to get old and annoying way quicker than you would want it to. To fix this, the new picker can change the target color on the fly, letting the user see the end result in the scene while modifying the values.

Another upgrade that speeds things up a lot is going as visual as possible. No matter which parameter you are editing, being able to change it via a slider or some other graphical tool that you can click on will always help tons. The drawback of these, though, is that it’s very likely that you spend more time on them than needed just because they are so fun to use.

After all this long-ish introduction, just say hi to our new color picker:

And see it in action in this short demo video: