Graphics in Video Games: Bloom

When developing a video game, regardless of the platform or if it will be 2D or 3D, the developers must consider how they will utilize the powers of different computer graphics effects. How developers manipulate image data to achieve different effects is absolutely critical for achieving different objectives in game development. An example of a powerful graphics effect is Bloom. Bloom is used to amplify light in a scene, create more realistic effects, or enhance the look and feel of something. This week, I will be discussing bloom in video games, and some GLSL shader code on achieving some bloom effects.

Syndicate: A newer game known for excessive bloom.

What is bloom?

Bloom, also known as glow or bloom light bloom is a computer graphics effect used to amplify light in a scene. As mentioned before, it can be used to enhance realism. The effect of bloom produces 'light bleeding' where bright light will extend onto other parts of the scene. In the image above, you can clearly see the light bleeding onto the play characters gun. This light bleeding effect adds to the illusion of a very bright light.

How can bloom change a game?

Bloom, in most cases, is simply used to enhance the visual appeal of something, whether it be an item, part of the environment, or otherwise. Bloom can also help define a games mood.While some games over-intensify their bloom, a good example of properly used bloom is Far Cry 3.

In Far Cry, the game takes place in a wild, dangerous, yet beautiful jungle across several different islands. In order to achieve different effects of beauty and wonder, as well as cool effects, bloom comes in handy.

In the image above, there are several cases of bloom being used here. Perhaps the first thing in this image someone may notice is the very well done explosion effect. However, using images for particles isn't enough. The light for the particles is being amplified to enhance the explosion so it draws more attention and looks more powerful. Along the outside is a glow to enhance the explosion. Also the interior is white very white. The excessive white we see is called threshold, an important component of bloom. Threshold, in a nutshell, makes colors closer to white more white and colors closer to black more black. This draws contrast and exaggerates (which is also a principle of animation) an effect a programmer is trying to achieve.

The other use of bloom is clearly in the background. The light is so intense that is whites out the background and adds blur as well. This is used to add realism to the scene and try and draw contrast between the mountains and sky. As mentioned with threshold, the mountains are made to be very dark while the sky very white. The light in so intense, you can see the bleeding of color onto the dark mountains.

Now that different uses of bloom in Fay Cry 3 have been discussed, let's move on to some GLSL fun.

How a programmer may code bloom

Seeing bloom and knowing generally how it works is one thing, but how to code it is another entirely. First and foremost, use FBO's to render everything out fast. Second, we want to understand what to use in order to get at least a basic bloom effect. We will need to control the lighting in the scene so we can rely on three different values: luminace, threshold, and middle grey. Middle grey is a controller for the strength of the luminace, allowing us to use different levels of luminace and threshold without going overboard.

Although controlling these would only be a very basic shader there is a problem. Just controlling the different light values is not enough to warrant for a proper bloom effect. Bloom also blurs the surroundings to give it that overbearing effect. For GLSL, we also have to consider down sampling and composite passes. Thus, we need a shader that handles the blur, down sampling, and composite passes.


Gaussian Blur

Gaussian Blur is a method in which to effectively blur in computer graphics. For bloom, it can be used to give the effect of an incredibly bright light that overcomes different parts of the scene, or the scene as a whole. For gaussian blur, we write our code so it is received in the fragment shader. We do this because we want to blur the pixel data itself, and not necessarily the have what is called a "kernel" that is a 3x3 square.

The kernel acts as a 3x3 matrix and our computer will filter the data. In the matrix, it takes the sum of all elements and divides it by the sum. So in the end, the source pixel is replaced by a weighted sum of itself and nearby pixels. This method is called convolution kernel filtering.

While we could take the weighted sum, this is not the only way either. An alternative is to have the kernel weighted stronger towards the center. So each pixel is not weighted on average, but towards the center means a larger number.

GLSL Code

Our original image

After discussing different variables we need to help control bloom, and an overview of Gaussian blur, we can start to work on some code. Using a fragment shader called brightPass, we need the following:

float luminance;
float middleGrey;
float threshold;

This will allow us to control the luminace of our scene or object, the "middleGrey" is used to control grey scale, and the threshold is used to control the white in the scene (as the Far Cry example). Then, we can calculate the lighting for our bloom scene using this shader.

void main()
{
vec3 colour = texture2D(inputImage, gl_TexCoord[0].st).rgb;

colour *= (middleGrey / luminance );
colour *= 1.0 + (colour / (threshold * threshold) );
colour -= 0.5;
colour /=(1.0 + colour);

gl_FragColor.rgb = colour;
}


The image below demonstrates a much more powerful illumination and blurs the scene slightly. This additional brightness, while may be ugly, can be useful in some gaming situations. For example, if the programmer wishes the player to be exposed to a massive amount of light very quickly, this type of high powered bloom can be useful.

However, this is only the brightPass fragment shader and in this, we have a blur effect as well down sampling and composite. For our blur fragment shader, as I mentioned before we need a kernel. However, we also need our image data and pixel size if we want to blur.


uniform sampler2D inputImage;
uniform vec2 pixelSize;


Now, we need a vector representing our box blur. In this vector, we will determine the strength of the blur at a location of x,y (hence vec2 pixelSize) and and output the 3D vector with the 2D blur values.


vec3 colourOut = vec3(0.0);

vec2 offset[9] = vec2[]( // Bottom of the kernal
vec2(-pixelSize.x, -pixelSize.y),
vec2( 0.0, -pixelSize.y),
vec2(+pixelSize.x, -pixelSize.y),

// Middle of the kernal
vec2(-pixelSize.x, 0.0),
vec2( 0.0, 0.0),
vec2(+pixelSize.x, 0.0),

//Top of the kernal
vec2(-pixelSize.x, +pixelSize.y),
vec2( 0.0, +pixelSize.y),
vec2(+pixelSize.x, +pixelSize.y)  );

for( int i = 0; i < 9; i++ )
{
colourOut += texture2D(inputImage, texCoord + offset[i] ).rgb;
}

return ( colourOut / 9.0);

In this, we take the pixel data at a pixel location, add the x and y values together and divide it so that the data at each part of the kernel is the weighted sum and it blurs. This is effectively our Gaussian blur shader. However, we are not done just yet.  We require just one more shader and this is the composite shader that will output for us the color of the scene and the bloom with the added blur. Because it outputs pixel data, it is a fragment shader.

First, we need the input data for the bloom and scene.


uniform sampler2D input_scene;
uniform sampler2D input_bloom;

Once these are declared, we can write our composite main.


vec3 screen(vec3 colour1, vec3 colour2)
{
return ( 1.0- (1.0 - colour1) * (1.0 - colour2) );
}

void main()
{
vec3 colourA = texture2D(input_scene, gl_TexCoord[0].st).rgb;
vec3 colourB = texture2D(input_bloom, gl_TexCoord[0].st).rgb;
gl_FragColor.rgb = screen(colourA, colourB);
}

In the end, this gives us a nice blur effect and a bloomed scene.

Before

After

In conclusion

Bloom in video games is a common method to enhance realism of effects and the environment the player is immersed within by amplifying the scenes lighting. It is a powerful tool that can hinder a game, or enhance the players experience.

I hope this blog taught you a thing or two about basic bloom and how you can implement it. Thank you for reading, don't forget to rate and comment!

Particles in Video Games: Part II

Last week I discussed the basics of particles in video games. This included a basic overview of what particles and particle systems are, different implementations,  and why particles are important in video games. However there are also limitations to what I discussed. CPU rendered particles are greatly inefficient and limited in their ability to look good. This week, I will discuss how to improve upon CPU based rendering by utilizing frame buffer objects (FBO's) and geometry shaders.

The premier way to render any graphics system is through the graphics processing unit, or simply the GPU. The GPU is designed to rapidly build images in a frame buffer to be displayed. GPU's are incredibly common in almost any modern day device such as: mobile phones, personal computers, game consoles, etc. GPU's are very effective at rendering computer graphics because of how it can take blocks of data and process it in parallel. For a particle system, being able to render out blocks of data in a fast and effective manner is critical to producing desirable particles. 

FBO's

The question is, for a particle system, why use FBO's to help render your particles? What is a FBO? A frame buffer object is an extension to OpenGL for doing off-screen rendering. This includes rendering to a texture. By taking what would normally be drawn to screen, it can be used to apply various types of post-processing effects and image filtering. That being said, FBO's are used primarily for post-processing of rendered images (for example bloom, blurring), and composition between different scenes.

Geometry Shaders

Geometry shaders also represent a way to improve your particle system.  A geometry shader, in short, is a GLSL shader that administrates the processing of primitives. In regards to the OpenGL pipeline, it happens after primitive assembly. They can also be used to create new primitives which in turn can be used to help render out textures to quads, a common practice for particle coders. Geometry shaders have several advantages:

• Unlike vertex shaders, a geometry shader can create new primitives.
• Can do layered rendering. This means it can render different primitives to different layers in the frame buffer.
• Excellent for rendering primitives.

How does this help programmers?

A particle programmer would be very wise to maximize the efficiency benefited from FBO's. Rendering to a texture will allow the programmer to create much more realistic effects, while applying different post-processing effects to give them the desired look. This is not the only benefit however, because of the additional speed and parallel rendering, a video game will be able to support many emitters handling many different particle systems that utilize all sorts of different effects. Not only will particles render faster, they can also achieve maximum appeal for the minimal cost.

A geometry shader is an incredibly powerful tool to a programmer. While FBO's assist with rendering, a geometry shader can create new primitives quickly. Many programmers will often render their particles to different primitives such as planes, spheres, or other shapes. This means that their emitter will render quickly because of FBO's and can also create new primitives using geometry shaders

Examples 

Crysis is a good example of a game that utilized geometry shaders in their particle systems.

In this image, the geometry shader could randomly generate the line bursts shooting outwards from the central explosion as well as trail size and intensity. This way, the game can have a unique explosion every time.

For similar reasons as above, the game can enhance player experience by adding in uniqueness to every explosion effect.

In Conclusion

To sum up this two-part blog about particles, they are a great method for achieving a wide array of different effects in video games. They can be used to set a mood in a game and to enhance a players experience. Particle systems, especially good ones, are most certainly a challenge to any proprogrammer. Thank you for reading, comment and rate!

Particles in Video Games: Part I

For our intermediate computer graphics class we have been assigned a various number of challenging homework questions. These questions range from Photoshop manipulations of our game, implementing different types of shaders, and many other game related tasks. Once question however greatly interests me. For it, we have to develop different particle systems. The only trick is that they have to look good! In this two part post, I will be discussing what are particles, what you need to build them, why they are important, and some coding applications.


What is a particle? A particle system?
 
Particles refer to a technique of simulating effects in computer graphics by the use of small sprites or other objects that would otherwise be hard to create. A particle can be represented by using:

• Points
• Sprites
• Different shapes (spheres for example)

With particles you can make any number of different effects such as fire, smoke, fireworks, hair, dust, stars in the night, rain, snow, and just about anything else you can of. In order to achieve these effects one needs a particle system.

A particle system refers to a collection of any number of particles that share attributes.  In video games, they are used to produce all sorts of amazing effects and some games are completely reliant on particles and  particle systems such as Geometry Wars.

An example of particles working with various emitters

Why are they so important?

Particles in video games can be used to achieve many different effects, but they can also be used to enhance or change a gamers experience. For example, a scene littered with fire and random explosions may suggest a war like area, or a gentle snow fall in a forest may give the player a feeling of serenity. They can set the mood, define an experience, make something epic.

With particles you can achieve just about anything. Many games will use them to their advantage to set a mood, such as the example video shows. It demonstrates the increasingly deadly threat of the current areas destruction with randomly explosions and fire.

While mood setting and achieving cool effects are some methods of using particles, they can also be used in a much more practical approach. Particles can be used for bullets, lasers, magical fireballs, and more. Often, in the case of games like Skyrim, when a character casts a magical spell, they are really firing a particle emitter that has collision detection (so we know when it hits an enemy). This type of usage can become a great tool for achieving many different effects for a game.

Coding application of particles


Particles and particle systems share different attributes in the 3D world. When developing a particle system, it is important to understand how each particle works and how they will be effect by particle physics.  Some attributes of a particle would be:

• Position
• Velocity
• Acceleration
• Life Span
• Size
• Color

The list goes on, but these would be amongst the most common you would see. With this, you can fully define an entire list of particles with different attributes to wield at your bidding (evil laugh here). You could implement a fire, water, or smoke style using all of these. Understanding this we can also define a proper particle system to achieve those effects. Particle systems control a group of particles that act randomly but share common attributes. They should be dynamic, not per-determined, changing form and moving over time to get a natural flow in your effect. Some common shared attributes of a particle system would be:

• Position
• Number of particles
• Emission rate
• Current state (is that tree burning or not?)
• Particle list (What type of particles are we using)

Typically particles are created from an 'emitter'. While not every single particle has to come from a single emitter, games will often use multiple emitters, if not potentially hundreds. Emitters determine (randomly) the rate, flow, etc., of particles.

Games like Empire: Total War would have taken hundreds
of particle emitters and thousands of particles into consideration.

Particle physics are reliant upon some kind of force. This could be gravity, wind, player influence, anything that may effect a particles acceleration. If we know the acceleration, we can calculate the velocity which then changes the particles position. Once again, this is very useful for the development of various effects.

Effective use of particles is trying to use as little particles as possible to achieve your desired effect. Many games will implement sprites to achieve a fire or smoke effect that uses a fade and color change to give it the proper effect that is needed. You can often see this effect with trees in games where a group of leaves are simply 2D sprites that are effected by the wind in the game. This saves on computation and can be used to make a realistic object in your game.

Sprites are an excellent way of getting a lot for very little. Smoke blooms are often generated by having a very small sprite that grows larger and fades over time to give the proper effect. Sprites were suggested to me by my TA for intermediate computer graphics, and after realizing this, it didn't take long to know just many games use sprites for their particles.

Conclusion

Particles are an excellent way of achieving literally hundreds if not thousands of different goals in video games. From smoke and fire, to mood, or more practical uses like a gun shot, game developers can use them to achieve many different things.

When developing a particle system, you have to take all of these attributes in consideration when creating it. Next week, I will finish off this two part talk about particles about discussing how FBO's and Geometry shaders can be used to improve a basic CPU particle system.

Thanks for reading, I hope you enjoyed it and learned something!

Lighting and Luna

As a UOIT Game Development and Entrepreneurship student it is a requirement to develop a game every year a team of other students. Over the years, Noble Studios, has developed games several and they have evolved from simple text-based, to 3D games. However, this year marks the first time we will implement shaders into the game to enhance the experience. In this post, I will discuss the possibilities of improving our current game through the use of lighting.

Noble Studios is developing a 2.5 game called Luna. As taken directly from our development webpage, "Play as a magical moon girl who journeys to the Earth searching for a little boy in hopes of finally finding friendship...but all is not right, as someone plots to make sure you will never return home." Currently Luna is a prototype and it features no shaders, only textures and no good lighting. See the example image below.

Our main character Luna with minimal lights.

In the image to the left, there are few shadows, character detail, or any sort of interesting lighting detail. This is because the lights are not properly set up to provide any higher level of detail. Currently, the scene looks very plain. We intend to increase the quality of our visuals by introducing vertex shaders to compute our lighting. We will be using various types of lighting including ambient, diffuse, and specular. By using these types of lighting, I will explain how we will improve our game.

Ambient

An ambient light is a non-directional light that determines the amount of lighting in a scene. In Luna, the game changes from broad daylight to night over the course of the levels. Knowing how our game progresses, we can alter the ambient light in order to achieve certain levels of light or dark between the stages.
In code, we can achieve this with a simple formula
Ambient = K * AmbientIllumination where k is a float value between 0 and 1.

Diffuse

Diffuse lighting is a method to apply shading to an object. While the ambient light applies lighting to an entire scene, a diffuse light will ensure a proper shade of that object. The image below demonstrates this.

We can use diffuse by remembering Lambert's Laws:

• The illuminance on a surface illuminated by a light falling on it perpendicularly from a point source is proportional to the inverse square of the distance between the surface and the source.
• If the rays meet on a surface at an angle, then the illuminance is proportional to the cosine of the angle with the normal.
• The luminous intensity of light decreases exponentially with the distance it travels through an asorbing medium

Using the following code, we can compute diffuse lighting

// Compute the diffuse term

  float3 L = normalize(lightPosition - P);

  float diffuseLight = max(dot(N, L), 0);

  float3 diffuse = Kd * lightColor * diffuseLight;

In our game, we can use this to apply realistic environment and character shading. Players will be able to determine the curvature of objects and the intensity of the lights. This is important because it adds a sense of realism to the characters, and it simply makes things more defined.

Specular lighting (also known as specular highlights)

Specular lighting is the bright spot of light that appears on shiny objects such as metal or water when illuminated. This is important in 3D graphics because it suggests the shape of an object and its position according to the light sources of the scene.

The image below is a good example of specular, diffuse, and ambient lighting. The brightest section of the water is the specular highlight. Along the edges of the highlights the light begins to diffuse. At the far right, it becomes darker because less light is reflecting off that surface. The ambient light is the sun, that provides light to the entire scene.
In our game, we can use specular highlights to show the brightest part of an object. This is useful because we can define an object not only will diffuse lighting, but add additional realism with specular highlights.
Using the following code, we can create a specular light

// Compute the specular term

  float3 V = normalize(eyePosition - P);

  float3 H = normalize(L + V);

  float specularLight = pow(max(dot(N, H), 0), shininess);

In conclusion

By using these three different types of lighting, we'll be able to achieve character and environment lighting that suits our games and its needs. In later posts, I will be describing more ways we will be able to improve our current prototype!

Introduction to Intermediate Computer Graphics and OpenGL

Hello, my name is Wesley Chartrand and I am a second year Game Development and Entrepreneurship student at UOIT. Here in Game Dev. we study a variety of different fields in computer graphics and the business world. In this blog, I will be focusing primary on my intermediate computer graphics class that focuses primarily on utilizing the power of OpenGL and shaders. I will also be demonstrating different programs I created throughout the program, as well as some videos from professionally made video games for your enjoyment and education. Let's get started.

Before I dive straight into talking about video games, fancy graphics, awesome particles, and more, I will first be talking about the foundation of in which many video games are programmed: OpenGL and shaders. Once we have a basic understanding of exactly what both accomplish we can understand how it is used to make video games look and feel awesome!

What is OpenGL?

OpenGL (Open Graphics Library) is a cross-language, multi-platform API used to render 2D and 3D graphics. What makes OpenGL so great is that it taps into the GPU to achieve high render speeds. With OpenGL we can create a 3D environment in which programmers can display whatever they wish to the viewer. In order to use OpenGL though, we use API like GLUT, GLEW, and SFML in order to effectively utilize the libraries in OpenGL

An example scene I programmed. Note: there is no lighting yet

As seen above, OpenGL can be used to create a scene, draw objects at specified x,y,z coordinates, add color, and more. All of these are rendered quickly and efficiently through the GPU. Below is an example of a particle system I developed.

These are just simple examples of some really easy programs that can be created with OpenGL. Many popular games have been created using this tool such as Half-Life, Minecraft, the Quake Series, Doom 3, and more. These are just a few examples of the power of the OpenGL tool. Utilizing shaders helps brings games to life.

What are Shaders?

Shaders, in computer graphics, are programs that run off of the GPU in order to do specialty tasks such as lighting, special affects, or post-processing. While most shaders calculate different levels of light or dark, there are numerous algorithms available or waiting to be discovered that allow for programmers to do all sorts of neat tricks. Some examples of where shading algorithms are useful: hue, saturation, brightness, contrast, blur, bloom, bump mapping, displacement mapping, and many more.

Overview of Shaders

Shaders are simple programs that describe the components of either a pixel or vertex. By describing their components, we can achieve many different goals. Two major different shaders we use are pixel shaders, also known as fragment shaders, and vertex shaders.

Pixel shaders compute things like color, z-depth, and alpha values of a pixel. Fragment shaders can do many things like bump mapping, applying light values, shadows, specular highlights, and other things. However, Fragment shaders have their limitations. They are restricted in how many effects they can do because it operates on a single fragment, and not necessarily the scenes entire geometry. Although, fragment shaders do know the screen coordinates that are being drawn, and can sample the screen and nearby pixels if the contents of the entire screen are passed as a texture to the shader. This technique allows for many two-dimensional post-processing effects, such as blur and cel-shading for cartoonists.

Phong Shading is often implemented using pixel or fragment shaders

Vertex shaders commonly describe position, color, texture co-ordinates, etc., of a vertex. Vertex shaders are called for each vertex in a primitive. Every vertex is then rendered out as a series of pixels that is transferred to the screen. It is important to note that vertex shaders influence currently drawn vertex's, they do not create new ones. Vertex shaders are a powerful tool for controlling position, lighting, color, and more.

The below images were provided by our Interactive Computer Graphics tutorial

Using a shader to create lighting

No lighting.

As of yet, we have learned about pixel shaders, also known as fragment shaders, and vertex shaders.

Alright so we know that shaders exist and they can do fancy things. Why is this so important?

The more games evolve, the better looking they get, the better they run, the more realistic effects we produce, and most importantly, the games simply get better. By utilizing these tools, we can achieve great levels of quality, detail, and speed.

Later in the year, I will be explaining more in depth of how these shaders work, and some sample programs of how they work. I hope you enjoyed this introduction, leave any comments you wish below! Thanks!