Advertisement
  1. Game Development
  2. Game Design
Gamedevelopment

Building a Dynamic Shadow Casting Engine in AS3

by
Difficulty:AdvancedLength:LongLanguages:

Dynamic shadows give game developers a way to recreate real life experience with lights and shadows. Every time we move, we cast shadows according to the position of the light sources around us. Dynamic shadows are no different than that. Our goal is to transfer this experience to the virtual world by creating an engine that will be able to cast these shadows.

These are common in 3D games. You have probably played a 3D game or watched a gameplay video and noticed that the shadows in most of them are dynamic. However, due to the complexity of the code, 2D games lack a good implementation of shadows, and often end up using a static alternative. The objective of this tutorial is to implement dynamic shadows in a 2D environment.


What Will You Need?

These two .swc files are used in this tutorial, so it's better to grab them now!

Light ball

Texture


Step 1: Defining Terms

During this tutorial we will use some terms that might not be common in your vocabulary, so here's a quick explanation of all of them:

  • Light source: any object that will create light. It will be represented by a single point and will have the following properties: range (radius of the circle created by the light), color (of the light), intensity (amount of light in the middle of the circle) and fall off (the amount of light that will still be present at the edges of the circle).
  • Light ball: if stated in the engine that they must be shown, the light ball will be in the center of a light source's circle and will represent the light source.
  • Solid object: any object that will cast shadows when illuminated by a light source. Will be defined by its vertices (defined in the counter-clockwise direction) and can have any shape you want, as long as it's a convex shape.
  • Display area: the area in which all the lights and objects are contained.
  • Light map: the sprite in which we will draw all the lights and shadows of the display area.
  • Objects map: the sprite in which we will draw all the objects of the display area.
  • Ambient light: it acts like sunlight, except that it can have any color you want. It will be applied to the entire light map, independant of the presence of lights and/or objects.

Important: Please note that the solid objects must be convex shapes and their vertices are defined in the counter-clockwise direction. This is very important and you may get different results if you don't do it (the reason for that will be explained later in the tutorial).


Step 2: An Example of Static Shadow

Static shadows are not what we want, but they are shown here to explain how they are done and in which cases having them isn't a problem.


Description: the circles below the characters are static shadows that only give the impression of a light source from the sky, like the sun.

Static shadows are simple circles with a radial alpha gradient (the most advanced ones have the shape of the object, for example) that lie below a character or an object. Sometimes their radii change if a character jumps or an object gets more distant from the gound, but they will always have a defined shape. Shadows like that are used when the light source is very distant from the objects, giving the impression that the shadows don't change. This is common in platform games or flying games, but not what we want to achieve here.


Step 3: Creating the Classes

Let's begin the code! First of all, we must set up all the classes that we will work with. For this tutorial, we will need three initial classes: ShadowEngine, Light and SolidObject. They must be created this way:


The classes don't need to extend any other class.


Step 4: Defining the Light

We will now start adding code to the classes. The first one will be the light. Here's the basic code:

We defined the _x and _y properties of the light, the _color, _range, _intensity and _fallOff. Since we will need to access the x and y properties, we created getter and setter functions for them. They might look useless, but we will add more content in the functions as we start giving more "life" to our engine. The drawing area is a Sprite in which the light will be drawn, and the shadow area is the sprite in which all the shadows related to that light will be drawn. The drawing area and shadow area will blend together later. The initializeDrawingArea() function sets the drawing area and shadow area up for drawing. More will be added into that function later.


Step 5: Defining the Solid Object

Now we must define the SolidObject class. This contains a Vector of all the vertices (corners) that make up a solid object:

As we did with the light, the getter and setter functions for the _x and _y properties will have more content as we keep adding more things in the tutorial. You must notice line 19, which is highlighted; in this line we call the concat() function of the vertices argument passed to the constructor in order to provide a new Vector of points to our class. This way, our class can make changes to this Vector without changing the original one, which is the Vector passed as an argument. The drawing area of our solid object will be the Sprite in which we draw its image.


Step 6: Starting the Shadow Engine

We must now start coding the mechanics behind displaying the objects and lights! Here is the code for it:

The _ambientLight and _ambientLightAlpha properties will be used for applying the ambient light in the light map. The _drawBalls property indicates to the engine whether or not to draw the light sources' balls (that is, whether to render instances of the LightBall class, contained in the SWC you downloaded earlier). The WIDTH and HEIGHT properties are the width and height of the display area of the engine. They are created as "public static" properties because they will be later accessed in our other classes. The drawing area is what all the maps will be added to.

If we head to the Main class and create a new instance of the shadow engine, we will see the nice message in the output window!

Note that we already added the engine's drawing area to Main's child list. This is the part of the code responsible for displaying everything in the screen.



Step 7: Adding and Removing Objects and Lights

Now we need to make our engine recognize objects and lights. This code will go into the ShadowEngine class:

And don't forget to add the following import statements:

In the code above, createObjectMap() and createLightMap() must be called when the ShadowEngine class is instantiated, because they are responsible for setting up all the maps used in the engine. The addObject(), removeObject(), addLight() and removeLight() functions do what their names suggest. Note that when we add or remove a light, we don't add their balls to the ball map yet, because they haven't been created in the Light class. This will be done on the later steps.

As said, in order to set up the light map and the object map, we must run createObjectMap() and createLightMap(). Let's add this code to the ShadowEngine constructor:

With this code you can already add objects and lights inside the engine! Let's give it a try. In the Main class we will add two lights and one solid object with the following code:

Use these import statements:

And this is what we get in the output window (don't worry, in the next steps we will actually draw the objects and lights)!



Step 8: Drawing the Object

It's time for some action: what do you think of drawing all the objects to the screen? In order to do that, we need to add draw() functions inside the SolidObject class.

The object will be drawn using Flash Player's drawing capabilities: we will begin filling the shape with a half-transparent red color, then run through each of the vertices creating lines, and then end the fill. Let's add this code to the SolidObject class:

The _iterator variable will be used throughout the class. It was created in order to help our engine's performance. Note that we add the object's current x and y positions to the vertices' positions before moving or drawing lines to them. That way we are able to draw in the exact position of our objects.


Step 9: Drawing the Light

In order to draw our light, we will need to create a gradient fill and draw a circle starting in our light's position. If you want to learn more about gradient fills, you can check this tutorial: How to Create Gradients with ActionScript.

Now, let's create some functions inside our Light class:

We created a gradient circle centered in the light's x and y positions in this code. The _lightGradientMatrix property needs to be inside a variable because we will later have to update it. We will also need to create the light's "ball" -- a graphic to represent the light source. Let's use the LightBall image provided in DynamicShadows.swc as the light ball. Inside the Light class, let's add a createBall() function and call it in the constructor:

Don't forget that we need to add some import statements in our Light class!

The ColorTransform was used here in order to give the same color of our light source to the ball. The getter function for the ball property was created in order to add the light's ball in the ball map, and later in the ShadowEngine class. Now, everything should be right for adding these objects in the screen.


Step 10: Adding the Objects to the Screen

Back in our ShadowEngine class, we need now to draw our objects in the screen. In order to do that, we need to update our engine every frame and then ask every object to draw itself. Since the objects are added to the object map, they will be automatically displayed on the screen.

Our engine will be updated through the updateEngine() function. It should be called every frame. Let's add that in the ShadowEngine class:

We can now go to the Main class and create an Event.ENTER_FRAME event listener, then update the engine. Add the following code in the Main class:

When compiling the project, we can see the result: our object (drawn in red):



Step 11: Drawing the Lights

Drawing our lights on the screen will be a bit more complicated. In the ShadowEngine class, let's create a drawLights() function and add the call to this function in the updateEngine() function. Let's also not forget to update the addLight() and removeLight() functions to make them add the Light's ball in the ball map:

This code clears the light map in order to apply the ambient light, and also loops through each light and makes the light draw itself in its drawing area. Since each light's drawing area is added to the light map, they will be automatically shown.

We can now add lights into the engine! Take a look at our result:


You can see the two light sources that we added back in Step 7: one cyan and one red. You can also just about see the solid object, but it's drowned out by the cyan light!


Step 12: Improving the Lights

In our last build, the lights are too much bright, and this makes it hard to see other objects. This happens because we are only drawing the lights in the screen, so the raw color data is being used. What if we wanted to have more than a light (like in our example)? Their colors would need to mix up a bit in their intersection. In our example, the red light can barely be seen. And we can't forget the ambient light, which is present at all the time, but is covered by the lights now. This can be easily changed by using Blend Modes.

The Blend Mode we will use in our engine at this time is the Multiply mode. The Multiply mode must be added in our light map, so that it affects every light added to it. Let's add it in the ShadowEngine class:

And let's not forget the import statement:

Hit the compile button and this is the screen we get:


It looks much better now! We can see that in the non-red corners the ambient light starts to kick in! However, the cyan light still looks too bright. What about reducing its range to 300 pixels and fall off to 0 in the Main class? Let's also increase the ambient light's alpha to 1. That way we can create a totally dark environment with only the two lights illuminating it.

Let's take a look at our result:


This is starting to take shape, isn't it?


Step 13: Introducing the Hard-Edged Shadow Casting

Now that we have our classes displaying in the screen properly and that our lights draw themselves correctly in the screen, it's time to begin our work towards shadow casting. The first method we will use is called the Hard-edged Shadow Casting. It is called that way because we assume the light is a single point and every light ray comes only from this point, creating a solid shadow behind the objects.


In order to draw the shadows, we need to set up a strategy. This is ours:

  • Draw a line connecting the light source and a vertex in our shape.
  • Identify whether this vertex is totally illuminated by the light (i.e. is in the "front"), is totally immersed in the shadow (i.e. is in the "back"), or is a boundary vertex. A boundary vertex lies in the intersection between front faced and back faced edges.
  • Create a Vector with all the back-facing vertices and both boundary vertices.
  • Draw triangles, connecting all the back facing and boundary vertices, in the light's shadow area.

All this code will be within the SolidObject class, because it is the only one that should know about its vertices. We will pass the light properties used in our class through a function called castShadows().


Step 14: A bit About Vectors

If you are not familiar with vectors yet, I suggest you take a look at my earlier tutorial, Euclidean Vectors in Flash.

In the next steps we will need to find the normal for every edge of our shape, then make a dot product between the normal and the vector passing through the light's point and the middle of the edge. If this dot product is less than 0, then the edge is back facing. If not, then it is front facing. We will not store the edges that are back and front facing, but instead we will store the index to the first point that makes the edge. This will make things easier later when determining the back facing vertices.


Description: Normals are shown in green. We will do a dot product between each normal and its corresponding orange vector (the vector from the light source to the middle of edge)


Step 15: Classifying the Edges

It is time to begin coding, and the first thing is to loop through each vertex and check if it belongs to an edge that is back facing or front facing. We will then store this in a Vector called _backFacing. We will be using the method described in the tutorial linked in the previous step in order to calculate the dot product and the normal of the edge, and we will have a function called checkEdgeBackFacing() to determine if an edge is back facing. In the SolidObject class:

In the above code, note the importance of defining the vertices in a counter-clockwise direction. When we loop through the edges, we are running in a counter-clockwise direction and storing the facing of the edges in the same direction. If there wasn't an order when defining the vertices of each object, then this code would all get messed up.


Step 16: Defining the Boundary Vertices

Now that we know the facing of each edge, we can determine which vertices are the boundary vertices. We will determine the "starting" boundary vertex and the "ending" boundary vertex. The starting vertex is in the intersection with a front facing edge followed by a back facing edge. The ending vertex is in the intersection with a back facing edge followed by a front facing edge. Take a look at the image below:


Let's go to our code. In the SolidObject class, we will add the following code to the castShadows() function.


Step 17: Determining the Number of Vertices in the Shadow and Looping Through Them

Now that we know the starting and ending boundary vertices, we can tell which vertices are in the shadow, so we can draw the shadows later. However, this task is not as easy as it sounds. One may infer that it is just about looping from the _startingBoundaryVertexIndex to the _endingBoundaryVertexIndex, but what if we have the _endingBoundaryVertexIndex lower than the _startingBoundaryVertexIndex? We need to take care of this issue, while still being able to loop through all the vertices. Here is the code for it:

The code added counts the number of vertices in the shadows, then creates a _currentHullIndex property (named after the idea of a convex hull). The _currentHullIndex will loop through our vertices in the shadow. This will be the index used to access the vertices.


Step 18: The Light-to-Vertex Vectors

As stated in Step 13, we will have to draw triangles in order to build our shadow. The triangles will be based on the vertices in the shadow, but they aren't enough. We need a few extra points, which we will call the light-to-vertex vectors, that will give the illusion that the shadow goes to the infinity. In order to do that, we need to first determine these vectors, and then multiply them (refer to the tutorial pointed in Step 14 to learn how to multiply vectors) by a very big scalar number, so that they give the illusion we want.

Here is how we determine them:

This code gives us the vector that goes from the light source to the vertex we want. In the next steps we will use it to draw the shadows we want.


Step 19: Details on drawing the shadows

We can now finally draw the shadows, but we must first create a more complete strategy for drawing them. We will need to draw a lot of triangles in order, so if we don't get it organized now, our code will become messy.


According to the image, we will start in the starting boundary vertex. Then a line will be drawn to the first light-to-vertex, and then another line will be drawn to the next shadow point. In order to complete the triangle, we will draw a line back to the starting boundary vertex.


Going to the next vertex, we will do the same thing, but now notice that there is a gap between both triangles. In order to fill this gap, we also must begin in the second vertex, then draw a line to the light-to-vertex of this vertex, and then draw another line to the previous light-to-vertex point. Then we complete the triangle drawing back to the second vertex.


Notice that we will need to hold the previous light-to-vertex as well, in order for this strategy to work.

When we reach the last vertex in the shadows, we will only draw a triangle using the light-to-vertex point for the last vertex and the previous light-to-vertex point.


Step 20: Drawing the Shadows

In order to draw the shadows, we will need the light source's shadow area to draw into them. Let's add a parameter to the castShadows() function and create our code!

Notice that we multiply the light-to-vertex points by 10000, a reasonable number to give the illusion of a shadow going to the infinity.


Step 21: Adding the Drawing Code to the Engine

In order to have our engine drawing the shadows, we need to add some code in the ShadowEngine class. Let's do it!

Don't forget to add the following import statement:

With that code in the function, all left to do is make the Light's shadow area visible. Let's head to the Light class and add this code in initializeDrawingArea():

Now, let's compile the project aaaaand...

Check out the result!

Congratulations!!! You have just made the first step to an awesome dynamic shadows engine! However, there are more things that we should do before using it.


Step 22: Creating Bounds

In order to optimize our engine, we must save processing time. Right now, our engine is looping through every light and making every object draw shadows in their drawing areas. Let's add bounds to every object and light and check whether the bounds of an object touch the bounds of a light. If this happens, then we draw the object's shadow into the light's drawing area. If not, then we skip the drawing. That will save a lot of performance.

Inside the Light class:

(AABB stands for Axis-Aligned Bounding Box.)

This code will create the bounds of our light as a square, centred on the light, and update it every time we change our light's position. Don't forget to import the Rectangle class:

Now, inside SolidObject:

Don't forget the import statement here, either:

The createAABB() function creates the bounds of our object based on the position the vertices were created. The reason we don't have an updateAABB() function in the SolidObject class is that, when the x or y properties change, all we need to do is bring the AABB rectangle to the new position, and that can easily be handled in the setter functions themselves.


Step 23: Simple Boundary Check

Now that we have our boundaries created in each object, let's do a quick check in the ShadowEngine class: if the bounds of an object intersect the bounds of a light, then the object's shadow should be cast in the light. If not, then nothing happens. The code:

With that code, we saved a lot of process time when our engine begins to have more objects and lights!


Step 24: Correction of the Light's Matrix

If we take another look at our code in our Light class, we will notice that our _lightGradientMatrix depends on the Light's x and y properties, and it doesn't update when these properties change. Let's fix that:

With this code, everything should be right for our Light class. Everything that depends on the x and y properties are updated.


Step 25: Creating a Mask for the Light

If you had the interest to try a few things in our current engine, you will notice that if you change the ambient light, you will notice something very weird.


Description: our engine with the ambient light set to full green (0x00FF00) with an alpha of 0.3 and a white (0xFFFFFF) background.

See that our object's shadow extends to the infinity and takes up the space (the red rectangle) that should be the ambient light's space, since the cyan light doesn't have "power" to illuminate everything in the stage? We need to fix that, and for that we can simply add a circular mask to the Light. The code:

The code part is done. If we compile our project (maintaining the properties in the description of this step's image), this is what we will see:


Still doesn't look quite right, does it? Time to fix those shadows!


Step 26: Using the Erase Blend Mode and Cleaning the Shadows

Right now our engine draws shadows in a full black (0x000000) color. The problem with this is that not always we have a full black ambient light and the shadows should be the absence of light. Our objective in this step is to completely remove the shadows from the visible area, while still leaving an empty space in its location. In order to do that, we need to simply add a BlendMode.ERASE in our Light's shadow area. However, this requires that the parent display object container (in our case, the Light's drawing area) has a blend mode set to Layer.

Let's do it. In Light.as:

And add the following import statement:

We also must notice that our Light's shadow area isn't being cleared on every frame. That way, if any object moves, the shadows will "sum up", which definitely isn't what we want. It also starts to lag after some time, due to not clearing the shadow area before drawing the new shadow. Let's go to ShadowEngine and add a single line of code to fix that:

If we compile our project now, that's what we should get:


Works like a charm! You see how the shadow of the solid object is now the absence of cyan light, rather than the absence of all light, including the ambient light?

Our next step is to improve the look of our shadows.


Step 27: Introducing Soft-Edged Shadow Casting

Our engine has been using hard-edged shadow casting until now. This isn't a problem, since it is, in theory, how shadows should work with lights represented as points. However, in reality there isn't a source of light that behaves as a single point: every light source has some kind of dimension, even if it's small. We can view these light sources as an infinity of other tiny light sources, so tiny that they can be considered points. That way, a real light source can be represented as being made up of many of these "tiny light sources". We would spend a lot of time adding this to our engine, since this would require a lot of processing, and Flash wouldn't handle that for a few light sources per frame.

Instead of taking the approach cited above, we are going to use a small trick to give the impression of soft-edged shadow casting. Compare both images below:



Can you guess what are we going to do? That's right: we will add a small blur filter to our lights, to give the same idea of soft-edged shadow casting. Let's do that in ShadowEngine:

And don't forget the import statement:

Let's change our ambient light back to full black (0x000000) with an alpha of 1 to see the effect better. When we compile our project, this is what we see:


Notice something weird around our object? Me too. Let's see what's causing the problem and how to fix that.


Step 28: Adding the Shadows on our Objects' Shapes

The problem with our engine right now is that we are only drawing the shadows behind our objects. However, we forgot that in the place where our objects are, there is also shadow! We can easily fix that by drawing their shape in the castShadows() function, in the SolidObject class. Remember the process used in the draw() function for our solid objects? We will do the same thing:

After compiling, this is what we get:


Did you notice that our object has disappeared? This happened because the light map is being added over the object map, and since our ambient light is totally black with an alpha of 1, our object can't be seen. Let's change that by making our ambient light a bit transparent. In the Main class:

If we compile our project now, we can see that everything works fine!



Step 29: Adding More Objects and a Problem

It's time to add some more objects in the screen. Let's create two more objects in the Main init() function:

Don't forget: the vertices must be always declared in the counter-clockwise direction!

After we compile our project, this is what we get:


Hey! Something is wrong! We added an object "around" our light source, and that caused it not to create light anymore! How can we fix that?


Step 30: Checking if the Light Source is Inside a Polygon

In order to fix our problem, we must, in addition to checking whether the bounds of a light intersects with the bounds of an object, check whether the light is inside the shape. If that is true, we simply will not ask that object to cast shadows on our light, giving the illusion that the light is over the object.

Do you remember the tutorial about vectors? It also explains a few methods of checking if a point is inside any shape (Steps 15-22).

We can implement any method that deals with convex shapes, since our engine uses convex shapes. Let's instead add the universal method of the even-odd check, because someone might want to change this engine to work with other shapes, and thus the rest of our code has to work with it.

Let's add a function checkLightInsideShape() in our ShadowEngine class:

And then, let's check this in our drawLights() function:

Notice that we must access the current object's vertices, but currently there's no way to do that. Let's go to our SolidObject class and add a small getter function:

With that, we can hit compile and see our result:

Check out the result!

Isn't it getting good?


Step 31: Further Improvements

Our engine is almost ready to use, but what do you think of making some improvements first? There's also a bug in our current code, so let's fix it first. In our Light class, we forgot to change the ball's x and y positions when our light moved too, so let's add this code in there:

Now our engine is free of bugs and you can already use it, but what's next? There is so much we could do with it! The classes in our engine were built in order to aid the creation of many types of objects and lights. For example, we could create a spot light, or we could create a RegularPolygon class that will represent a solid object in a regular polygon shape with the amount of sides we want. We could create support for rotation, and even add support to using different images for the solid objects, other than only drawing a half-transparent red shape!

In the next steps, you will learn how to create different objects and add support for different images, as well as for rotation.


Step 32: Creating a RegularPolygon Class

Our RegularPolygon class will be created to ease the process of creating regular polygons for the shadow engine. The class will create automatically a polygon once we pass the required parameters to it.

Regular polygons share the property of having all of its vertices defined in a single circle. We will use that to create our class. Basically, we will loop through the circle, defining the vertices as the current cosine and sine values of the angle. We will need the circle's radius in order to do that, as well as the number of sides. Take a look at the code:

In the code, we define the angleStep property. This property is the amount of radians we will "walk" in each iteration of the loop, completing the circle on the last iteration. Take a look at the image:


Our Regular Polygon class is almost done! It would be really cool if we could define the radius of our circle based on the amount of sides our polygon has and the size of each side. Here is a quick formula for getting the circle's radius given the size (length) of a side and number of sides:


Let's add it to our code and make our class able to receive either the circle's radius or the size of each side:

And that's our RegularPolygon class! We can add a regular polygon to our engine with the following code in Main:

And here's our result:



Step 33: Rotating our Shapes

It's about time to let our shapes rotate, but as you may have seen, we will have to rotate the vertices according to the origin point of our shape! Let's take another look at the Euclidean Vectors tutorial in order to get an idea of how to rotate our shapes.

Did you see it? Let's create a applyVerticesRotation() function in our SolidObject class:

After that, let's do a quick change in our init() and engineLoop() function in the Main class, so that we rotate a shape 5 degrees every frame. Here is the code:

And here's our result!

Check out the result!


Step 34: Adding Textures or Another Image to Your SolidObject Class

So I heard you are getting sick of the half-transparent red shapes in our engine. That's serious. What about making it better by adding any kind of image you want? In this example, I will use one of the great texture packs made by Georges Grondin, which are free to use by anyone. The texture I'm using is available in the Texture.swc file.

In order to draw any image we want, we must use what is currently being drawn in the object's drawing area as a mask. To do that, let's create two other properties, another function and modify a few functions to make it work. In the SolidObject class:

Let's add a piece of code to set the image of a shape in the Main class and remove our rotating code:

(The DemoImage is from Texture.swc.)

After hitting compile, this is our result:


We can see that our texture works, but the default image for the other shapes doesn't. What could be wrong? Let's fix it in the next step.


Step 35: Fixing our Solid Objects' Image

In the previous step, we noticed a problem after adding custom images to a shape: the others lost their default image. What's wrong? Let's take a look at this part of code from our SolidObject class:

Did you notice that we are only drawing on our _imageMask property? In order to have the default image, we will also need to change a bit of code to set a default image. Let's do it:

The createDefaultImage() function is just a modified version of draw(). It runs only once and creates the default image. If we set another image, the default one will be lost and the new image will be added, and so everything will be working again. Let's hit compile and see what we get:


Success! We can now use custom images in our engine.


Step 36: Making an Object Move With the Mouse

If you would like to have a moving object so you can play with its shadows, let's add a very simple code to make an object follow the mouse's position! In the Main class:

This is the result:

Check out the result!


Conclusion

If you are reading this, congratulations. You have just created your very own Dynamic Shadows engine, and can now use it freely for any project you want! That was a tough task, and I hope you have learned a lot from this. Feel free to post anything you make with this engine in the comments, and don't forget to put a link so that everyone can play with it! Thank you for reading this tutorial.

Advertisement
Advertisement
Looking for something to help kick start your next project?
Envato Market has a range of items for sale to help get you started.