[Godot 2] Lighting not working properly in manually built mesh

woopdeedoowoopdeedoo Posts: 109Member
edited August 26 in 3D

I'm making a map editor for my game, and that means I'm having to build the map's 3D mesh through code, using the SurfaceTool, but I came across the problem of lighting not working as it should.

I'm calling SurfaceTool.add_smooth_group(true) before building the entire mesh (code below), and then SurfaceTool.generate_normals() before committing the mesh. I'm not sure how to use the former. It smooths the lighting within each tile this way (each tile is two triangles), and I couldn't get any different results by calling it before adding each vertex, or in any other way that I tried.

The result is what you can see in the image below. When I pull a vertex up the lighting creates seams between tiles. However, within each tile, the lighting is smooth.

I don't know what I'm doing wrong...

The map is a 2D array of "cells" (stored in 1D), and when I create or make changes to a cell this function gets called (and the resulting mesh fed into the map's MeshInstance):

func make_map_mesh(cells, w, h):
    var st = SurfaceTool.new()

    for c in cells:
        if c != null:
            create_cell_mesh(st, c)

    return st.commit()

Each cell object stores the positions of its 8 vertices: 4 for the ceiling, and 4 for the floor. The walls are computed depending on neighbors (I excluded the code for the walls and ceilings for testing).

The create_cell_mesh() function takes each cell and builds its part of the mesh.

# I'm handling vertices in this order (not sure if it's ideal, but...)
#             A ----------- B                    
#            /             /        top face     
#           E ----------- F                      
#             D ----------- C                    
#            /             /        bottom face  
#           H ----------- G                      
func create_cell_mesh(st, cell):
    # (...) - compute uvs according to cell's tile textures

    var a = cell.a  # ceiling vertices
    var b = cell.b
    var e = cell.e
    var f = cell.f

    var c = cell.c  # floor vertices
    var d = cell.d
    var g = cell.g
    var h = cell.h

    # (...) - build walls
    # (...) - build ceiling

    # build floor
    create_face(st, d, c, g, h, u, v, u2, v2)

And this is where I create the vertices.

func create_face(st, a, b, c, d, u, v, u2, v2):
    var a_uv = Vector2(u, v)
    var b_uv = Vector2(u2, v)
    var c_uv = Vector2(u2, v2)
    var d_uv = Vector2(u, v2)

    # top right triangle



    # bottom left triangle




  • TwistedTwiglegTwistedTwigleg Posts: 902Admin

    I looked at the documentation description for the function and I think it is saying that after you add a vertex/index, you can use add_smooth_group to specify whether the recently added vertex/index should be using smooth normals.

    I would try adding add_smooth_group after each add_vertex call and see if that works.

  • MikeWieringMikeWiering Posts: 3Member
    edited August 26 Answer ✓

    I was having a similar problem and ended up adding the normals manually (instead of generate_normals) to get smooth shading. I think it did work before though (a few months ago).

    EDIT: I now see [Godot 2], I'm working with 3.

  • woopdeedoowoopdeedoo Posts: 109Member
    edited August 27

    @TwistedTwigleg said:
    I would try adding add_smooth_group after each add_vertex call and see if that works.

    Nothing gets smoothed that way. And it throws a million of these into the output:

    SMOOTH AT 3: 1
    SMOOTH AT 6: 1
    SMOOTH AT 9: 1
    SMOOTH AT 12: 1
    SMOOTH AT 15: 1
    SMOOTH AT 18: 1

    @MikeWiering said:
    I was having a similar problem and ended up adding the normals manually (instead of generate_normals) to get smooth shading. I think it did work before though (a few months ago).

    I was trying to avoid that, but seems like it's what I'll have to do. Problem is I don't know how to deal with the fact that my cells can have their floors disconnected. Like this: (the vertices can only be moved vertically though)

    Maybe I'm thinking of it backwards, but it seems like I need a lot of conditional branching to determine whether to join floor vertices or create those walls, and whether to calculate the normals based off of the next floor or that wall, etc (and there may be empty gaps in between cells). It's been two days since I tried it, so I can't quite tell what exactly I was getting stuck on, but the complexity of my code seemed like it was growing exponentially... I couldn't just deal with the vertices in loops, because each vertex seemed to require different considerations...

    Although, I was trying to index the vertices, and that was the part that just killed me. But maybe I could have each cell know which triangles belong to itself, so I can refer to them to calculate the normals.

    Or maybe I'm just overcomplicating everything, but I can't see how to make it simple to deal with...

  • woopdeedoowoopdeedoo Posts: 109Member
    edited August 28

    So I managed to work it out. And so far it seems ok, as far as smoothing goes. The code is convoluted as hell, but works as far as I can tell. Calculating the normals manually, btw, removes the need to call add_smooth_group().

    I have another problem with the lights though, and it doesn't seem related to any mistake in my calculations of normals, because it also happens in a map I imported from blender (second pic), unless I made some similar mistake in blender. As you can see in the pics below, the light goes through the walls (and the more I raise them, the larger that gets).

    It may be related to the fact that the bottom vertices of the walls (and the block on the second pic) aren't shared with the floor. I did this so that the normals of the walls (and the block) could point perfectly perpendicular to the walls, so that the light doesn't make it look like the walls are rounded around the edges. So basically, there's a gap into the void beneath them.

    What I find confusing is that it only happens in the x axis. The shadows on the z axis seem fine, as you can see in the first pic.

    The shadows are also rounded. Is this normal? The light is an OmniLight.

  • MegalomaniakMegalomaniak Posts: 1,001Admin
    edited August 28

    Shadows for omni-directional lights are the most difficult case to handle. One way is to use a cubemap, 6 textures is obviously costly though. Another way to go is dual-paraboloid shadow mapping, cuts down from 6 to 2. Still uses more than 1 texture and you are still dealing with a seam between the 2 shadow-maps.

    My point being I suspect that the single axis along which you have light leakage is the seam and the light leakage is the product of the error margin. But without looking at the source I can't claim that this is definitely it, I'm not even really sure how they handle shadows for point lights. So it's just a guess on my part.

    you might also want to see the following:

    the latter confirms what I was saying above.

  • woopdeedoowoopdeedoo Posts: 109Member
    edited August 28

    Hmm, if by seams you mean what I think, there are no seams in the block in the 2nd pic (all vertices are shared, except the top face, and the floors around it), and there are seams in all walls in the first pic, as all the walls are 1 grid unit in x and z, so those walls you see there are actually two in each direction. I'll see how the light reacts if I attach all the vertices in that block. I'll also take a look at that link.

    This is the entire code with which I generate the geometry. It's kinda cryptic...

    The map class just calls the make_map_mesh() function (and draw_normals(), if enabled):

    func update_mesh():
        map_mesh.set_mesh( cf.make_map_mesh(cells, w, h) ) # cf for CellFactory
        if show_normals: cf.draw_normals()

    I would share the whole project, but it's a HUUGE mess of insanity, since it's the result of a whole lot of experimentation...

    EDIT: seen the two latter links now. I'll give them a look too.
    Thanks btw.

  • woopdeedoowoopdeedoo Posts: 109Member
    edited August 29

    How do I make an OmniLight cubemapped? The 2.1 docs don't seem to mention it, or I overlooked it.

  • MegalomaniakMegalomaniak Posts: 1,001Admin

    Ah, my bad. I was thinking in terms of godot 3. Not sure if godot 2 even has the option to switch. Version 3.1 will likely be your solution once it's released.

  • woopdeedoowoopdeedoo Posts: 109Member
    edited August 30

    Yea, I overlooked this in the 2.1 docs:
    "The second option is to simply not use a shadow map, and use a shadow cubemap. This is faster, but requires six passes to render all directions and is not supported on the current (GLES2) renderer."

    I'm finding lights a bit too much of a hard beast to tame... Gonna see if my PC can handle the latest 3.1 build.

    I do wonder though: I used to make maps in Valve's Source engine (HL2dm/CS:S version), and when it came to point lights I never had to worry about anything except for specifying the resolution of the lightmaps in certain walls, which only made the difference in how sharp the shadows were at the edges (and how taxing they were on performance). The lights always seemed to fit perfectly to the geometry and all that. I wonder what lighting algorithm that engine used, and whether Godot couldn't benefit from having it available. Or maybe it was just also cubemapped.

  • MegalomaniakMegalomaniak Posts: 1,001Admin

    That was static/baked lighting. I.e. no real-time shadows at all.

  • woopdeedoowoopdeedoo Posts: 109Member
    edited August 30

    True. There were dynamic lights too, but you had to use them very sparingly as they were quite heavy (I just tried to test one in Hammer, but it didn't work at all for some reason... so nevermind.)

    I was looking at Thief Deadly Shadows gameplay, though, and I noticed that every single light in the game makes the player (and at least some other actors -- doesn't seem to be all of them) cast a shadow. It's easy to see in 3rd person playthroughs.

    I wonder how they did that. I wouldn't suppose all the lights are dynamic; although it seems there's hardly ever more than one or two lights in the same place, so that might still be the case.

  • MegalomaniakMegalomaniak Posts: 1,001Admin
    edited August 31

    @woopdeedoo said:
    I was looking at Thief Deadly Shadows gameplay, though, and I noticed that every single light in the game makes the player (and at least some other actors -- doesn't seem to be all of them) cast a shadow. It's easy to see in 3rd person playthroughs.

    They probably took very deliberate and direct control over the dynamic lights and switch(on/off) between them in a very controlled fashion. The game is old enough now I'm having a hard time finding any papers or talks on it but I'm sure there must have been some at around the time of the games release.

    edit: Actually now that I think about it they might have used a modified doom 3 engine..? In which case I think they might have relied on shadow volumes?

  • woopdeedoowoopdeedoo Posts: 109Member
    edited August 31

    It seems easier to find papers on the first two games than on the third one, as Sean Barrett wrote a few things (and gave a talk with some other guy also called Sean, many years ago about Stim & Response). The Dark Mod uses the Doom 3 engine though.

    According to wikipedia Thief 3 used a modified Unreal Engine 2.

    Anyway, it seems my pc is handling the latest nightly build of Godot 3.1, so I'm porting my project to it to see how it goes.

    EDIT: It seems, from reading media articles from ~2004 (just search for "lighting"), that T3 was becoming known for using dynamic lighting. So I guess they made some clever use of it. Still no clue what were the algorithms, though.

Sign In or Register to comment.