Mysterious and Horrifying normals

[Balto-the-Wolf-Dog]

I figured out UV unwrapping about six months ago and have been churning out parts ever since. For the most part it's been smooth and I've gotten fairly good at it, but today (while working with the single simplest model I've made yet) I ran into an issue with normals in unity. The model looks fine in 3ds Max and sketchup, but even after normal smoothing in unity it develops harsh, warped seams apparent both in unity and in game in most lighting conditions. No amount of normal smoothing/other alteration to the import configuration has proven to resolve the seams regardless of what else is damaged. They will remain present even with 180 degree normals. 

 

 

The seams seem to correlate with where the triangles get narrower to deal with the curvature in the windows, but I've never had this problem to this extent before and never with a model that didn't have a huge number of awkward faces. Note that the pod attached to the part in question fits the formfactor (the faces on both match properly with standard 2.5m parts), and its normals are fine despite far more complicated geometry. The only explanation I've been able to come up with is the boarder between the broader, simple triangles and the complex ones. I'm skeptical as to that actually being the problem, though.

It should be noted that while the windows are inset into the side of the fuselage, the model is otherwise a normal, linear cylinder with flat faces like any fuel tank. As such the issue at least shouldn't have anything to do with actual geometry, as even the complex triangles should be coplanar. 

 

Does anyone have any ideas as to the cause and/or resolution of the issue? I've only seen it one other time, in which it appeared as a minor imperfection on one side of a very complex pod. This case is much more severe and at least appears to be symmetrical based on the angle of lighting. Presumably I'm doing something wrong somewhere, but I'm not sure what it could be.

 

Other details:

The windows and fuselage are skinned separately (planar vs cylindrical), but the seam appears to be consistent across the two anyway. The actual planar faces should be identical to those around the rest of the model. 

Edited March 4, 2016 by Balto-the-Wolf-Dog

[nli2work]

show the wire frames

[Balto-the-Wolf-Dog]

7 hours ago, nli2work said:

show the wire frames

[Bonus Eventus]

I'm not sure what your 3d modeling software package is (I use modo) but have you tried baking your normals from a higher resolution mesh? in modo you can transfer your vector normals from one mesh to another similar to normal map baking. People usually use this technique for making the separate leaves on a tree a soft shadowed appearance, but I've found it can do wonders for when you have a mesh with complex form changes!

[nli2work]

yeah all those edges going into a single vertex is bound to give shading issues. Your design is better accomplished using simple cylinder geometry with baked normal maps for the window shapes.

Edited March 5, 2016 by nli2work

[Balto-the-Wolf-Dog]

 

Thanks guys, I've mostly dealt with it now. 3ds Max was producing parallel normals natively, which was good for triangle smoothing but awful for face smoothing. Unity autosmooth just wasn't precise enough to deal with it, so I averaged the normals in 3ds max and made sure they were parallel on the seams. Unfortunately that's left the actual triangles not totally coplanar as far as the normal map is concerned (since one is smoothing the face above and the other the face below) but the imperfection is very slight.

30 minutes ago, Bonus Eventus said:

I'm not sure what your 3d modeling software package is (I use modo) but have you tried baking your normals from a higher resolution mesh? in modo you can transfer your vector normals from one mesh to another similar to normal map baking. People usually use this technique for making the separate leaves on a tree a soft shadowed appearance, but I've found it can do wonders for when you have a mesh with complex form changes!

I may give this a shot to see if it can do a better job with the normals than I can. I know the process can be done in and is similar in 3dsmax, though I don't know how as of yet. 

[Porkjet]

View your mesh in shaded mode in 3ds and not consistent colors, you have no chance of judging the normals without shading.
The topology in your mesh is bound to give troubles tho, those insanely high density areas next to much lower desnity mesh is not a good idea, try to make poly density as evenly as possible, you dont need that super high density at the roundings at all. But for such fine bavels, consider creating those with a normal map instead, those 0.5cm insertions really aren't worth the trouble of poly modelling them.

[InsaneDruid]

You may want to take a look here:

as this thread and the thread the tutorial was based off:

might give you some insights to fix this problems by manipulating the normals. Not everytime you can just use some normal maps.

Edited March 10, 2016 by InsaneDruid

[Balto-the-Wolf-Dog]

I got the fuselage sorted, but I'm playing the game again now.

 

The windows are once again inset to avoid looking painted on, but the window frames and windows themselves are technically separate models with the intent of avoiding interference when smoothing, a semi-successful measure. I hand smoothed most of the model in 3ds max via edit normals, but I can't seem to get it much better than above. My current idea is to attempt to project normals from a version without any window bevels at all, though given the nature of the artifacts I'm not entirely sure it will help.

I saw in the window geometry thread the projection method being used to resolve a mesh with similar to this one, but, to be clear, my previous success (and all of mine thus far on this project) has been by manual manipulation. I'm fairly new at this and to be honest I have no idea how to do it the "easy way", be it by normal mapping or projection, nor am I particularly clear on if either of those approaches is applicable in this situation. Any guides/pointers would be much appreciated; reckon I'll be dealing with this a lot from here on. 

Assuming the viability of projection in this case, with what I gleaned from the thread the noise could be mitigated by projecting from a mesh in which the walls of the cylindrical component had a similar resolution to that of the window edges. I suspect automatic subdivision won't do me many favors at this point; would it be reasonable to try to project normals from a freshly generated cylinder with an appropriately high resolution? In theory I could also re-preform the lofting in sketchup from a higher resolution circle, though I have my doubts as to the resulting accuracy.

 

So right now I'm looking primarily at two options:

• Generate and project from a higher resolution mesh where the cylinder walls are of matching resolution to the windowframes
• Generate a windowless mesh otherwise identical to the "real" one and project the normals from that

How should I proceed, and where can I find some 3ds max normal projection tutorials? 

Note: Since I do the legwork for modeling from within sketchup, in theory I should  be working with quads. In practice I haven't been able to get a non-triangulated version, though vanilla .3ds exports seem to get close. Then again, since sketchup deals in planes, if I didn't triangulate I'd probably be getting n-gons as well.

 

Edit: I'd really, really like to keep the inset windows on the actual model. My first attempt at a pod had carefully painted on windows and, while they looked alright, they still didn't look real. After looking around a bit I saw all the stock pods had inset windows and adopted the style. It's worked very well for me hitherto, and I found it helps tremendously when perfecting the alignment between exterior and IVA models. 

 

 

Edited March 14, 2016 by Balto-the-Wolf-Dog

[NecroBones]

 

I have a huge amount of respect for all of you who take the time to work these out. My solution (and another option for you) would be to take the easy way out and paint the windows on, but create a raised rim around the outside edge of it that is non-manifold. Since the rim would be a detached mesh, the normals wouldn't interact. As long as the rim is wide enough to hide the aliasing of the window edges, it'll look nice and sharp. Lazy, for sure, but it gets the job done.

[InsaneDruid]

You want to go the "Generate a windowless mesh otherwise identical to the "real" one and project the normals from that"-route, as you need the normals to behave like there is no window. That this (windowless) version can be autosmoothed to a higher resolution. Then, like @nothke stated over in the original windows thread (linked in my first post here) youc an use the Normal Thief addon for max to transfer the normal data to the vertices that form the outermost edge of your window inset (the ones sitting on the original shape).

As discussed there, too, you could avoid some of these problems with a better consistency of size of the polygons and a better topology. Given that the hull has only (about, hard to count from these pics) 20-30 sides, the windows a bit too highres (or the hull too low-res). Why are the super small upper edges of the front window 5 times smoother than the nose of the whole vehicle? It looks unbalanced.

[Balto-the-Wolf-Dog]

14 hours ago, NecroBones said:

 

I have a huge amount of respect for all of you who take the time to work these out. My solution (and another option for you) would be to take the easy way out and paint the windows on, but create a raised rim around the outside edge of it that is non-manifold. Since the rim would be a detached mesh, the normals wouldn't interact. As long as the rim is wide enough to hide the aliasing of the window edges, it'll look nice and sharp. Lazy, for sure, but it gets the job done.

That's a damn good idea actually. Hadn't occured to me.

 

9 hours ago, InsaneDruid said:

You want to go the "Generate a windowless mesh otherwise identical to the "real" one and project the normals from that"-route, as you need the normals to behave like there is no window. That this (windowless) version can be autosmoothed to a higher resolution. Then, like @nothke stated over in the original windows thread (linked in my first post here) youc an use the Normal Thief addon for max to transfer the normal data to the vertices that form the outermost edge of your window inset (the ones sitting on the original shape).

As discussed there, too, you could avoid some of these problems with a better consistency of size of the polygons and a better topology. Given that the hull has only (about, hard to count from these pics) 20-30 sides, the windows a bit too highres (or the hull too low-res). Why are the super small upper edges of the front window 5 times smoother than the nose of the whole vehicle? It looks unbalanced.

I made the base of the hull to match the side count of stock KSP 2.5 m parts. The actual geometry is the result of my modelling style. I love me some lofts. For whatever reason the line that formed the nose contour didn't wind up that high res so the skin didn't either. Didn't worry about it because that kind of thing usually ends up looking fine. The windows are derived from straight up curved lines. 

Going to give noemalthief a whirl. Regarding that, why would I crank the res of the windowless mesh? Why not just smooth its normals as it is and project those? As I understand in this case, if I can smooth the windowless version effectively then the result of the windowed version should be basically identical.

 

Edit: Normalthief seems to be crashing after the first vertex. Am I missing something? Does it just act crashed while processing? Sketchup has that habit.  

 

It seems to be working, but it's not helping much. Transferring the normals has reintroduced several problems I spent hours fixing manually, and (strangely) doesn't appear to have fixed the target anomolies. I'm going to give it a few more whirls and then post comparison pictures. Are there any ways I can convince the 3ds max environment to look more like the unity renderer? It'd be easier to fix things manually if I could see what was wrong, but the model looks fine in 3ds, even running realistic lighting. Part of the problem seems to be the quantity of normals associated with each vertex, some of which seem to have minds of their own. The Noise visible on the ingame image is comprised mostly of half-fixed remnants of much harder edges. I can't seem to track down the remaining troublemakers though.

<It wasn't actually doing anything at all. Turns out I was just running a global unify on the model>

 

More edit: I was fooling myself, I didn't actually have it working, I was just unifying the normals on the regular model and tricked myself into thinking it did something. This is the result when I transfer normals from any object if I break them on the target object to begin with (as suggested in one of the threads). It looked identical when I transferred from the windowless skin. If I transfer onto non-broken normals they will all gather together and point in the same (seemingly random) direction.  Turns out this isn't actually the case, but comes closer to being the case as more and more complex models are projected. The most intelligible projection I've had has come from projecting a simple unsmoothed box onto the model with unified normals. Not a good projection, per se, but intelligible. The upside of all this is it means transferring normals from a windowless mesh may still totally fix it if I could get it to cooperate.

(Close look at the normals on the triangle pattern. Something to do with the extra normals all facing straight up?

Transfer from a capsule was for science, after studying the effects of normals all facing the same direction. I don't want to play the "please do it for me" game, but if anyone wants to look at this thing directly and see if they come up with anything I'd be happy to provide the files. I've been doing all sorts of random work on it; so far the most success I've had has come from grouping the model into ~four sections and having all those normals face the same direction with a little rounding in between them so the model only had about six "sides". So far I haven't been able to actually top the one in the images though.

 

Further science: Projecting cylinders and other complex primitives has been resulting in normals all pointing in one direction, with no outliers. Projecting onto ununified normals has produced the same traingle pattern depicted.

 

Edited March 15, 2016 by Balto-the-Wolf-Dog

[InsaneDruid]

I don't use max anymore since some years, only blender. So i can't speak much in terms of the actual workflow there.

You should have only one shared normal for each vertex (smooth groups?!)

Maybe source and target mesh should sit in the same position, as the script needs to know which vertex of the source is nearest to those on the target. This could be the reason why everything point into one direction if the source and target are located next to each other, not in the same place.

[Balto-the-Wolf-Dog]

yeah as I understand coincident normals are a smoothing group thing. Thing is, I never created any smoothing groups, at least not to my knowledge. Sketchup export so who knows. 3ds places normals on vertices not faces, so I would expect most verts to have one for each face that meets it. For whatever reason these seem to have an in-congruent number, and various reading seems to suggest it shouldn't be that way anyway. Think you may well be right about the alignment; going to give that a shot.

Edit:

It would seem you're right. Unfortunately transferring from the windowless mesh doesn't do as good a job as I've been able to manually. I think the problem lies in those stray extra normals. In general there's a lot of weird behavior with this thing. If I select all the normals and unify them a few errant normals will appear on the side of the fuselage perpendicular to the rest for no apparent reason. Some sets of normals seem to parent to each other and can no longer be moved independently, often coinciding with sets that don't have this relationship. I'm sure I'm misusing a feature I don't understand, but I'm fairly lost as it is.

Edited March 15, 2016 by Balto-the-Wolf-Dog

[InsaneDruid]

The windowless mesh has to be very high res. The thing is, you need to get the normals on these vertices right, that are added between the 24 faces of the base mesh. Imagine a circle with 24 faces, the normals of each vertex points otwards on a line that goes from the origin of the circle through the vertex. If you add some vertices between a pair of these 24 base meshes, their normals also should sit on such a line. If your windowless mesh only has 24 faces for its perimeter, and normal thief copies the normal from the nearest vertex this will not be the case.

[Balto-the-Wolf-Dog]

21 hours ago, InsaneDruid said:

The windowless mesh has to be very high res. The thing is, you need to get the normals on these vertices right, that are added between the 24 faces of the base mesh. Imagine a circle with 24 faces, the normals of each vertex points otwards on a line that goes from the origin of the circle through the vertex. If you add some vertices between a pair of these 24 base meshes, their normals also should sit on such a line. If your windowless mesh only has 24 faces for its perimeter, and normal thief copies the normal from the nearest vertex this will not be the case.

Doesn't look like that particular move is going to work out for me. So far the closest I've come is level 3 tessellation but it still didn't come out as well as the manual pass. Outright geometric smoothing (as opposed to face subdivision) didn't agree very well with the windowless version. It developed many small wrinkles that interfered, or I believe they did. I've also tried subdividing the object itself and resmoothing it. The wireframe makes it look promising as the polygons can be made roughly congruent to the window polys, but whenever I smooth I can't escape an interference pattern in the normals following the shape of the tessellation/subdivision. So far the results have been fairly messy. I'm going to take a shot at it with normal mapping as described here, though the preliminaries don't look all that promising. This thing's really turning out to be quite a chore.

 

The diffuse map is promising (aside from the issues on the flank, but those have been there and should be easy to correct. They result from those odd perpendicular normals. Unfortunately the normal tga is still showing faces even though the projection source was smoothed. Not sure why yet. 

 

Edit: Results

3ds max after preforming the normal map. Weird, but promising! No ugly window deformities.

Unity. Monstrosity. 

(When rendered in 3ds it looks similar to unity, though not quite as bad. I'm clueless as to what partial effect causes it to look better after the operation despite not being rendered.)

 

 

 

 

Edited March 16, 2016 by Balto-the-Wolf-Dog

[Porkjet]

The core problem is just bad topology. Custom vertex normals can't fix everything, If you do a really good job at making the topology nice and evenly you won't even need to resort to tricks like that

Edited March 16, 2016 by Porkjet

[nli2work]

Good low res mesh and good UV is absolutely critical. so fix those things first before anything else.

dont mess with per vertex normals. Put low mesh into 1 smoothing group,  moderate division is enough, ~32 cylinder. Lower you go the wavier your normal bake will be. Add projection modifier, add the high res to the projection list. push the cage out little by little until all the high res is just inside the cage volume. don't hand adjust the cage vertices unless you are absolutely sure the low res mesh will stay the same for all eternity. You want to add some support edges near sharp corners for better bakes, see below links.

do the rest like the autodesk walk through. Set Green to Up instead of default Down. 

I don't know what version of Max you're running, but 2015 Max occasionally jumbles UV on FBX export. no idea why, it's a known issue. Your Unity screen cap looks like that's what happened. Add a STL Check modifier on the mesh before export and the UVs should stay intact. Export your Tangents and Binormals with the FBX, and import that in Unity along with the model.

Read these two posts by Joe Wilson, multiple times preferably, if you want to really get a handle on baking good normals.
http://polycount.com/discussion/81154/understanding-averaged-normals-and-ray-projection-who-put-waviness-in-my-normal-map
http://polycount.com/discussion/107196/youre-making-me-hard-making-sense-of-hard-edges-uvs-normal-maps-and-vertex-counts

Edited March 16, 2016 by nli2work

[Balto-the-Wolf-Dog]

7 minutes ago, nli2work said:

Good low res mesh and good UV is absolutely critical. so fix those things first before anything else.

dont mess with per vertex normals. Put low mesh into 1 smoothing group,  moderate division is enough, ~32 cylinder. Lower you go the wavier your normal bake will be. Add projection modifier, add the high res to the projection list. push the cage out little by little until all the high res is just inside the cage volume. don't hand adjust the cage vertices unless you are absolutely sure the low res mesh will stay the same for all eternity. You want to add some support edges near sharp corners for better bakes, see below links.

do the rest like the autodesk walk through. Set Green to Up instead of default Down. 

I don't know what version of Max you're running, but 2015 Max occasionally jumbles UV on FBX export. no idea why, it's a known issue. Your Unity screen cap looks like that's what happened. Add a STL Check modifier on the mesh before export and the UVs should stay intact. Export your Tangents and Binormals with the FBX, and import that in Unity along with the model.

Read these two posts by Joe Wilson, multiple times preferably, if you want to really get a handle on baking good normals.
http://polycount.com/discussion/81154/understanding-averaged-normals-and-ray-projection-who-put-waviness-in-my-normal-map
http://polycount.com/discussion/107196/youre-making-me-hard-making-sense-of-hard-edges-uvs-normal-maps-and-vertex-counts

Good to know, I'll give that a whirl.

 

40 minutes ago, Porkjet said:

The core problem is just bad topology. Custom vertex normals can't fix everything, If you do a really good job at making the topology nice and evenly you won't even need to resort to tricks like that

The worst of it, if not all of it seems to stem from the triangulation of otherwise coplanar faces. I'm not entirely clear on why the normals in such a situation wouldn't treat a collection of triangles as coplanar even at unusually high aspect ratios. Regardless, this particular model has been a learning experience. Do you figure it's salvageable with tricks/manual retriangulation of the problem areas or should I just up and remake it? Assuming I do, is there a trick to adapting between the relatively low resolution standard profile and the detail in the windows or should I just experiment?

[Porkjet]

There are a few tricks yea. One thing that often works quite well it model a loop of polies around the window shapes that adapts to the regular mesh. Also you can reduce a lot of polies on those window curves. 3-4 sides for a 90° turn are plenty at that size. This will also make it easier to adapt it to the lower density. I can make you a paintover tomorrow.

You're right tho, normals on relatively coplanar faces shouldnt be that bad even with bad toplogy. Taking a closer look there might also be problems with smoothing groups and hard edges. here in the upper corner of the long window it looks like the hard edge that should go all the way along the window edge is smoothed, making perpendicular shading from the window edge bleed into the hull. Not 100% sure, but make sure smoothing groups are correct.

BTW 3DS smoothing groups are a weird concept tbh. Even tho it lets you control shading through faces, vertex normals really are a per vertex thing.
Think of them as a vector on each vertex perpendicular to the face, by default when mesh is smoothed, each vertex normal gets it's direction interpolated between neighbors. When you break the smooth shading with smoothing groups, what actually happens is the vertices at this break are duplicated and there's basically a cut in the mesh. The modelling software just treats it as if it was one vertice still.

[Balto-the-Wolf-Dog]

On 3/16/2016 at 2:39 PM, Porkjet said:

There are a few tricks yea. One thing that often works quite well it model a loop of polies around the window shapes that adapts to the regular mesh. Also you can reduce a lot of polies on those window curves. 3-4 sides for a 90° turn are plenty at that size. This will also make it easier to adapt it to the lower density. I can make you a paintover tomorrow.

You're right tho, normals on relatively coplanar faces shouldnt be that bad even with bad toplogy. Taking a closer look there might also be problems with smoothing groups and hard edges. here in the upper corner of the long window it looks like the hard edge that should go all the way along the window edge is smoothed, making perpendicular shading from the window edge bleed into the hull. Not 100% sure, but make sure smoothing groups are correct.

BTW 3DS smoothing groups are a weird concept tbh. Even tho it lets you control shading through faces, vertex normals really are a per vertex thing.
Think of them as a vector on each vertex perpendicular to the face, by default when mesh is smoothed, each vertex normal gets it's direction interpolated between neighbors. When you break the smooth shading with smoothing groups, what actually happens is the vertices at this break are duplicated and there's basically a cut in the mesh. The modelling software just treats it as if it was one vertice still.

Spoiler

After some retriangulation. It's not perfect, but its much, much better. I'm going to play around with it a little more, but I'm pretty happy. Thanks for all your help guys. Probably be back here next time I manage to make an even worse mess of something's normals.

Edited March 19, 2016 by Balto-the-Wolf-Dog

[Balto-the-Wolf-Dog]

Update with a fresh normal issue: attempting to normal map edition

In Unity with the "generate from grayscale" box checked:

In KSP after export

Unity or KSP without the from grayscale box checked

The map:

 

What gives? 

[nli2work]

I don't know how you got the map you got, but it's all wrong. I don't know you could have got it out of a grayscale map. Can you list your steps? i want to see if I can replicate it.

correct normal map should look like this

Edited April 2, 2016 by nli2work

[Balto-the-Wolf-Dog]

41 minutes ago, nli2work said:

I don't know how you got the map you got, but it's all wrong. I don't know you could have got it out of a grayscale map. Can you list your steps? i want to see if I can replicate it.

correct normal map should look like this

Truly this thread is my savior. This could have been hours of confusion; thanks so much.

 

 

To be honest I'm not exactly sure how I got the map I linked, as I tried quite a few things before coming here. I know this one produced virtually identical results, however:

It's simply what I understood (by way of some googling) to be the neutral blue with the door highlighted by photoshop's native bevel and emboss function.

Edited April 2, 2016 by Balto-the-Wolf-Dog

[nli2work]

the 2nd one isn't exactly correct either. but at least it won't give you the dark flat look from the first one, the one that looks more violet than blue. I suggest doing some more reading up on Normal Maps before attempting to utilize it. Specifically Tangent Space Normal Maps, which has the characteristic Blue back ground. There's maybe 1 in 1 million occasion where you attempt to create a normal map by editing RGB channels manually, the other 999999999 times it's tantamount to mental suicide.

The short of it is the R channel pushes the pixel normals left/right; G channel pushes up/down; blue pushes in/out. Problem is every 3d package and game engine have some different way of implementing Tangent Normals and some use R-255 for Right, some use R-255 for Left, same goes for Green. And for the most part, game engines don't even care about the Blue channel. Good thing is game engines can handle the internal conversions it needs as long as you give it the correct normal map.