Dealing with complex geometry inside simple geometry

[Balto-the-Wolf-Dog]

Since I started modelling a few years back I have consistently had trouble with bad lighting by virtue of poor triangulation/quadification, usually around inset windows. I know the problem has to do with congruence (or lack thereof) between the window edges and the surrounding faces, but I have yet to develop an effective means of dealing with it. Most of my projects have allowed me to worm my way around one way or another, but this time my desperate attempts at retriangulation haven't brought me any luck.

The terrible in Unity:

 

The worst offender is the aft window (though the middle window is surprisingly bad too) . The root of the problem seems to be that the window in question is cut into an otherwise flat, geometrically simple face, so when the mesh triangulates the resulting geometry involves many elongated, incongruent triangles. I've tried cutting the mesh around the window and retriangulating it in several patterns, but I have yet to produce a serviceable result. Is there a trick to this kind of work? If not, how might I approach the problem better than I have been?

[InsaneDruid]

Use more geometry for the base mesh. Ideally, you should only use quads (use at-j in blender to automatically convert tris in quads).  You seem to understand the problem, but you also seem to "fear" to use some more vertices. Don't be afraid. Use them where they are needed. You can only do so much with ultra low polycount. Also you use different vertice count for the inner and outer curves of a window frame. This leads to triangles, and usually to elongated ones. Always seek a mesh that avoids triangles by having egqual amounts of "corners". See your screenshot:

 

 

each of the blue dotted line sould be an uninterrupted edge, yet your amount and position of vertices is never consistent, aways wildly mixed with vertices on positions where you dont need one and other positions in need of vertices left empty.

 

Seek for tutorials with "mesh topology" in the title. Most of them are for modelling organic shapes, but the principles remain. Only if you understand what a perfect mesh topology would be you can decided where to break such a perfect topology to save some polygons.

 

for example CG cookie has some older videos (old but perfectly fine and still true in all details) for free (only needs a free account on cg cookie) https://cgcookie.com/archive/learning-mesh-topology-collection/