This is an extremely technical bug relating to how the thrust-at-atmospheric-pressure thrust values are calculated.
To explain what is happening in (poorly attempted) layman terms, all non-jet engines are weaker in thicker atmospheres. This is done by using what is known as a curve, which is a way to smoothly "draw a line" between two or more points.
The Ant engine has three of these points. The first point is at 0 atm of atmospheric pressure, and gives an Isp of 330s. The second is at 1 atm (Kerbin sea level), and gives an Isp of 80s. Between these two points, the game engine (Unity) will smoothly increase/decrease your Isp — and by extension your thrust — to be between these two values when you're between 0 and 1 atm of pressure. For example, even though only 0 atm and 1 atm have known values, the game will still give you an Isp at 0.5atm, that's somewhere between the two (I'll edit in the exact value later once I can properly calculate it). There is also a third point at 3 atm of pressure and 0s Isp.
Part of how the spaces between these points are calculated involves additional information, which I will broadly give the name "tangents" (these include the in-tangent, out-tangent, in-weight, and out-weight). I personally don't fully get exactly how these work, so I won't even try to dive into explaining them thoroughly, but generally their purpose is to define the shape of the curve between the points. There are many possible shapes these tangents can create.
However, here is where the issue with the two engines come in. The points and tangents of the Ant and Dawn are defined in such a way that there exists a range where this curve drops below 0 and becomes negative. Effectively, this means that the Isp and thrust of the engine become negative. Throttling up the engine while pointing prograde would make you slow down instead of speed up. This completely violates physics, and is unrealistic.
Does this actually matter in the slightest? No, not even remotely. These negative thrusts are absolutely negligible (like ~1 newton (not kilonewton) of thrust per engine), do not generate fuel, and use two deep-space engines in atmosphere, so this will have no realistic impact on gameplay in basically any scenario. But I spent 2 hours testing it anyway.
This could be solved by adjusting the tangents so that the curve never goes below 0. Again, I don't know much about tangents, but if every single other engine in the game is unaffected I'm sure these engines can be fixed too.
I'm probably forgetting something. If you're confused, ask a question. This isn't a high-priority bug by any means, but it's definitely something curious that I don't think has been noticed before.
