It seems from yesterday's dev notes there were a lot of pending doubts as to how the cargo bays would function excatly, so here's my attempt to explain it in more detail.
First off, I should make it clear that with Cargo Bay, I mean the cargo bay Part Module, as in the bit of code used to make them functional. This module mostly deals with containment detection, and it should be possible to apply it both to the existing cargo bays and to new fairing parts, when we get to that. For now though, the purpose of the cargobay module is twofold: To detect (as efficiently and accurately as we can) which parts are inside the bay and which aren't, and to flag those parts as not exposed to the outside world (aka shielded).
This cargo bay module isn't entirely new in fact. It was first written sometime last year, during the blur that was the last months of the 0.90 release, and it was pretty much working already. The problem then was that by itself, it was deemed to be too little to count as an improvement over the existing (old) aero model, so we decided to leave it out until we got to a point where we could tackle aerodynamics properly (aka now).
Another thing that needs to be made clear is that the following method is used to detect containment relative to Cargo Bays, and cargo bays only. This is not the same method used to handle aerodynamic occlusion for the new drag system. That is an entirely separate thing, which would be the subject of an entirely separate (and equally long) post.
So, the first problem to solve is that of containment. There were a number of potential pitfalls we needed to have our solution avoid:
The first issue is that we saw early on that we couldn't rely on any of the 'standard' ways we usually use to work with part relationships. A cargo bay can contain parts attached to the same vessel, but then again, it might contain detached debris (or Kerbals taking a zero G joyride), so we can't rely on the vessel hierarchy, or any of its lists of parts to detect which are contained in the bay, as that wouldn't include parts that don't belong to the vessel at all.
Also, we have the problem of parts which may be partially inside, or surface-mounted to the internal or external walls of the cargo bay. That means we can't rely on the part transform.positions either, as those often are placed at extreme ends of parts, which means they aren't accurate representations of where parts really are.
Furthermore, mesh containment is in itself a complex problem to solve with full accuracy in software. A complete solution would require going through every vertex in every mesh, in every part against again, all verts in all meshes of the cargo bay. Not exactly fast... needless to say. That means our solution must also not be computationally impractical, so we're looking for something that falls in the 'good enough' zone, where it works as you'd expect without reducing the game to a slideshow. Fortunately, we don't have to solve for contained parts every frame, so we have some headroom, albeit not a lot.. But a very complex solution would also give us very complex results to deal with, which would become an issue all of their own... We could easily get caught in a never-ending spiral of complexity there, and that's the most dangerous pitfall of them all, because there is no limit to how complex a system can be made.
So taking a step back, what we really need is a solution that hits the 'predictability' zone, where if something looks like it should be contained, it should be contained. The emphasis is on the word 'looks' there, you'll see why in a bit.
Here's how we tackle the problem of detecting containment:
* The cargo bay first detects all parts inside a spherical radius that entirely encloses it. Those are the nearby parts that will be tested. This detection method is independent of vessel relationships, being solely dependent on the position of parts. A lot of the parts in this first list will be outside the bay, and that's fine. We just do this to avoid having to iterate through every single part in every single vessel.
* Next, we compute a centroid point for the part, based on its renderer bounds. That centroid is then a product of the part's visual mesh, not dependent on attachment position, center of mass or even colliders. It's the visual center of the part. This is now our reference to test the part in a visually accurate way.
* We then rule out any part whose centroid lies outside the merged visual bounds of the cargo bay itself. This is just a sanity check to avoid performing needless calculations on parts that are clearly outside the cargo area already. The merged bounds are a bounding box we calculate off all renderer bounds, to encompass the entirety of the part. This is necessary as cargo bays (or fairings) can potentially be made out of multiple objects, meaning multiple renderers, and multiple bounds, hence the need to merge them.
* So, we now end up with a much shorter list, comprised of nothing but parts which are in a reasonable range to warrant further testing. Now it's time to really check whether we are in or out of the bay.
[a quick note here to explain that all this is done with the assumption that the bay doors are closed. If they are open, no part would be considered as shielded by that cargo bay, so that's something we can safely not worry about]
* The test itself is simply a raycast, from each part's visual centroid to the bay's own centroid. Now, the thing with raycasts is that they can (and do) hit any parts along the way, and we don't want that. We are just concerned with the bay itself and each part, in isolation. This testing then is done against only the bay's own colliders.
* From this we can now determine if a part is inside or outside of the bay with a pretty decent level of accuracy. If the raycast reaches the bay centroid without hitting anything, we can be quite sure it's inside the bay. If it hits anything, it must by necessity have been a wall of the bay itself, and therefore must be outside the bay.
Now, you may be wondering then, what of the open sides of the bays then? Surely there is no wall there. Indeed there isn't, but that doesn't mean we can't add a 'wall' that only exists to catch a raycast. Trigger type colliders will do just that in fact, so for all testing purposes the open ends of the cargo bays are effectively closed. This still lets you pass though the open spaces normally, but catches the raycasting so we have a fully enclosed volume which defines the cargo bay.
As for the shielding itself then, after the containment is solved for, it becomes much simpler. We can now flag parts as being shielded, which is then used by each part's modules to determine how a part should behave if shielded. This way, we can prevent surfaces from generating lift while inside a closed bay, protect them from any heating from atmo friction, prevent them from receiving drag forces, and keep some parts (like parachutes and such) from functioning at all until the bay (or fairing) is open.
The only limitation of this method is that there is no support for partial shielding. A part is either inside or outside the bay. This I believe is fine, as it still means we meet all the initial requirements for the parts: We can protect payloads from being exposed to drag forces and air friction, we allow you to pack lift-producing parts inside the cargo area and not have to worry about those affecting the ship's handling (especially useful if launching a flying thing meant to do its flying off-Kerbin), and we can exclude the enclosed parts from being affected by drag, which is the primary purpose of cargo bays (and fairings). Plus, the solution is very simple to work with, which is also a good thing as we get to keep our sanity.
This method also works independently of vessel hierarchy relations (or parts even belonging to the same vessel), and reliably works in tricky cases, like parts being side-mounted to the bay, which could potentially have their transform positions at the wrong side of the bay walls. It works based on the visual mesh, so if it looks like it should be shielded, it probably will be.
Hopefully that explains the process enough to answer most of the questions about how cargo bay occlusion will work.