KerikBalm

Pretty much every part except full ore tanks in buoyant. Interestingly the medium sized rotor sinks while the other two float. Aside from it, I haven't found any other non'ore tank part that sinks. Even structural girders float (hmm, maybe I should try I-beams).
When operating on kerbin, I'd say the best engines for a boat are goliaths, due to their high Isp. Air intakes will work above and below water, as will jet engines (that's KSP for you).
If you want, you can use the breaking ground rotors to make screw propellers:

But Jet engines are simpler and more powerful, particularly if you don't care about neutral buoyancy as with a sub. Before breaking ground I used them to make subs (wouldn't work on Eve, or a mod world like Tekto though) where the sub would start slightly heavy and would slowly sink when full of liquid fuel, and would slowly rise when empty. The low fuel consumption of jets meant that short trips had very minimal buoyancy changes.


boats are much easier, just about anything floats. You have to try to not make it float. So just take some mk3 fusalages (high impact tolerance), and put some jets on them... like this without wings, and more efficient jets:

Paddlewheels can work too, but they are slow, as this pre-breaking ground (using the old separate craft for rotors trick) demonstrates.

If all you want to do is pick up a capsule, then the current robotic parts are plenty. I tried to make something that could lift a half fueled Rockomax jumbo 64... it was... bendy.
I could have stengthened it, but my design goal was fitting it in a mk3 cargobay:



A recovery boat doesn't have the same restrictions
 

I've made stock propellers that work on Duna and Eve, in the atmosphere... but they don't work in the ocean (so much for an Eve submarine without mod parts). They seem to just create drag vectors in the water without any lift vector. The atmospheric forces should still be present though, maybe if I did two counter rotating ones, the water drag would cancel out and I'd have very weak aerodynamic thrust...
The OP seems to have a similar goal that I had in this thread:
I've made some re-dockable (the ship needs to undock to have one part propel it, but that part can rejoin/redock) paddle-wheel style boats, but I couldn't get them much over 1 m/s... less than 4km/h... not practical at all.
On that thread, someone posted an air-boat or fan-boat design that go a decent speed on kerbin. That may be the way to go.

This one initially had docking ports at the front and back. You could undock from the front, fly around, then stop the prop and use air resistance (or put it in reverse on the ground) to have it slide back and dock at the back. (a quick save and load helped here as I sometimes had problems with the docking ports resetting).
I also made a design with 2x structural fuselages instead of a fairing:
To try and improve its usability with physics warp (but mostly: nope, going over 2x physics warp still broke it). Also note that the prop blades sort of stretch farther out at high RPM, they don't start with such a gap between themselves and the rotor.
That design had a single docking port part, but I could move the rotor far enough forward and back to have the docking ports reset and enable them to reconnect.
So I was thinking of using that as a sub-assembly, and mounting a pair of them (counter rotating of course) on some sort of air-boat on Eve.
 
 

Well, there are 2 basic ways to make a submersible in KSP.
1) is a dynamic diver, basically its got positive buoyancy, and you just use wings (hydroplanes) to keep it down and dive. Stop the engines, and you rise
2) is a neutrally buoyant submarine. For this you need to use lots of heavy parts: ore tanks basically.
The problem with this is that there aren't any good electric propulsion solutions in stock, so you'll want to use a high efficiency turbofan (wheesley, panther, or goliath) for propulsion, so it won't work in Eve's seas. Also its buoyancy will change as you consume fuel, so you may want to use the open/closed cargobay trick. I tend to make the sub slowly sink with a full load of fuel, so when its nearly empty, it will slowly rise (of course, ore jettison is an option, but then its more of a hassle to resupply)
If I recall correctly, unmodded, there is no ore in the ocean biomes, and never will be any (I do mod this though), so this will only ever work near the shores, while underwater but still in the shores biome.
Here's a stock submarine that I made (on a mod world):

Its got a wheesley for propulsion, and small ore tanks for transfering ore to maintain balance as fuel burns off. Its got rover wheels, but it doesn't really drive well on the bottom (almost no grip, near neutral buoyancy with its standard ore load).

The wheels are for driving onshore to meet up with a surface base near the shore to refuel, or for driving up a cargo ramp for airborne transportation
Testing on laythe (before changing the front docking port to a shielded one):






 

Low tech pre-whiplash/rapier SSTO spaceplanes probably aren't worth it...
But high end SSTO spaceplanes are definitely economical for funds... maybe not player time. Their 4000/3200 Isp offsets the mass penalties easily.
SSTO rockets on the other hand do consume more fuel than a conventional staged rocket, but they still come out ahead in terms of funds.
SSTOs are very practical once you have a working design.
You can put together pretty much any mission you want using only spaceplanes to get payloads to kerbin orbit
 
Just 2 launches ot get this payload into orbit with a spaceplane (a moho mission, where everything can be reused or recovered):

http://i.imgur.com/EZ6XwJt.png
http://i.imgur.com/JiHWstz.png
http://i.imgur.com/FOfPjuO.png
3 launches for this:

(not including the SSTO laythe lander docked to it)... well.. really only 2, as the lower KR-2L stage is just for ejecting the craft, and it gets refeuled from ISRU at mun (after a retroburn to keep it in kerbin's SOI and aerobraking to lower Ap to Mun orbit)... so its the same launch as the KR-2L part of the first payload.
You can even launch laythe landers for less than 180 funds/ton
http://i.imgur.com/h5QS5sP.png
My version as of 1.1.3 (sans payload)  http://i.imgur.com/BJLhFLs.png
Also note that you can do vertical launch rockets that land like spaceplanes... and you can even stage them using a flyback booster... although the fuel required for the booster retroburn is so great (because the booster needs to get an apoapsis over 70km to enable you to switch back and forth between it and the orbiter as you need to recover both), that its payload fraction is about the same as if it doesnt stage and just SSTOs... still, its cool.
This can SSTO without staging, and thus would count as a panther SSTO, even though the panthers have only a small contribution towards making back ot orbit and are mainly for the booster recovery.

Booster retroburn after separation: http://i.imgur.com/utRkpeF.png
Orbiter insertion burn + orbiting with payload: http://i.imgur.com/Hfx71vf.png
http://i.imgur.com/e1JK9ul.png
Booster approach and landing: http://i.imgur.com/vaDw4vc.png
http://i.imgur.com/bpWg19y.png
 
I will also note that this surface base was entirely SSTO'd... not just single stage to orbit, but single stage to Mun orbit:

The Mun dropship at right was single staged to orbit of kerbin without any airbreathing engines, and without any spaceplane stuff:

Just mammoths and LFO, and then an infaltable heatshield as an airbrake + chutes for landing... I manages >97% recovery from landing the boosters close enough to KSC. The other modules were brought in a spaceplane... I could have brought them one at a time with a craft like this:
http://i.imgur.com/Fvfy9vY.png   (with a slightly extended cargobay) or this: http://i.imgur.com/akhX6tm.png ,   but I made a spaceplane to take all three modules at once (even though I then ferried them down 1 at a time with the mun dropship and an orbital tug (loading it in orbit was a pain though)