I can't see your RCS placement, so I can't help you there. Make sure you have selected the other ship as your target. Then, burn retrograde until you are basically at 0 m/s. Then point toward the target (the pink-ish looking icon on the navball) and burn very slowly towards it. Make sure you have RCS on to do this so you don't smash the ships into each other. After that, I find that pointing the two ships in opposite directions really helps. I line one up facing North and the other facing South. Then, using the RCS translation mode (or the translation keys, whichever you prefer), very slowly push the ships closer to each other. Once you get a hang of the directions, it is a bit easier. After you line up in the x and y directions, it's just a matter of coasting in to the docking port. Here's a video by Scott Manley. He's really good at explaining things, so maybe he will be able to help you out better than I can. http://youtu.be/AHkY3FusJIQ?t=9m49s