Do you hate it when your craft is tugging along on SAS and you want to correct the course a bit? You touch pitch controls and it gets all wobbly.
There is a simple solution to your problem: SAS should not control the control input, but work via the TRIM settings. You control input would be combined with the TRIM and you would have perfect controls even with SAS activated.
Better yet: Allow the TRIM settings to be kept when leaving SAS (maybe with a modifier key or so) and move your craft from SAS to full manual control without even the slightest bump.
Why we should do this? Because usually it is done like that. Your input is combined with stabilisation/trim.
If it is still unclear what I mean exactly, move down three posts. User OHARA does explain it better!