If your temperature didnt go berzerk this sound to me as lacking NB voltage or to low freq. Whenever I had stabile OC but running applications (games) had issues, to slow, not working etc, it was NB not able to cope with OC CPU. Happily runing CPU with stock 3,5+ mhz at stable 4,7 mhz OC, for that +3 fps at KSP ship lanch