The KSPI reactors will all match their output to the power demand, so that no fuel is wasted, so yes, if you don't have anything requiring power hooked up to it, the reactor will produce minimum power, which IIRC is zero for the antimatter ones, as they do not need a minimum sustaining reaction. To test whether the reactors are working, I would recommend that you try hooking up a high power device to them, activate the device, and see whether the reactor outputs power. As long as that is the only reactor on your ship, you know that if the power using device works, then the reactor works, as long as the reactor is the only source of electricity.