Make sure there’s a DC blocking cap on the output. You shouldn’t have DC on the voltage output, that can misbias the input stage of whatever you’re plugging into.
You should propably not use any electrolytic caps in AC situations, the can “take” only 1 polarity and may get damaged or even explode if voltage is applied in the wrong way. A ceramic cap in series (parallel wiring might give you plenty of DC and be pretty pointless - Lowpass) with the signal line should block DC good enough. If touching the ground changes stuff, something else might be touching the ground when it shouldn’t, so check for shorts.
When checking with an osc., be sure to have DC adjusted to zero before checking, and that all grounds are wired properly.