Sounds like the controller has a bad contact in there. You may want to look into a new controller (maybe borrow one from a friend first just to be sure before handing over the cash for a new one).
If you know someone who is somewhat competent with electronics repair you may be able to get the controller taken apart and loked at- it could even just be a bit of dirt inside causing this.