If you lost downrigger balls that easy you must be using those Scotty plastic downrigger clips. Throw them out if you are, and get the big snap swivel kind, and set the clutch on the downriggers. I havn't lost a ball since i switched over. (knocks on wood) I even snagged submerged pile of something near the Red container cranes last summer, and the clutch released cable like it should, i yarded the other lines in real fast. Then gave a little slack on the down rigger, and slowly back straight back where i had come from. The ball came undone. If i had been using those plastic clips, i would have lost that one.
My intermitant problems have been related to those craptacular Bosch/Hella 12v plugs. They suck, and must be change out for something else.
Good luck.