Peter James replies: The problem may not be the adhesive, but the surface perpetration of the silicon surface. It may need to be cleaned. One product often mentioned for this is Comet Bathroom Cleaner. Another is ketone solvent.
Bob Oswald replies: CA glue has pretty specialized applications and this would not be one of them. Gorilla glue bonds anything to anything, as they claim. And I’ve had good results with it.
Steve Costain replies: I’ve tried CA glue but it doesn’t work that well. I would try a silicone based adhesive, think silicone caulking.