Stupid? No. Annoying- definitely.
You may want to go into your router settings (you are connecting through a router?) and go into the port forwarding section (usually in Advanced or Security). In here, make sure you have the necessary ports set up to be open. They are:
TCP Ports: 80, 443, 5223, and 10070 - 10080
• UDP Ports: 3478, 3479, 3658, and 10070
Once you have these all in and activated you should have no more problems.