First, ignore the old modem and router from upstairs- they aren't needed.
As for the router downstairs, you may need to add some settings in there. Add your PS3's MAC Address to the client list of that router and, if it has it, set up a reserved IP address for the PS3 using that MAC Address. I would also go into the settings of it and get some of the connection information such as the router IP address, subnet mask and DNS server addresses to add into your PS3's settings. If you have encryption set up you will also need the key to add to your PS3's settings.
Finally, set up the port forwarding on the router- I have the ports needed listed in the pinned thread at the top of this section.