A router is actually preferred so that your computer can stay online at the same time.
Your router should have ports on the back for wired connections- run some regular ethernet cable from one of those to the port on the back of the PS2. Once that is done you will have to set up a connection file either using the online set up disc that should have come with your system or just use any online-enabled game. When you choose the online section of a game and select "make a new connection" you will be taken to the set up routine.