Aside from the normal glitches like floating books, floating people and being able to run through objects there are also some intended funny (but useless) things to see- and lots of items to obtain!

Dancing soldiers
At least now we know what the Galbadians do when they don't have a war to fight... When you're in the missile base, go to the control pad. Hold Square and Triangle (keyboard: W and A) and press up or down to see a dancing Galbadian or Elite Soldier.

Dancing Robot
Soldiers aren't the only dancers in the game. Go to Mayor Dobe's house (second floor) and check the robot next to the hidden Ultima draw point (both to the right).

A melodic concert
Choose the following instruments for the concert in Fisherman's Horizon:
Saxophone
Piano
Bass guitar
Electric guitar

or
Guitar
Tap dance
Flute
Violin
and you will have a different conversation- and different songs.

Free Item giveaway
...is not as rare as you may think. After clicking away several "There's nothing here"- messages on a pile of magazines in Dollet, you will find up to three items! Re- enter the pub and click on the same pile of magazines again... and re- enter again... and again... and again... the supply of random items is unlimited. I found everything from an Aura- stone over Gysahl Greens to Potions and Ammos there.
Also in Dollet: you'll find a guy looking at the trains who gives you a potion every time you see him if you have Quistis or Rinoa in your party.
When Edea gives you her Sorceress letter, you can sell it for some money and get a new one from her as often as you want. This is completely useless though, because the amount of money is incredibly low and you probably gain a lot more alone by beating the monsters on the way to a shop (unless you have the Tonberry GF, but even then the letter- selling is useless because it will take forever).
More useful is this tip: In the second Laguna- sequence (Lunatic Pandora, but you don't know this yet), go to the room with the three doors on the ground. Break the handles on each one, then continue to the next room where a little red thingie will be on the ground ( I haven't quite found out what that's supposed to be, yet..). Hit it, then hit the blue one so the doors open. You can't get in there yet though. When you enter the Lunatic Pandora with Zell and Edea, go to the room with the three elevators and take the right one. Follow the path until you reach the room with the three doors- NOW you can go through and collect your free items.

Laguna is bad for transportation
Attention: Laguna may cause traffic jams! Actually, he does. Once you enter Deling City as Laguna for the first time, go straight to the hotel and leave again- and acknowledge the huge traffic jam Laguna caused, because he parked on the open street....

Is Rinoa really Ultimecia?
I didn't know where else to put this, but I suppose it would fit into this section. While unfortunately the original page with the original Rinoa/Ultimecia theory is down by now, people are still speculating whether Rinoa is or is not Ultimecia. There is a quite interesting thread about it on the VNboards for example which has some valid arguments, and another anti- statment to the original theory on RPGamer titled Rinoa is not Ultimecia.
I think they present very interesting arguments and I would have supported the theory myself; however Square-Enix stated in the popular "The Final Fantasy VIII Ultimania Guide" that witches have normal lifespans (as in, they live as long as a regular human) and considering Ultimecia lives in the far future, it's about impossible for Rinoa to be her, unless she drank some kind of life elixir. Nothing about that is mentioned or hinted in the game though, so it's highly unlikely. Admittedly, Rinoa being Ultimecia would have been a deep and interesting twist in the storyline- and given Ultimecia more depth. Instead she is just another random superevil monster who wants to take over the world. A pretty one, though. But that's just my opinion.