1. Line all your units up so there are no gaps between them
2. Put your most experienced units on the flanks (they will usually have to withstand the heaviest attacks)
3. When you want to move a phalanx, order it to move first, then take it out of phalanx mode. Try not to move them around a battlefield in phalanx mode, they are far too slow and it breaks the formation (but see point 4 below)
4. Never leave it to the last moment to put it back into phalanx mode, sometimes the unit mills around for a good 30 seconds when ordered into phalanx, and if you get attacked during that time you are in deep do-do
5. Never let anyone attack your phalanx from the sides or rear (especially archers, velites, slingers and cavalry)
6. Guard mode has a big effect. With it off your phalanx will fight better, but your phalanx will also chase after any retreating enemy which pulls it out of line and breaks up your formation, exposing the phalanx's flanks (see rule no. 5). So, I tend to leave it switched on when in purely defensive mode (ie. enemy are attacking me) and switch it off (but keep a close watch on them) when attacking
7. Where at all possible, support them with archers. Enemy missile troops can cause horrific casualties to a phalanx and you are too slow to capture them. Use the archers to kill them instead (particularly Cretan archers)
and always, always remember that the whole point of the phalanx is to act as the anvil. You need to use it to engage enemy troops, tie them down and then attack them from behind with cavalry/missile troops. Phalanxes by themselves are only usefull for defending bridges and streets in towns. Anywhere else and they are easily destroyed.
ps. I'm planning on doing a walkthrough of a phalanx army vs. Rome this weekend. If I do, I'll put the link in here.