Vanilla
Melee:
Spear and two handed axe combo has proven very effective in every test I have done, even against Olog-Hai (army size huge).
Ranged:
Catapult and ballista will get two or three shots off before they close and enter your line, making them nearly ineffective (maybe one troll killed, accuracy is horrible.... castle tower ballista is even worse). Arrows will help in knocking off a few of their hitpoints, but in vanilla bowman range means only a few volleys and then you have to redirect fire to avoid friendly fire incidents.
Crossbow are pretty effective, with one Olag-Hai dying before reaching the line. Javelins are the ranged weapon of choice, for troll or Oliphant, but simply placing skirmishers behind the line ( a common player tactic) won't work to counter trolls.
Rohan is probably the most effective with their mounted javelin skirmishers and succeed with fewer losses than any other faction (at least from the custom battles I have played).