What they really needed to do is show us (or at least tell us) that this new group suffered at the hands of other survivor groups, and then I'd buy their motivation for following the Governor more. There were two major clues we saw to their group's mentality. First, the tank-driver and his brother got into an argument over raiding that random camp for supplies. This tells us that this group hasn't had to do anything evil yet to survive. They were still in the "humane" stage of survival, in that they didn't need to hurt others to be safe. Second, the group raised a ruckus when Martinez was killed, upset that they weren't going to vote for leader. We have a group that hasn't needed to do "evil", following the orders of a man they didn't elect. The Governor's charisma is another variable, sure, but Woodbury actually did suffer at the hands of Rick's group, these new people didn't. With all that in mind the attack still feels weirdly out of place to me.
As for Herschel: If Guv believed Herschel was the more valuable hostage (which could certainly be true, that's a very good thought I hadn't considered), then why doesn't the Governor put one in Michonne's head before attacking the prison and simply lug Herschel along as the sole hostage? In the circumstance the writers created, Michonne does not live if the Governor behaves like the Governor.
I agree with you concerning the battle itself, but I too can forgive these as simple television action tropes.