If you are having problems because barbarians are knocking down your walls don't give up. Build your walls with wells. They cost less and the enemy can't get through it.