Oil Refinerys. They are almost useless. Oil shouldn't be a problem if you have about 4 oil tankers. If you do build your shipyard too far from the oil rig, build another shipyard instead of an oil Refinery. If you do get into serious ship battles, with 10+ ships and find even 8 tankers don't get you enough oil, THEN get a refinery.
The 4000 gold level 3 upgrade for Catapults.Come on! 4000 dollars? Do you know how much crap you can buy with that? You only get 15 more damage. Not even worth it. Paying 4000 to upgrade a catapult is the kinda of thing that can lose the game for you. The 1500 gold upgrade is the only one you should pay. The only time it's useful is on Nowhere to Run.
Berzerker Regeneration It's just too slow and costs too much. Usually you don't have Berzerkers survive a battle too often either. If you got a huge amount of money and a bunch of 1/2 dead Berzerkers, that is the only time, I might get it. That's never happened in a game either.
Always stay a level ahead of your enemies in both ships and men. If you see they have level 3 ships build level 4 or 5. The same with men, build them up in the blacksmith. This has won many games for me. I would take out four level 1 knights with four level 4 grunts and the people playing me would be so surprised. Every level you up upgrade ships/men will pay off more. Its the best investment you can make, almost.