Polearm accuracy penalty applies to all classes regardless if they are mage or fighter. There are no penalties for using physical attack to kill mobs on mages.
You just lack all useful skills (including passives) and have garbage stats that matter (STR, DEX, CON).