Louis Dodds and Robert Taylor had turned things around and put Vale in full control but Marc Pugh and Tristan Plummer were given soft chances and they took them to bring the home side level.
Hereford took the lead against the run of play during the first period when Pugh charged in unchallenged and fired home.
Second-half strikes from Dodds and Taylor put Vale in front before John McCombe conceded a late penalty, which was converted by Plummer.
Adams told the Sentinel: "We should have won the game.
"There were two lapses of concentration, with one in each half costing us a game we should have won quite comfortably.
"We didn't take our chances at the Hereford end and certainly we've got to be disappointed with the two goals we've conceded.
"So, all in all, we're disappointed."