Jamie Cureton netted a late, late winner to keep Rangers on the tails of the top two - seconds after Adrian Littlejohn had broken the home side's hearts.
Cureton struck in injury time just after fellow substitute Adrian Littlejohn bagged what he thought was a point for Port Vale.
Former Reading striker Cureton had scored just two minutes after coming on for Kevin McLeod after Marc Bircham had cancelled out Stephen Brooker's early opener.
Rangers, who have not lost a home league match since Wigan defeated them in March 2003, went one down in the first minute through Brooker who had the easiest of near-post headers off Marc Bridge-Wilkinson's corner.
But blue-haired Rangers fan Bircham equalised in the 64th minute from Kevin Gallen's free kick before Cureton, who signed in January from Korean side Busan Icons, converted Martin Rowland's run through midfield.
The goal could have come much earlier. Port Vale goalkeeper Jonathan Brain was forced into two diving saves from shots by Rowlands and Tony Thorpe following sustained pressure from Rangers early on.
But play off-chasing Port Vale were more than a match for Ian Holloway's side, and they again rattled Rangers when Brooker was left unmarked to get a shot off from just outside the penalty box.
Kevin Gallen came close for Rangers after 12 minutes when his shot went inches wide, just before Clarke Carlisle headed over the bar.
And Arthur Gnohere should have done better midway into the second half when he was unmarked at the edge of the six-yard box only to head over.
Rowlands came close to scoring Rangers' second a minute after Bircham's goal, but his shot went just wide of the near post.