The visitors made a lively start and it was no surprise when Elding headed them ahead from 12 yards after Matt Mainwaring lofted the ball into the box.
Vale lacked confidence and ideas and only a fine tackle from Lee Collins denied Stockport winger David Poole a simple chance to add number two.
Marc Richards should have equalised early in the second half before substitute Sean Rigg did so just three minutes after his introduction.
Rob Taylor lobbed in an inviting cross from the left and Rigg reacted smartly to direct a firm header into the roof of the net from six yards.
Taylor then headed against the bar with Marc Richards crashing the rebound a foot too high with Vale rampant. But Stockport stemmed the tide after that and snatched an 86th-minute winner when Elding dived forward to head in a quickly taken free-kick by James Wallace.
Source: PA
Source: PA