Kris Boyd was the hero after scoring the winning goal from the penalty spot following his two equalisers in normal play.
Twice Dundee United took the lead, but Rangers' never-say-die spirit saw them pick themselves up to draw level.
For a long time it looked as though Noel Hunt's first-half goal would be enough to see United through.
But a shocking mistake five minutes from time by Mark Kerr allowed Boyd to seize on his weak pass-back and shoot past Lukasz Zaluska.
The visitors went ahead again in the first period of extra-time when Mark De Vries drilled his shot low past Allan McGregor
But back came Rangers and once again it was Boyd who rescued them with a header at the back post from Steven Davis' cross.
It was almost written in the script that Boyd would win the cup for Rangers with the last penalty of five after both sides had missed two spot-kicks.
With the penalties level at two each, Lee Wilkie's spot-kick was saved by McGregor diving to his left before Boyd lashed his penalty kick past the United keeper.
It was cool luck on a United side, who deserved to win this match.
Leading at half-time through Hunt's goal against the tired-looking Rangers team, the Terrors had a good penalty claim turned down when Carlos Cuellar hauled Christian Kalvenes to the ground.
Rangers sent on Jean-Claude Darcheville and Boyd in an attempt to boost their attack, but it was Kerr's shocking pass back that allowed the Glasgow club to equalise.
In extra-time, Craig Conway's pass was touched on by Prince Buaben to De Vries, who swept past Kirk Broadfoot before firing in an unstoppable shot.
Then came Boyd's equaliser to set up the penalty shoot-out which Rangers eventually edged by a 3-2 scoreline.