The game turned on a controversial decision by referee Brian Winter in the 77th minute when he brandished a red card at Christian Kalvenes after the full-back had slid in hard on Killie's Steven Naismith.
Slippery underfoot conditions may have made the tackle appear worse than it was, but even a yellow card would have appeared harsh.
The extra man gave the home side the impetus they needed to remove the shackles with which United had smothered them, and goals from Willie Gibson and Colin Nish completed the Terrors' misery.
United created by far the better chances in a dire first half, with Barry Robson celebrating his first international call-up by initiating all of his side's best moments.
The visitors could have been comfortably ahead by half-time, but Noel Hunt failed to convert from four yards after Craig Conway had burst past David Lilley and Fraser Wright, and the Irish striker then headed Robson's perfect cross wide.
Hunt made up for his earlier misses five minutes after the break when he put United in front. Simon Ford gave away possession easily in his own box and allowed Robson to progress to the dead-ball line where his scuffed cross reached Hunt four yards out and this time he made no mistake.
The home side failed to create any chances until the 67th minute when Naismith fed the onrushing Wright who had a clear sight of goal but blasted over.
Then came the game's talking point when Kalvenes slid in on Naismith and was promptly dismissed to his and both managers' astonishment.
It provided the home fans and players with the encouragement they needed to press forward and they were level two minutes later.
David Fernandez held the ball up in the box and it was eventually cut back to Gibson on the edge of the box. He struck his shot well, but it still needed a wicked deflection off a United defender to beat the keeper.
It was only a matter of time before resurgent Killie stole the match, and the winning goal duly arrived with three minutes to spare.
Wright did well to control a waist-high bounce to fire his shot beyond Grzegorz Szamotulski, and Nish showed a striker's instinct to follow the ball in and make sure it crossed the line.