However the manager was forced to endure a nervy 90 minutes as his team missed a series of chances including a second half penalty.
Following a cagey opening to the match, Alan Thompson fired Celtic ahead.
Craig Bellamy won a free-kick on the right flank when he was brought down by Alan Archibald. Thompson smashed the set-piece towards the near post and his low delivery deceived everybody as it flew into the back of the net with the aid of a deflection.
Bellamy had an excellent chance to increase the Bhoys' lead midway through the first half but the Welshman saw his shot saved by United keeper Tony Bullock.
Then 60 seconds later, Stilian Petrov pulled an effort wide of target as Celtic created the better chances in the opening stages.
A tremendous block from Archibald denied John Hartson at the start of the second half. Bellamy cut the ball back to his fellow Welshman and Hartson looked certain to score from close range, but Archibald flung himself in front of the ball, deflecting it behind for a corner kick.
Celtic should have increased their lead as the second half progressed but they spurned a series of chances as Bellamy, Thompson and Hartson all went clear but failed to net.
Then with five minutes remaining, Celtic were awarded a spot-kick when Bellamy was tripped inside the box by Garry Kenneth.
Chris Sutton stepped up to take the spot kick but he slipped as he struck the ball and blasted it high over the bar.
However the Parkhead outfit held on in a nervy five minutes of injury time to the chorus of victory that brings down the curtain on O'Neill's reign as Celtic boss.