Hanno Dirksen's try and eight points from the boot of Matthew Morgan looked to have teed up the Ospreys for their first win of the new RaboDirect PRO12 season.
But Jackson shook off the calf injury he suffered against Glasgow Warriors and Marshall continued to step up his recovery from elbow surgery as the duo combined to deadly effect.
Replacement scrum-half Marshall crossed to give Ulster a 75th-minute lead while Jackson contributed eight points making it back-to-back wins for Ulster.
And Anscombe admitted he was a relieved man.
"The subs came on and really lifted the performance. Some of them haven't played a lot of rugby recently but it was great to be able to bring players of that calibre on," he said.
"I'm very relieved to come away with the win. It's never easy to go to Swansea and come away with four points but we've achieved that so we have to be pleased.
"I was disappointed with the performance in the first 65 minutes. We didn't look after the ball well enough, we were turned over a lot and we didn't play territory like we had talked about.
"We seemed happy to amble through the game and I thought we generally lacked a bit of fizz and didn't show enough purpose.
"But I thought we played very well in the last quarter of an hour and it was great to get the win in the end.
"The scrums were very good in the second half; Nigel [Brady], John [Afoa] and Tom [Court] were very strong and put their front row under pressure.
"Allen Clarke has done a lot of work on our set pieces and we are reaping the rewards."
63x46.png){kind=logo}