Gareth Bowen missed four first half penalty attempts while in contrast Adam Larkin was on form to give the visitors an early 6-0 lead.
Bowen finally found his range at the close of a scrappy first period which had seen both Scarlets lock Chris Wyatt and Ulster flanker Neil Best sin-binned.
The second half looked to be following the uninspiring form of the first as Bowen and Larkin exchanged penalties and then Bowen levelled things up with a 54th minute kick.
Bowen was certainly redeeming himself after his early misses and made it 12-9soon after when Ulster, not for the first time, were penalised for offside.
Then came the game-clinching try on 68 minutes with Phil John rounding off a powerful forward surge and Bowen adding the extras.
The fly-half landed his fifth penalty in injury time to seal the win.
63x46.png){kind=logo}