Ospreys looked to be cruising to their first win of the new season after Hanno Dirksen opened the scoring with a try in the 24th minute.
And, despite a Niall O'Connor's penalty seven minutes later, Ospreys increased their lead to 13-3 after the break thanks to two kicks from Matthew Morgan.
Ulster gave warning of their intent to get back into the game when replacement Paddy Jackson kicked two penalties himself to make it 13-9 with 13 minutes to go.
And the Irish outfit duly completed the comeback when another replacement Paul Marshall dived over in the 74th minute and Jackson converted to make it 16-13.
Ospreys backs coach Rees was in no doubt that the Welsh region had the chances to have put the game to bed by then and insisted not doing so ultimately proved costly.
"We showed glimpses and patches of where we want to get to but it was too fleeting for our liking," he said.
"We couldn't sustain pressure for long periods. We were 13-6 up and just couldn't put the game away from there.
"There were a few key moments at that time, 13-6, 13-9, a couple of big set-piece moments that went against us that we'll have to look at.
"Ulster have got decent squad depth and they brought decent players on but there are perhaps other issues around those scrums that we need to look at again in terms of that call that's been made.
"Those incidents aside though, in many ways we were architects of our own downfall. We were pressurising in their 22 and didn't come away with points."
63x46.png){kind=logo}