The Welsh region arrived at Ulster's Kingspan Stadium boasting seven wins from seven and sitting top of the table.
However Ospreys, without ten players to international duty, struggled to find their early rhythm and trailed 18-3 at the break - Sam Davies' long-range penalty their only score.
They were much improved after half-time, with Martin Roberts crossing four minutes after the restart, before two quick-fire penalties moved the visitors to within two points of their opponents.
But it would be Ulster who would emerge victorious with Darren Cave's second try of the night a minute from time making sure of the 25-16 win, leaving Gibbes to rue his side's poor start.
"It was a great effort in the second half, yes, and we showed great spirit, but ultimately we've only really got ourselves to blame about losing this game. That's what is most frustrating," he said.
"In the first half we didn't put it in the areas where we wanted to and that meant we found ourselves under pressure.
"I think we probably did do enough to win the game but ultimately, we gave them too big a head start in the first half and just couldn't keep going in those last five minutes.
"That's a mark of the standards expected of the Ospreys that coming here and being 15 points behind at half-time, we are bitterly disappointed about not closing the game out at the death and leaving here with nothing.
"We've got to keep focusing on getting the performance right and we've got to understand that if you don't take your opportunities then you'll lose games."
Click here to view the best images from Ulster's victory over the Ospreys.
Follow us on Facebook, join the conversation on Twitter, sign up to our YouTube channel for extensive match highlights and sign up for our newsletter for regular updates on the GUINNESS PRO12
Guinness PRO12
Suite 208, Alexandra House,
The Sweepstakes
Ballsbridge, Dublin 4, Ireland