The Welsh side led early on through No.8 Joe Bearman's try but after Dan Biggar was sin-binned Ulster took control.
Trailing by eight points with two minutes to go Ospreys attempted to stage a fightback but although Eli Walker's try secured a bonus point it was too little too late to turn the game.
Ulster climbed up to fourth after the win while Ospreys remained second and Humphreys accepted they'd missed the chance to close the gap on leaders Leinster.
"It was a fantastic effort against a very good team but ultimately we came up just a little bit short which is disappointing," he said.
"I couldn't have asked for more from the team in terms of effort and endeavour, but the reality however is that we made too many mistakes and when that's the case you're always going to be up against it.
"We need to improve our accuracy. If we want to come to places like Ulster and win, if we want to win the league, then we need to remove these unforced errors when we have the ball.
"We need to take out accuracy levels up to our endeavour and effort. That's what we'll take from this game, and if we can do that, then we could be a very good team.
"There's no doubt that the penalty try and yellow card was a turning point in the game. We were just about at half-time and although they'd had a lot of possession and territory, we had worked very hard defensively to limit their chances, which makes it even more disappointing."
63x46.png){kind=logo}