Ospreys have also won their last two games against Ulster and have won at Ravenhill on four occasions in this tournament - only Leinster have done as well in Belfast.
But with his side still without their international stars, Humphreys has called on the Ospreys to step up another gear as they prepare for the trip to Ulster.
"It's good to go to Ulster off the back of two wins but we know that there's going to have to be a real improvement from last week if we are going to make it three from three," he said.
"Ravenhill is always a difficult place to go, even more so when they are in the kind of form they've shown during their European campaign this season.
"It's likely to be a full house, a passionate atmosphere, and a difficult evening for what is largely a young Ospreys team, with a sprinkling of experience. It's nights like these where you learn a lot about a team, about the character and spirit of young guys. It's going to be a great test.
"We've got ourselves in a good position in the PRO12, mainly down to these guys, who've been here during the World Cup and Six Nations. We need to maintain our focus now and make sure we finish the job in the coming weeks and secure that home semi-final."
Ulster's three game winning run in the RaboDirect PRO12 ended at the Arms Park against Cardiff Blues on Friday.
The Ulstermen's only defeat at Ravenhill in their last 17 matches in all competitions was 12-23 to Treviso on 7 October.
The Irish side are boosted by the return of Pedrie Wannenburg but after falling to Cardiff last time out, head coach Brian McLaughlin insists they must rediscover their killer touch.
"I think if you look at the game, whenever we got into the opposition 22 we weren't ruthless enough," he added.
"We had chance after chance but we weren't able to convert the pressure that we had into points.
"And as a result that opened the door for Cardiff.
"But we have to learn - we will get on the training paddock this week and make sure we are back on form for Ospreys next weekend."
At Ravenhill, 7.05pm
Referee: Neil Henessey (WRU, 9th competition game)
Assistant Referees: Dudley Phillips, Sean Gallagher (both IRFU)
Citing Commissioner : Peter Ferguson (IRFU)
TMO: Kevin Beggs (IRFU)
Live on BBCNI














