Jon Welsh's third try in four games and a Colin Shaw score helped them achieve the previously impossible with the Warriors even surviving a late stint with just 13 men on the field following Kellock and Scotland teammate Johnnie Beattie's sin-binning.
After their victory on their travels the Warriors now have a mini-break from RaboDirect PRO12 action with their next action back-to-back home games against Aironi and Cardiff Blues, beginning on March 23.
And Kellock is urging his side to make sure they don't undo their hard work against the Ospreys in the comfort of home surroundings.
"It is hugely pleasing to get a win at Ospreys," Kellock told www.glasgowwarriors.org. "The way it happened as well was great because I thought for 70 minutes we were outstanding and really difficult to play against.
"Then myself and Johnnie had a rush of blood to the head which made it a bit more difficult. But the way the boys dug in was brilliant so to come out with a victory down here is huge.
"The way we closed that game out at the end was intelligent, I'm delighted to get four points. We have got a couple of huge games coming so the four points that we got today is not only important but it is also about the confidence we have got to take from it.
"The performances of the young lads were great but we have got to back up these performances now."