Seemingly out of it when they trailed 20-3 at half-time, the Ospreys fought back with two quick tries early in the second half from Justin Tipuric and Eli Walker.
Clermont responded to seal the five points through David Strettle and Wesley Fofana, but the Ospreys refused to give up, and claimed both a losing bonus point and a try bonus point thanks to scores in the last two minutes from Sam Parry and Tom Habberfield.
That was enough to push them one point clear of their hosts, having played a game more, before a crucial double header against Bordeaux in December.
Before that they will return to GUINNESS PRO12 action and a derby against Cardiff Blues, and fly-half Dan Biggar believes they showed exactly what the side is capable of out in France.
"I thought we showed a tremendous amount of bravery out there," said the Wales fly-half Biggar.
"We knew coming here that it would not be enough to just play for penalties and drop goals. We had to be positive, we had to be confident enough to play and try and put them under pressure.
"We did that but they stood tall towards the end of that first half and we could easily have let things slide away there.
"But we knew we had to keep playing our game and chance our arm. It if had ended with us losing by 40 points then at least we would know we had given it a go.
"We managed to get the balance right and took our chances well. We could have dropped our heads and it is a huge testament to our spirit that we finished those chances at the end.
"To come away from one of the most difficult places to play in Europe with two points is like a win for us, really."
On a difficult weekend for GUINNESS PRO12 sides in the Champions Cup Leinster, Scarlets and Benetton Treviso all slipped to their second successive defeats in the tournament.
Leinster were narrowly beaten at Bath, with a late George Ford penalty giving the English side a 19-16 win, while Scarlets went down 29-12 at home to French giants Racing 92 and Treviso lost 36-3 against Leicester.
For their opening games of the tournament, Glasgow and Ulster were also beaten, losing to Northampton and Saracens respectively.
There was better news in the Challenge Cup with the performance of the weekend coming from Zebre, who comfortably saw off Worcester in Parma.
They were 27-8 winners thanks to tries from Marcello Violi and Tommaso Boni, while Carlo Canna chipped in with 17 points.
GUINNESS PRO12 leaders Connacht continued their fine start to the season with a second successive European win, beating Brive 21-17.
On Friday Edinburgh also maintained their unbeaten record in Europe as they cruised past French opposition for the second week running, beating Agen 27-6.
Newport Gwent Dragons almost pulled off a great win away in France, before eventually going down 32-29 at Castres, while on Thursday Cardiff Blues lost 32-20 against Harlequins.
To buy tickets for the GUINNESS PRO12 Grand Final click here. Also follow us on Facebook, join the conversation on Twitter, sign up to our YouTube channel for extensive match highlights and sign up for our newsletter for regular updates on the GUINNESS PRO12.