Ulster led by a single Paddy Jackson penalty at half-time and though Alan MacGinty levelled the scores, man-of-the-match Williams ploughed over the line in the 76th minute to clinch the victory.
Indeed the Ulstermen are well and truly finding their feet under new boss Kiss, completing a stunning double over Toulouse in Europe while Saturday's win was the first home defeat inflicted on Connacht all season.
And Kiss' attention has already turned to next Sunday's visit from Munster, who he expects to be a menace back at the Kingspan.
"We didn't expect this to be easy and it wasn't but it's nice to come out the other side with the win," Kiss told RTÉ Sport.
"I thought the defence was very good. At times we gave them a bit of access in the wide channels but overall our work ethos was vital.
"It was a game where both sides had ambition and intent, but both were guilty of their skills not matching that ambition. Nick Williams is a great rugby player. He's got a lot of heart and soul.
"Munster is going to be a torrid affair. They beat us last time; we know that they're a team that are probably hurting at the moment and want to get a result - hopefully they get that result against Leinster and we haven't got a dangerous beast the following week!"
To buy tickets for the GUINNESS PRO12 Grand Final click here. Also follow us on Facebook, join the conversation on Twitter, sign up to our YouTube channel for extensive match highlights and sign up for our newsletter for regular updates on the GUINNESS PRO12.
63x46.png){kind=logo}