A Paddy Jackson penalty had put Ulster ahead at Kingspan Stadium and he added another shortly after before Olding picked up his first try of the night.
Jones' men got themselves on the board before the break but the second half was more of the same with Olding adding his second try.
And Jones admitted his side weren't good enough to win on the night but praised his side's character in defeat.
"It was a game that should have been well within our reach but on the night we just weren't good enough," he told the South Wales Argus.
"There were some really good performances but we failed to manage the game as well as Ulster.
"Our kicking game was off, that meant we were on the back foot and didn't get field position to put pressure on them.
"I was pleased with the spirit; we showed the attitude and desire to fight hard.
"We had to bring two teenagers off the bench in Ollie Griffiths and Barney Nightingale but showed lots of guts until the end and didn't fold under relentless pressure."
Follow us on Facebook, join the conversation on Twitter, sign up to our YouTube channel for extensive match highlights and sign up for our newsletter for regular updates on the GUINNESS PRO12
63x46.png){kind=logo}