Australia had edged a cagy first half, with four Berrick Barnes penalties to three from Leigh Halfpenny.
But after a scorless start to the second half, Wales finally got the first try on the hour through Ryan Jones, Halfpenny's conversion making it 16-12 to the Welsh.
Australia reacted immediately with a try through Rob Horne to take a one-point lead despite Barnes missing the conversion.
A powerful Welsh scrum allowed Halfpenny to put his side back in front but with five minutes to go Wales were penalised about 40 metres out and Barnes held his nerve to seal the win for the Wallabies.
But with another clash against the Australians to come in the Autumn, Lydiate insists Wales will be after some revenge.
He said: "We had our chances but fair play to Australia. They had chances and took them well. They kicked their points," said Lydiate, who featured seven times for the Dragons in the RaboDirect PRO12 this season.
"I can't really talk at the minute, I am just absolutely gutted.
"We're getting closer but it was our last chance and it wasn't good enough.
"Too many errors on our part but credit to the Aussies, they're a good team. These games are won and lost by one or two points but we'll get there and we'll look forward to the autumn series at back at our place."
63x46.png){kind=logo}