There has been no such trouble on that front this year with a quarter-final against Ulster in the bag after an impressive qualification campaign, featuring, most notably, a 51-36 win over Northampton Saints.
However while silencing the critics is a priority for Varley and co the hooker insists their European adventure won't be prejudicing their RaboDirect PRO 12 aspirations.
"We are in a good position in all competitions," he said. "We've worked hard and probably some results haven't gone the way we would have liked them to have gone.
"But we can't complain at where we are in the Heineken Cup and the RaboDirect and the lads are doing a great job in the British and Irish Cup.
"There has been a lot of bad feeling out there towards where we are, particularly after our Heineken Cup campaign last season but it doesn't really impact on us. We discussed it as a squad and went into this season knowing what we wanted to achieve out of the season.
"We are in a good position even though we haven't been playing well on a consistent basis. What can you say about the guys who criticise you openly? Let them do what they want to do and we'll keep doing what we're doing.
"We have said something but there is a lot more to say yet on the field. The Heineken Cup is the standard you want to set, it's the competition where every team wants to set their standard and want to win. But equally we want to win the PRO12 as much as the British and Irish Cup and the Heineken Cup so we are not just relying on that.
"We have a lot of areas and preparation that we want to take forward in the next few weeks in the big PRO12 games coming up."