A bumper crowd of 8,000 watched prop Sam Hobbs score the decisive try, and 17 points from Ben Blair's faultless kicking display secured the victory.
Although the Blues have agreed to pay £500,000 per year rent on the Cardiff City Stadium until 2029, Holland admits they still could finish the RaboDirect PRO12 season at Arms Park and revealed a decision could be made on Monday.
"It was a great evening, great result and great crowd," said Holland.
"We had to get a lot of boxes ticked before we got the match played at Arms Park but I'm delighted that we managed to pull it off.
"It's my brief to make the business sustainable, which is important. There may be some more options to play games away from the Cardiff City Stadium if it makes it more financially viable.
"The contract for Cardiff City Stadium is in place and that is our home but we'll look at it going forward.
"I'm quite keen to make a decision on the remaining four home games on Monday.
"We'll look at next week's game against Ulster on Monday morning and we'll take it from there. I'll meet with my team and assess where we go for the rest of the season.
"We had to listen to the fans, they wanted to come here for a game, and they have wanted to come here for a while. We bought the game here, it's been a success so lets have a look on Monday and we'll take things forward.
"The facilities are second-class in comparison to those at the Cardiff City Stadium. If we are looking at having more games back at Arms Park, we'll look into the investments that are required if we are looking to come back - if it's not the case, then the investments will not be required.
"We are still paying the rental to Cardiff City but if it makes it more cost effective to play elsewhere, bizarre as it might sound, we will have to look at it."