Swinson had only been on the field eight minutes when his punch in a driving maul was spotted by the linesman, leaving referee Romain Poite with little choice but to show a yellow card.
Glasgow lost the ball and good field position and Saints bounced back in the closing stages to punish them - keeping their own hopes of the knockout stages alive at the expense of their opponents.
"It always hurts to lose in the last minute, but it is the players who put in all the effort and they are hurting the most," said the Glasgow head coach Townsend.
"I was really proud with the way the players stuck to their task. Tim Swinson is really disappointed at his yellow card because he knows it cost us. It was out of character for him - he plays tough, but always within the rules. But sometimes when people are hanging on to you things can happen.
"We were in control of the ball at that stage and we could have got a penalty or a scrum. We had just picked up a penalty at the previous scrum and we might have been able to see it out.
"But we always knew how difficult it was going to be to win at Franklin's Gardens. They hadn't gone three games without a win in Europe since 2000.
"We defended really well against their big runners and after a few early problems at the scrum we did well in that area in the end and it was pretty equal. We have shown some good improvement in the last three games, but this will hurt more than the other two defeats.
"We've still got a game to go in Europe and we will have to see who we play against Racing 92. We'll be playing a quality side and we will be trying to get a win - our fans deserve that."
To buy tickets for the GUINNESS PRO12 Grand Final click here. Also follow us on Facebook, join the conversation on Twitter, sign up to our YouTube channel for extensive match highlights and sign up for our newsletter for regular updates on the GUINNESS PRO12.
63x46.png){kind=logo}