Cfrgtkky

When talking to learners of my mother tongue, Swedish, I've sometimes had to explain how using too polite language can be taken as rude or insulting, as it creates a certain distance between the speakers.
I guess this is a mechanism in many languages/cultures. Is there a linguistic term for this phenomena?

It depends on the exact theoretical framework used and the exact nature of the language's politeness / rudeness system, but following Brown and Levinson's 1987 framework, Culpeper's 1996 Towards an anatomy of impoliteness provides a few answers. I think what you are referring to is (unintentional) mock politeness / sarcastic rudeness.

However, depending on the usage, it can be perceived as positive impoliteness or as a positive face-threatening act. This is paralleled in the use of inappropriate vouvoiement in most French-speaking communities, but especially salient in African French. This study on Cameroonian French breaks this down into the vouvoiement de distanciation and the vouvoiement de discrimination, where one makes the hearer feel "distant", and the other makes the hearer feel "discriminated against".

The other categories as categorised by function are negative impoliteness/face-threatening acts and withholding politeness. However, all these categories can be very fuzzy, and one feature can have components of than one category.

A different way of looking at these impoliteness "strategies" is via form, as per Bousfield (1998). These would split them down into on-record and off-record impoliteness, which correspond to explicit vs implied impoliteness. Even so, categorising a form of address into explicit or implied can be trickier than it first appears, especially when the impoliteness is accidental.

In English, overly polite language could be perceived as patronizing, or characterized by condescension via insincere kindness.

The source of the perceived rudeness might very well be that overly polite language often seems subservient, and unless there is a good reason (e.g. the speaker is apologising for sth), the excessive submissiveness conveys a context of passive-aggressiveness and concealed hostility.

Example: "Wouldn't You, Good Sir, agree that this might be the reason?"

(Even though I wrote that sentence as an example only, and even though it is outwardly very polite in form, I think I'd better apologise for my language in advance; that's how rude it sounds to my ear, at least.)

If the person feel as though you're politeness is to create distance instead of being friendly the person you're talking to might take it as sarcastic.

There's a Swedish saying/quote, "du är inte Ni med mig", or translated: "you are not You(formalized) with me". It's basically saying that you think they are trying to be too formal for your taste.
I think this mostly comes down to the law of Jante that most Swedes live by, which states that noone should think they are more special than anyone else. Everyone should be equal. And it can be considered rude if someone is being very posh towards you, when you're not in that mindset yourself.