Whether it's provable or not, I've always felt that the actual Swedish word for "accent" may have something to do with this. It really surprised me - and irks me to this day - that the word for accent in Swedish is brytning, which comes from the verb "to break": bryta. So essentially instead of saying that you speak with an accent, in Swedish you say you speak broken Swedish (bruten svenska) or with a break/ing (brytning). I'm sure no Swede thinks about this at all on a conscious level, but the very nature of the verb "to break" seems negative. I mean when is it ever good when something breaks or is broken?
Don't worry, the point of this post is not to debate the sociolinguistic situation of having an accent in Sweden and whether this has anything to do with the word for accent in and of itself. The point is to ask what the word for "accent" is in other languages? Furthermore how would you say someone "speaks with an accent"? What are the literal translations of these into English?