I actually run into issues based on my height in addition to my bulk. At a little above 6'4", when it comes time for performance appraisals at work, talking to people at the bank, or anywhere else for that matter, I have to take a very calm tone or I get accused of being belligerent and threatening. On the other hand, anywhere I walk into, people assume I'm in charge. Stores, warehouses, university functions, I'm the first one people go to. It has its ups and downs.
But no one every thinks I'm dumb. I dress professionally at all times unless I am lifting or in the outdoors. I work on speaking without an accent. I strive to be articulate, observant, and knowledgeable, and it has never failed me. So, yes, people think I might be about to go for their throat, but they rest easy in the knowledge that at least it was someone of extraordinary intelligence.