Johnson appeared on Broadway six times, including New Faces of 1933, Too Many Girls, and the original production of Pal Joey, in which he understudied the title role. His largest roles were when he took over the part of Mark Bruckner in On a Clear Day You Can See Forever and when he took over the role of Georges in La Cage Aux Folles. Johnson also worked extensively in regional theater and in London, where he starred in The Music Man.
He was best known for his film work, however, where he spent many years under contract at MGM, and appeared in such musicals as In the Good Old Summertime with Judy Garland, Brigadoon with Gene Kelly and Cyd Charisse, and the Jerome Kern biopic Till the Clouds Roll By. His many other credits included A Guy Named Joe, The Caine Mutiny, The Last Time I Saw Paris, and Wives and Lovers. During the 1970s and 1980s, Johnson appeared on numerous television series and miniseries, and earned an Emmy Award nomination for Rich Man, Poor Man.