Colavito grew up in the Bronx as a devoted fan of the New York Yankees, particularly Joe DiMaggio. By age nine he was playing semipro baseball, and he dropped out of high school at age 16 to pursue his career; major league rules required him to wait until his school class graduated before signing, and only a special appeal allowed him to go pro after a one-year wait. The Yankees expressed little interest in him, and the Philadelphia Athletics had to bow out due to financial problems; the Indians finally signed him in 1950, with two-thirds of his signing bonus deferred until he progressed in their system. He spent most of the next six years working his way up, often hampered by his own desire to emulate DiMaggio's style rather than develop his own.

After breaking in with the Indians briefly in 1955, he started 1956 in the Pacific Coast League, once showing off his throwing arm by hurling a ball over the center-field wall from home plate, with his longest mark at 436 feet. He returned to the Indians in July, and after batting .276 with 21 home runs he earned one vote in the AL Rookie of the Year voting. After slipping to a .252 average in 1957, in 1958 he batted .303 with 41 home runs (one behind Mickey Mantle's league lead) and 113 runs batted in, and finished third in the MVP balloting; he also led the AL in slugging with a .620 average, the highest by a right-handed Indians hitter until Albert Belle in 1994. One year later Colavito became the first Indian to have two 40-HR seasons; with 42, he tied Killebrew for the AL lead and was one short of Al Rosen's club record. He finished fourth in the MVP vote, and also made the All-Star team for the first time; he was eventually chosen in six years, including both All-Star Games in 1959, 1961 and 1962, when two were played annually. On June 10, 1959 he smashed four homers in consecutive at bats in a single game at the Baltimore Orioles' cavernous Memorial Stadium. The Indians finished five games behind the Chicago White Sox in the 1959 pennant race, the closest he would come to a title until 1967. Colavito would hit 30-plus homers seven times, establishing himself as a major power hitter and excellent fielder with a strong arm despite being flat-footed.

Colavito was easily the Cleveland fans' favorite, with his handsome appearance and approachability, always accommodating the hundreds of autograph seeker after each game even if it took a few hours. But just before the 1960 season, Indians general manager Frank Lane traded him to the Detroit Tigers for Harvey Kuenn, who had won the 1959 batting title. The trade proved to be a good one for the Tigers but a terrible unpopular one for the Indians, whose fans lost their favorite player and best hitter; Kuenn had a minor injury early in the season and was traded away by the end of the year. Lane further irritated fans by saying, "What's all the fuss about? All I did was trade hamburger for steak."; Tigers GM Bill DeWitt jokingly responded that he liked hamburger. In 1961 with the Tigers, Colavito enjoyed career highs of 45 home runs, 140 RBI and 129 runs scored as the team led the major leagues in scoring; he placed eighth in the MVP race.

But Tiger fans didn't take to him the same way as had those in Cleveland, preferring the more consistent Kuenn; and sportswriter Joe Falls, who viewed Colavito as a "self-ordained deity," started a feature chronicling the runs he failed to drive in. In one game, Falls "acting as the official scorer" charged Colavito with a controversial error, and the outfielder tried to attack him; and on May 12 of 1961, he was ejected from the game after climbing into the stands to go after a drunken fan who had been harassing his wife and father. After his excellent 1961 season, he drew the local fans' criticism by holding out for a higher salary than established team star Al Kaline. He was traded to the Athletics (by now in Kansas City) after the 1963 season, but spent only one year with the team, becoming one of the youngest players to reach the 300-home run mark. In 1965, with Gabe Paul running the Indians, Colavito was brought back; but to obtain the slugger in a three-team deal, Paul had to send the White Sox pitcher Tommy John, who would play until 1989 and win 286 games after the trade, as well as Tommie Agee, who won the Rookie of the Year award in 1966, then became the New York Mets' top hitter in 1969 as they won their first pennant.

Colavito made his last years in Cleveland worthwhile, making the All-Star team in 1965 and 1966 and placing fifth in the 1965 MVP vote after leading the league with 108 RBI and 93 walks while finishing among the AL's top five in home runs, hits and runs, as well as playing without an error in the outfield. In July 1967 he was traded to the White Sox after hitting only .241; he hit .221 with the Sox, who finished three games out of first, and was sent to the Los Angeles Dodgers before the next season, batting .204 with 3 home runs in his National League debut. Later in that final season of 1968, with his last team, his boyhood idol Yankees, he became the last position player until Brent Mayne in 2000 to be credited as the winning pitcher in a game, earned in a scoreless two and two/thirds inning relief appearance in the first game of a doubleheader. He also homered in the second game.

In his 1,841-game career, Colavito batted .266 with 374 HRs, 1,159 RBI, 971 runs, 1,730 hits, 283 doubles and 21 triples. As an outfielder, he recorded 3323 putouts, 123 assists, 26 double plays, and committed 70 errors in 3516 total chances for a .980 fielding percentage. In 1976 he was voted the most memorable personality in Indians history, and in 2001 he received a huge ovation at the introduction of the Indians' All-Century team.