In 1946 he started to practice with a local baseball team called the Senadores de San Juan (San Juan Senators) where he learned many of his baseball skills. He was later invited to play for the Criollos de Caguas, where he was spotted by a New York Yankees scout. In 1949, he left for the City of Chicago and was sent to play for a team in Drummondville, Quebec.

He was signed by the Yankees in 1951, but was sent to the Triple-A Syracuse. In 1953, even though he was the minor league batting champion, he was not invited to spring training. If the Yankees wouldn't have picked Elston Howard over him, he would have been the first black and Puerto Rican to have been a Yankee. Many Blacks and Puerto Ricans protested in front of Yankee Stadium in response to what they believed was a racially motivated decision. In 1954, he was recruited by the Philadelphia Athletics and thus became the first Puerto Rican to play for that team. Suffering from the racial discrimination which was so rampant in the nation at that time, he could neither stay with the rest of the team at the same hotels nor be allowed to eat at the same restaurants. The Athletics moved to Kansas City in 1955, where he finished in second place in the batting race that season. In 1958, he was then sent to the Cleveland Indians. In 2000, the Cleveland Indians honored him by declaring him to be among its 100 all-time greatest players.

Since it was hard for the sports newscasters to pronounce his last name, Pellot, Victor decided to use his mother's maiden surname, Power. Many Puerto Ricans at first falsely thought that Pellot was ashamed of using his Hispanic surname and accused him of using an American sounding name for his own selfish advantage; this, however, was not the case. Also, he was often criticized by the press and his peers for fielding the ball using only one hand instead of using both hands; this was to become his trademark. In his 12-season career, he played with the Philadelphia and Kansas City Athletics (1954-58); Cleveland Indians (1958-61), Minnesota Twins (1962-64), Los Angeles Angels (1964), Philadelphia Phillies (1964) and California Angels (1965). Before retiring, he won seven Gold Glove Awards and was selected a seven-time All-Star. He was also voted the Minnesota Twins MVP in 1962. He has the record of having made one or more assists in 16 consecutive games; he shares the record of making two unassisted double plays in one game, and he is one of 11 players to steal home plate twice in one game, and he also shares the record of being assists leader for six years in a row and of double plays in a single game.

Among his career totals are the following: 1,716 hits and 126 home runs, and he was only struck out 247 times out of 6,046 at bats. Currently, he lives in Guaynabo, Puerto Rico. The City built a ballpark and named it after Pellot, which can be seen from the window of his apartment. During the months of summer, he likes to help the youngsters of the community in their sports skills development. He believes that many of today's youth will stay away from trouble and will have the chance to enter college on sports scholarships.