Nicknamed "The A-Train" for his unprententious but dominating style of play, the durable 7 ft 2 in Gilmore once played in 670 consecutive games. Gilmore played college basketball at Gardner-Webb Junior College and Jacksonville University, leading the Dolphins to the NCAA title game in 1970. During the two years Gilmore played NCAA basketball at Jacksonville, he became one only five college players ever to average at least 20 points and 20 rebounds over a career. He led the NCAA in rebounding both years, and his career average of 22.7 rebounds per game is still the highest in NCAA Division I history.

He began his professional career with the Kentucky Colonels of the American Basketball Association and played there for the Louisville team until the league folded in 1976. As a rookie in 1972, Gilmore had the rare distinction of being selected both the league Most Valuable Player and Rookie of the Year. Gilmore led the ABA four times in rebounding average, twice in both field goal percentage and blocks per game, and once in personal fouls. He was named to the All-ABA First team five straight seasons, and was named to the ABA's All-Defense team four times. He played in the ABA All-Star Game all five years he played in the ABA, and was named the 1974 game's MVP. In 1975, he led Kentucky to the ABA championship and was named the Playoffs MVP. During his ABA career, Gilmore established ABA records for career field goal percentage (0.557), career blocked shots (750), blocked shots in a season (287 in the 1973-74 season), and rebounds in a game (40).

After the ABA folded, Gilmore was chosen with the 1st overall pick in the 1976 ABA dispersal draft by the Chicago Bulls. Gilmore played in a total of six NBA All-Star Games. He led the NBA in field goal percentage in four consecutive seasons, including a career best .670 during the 1980-81 season - the third highest percentage in NBA history. Gilmore was also the NBA's career leader in field goal percentage (minimum 2000 shots made) with a 59.9 percentage. In addition to being pro basketball's all-time most successful field goal percentage shooter, Gilmore ranked among the top 25 all-time point scorers, as well as in the top ten in rebounds, blocked shots, games and minutes played at the time of his retirement.