Irving committed to Duke on October 22, 2009, in a television broadcast on ESPNU. Irving started playing with Duke in the 2010–11 basketball season under the guidance of head coach Mike Krzyzewski. He led the team in scoring at 17.4 points per game on 53.2% shooting through the first eight games of the season. In addition, he added 5.1 assists, 3.8 rebounds and 1.5 steals per game as he quickly began to make his case for NCAA Freshman of the Year.

In Duke's 8th game of the season, Irving suffered a severe ligament injury in his right big toe that sidelined him indefinitely. Irving announced that he would forgo his final three seasons of eligibility and enter the 2011 NBA Draft where he was the number 1 pick by the Cleveland Cavaliers. 

Irving got off to a fast start, and through twenty-three games, had ten games with 20+ points. Irving was named to the 2012 Rising Stars Challenge where he scored 34 points, earning MVP honors. Irving won the 2012 NBA Rookie of the Year Award, receiving 117 of a possible 120 first-place votes. Irving was also the only unanimous selection to the NBA All-Rookie First Team.