Kemp made his International debut in an ODI against Sri Lanka on 14 January 2001. He made his Test debut a week later also against Sri Lanka. He was taken on the tour of the West Indies in the 2000/2001 season but he did not score many runs and was involved in a controversy when he admitted smoking marijuana with 6 South African players. He was selected for eight more One day games but again he disappointed and was dropped. He was out of the South African side for nearly three years until he was selected to play in the home series against England. In this series Kemp impressed with his big hitting with him smashing 80 runs from 50 balls in an innings that included 7 sixes. Following his performances he was picked for the series against Zimbabwe and in the match at Durban he hit 53 from 21 balls in an innings that included 5 sixes. He was then taken on his second tour of the West Indies where he made an impressive 65 at Trinidad.

He was then selected to represent Africa in the Afro-Asian Cup although he batted lower down the order and struggled. In the South African season of 2005/2006 he showed that he is a match winner when he blasted 73 from 64 at Bloemfontein. Kemp then went on his first tour of India where he showed he was able to bat on slower pitches. He went on the South African tour of Australia where he made a rare appearance in a Test Match when he and Jacques Rudolph formed a partnership that rescued a draw for South Africa at Perth. He was not picked for the second Test and has not been picked in a Test Match since. He struggled in the VB Series that involved Sri Lanka, Australia and South Africa with him averaging just 17.00. He quickly returned to form when Australia toured South Africa as he made 51 from 41 balls at Newlands Cape Town.

He was selected in South Africa's squad for the 2006 ICC Champions Trophy in India where he showed he is more than just a big hitter when he made 64 from 110 balls in a partnership with Mark Boucher where he along with Mark Boucher set the record for the highest 6th wicket stand in ICC Champions Trophy(131) 26 November 2006, in the third ODI of the series between India and South Africa, his side were at 137 for 7 when Kemp came in and smashed his way to his maiden ODI century with the help of Andrew Hall. Together they put on a world record 8th wicket stand of 138^[6] and took their score to 274, with Kemp named Man of the Match.^[7] He also showed that he is a useful medium pace bowler when he took three wickets at St George's Park Port Elizabeth. He was selected in the 30 man provisional squad for the 2007 Cricket World Cup.