Rudolph was chosen for South Africa's Test team in 2003 having previously had his selection for the 2002 New Year's Test against Australia in Sydney overruled by cricket administrators due to the South African practice of obeying a 'quota' of non-white players; the board overruled the selectors and installed Justin Ontong in the team. The selectors noticed him during one of his performances in domestic cricket.

Rudolph was named the South African Cricket Annual Cricketer of the Year in 2003.

In his debut Test match against Bangladesh he scored 222 not out, his best first class score. It is still the highest Test debut score by a left-handed batsman and a Test opener as of 2013, when Hamish Rutherford made 171 for New Zealand against England.

He also broke the South African third-wicket partnership record, alongside Boeta Dippenaar, reaching 429* against Bangladesh at Chittagong. He saved a Test for South Africa in the Perth Test match in December 2005 after defying the Australian bowlers for more than a day to score 102* and ensure a draw. After declaring himself unavailable for selection by South Africa after agreeing to join Yorkshire under the Kolpak ruling he stated he did not want to represent South Africa again and hoped to possibly represent England, qualifying by residency, after deciding to buy a house and live permanently in Yorkshire.

Rudolph was recalled to the South African Test squad in October 2011, for the two-Test home series against Australia.

Rudolph plays for the Nashua Titans in South Africa. He represented the Gestetner Diamond Eagles and Northerns.

Although still centrally contracted, Rudolph signed a three-year deal in 2007 to play for Yorkshire under the Kolpak ruling. In doing so he declared himself unavailable for selection for the duration of the contract. Rudolph wore squad number 24 for Yorkshire and in October 2007 further committed himself to the county, signing a 4-year deal which would keep him at the club until 2011.

In 2008, Rudolph was awarded the Players' Player, and fans' Player of the Year awards for Yorkshire.