Woolmer was born in the Georgina McRobert Memorial Hospital across the road from the Green Park Stadium in Kanpur (earlier 'Cawnpore'), India on 14 May 1948. His father was the cricketer Clarence Woolmer, who represented United Provinces (now Uttar Pradesh) in the Ranji Trophy. At the age of 10, Woolmer witnessed Hanif Mohammad scoring 499, and thereby setting a world record for the highest score in first-class cricket. Some 35 years later, Woolmer, as coach of Warwickshire County Cricket Club, was watching when the county's batsman Brian Lara passed that mark, setting a new record of 501 not out.

Woolmer went to school in Kent, first at Yardley Court in Tonbridge and then The Skinners' School in Tunbridge Wells. When he was 15, Colin Page, the coach and captain of the Kent second XI, converted him from an off-spinner to a medium pace bowler.

Woolmer's first job was as a sales representative for ICI, and his first senior cricket was with the Tunbridge Wells club and with Kent's second XI. In 1967 he broke the Kentish Leagues' Bat and Trap record for most consecutive strikes between the white posts: 13 in one game.

In 1968, at the age of 20, Woolmer joined the Kent cricket staff and made his championship debut against Essex. His ability to move the ball about at medium-pace was ideally suited to one-day cricket at which he became a specialist. He won his county cap in 1969. Woolmer began his coaching career in South Africa in 1970–71 at the age of 22 and by 1975, when he made his Test debut, he had become a teacher of physical education at a prep school in Kent as well as running his own cricket school – at the time one of the youngest cricket school owners anywhere.

Bob Woolmer played English county cricket for Kent, initially as an all-rounder. He graduated to Test cricket with England in 1975 again, at first, as an all-rounder, having taken a hat-trick for MCC against the touring Australian cricket team with his fast-medium bowling. But he was dropped after his first Test, only reappearing in the final match of the series at The Oval where he scored 149, batting at number five, then the slowest Test century for England against Australia. Further batting success followed over the next two seasons, including two further centuries against Australia in 1977. Rarely for an Englishman since the Second World War, all his Test centuries were made against Australia.

Woolmer was also a regular in England ODI cricket from 1972 to 1976. But Woolmer's international career stalled after he joined the World Series break-away group run by Kerry Packer. Though he appeared intermittently in the Test team up to 1981, he never recaptured the form of the mid 1970s. He also took part in the South African rebel tours of 1982, a move that effectively ended his international career.