Richards was born to Malcolm and Gretel Richards in St. John's, Antigua, then part of the British Leeward Islands. He attended St. John's Boys School and then Antigua Grammar School on a scholarship.

Richards discovered cricket at a young age. His brothers, Mervyn and Donald, both played the game, representing Antigua as amateurs, and they encouraged him to play. The young Viv initially practiced with his father and Pat Evanson, a neighbour and family friend, who had captained the Antigua side.

Richards left school aged 18, and worked at D'Arcy's Bar and Restaurant in St. John's. He joined St. John's Cricket Club and the owner of the restaurant where he worked, D'Arcy Williams, provided him with new whites, gloves, pads and a bat. After a few seasons with St. John's C.C., he joined Rising Sun Cricket Club, where he remained until his departure to play abroad.

Richards made his first-class debut in January 1972 when he was 19. He took part in a non-competition match, representing the Leeward Islands against the Windwards: Richards made 20 and 26. His competitive debut followed a few days later. Playing in the domestic West Indian Shell Shield for the Combined Leeward and Windward Islands in Kingston, Jamaica versus Jamaica, he scored 15 and 32, top-scoring in the second innings in a heavy defeat for his side.

By the time Richards was 22, he had played matches in the Antigua, Leeward Islands and Combined Islands tournaments. In 1973, his abilities were noticed by Len Creed, Vice Chairman at Somerset, who was in Antigua at the time as part of a West Country touring side.