After posting a thread on RMWeb I've been informed that the likely source for the ballast during S.R. days would be Meldon Quarry on the edge of Dartmoor. If anyone else in interested the three links below offer a reasonable background information about this quarry and include photos.
If there was any ballasting completed after being transferred to WR, Stoneycombe quarry would possibly have been used.
I'm still trying to track down if any of the branch was re-ballasted during the 1950's and 1960's but I am now more confident about the colour of two photos on this website taken in 1986. To me this looks like ballast from Meldon. But I do believe ColourRail have a number of slides of this branch which I am also going to try and purchase as reference.
The thread on RMWeb is still running so more information may well surface.