Just been checking out some of Malcolm Smiths photos of the line and come across an good photo of the level crossing at Boscarne. Bottom Photo on this page
It shows that the land fell away from the level crossing very steeply down towards the river. The buildings in the left of the picture must be what was the mill etc and are well below rail level. So I doubt whether the Wharf siding would have connected directly with the mill, not unless the mill was extended. Would think that the Wharf siding would have been at a slightly lower level than the main running lines. With this evidence I think I may well leave the buildings off the layout, but would really like to visit the site again, if possible, taking photos of these buildings.