My suggestion would be:
Make every refining instant
Make the stones give less materials
Make the player spend the same amount of time that he would in the current mechanic but instead walking around and mining different stones
The danger would be clearly while he's mining or coming back from...
