As someone who is addicted to reference, I have thought about this question a lot. In addition to Matt’s comments, one of the best explanations I’ve stumbled across belongs to Monika Zagrobelna, a Polish artist who appears on various drawing tutorial sites across the web.
Most of her postings are animal drawing tutorials, but if you dig around you see she’s something of a drawing theory Socrates. She has a number of articles about topics like why we draw, talent and a fascinating, complicated article about perspective. She also writes frequently about the question of imagination. You can read about it here:
Or a lighter version here:
Her basic theory is that it’s a lot of very specific work in teaching your mind what things look like, not by drawing them over and over, but by learning how they fit and function and then drawing them over and over. The theory isn’t necessarily novel, but the way she explains it really is. I recall, though I can’t seem to find it now, an article where she described her own path from drawing from reference to drawing from imagination, and it is encouraging to think of it as a process rather than feeling that you’re either born Kim Jung Gi or not.
In that article she is actually a little tough on reference drawing, describing it as more of a step in the process of learning to draw from your mind. I don’t agree — I think of them as two related but distinct branches of the same tree; probably half my instagram feed is artists who do stunning work from life, like Guno Park and Tito Merello. And I think the real world is a complicated mix — artists use reference in many different ways — Mort Drucker, the wonderful Mad magazine caricaturist, had a very original style, but would use the promotional stills from movie companies for his movie parody reference.
Anyway, I recommend you check out her postings. They offer an interesting perspective.