These days I try to do both, but recognize it's an on-going thing that will never be done.

Sometimes folder structure can be a challenge because of extensive metadata. Where do parts of a compilation go, for example. At least with metadata, music players can show the tracks correctly.