As long as you have enough people doing it that you can get a pretty good agreement on everything, then volunteers would work. Perhaps by comparing everything to one map, classic for example, you could do it more objectively. It is easier to analyze how much harder something is than something else than to just come up with a number.
Similar to rating a player after a game, you are allowed to leave a rating for the map. Complexity is my main concern, but other categories would be possible.
Finish a game, leave a rating. 1-5 Stars. When browsing maps, the rating shows up.
How this will benefit the site and/or other comments:
This would be an easy way to judge a map at a quick glance. This could also be integrated into the introduciton of new recruits, allowing them to only play on maps below a certain "complexity" rating. This would allow the community to dictate which maps would be accessible to new recruits.
Just a thought.
[MOD EDIT: I had a tough time deciding whether to move this to REJECTED for lack of support or IMPLEMENTED because something similar to this already exists. For now, if you would like to rate a map, please go to viewtopic.php?f=358&t=169263. I read this suggestion as a way of making the rating of maps easier. The OP, I believe, would like the rating of maps to occur immediately after playing those maps, instead of having to trudge through the Foundry to the Gallery to find the thread that I have linked to. I agree that would be nice and I consider it different enough from the Status Quo that this should be viewed as a distinct but rejected suggestion. It is nice that we now have the option to rate maps (Go Foundry!), but it is not as easy for the typical user as the OP would like. --agentcom]