Beta Stage For New Maps

[edbeard]

Concise description:

• After Quenching Maps go into Beta Stage listed separately from other maps

Specifics:

• At the bottom of the Map list, a new section of Beta Maps is listed. Instead of NEW, have them say BETA
• Users are also told these maps are in Beta Stage so there map be bugs in gameplay and also changes to the map can occur at any time. Play these maps at your own risk as points are still up for grabs

This will improve the following aspects of the site:

• appease those of us in the map foundry about a 'testing facility' while not having to worry about point free games or how people will join
• more importantly it allows XML and gameplay to be finalized before the NEW tags go up
• standard users will not have a bad experience on buggy or unbalanced maps
• improves the quality of new maps

[Incandenza]

Given that it's been quite often stated by various members of Management that a test site for new maps is unfeasible, I think that ed's suggestion is an excellent stopgap until such time as a test site becomes feasible. But I would add that it shouldn't necessarily be ALL new maps, or at least the time in beta should be variable given the complexity (i.e. a standard geographical map would be in just long enough for people to play and ensure the xml works, whereas a complicated map would be more about determining if gameplay is imbalanced).

I suppose then the question is, who approves the move from beta to finalized in the case of complicated gameplay?

[lancehoch]

I agree with this suggestion. After the incidents of the past two weeks where both "Arms Race" and "Age of Realms: Mayhem" had older versions of the XML uploaded, this seems like it would alleviate some issues.

[DiM]

I agree with this suggestion. After the incidents of the past two weeks where both "Arms Race" and "Age of Realms: Mayhem" had older versions of the XML uploaded, this seems like it would alleviate some issues.

mayhem didn't have an older xml uploaded it had the right xml but nobody knew 0 reinforcements per turn from the number of terits owned is impossible. so the system game 3 armies instead of zero. since zero was impossible the xml had to be changed to give 1 instead of 3.

but i agree with the idea. if a proper testing facility is impossible then beta maps are a must.

i don't think there was a single map in this batch that didn't have some xml isues and that's worrying cause everytime maps are uploaded there's bound to be some issues, either minor (like a missing connection) or more serious (like imbalance)

also i don't think there needs to be a separate beta testing period for complex or simple maps. i think 1 week or even less is enough.

usually minor things like a missing connection or a bad bonus are discovered within the first couple of games. and the imballance issues (if any) within 1-2 days. so i thing 4-7 days should be more than enough.

[ParadiceCity9]

Ya definitely need this.

[oaktown]

beta testing has, and has always had, my vote.

[wicked]

Suggestion of the Year.

[AndyDufresne]

We'll need to continue discussion on length of time in BETA. Also, eventually we'll need input on whether Lack would agree to such a thing (most of it I suppose would fall on redesigning the Start A Game/Game Finder) -- but don't worry about the second part for a while.


--Andy

[lancehoch]

Would it be possible to have the head cartographer start games on BETA maps in a similar manner to tournaments? Maybe have the maps be hidden on the game finder/start a game pages so that the average member cannot start these maps. I think DiM's suggestion of a week or so is a good time frame, however there should also be a number of games (around 50 or so? maybe 100) to make sure that as many different configurations of drops and options as possible are seen.

[edbeard]

I think a one week minimum is a good idea. And, additional week(s) added in extreme cases where not enough games have been played and/or if gameplay undergoes major changes.

Obviously quite a few maps are going to be in beta for much longer than they need but in the cases where changes are needed it'll help. The only problem with all this is if the map maker is not able to make changes in a short amount of time. This probably won't come up often though as the only time major changes (i.e. not XML) are necessary are with the very complex maps.

[James Vazquez]

sounds like a great idea especially since the quality and complexity of the maps are ever increasing.

Maybe a new thread in the foundry for comments from beta players {testers} for each map.

[TaCktiX]

Definitely go forward with the idea. As mapmakers seek to create new and different experiences, the realm of the known is being left in the dust, and bugs are going to happen. Letting other people know that this is a possibility is a must.

As for the other suggestions of hidden games, etc., they also have my vote. It wouldn't be a "testing area" per se (still need to finish the entire Foundry process), but it would keep the people not willing to risk their points on something that COULD have serious bugs from getting screwed.

[edbeard]

I think Lack has a problem with doing the hidden area thing as it'd probably be much harder to code.

This idea is good because it'd probably be simple. It'd just be an edit to the layout of the game starter/finder page.


It'd also just add another stage to the map making process so we can't just make a map and when it gets quenched be done with it. Anything from XML, territory names, and gameplay are up for editing.

One more line of defence before going out to the public. Though they can still access it at their own risk.

[blakebowling]

I suppose then the question is, who approves the move from beta to finalized in the case of complicated gameplay?

It could be that there has to be a certain time, two weeks?, that a map has to go without a bug being discovered.

[lackattack]

It'd also just add another stage to the map making process so we can't just make a map and when it gets quenched be done with it. Anything from XML, territory names, and gameplay are up for editing.

That's gonna be a problem - if you change the # of territories or continents in an xml that has already been in play, the older games will be messed up with log entries for non-existent places. Then I have to write a script to alter the older games so they don't cause errors. That's not something I want to do often.

So would the only difference be how the map listed, not how it's made?

Will enough people play hidden beta maps to discover the bugs quickly? If not there is no point to this.

[DiM]

Will enough people play hidden beta maps to discover the bugs quickly? If not there is no point to this.

well i'm certain at least the map makers and the foundry regulars will play the maps. that means ~40-50 people. not a lot but more than enough to play some games and solve the problems.
and i'm 100% sure other people will also join in on the testing.

[fireedud]

Well, I'm pretty sure most foundry regulars wouldn't mind playing; I know I definitely wouldn't mind playing a test map. And all we need is at least eight players.

[t-o-m]

That's gonna be a problem - if you change the # of territories or continents in an xml that has already been in play, the older games will be messed up with log entries for non-existent places. Then I have to write a script to alter the older games so they don't cause errors. That's not something I want to do often.

then you stop all the beta stage maps, when you play them you dont gain points or loose them?
or when you change the # of cont you could stop all games, or wait for them to stop - also stop anyone from joining anymore of those games - then alter it and re-live it?

maybe too much work.

So would the only difference be how the map listed, not how it's made?

yes i think thats what he's getting at, like how you were thinking of categorising the maps you could have a "BETA" category and a description to play at your own risk, no pun intended

Will enough people play hidden beta maps to discover the bugs quickly? If not there is no point to this.

i would
lol

or you could have active foundry members that are selected to play the map?

[wicked]

I would hope everything would stay the same (or improve) with the map development, and people would be itching to play the new maps, so yeah, everything should stay the same except for how it's listed. Right now for example, some maps have gameplay issues (mainly unbalanced), so might be changed anyway. At least if it's listed as a beta map, people would expect some possible bugs, and would choose whether to play it now or wait for the final version. But i see what you're saying, that people shouldn't use the beta stage as a fallback to not properly completing the map in the first place.

I don't think you need to hide the maps though, just list then at the bottom of the create a game page in a separate Beta section and slap a beta tag on the map image.

[lord voldemort]

if there was to be a map testing group id be down for it...i sort of a half foundry regular

[edbeard]

It'd also just add another stage to the map making process so we can't just make a map and when it gets quenched be done with it. Anything from XML, territory names, and gameplay are up for editing.

That's gonna be a problem - if you change the # of territories or continents in an xml that has already been in play, the older games will be messed up with log entries for non-existent places. Then I have to write a script to alter the older games so they don't cause errors. That's not something I want to do often.

So would the only difference be how the map listed, not how it's made?

Will enough people play hidden beta maps to discover the bugs quickly? If not there is no point to this.

The only difference between my idea and what happens currently is where 'new' maps are listed in the Start a Game or Game Finder pages. Just list them below the other maps with a warning about them being 'Beta' and possible bugs. There'd be no other changes except in the Foundry where after quenching we'd add another step to the process. Maps would still be considered finished when quenched but if any problems show up in the Beta Stage, they need to be fixed. I believe this happens now but it's not really officially listed. If it was its own step then it'd be more official.

Anyway, like I've said, it'd only separate new maps from the current ones with a warning that these are not totally finished and there could be a couple bugs. I don't think they should be hidden otherwise like you said we wouldn't have enough people playing. Just a separation and a warning so people are aware that it could be buggy. I think plenty of people will still play the maps. And, in most cases, the only problems will stem from XML and those can be found in only a few games.

[lancehoch]

That's gonna be a problem - if you change the # of territories or continents in an xml that has already been in play, the older games will be messed up with log entries for non-existent places. Then I have to write a script to alter the older games so they don't cause errors. That's not something I want to do often.

Couldn't you do something similar to the change for Classic, add characters (~ or !) on the end of the map name when it is in the BETA stage? Would the logs get messed up that way?

[lackattack]

I'm fine with using "beta" instead of "new".

Couldn't you do something similar to the change for Classic, add characters (~ or !) on the end of the map name when it is in the BETA stage? Would the logs get messed up that way?

Unfortunately that won't help, it's only to prevent image caching.

[blakebowling]

instead of changing the map could you just make a new one and take the old one off of game creation and game finder, so the old games would stay the same. you could just add the version to the end, until it gets done

[edbeard]

I'm fine with using "beta" instead of "new".

Just to make sure, are you going to separate them on the page with a warning?

There's no point just changing from labeling them New to Beta if they're not separated from the other maps and there's no explanation as to why they are separated/what Beta means.

I do think this explanation should be right there on the page. So after all the normal maps are listed you have a small gap to separate the maps. Then, a couple sentences saying, "The following maps are in the Beta Stage. Points are still up for grabs but XML and gameplay issues are possible. If you do not want to play a map that potentially has bugs, then wait until the maps have left the Beta Stage." Or something of that nature.