Game History XML For Plugins and Script Writers

Archival storage to keep things organized and uncluttered. Can't find what you need? Search for old topics here.

Moderator: Tech Team

Forum rules
Please read the Community Guidelines before posting.
User avatar
chipv
Head Tech
Head Tech
Posts: 3135
Joined: Mon Apr 28, 2008 5:30 pm

Game History XML For Plugins and Script Writers

Post by chipv »

lackattack has done terrific work in producing an interface for script writers to access the finished game logs of players without resorting to hideous amounts of AJAX calls. There is a new dynamic XML for each player (this is used in Map Rank GL) which produces an entire game history for the player.

It is a set of XML files divided into pages of 200 games each. (You'll have to manage this carefully).

Call it by using the url

http://www.conquerclub.com/game_history.php?username=

or

http://www.conquerclub.com/game_history.php?user_id=

That will get you the first page. For subsequent pages add

&page=2 for page 2 etc

Brief description of what is available:

<user_id>42535</user_id> : User Id of subject invoked in the calling URL
<page>1 of 4</page> - this xml page number and total number of pages
<games total="702"> - total number of games

<game_number>160306</game_number>
<tournament/>
<private>No</private>
<speed_game>No</speed_game>
<map>British Isles</map>
<game_type>Triples</game_type>
<play_order>Freestyle</play_order>
<bonus_cards>Escalating</bonus_cards>
<fortifications>Unlimited</fortifications>
<war_fog>No</war_fog>

Self explanatory descriptors for the game type etc.

<players>
<player state="Lost">16836</player>
<player state="Lost">21000</player>
<player state="Lost">24215</player>
<player state="Won">27346</player>
<player state="Won">9063</player>
<player state="Won">42535</player>
</players>

List of player User Ids together with whether they won or lost.

<events>
<event timestamp="1168099246">4 eliminated 2 from the game</event>
<event timestamp="1168181834">6 eliminated 3 from the game</event>
<event timestamp="1168181913">6 eliminated 1 from the game</event>
<event timestamp="1168181913">4, 5, 6 won the game</event>
<event timestamp="1168181913">1 loses 16 points</event>
<event timestamp="1168181913">2 loses 16 points</event>
<event timestamp="1168181913">3 loses 16 points</event>
<event timestamp="1168181913">4 gains 16 points</event>
<event timestamp="1168181913">5 gains 16 points</event>
<event timestamp="1168181913">6 gains 16 points</event>
</events>

Cut down elimination summary including timestamps - note this is in seconds not milliseconds since 1 January 1970
The numbers in the elimination summary correspond to the order of the <player> list starting its index at 1.

if you need any help in trying to write a script to use this, feel free to ask any questions.

Note that this is a work in progress and more is to come.

Once again, awesome work from lackattack, thanks ever so much for this.
User avatar
InkL0sed
Posts: 2370
Joined: Sat Jun 23, 2007 4:06 pm
Gender: Male
Location: underwater
Contact:

Re: Game History XML For Plugins and Script Writers

Post by InkL0sed »

This seems like an awesome addition. Cool.
User avatar
Gilligan
Posts: 12478
Joined: Thu May 11, 2006 4:59 pm
Gender: Male
Location: Providence, RI

Re: Game History XML For Plugins and Script Writers

Post by Gilligan »

Awesome work!

Just one question.

How do you get to the next page?
Image
User avatar
chipv
Head Tech
Head Tech
Posts: 3135
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Post by chipv »

Gilligan wrote:Awesome work!

Just one question.

How do you get to the next page?


Add &page=2 to the url etc etc

Good question, I have modified the head post.
User avatar
Gilligan
Posts: 12478
Joined: Thu May 11, 2006 4:59 pm
Gender: Male
Location: Providence, RI

Re: Game History XML For Plugins and Script Writers

Post by Gilligan »

Hmm...

I have typed this in:

http://www.conquerclub.com/game_history ... an&Page=22

But I'm still at page 1.
Image
User avatar
chipv
Head Tech
Head Tech
Posts: 3135
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Post by chipv »

Gilligan wrote:Hmm...

I have typed this in:

http://www.conquerclub.com/game_history ... an&Page=22

But I'm still at page 1.


Case sensitive - try &page=22 that works.

This is worth correcting, though, thanks.
User avatar
pimphawks70
Posts: 1102
Joined: Fri Aug 31, 2007 10:21 am

Re: Game History XML For Plugins and Script Writers

Post by pimphawks70 »

I enjoy using plugins and addons but I have no idea how any of it works or anything. In laymans terms could you please explain what this update does?
Image
User avatar
chipv
Head Tech
Head Tech
Posts: 3135
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Post by chipv »

pimphawks70 wrote:I enjoy using plugins and addons but I have no idea how any of it works or anything. In laymans terms could you please explain what this update does?


Sorry, it was aimed at code writers but does deserve a better explanation.

Scripts and plugins may wish to use data from finished game logs, say printing out the number of players you played against.
To do this you need to look at all of the game logs to see whom you played against.

Before this xml the only way was to ask the server to deliver one page containing one log at a time which puts stress on the server.
This xml puts 200 logs all on the same page thus reducing the amount of pages asked for.

So instead of asking the server for 200 separate pages for game logs, we can now ask for one page which contains all of the data for those logs.

To further increase the speed of downloading these xml files the data is cut down to the bare bones so it's lighter and faster.

This is now the best way for a script or plugin to get finished game data out of the CC database. More info may come later as well, depends on what people ask for.

So really this is for speed and saves stress on the server but also provides a standard interface for code writers to use.
whitestazn88
Posts: 3128
Joined: Mon Feb 05, 2007 2:59 pm
Gender: Male
Location: behind you
Contact:

Re: Game History XML For Plugins and Script Writers

Post by whitestazn88 »

i don't know what this is... but i looked at my page very briefly
User avatar
yeti_c
Posts: 9624
Joined: Thu Jan 04, 2007 9:02 am
Gender: Male

Re: Game History XML For Plugins and Script Writers

Post by yeti_c »

Out of curiousity - what happens for games with missing logs?

C.
Image
Highest score : 2297
User avatar
chipv
Head Tech
Head Tech
Posts: 3135
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Post by chipv »

yeti_c wrote:Out of curiousity - what happens for games with missing logs?

C.


<events></events>

The events tag is blank.
User avatar
pimphawks70
Posts: 1102
Joined: Fri Aug 31, 2007 10:21 am

Re: Game History XML For Plugins and Script Writers

Post by pimphawks70 »

chipv wrote:
pimphawks70 wrote:I enjoy using plugins and addons but I have no idea how any of it works or anything. In laymans terms could you please explain what this update does?


Sorry, it was aimed at code writers but does deserve a better explanation.

Scripts and plugins may wish to use data from finished game logs, say printing out the number of players you played against.
To do this you need to look at all of the game logs to see whom you played against.

Before this xml the only way was to ask the server to deliver one page containing one log at a time which puts stress on the server.
This xml puts 200 logs all on the same page thus reducing the amount of pages asked for.

So instead of asking the server for 200 separate pages for game logs, we can now ask for one page which contains all of the data for those logs.

To further increase the speed of downloading these xml files the data is cut down to the bare bones so it's lighter and faster.

This is now the best way for a script or plugin to get finished game data out of the CC database. More info may come later as well, depends on what people ask for.

So really this is for speed and saves stress on the server but also provides a standard interface for code writers to use.

thank you very much
Image
cspare
Posts: 101
Joined: Sun Apr 08, 2007 12:56 pm

Re: Game History XML For Plugins and Script Writers

Post by cspare »

I'm currently developing a plugin that uses this logfile to create a chart that shows a player's score over time.
<event timestamp="1168181913">6 eliminated 1 from the game</event>
<event timestamp="1168181913">4, 5, 6 won the game</event>
<event timestamp="1168181913">1 loses 16 points</event>

Processing this kind of log is rather difficult, because player numbers change each game for a given player. So I first have to lookup the relative position in the <players> element for the player I want to compile the stats for.
I would prefer something like this:
<event timestamp="1168181913">42535 eliminated 16836 from the game</event>
<event timestamp="1168181913">27346, 9063, 42535 won the game</event>
<event timestamp="1168181913">16836 loses 16 points</event>

This would save alot of time processing the log, it would allow me to directly select the right information I need.
I can't imagine a use for knowing the relative player indexes anyways, but if someone wanted to know, they could still retreive it through the <players> element.
timme10
Posts: 16
Joined: Mon Jul 09, 2007 5:32 am
Location: Australia

Re: Game History XML For Plugins and Script Writers

Post by timme10 »

cspare wrote:I'm currently developing a plugin that uses this logfile to create a chart that shows a player's score over time.
<event timestamp="1168181913">6 eliminated 1 from the game</event>
<event timestamp="1168181913">4, 5, 6 won the game</event>
<event timestamp="1168181913">1 loses 16 points</event>

Processing this kind of log is rather difficult, because player numbers change each game for a given player. So I first have to lookup the relative position in the <players> element for the player I want to compile the stats for.
I would prefer something like this:
<event timestamp="1168181913">42535 eliminated 16836 from the game</event>
<event timestamp="1168181913">27346, 9063, 42535 won the game</event>
<event timestamp="1168181913">16836 loses 16 points</event>

This would save alot of time processing the log, it would allow me to directly select the right information I need.
I can't imagine a use for knowing the relative player indexes anyways, but if someone wanted to know, they could still retreive it through the <players> element.


Yeah I agree, it would be a lot easier to select the right information. Good suggestion
cspare
Posts: 101
Joined: Sun Apr 08, 2007 12:56 pm

Re: Game History XML For Plugins and Script Writers

Post by cspare »

I was wondering if anybody could give an indication whether my suggestion above will be implemented or not? And if so, when it will be released.
I'd like to start developing some new scripts that rely on the new XML.
User avatar
Night Strike
Posts: 8512
Joined: Wed Apr 18, 2007 2:52 pm
Gender: Male

Re: Game History XML For Plugins and Script Writers

Post by Night Strike »

cspare wrote:I was wondering if anybody could give an indication whether my suggestion above will be implemented or not? And if so, when it will be released.
I'd like to start developing some new scripts that rely on the new XML.


More than likely, chipv will have to see this and work with lack on it, but chipv is away for a bit.
Image
User avatar
chipv
Head Tech
Head Tech
Posts: 3135
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Post by chipv »

cspare wrote:I was wondering if anybody could give an indication whether my suggestion above will be implemented or not? And if so, when it will be released.
I'd like to start developing some new scripts that rely on the new XML.


Yes, this is do-able, but you'll have to wait as it may not be done straight away, I'm afraid, so you'll have to work around for now.
cspare
Posts: 101
Joined: Sun Apr 08, 2007 12:56 pm

Re: Game History XML For Plugins and Script Writers

Post by cspare »

Ok, cool. thanks.
I implemented a work around that works pretty good as well. I improved a Xpath query so that I could eliminate 2 for loops, I'm getting a pretty good performance now. Having to download all xml files still stays the main bottleneck though.

I was also wondering if you guys also keep record of a player's actual score over time, or only the events that cause the score to change (so effectively only storing the score delta)? This could help fill in the gaps of the missing game logs.
User avatar
chipv
Head Tech
Head Tech
Posts: 3135
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Post by chipv »

cspare wrote:Ok, cool. thanks.
I implemented a work around that works pretty good as well. I improved a Xpath query so that I could eliminate 2 for loops, I'm getting a pretty good performance now. Having to download all xml files still stays the main bottleneck though.

I was also wondering if you guys also keep record of a player's actual score over time, or only the events that cause the score to change (so effectively only storing the score delta)? This could help fill in the gaps of the missing game logs.


You can make one saving by preventing downloading of all the xml files if the number of games hasn't changed by using persistent data.
Store the data you need as well as the number of games. Read page one of the xmls and compare game numbers to decide whether to use any more of the xmls.

The event tags store all points changes available except for games with missing logs. We've been looking at missing logs for a while now and unfortunately there is no way to accurately recalculate all of the points - the logs are literally gone. Since we can't guarantee finding missing logs sandwiched between visible logs we cannot recalculate all point changes. Pity.
cspare
Posts: 101
Joined: Sun Apr 08, 2007 12:56 pm

Re: Game History XML For Plugins and Script Writers

Post by cspare »

While testing ConquerStats I came across some empty user XML pages:
http://www.conquerclub.com/game_history.php?user_id=107287:

Code: Select all

<game_history>
   <user_id>107287</user_id>
   <page>1 of 5</page>
   <games total="1000">
   </games>
</game_history>

http://www.conquerclub.com/game_history.php?user_id=42178

Code: Select all

<game_history>
   <user_id>42178</user_id>
   <page>1 of 4</page>
   <games total="652">
   </games>
</game_history>
User avatar
chipv
Head Tech
Head Tech
Posts: 3135
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Post by chipv »

Yes this affects Map Rank also:

viewtopic.php?f=59&t=64678&start=60#p1627523

Problem was reported, awaiting response.
User avatar
chipv
Head Tech
Head Tech
Posts: 3135
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Post by chipv »

RECENT SITE CHANGES REQUIRE CHANGES TO MOST SCRIPTS.

Thread is being locked until they're fixed.
User avatar
chipv
Head Tech
Head Tech
Posts: 3135
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Post by chipv »

XML is fixed now, sorry for the delay.
User avatar
chipv
Head Tech
Head Tech
Posts: 3135
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Post by chipv »

Apologies in advance but the archived logs are about to be cleaned up which means the XML files will be broken again.
This will begin tomorrow morning (EDT) and last approximately 24 hours.

As soon as the logs have been cleaned up, the XMLs should work again and I'll post again to say so.

Apologies for the inconvenience.

For the avoidance of doubt this will currently affect Map Rank GL and Conquerstats charts.
User avatar
chipv
Head Tech
Head Tech
Posts: 3135
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Post by chipv »

XML is back in business now.
Post Reply

Return to “Tool Archives”