Conquer Club

Game History XML For Plugins and Script Writers

Archival storage to keep things organized and uncluttered. Can't find what you need? Search for old topics here.

Moderator: Tech Team

Forum rules
Please read the Community Guidelines before posting.

Game History XML For Plugins and Script Writers

Postby chipv on Tue Sep 23, 2008 5:19 pm

lackattack has done terrific work in producing an interface for script writers to access the finished game logs of players without resorting to hideous amounts of AJAX calls. There is a new dynamic XML for each player (this is used in Map Rank GL) which produces an entire game history for the player.

It is a set of XML files divided into pages of 200 games each. (You'll have to manage this carefully).

Call it by using the url

http://www.conquerclub.com/game_history.php?username=

or

http://www.conquerclub.com/game_history.php?user_id=

That will get you the first page. For subsequent pages add

&page=2 for page 2 etc

Brief description of what is available:

<user_id>42535</user_id> : User Id of subject invoked in the calling URL
<page>1 of 4</page> - this xml page number and total number of pages
<games total="702"> - total number of games

<game_number>160306</game_number>
<tournament/>
<private>No</private>
<speed_game>No</speed_game>
<map>British Isles</map>
<game_type>Triples</game_type>
<play_order>Freestyle</play_order>
<bonus_cards>Escalating</bonus_cards>
<fortifications>Unlimited</fortifications>
<war_fog>No</war_fog>

Self explanatory descriptors for the game type etc.

<players>
<player state="Lost">16836</player>
<player state="Lost">21000</player>
<player state="Lost">24215</player>
<player state="Won">27346</player>
<player state="Won">9063</player>
<player state="Won">42535</player>
</players>

List of player User Ids together with whether they won or lost.

<events>
<event timestamp="1168099246">4 eliminated 2 from the game</event>
<event timestamp="1168181834">6 eliminated 3 from the game</event>
<event timestamp="1168181913">6 eliminated 1 from the game</event>
<event timestamp="1168181913">4, 5, 6 won the game</event>
<event timestamp="1168181913">1 loses 16 points</event>
<event timestamp="1168181913">2 loses 16 points</event>
<event timestamp="1168181913">3 loses 16 points</event>
<event timestamp="1168181913">4 gains 16 points</event>
<event timestamp="1168181913">5 gains 16 points</event>
<event timestamp="1168181913">6 gains 16 points</event>
</events>

Cut down elimination summary including timestamps - note this is in seconds not milliseconds since 1 January 1970
The numbers in the elimination summary correspond to the order of the <player> list starting its index at 1.

if you need any help in trying to write a script to use this, feel free to ask any questions.

Note that this is a work in progress and more is to come.

Once again, awesome work from lackattack, thanks ever so much for this.
User avatar
Captain chipv
 
Posts: 2750
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Postby InkL0sed on Tue Sep 23, 2008 6:15 pm

This seems like an awesome addition. Cool.
User avatar
Lieutenant InkL0sed
 
Posts: 2370
Joined: Sat Jun 23, 2007 4:06 pm
Location: underwater

Re: Game History XML For Plugins and Script Writers

Postby Gilligan on Tue Sep 23, 2008 6:50 pm

Awesome work!

Just one question.

How do you get to the next page?
Image
User avatar
Captain Gilligan
 
Posts: 12478
Joined: Thu May 11, 2006 4:59 pm
Location: Providence, RI

Re: Game History XML For Plugins and Script Writers

Postby chipv on Tue Sep 23, 2008 6:51 pm

Gilligan wrote:Awesome work!

Just one question.

How do you get to the next page?


Add &page=2 to the url etc etc

Good question, I have modified the head post.
User avatar
Captain chipv
 
Posts: 2750
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Postby Gilligan on Tue Sep 23, 2008 6:56 pm

Hmm...

I have typed this in:

http://www.conquerclub.com/game_history ... an&Page=22

But I'm still at page 1.
Image
User avatar
Captain Gilligan
 
Posts: 12478
Joined: Thu May 11, 2006 4:59 pm
Location: Providence, RI

Re: Game History XML For Plugins and Script Writers

Postby chipv on Tue Sep 23, 2008 7:02 pm

Gilligan wrote:Hmm...

I have typed this in:

http://www.conquerclub.com/game_history ... an&Page=22

But I'm still at page 1.


Case sensitive - try &page=22 that works.

This is worth correcting, though, thanks.
User avatar
Captain chipv
 
Posts: 2750
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Postby pimphawks70 on Tue Sep 23, 2008 7:28 pm

I enjoy using plugins and addons but I have no idea how any of it works or anything. In laymans terms could you please explain what this update does?
Image
User avatar
Brigadier pimphawks70
 
Posts: 1102
Joined: Fri Aug 31, 2007 10:21 am

Re: Game History XML For Plugins and Script Writers

Postby chipv on Tue Sep 23, 2008 7:41 pm

pimphawks70 wrote:I enjoy using plugins and addons but I have no idea how any of it works or anything. In laymans terms could you please explain what this update does?


Sorry, it was aimed at code writers but does deserve a better explanation.

Scripts and plugins may wish to use data from finished game logs, say printing out the number of players you played against.
To do this you need to look at all of the game logs to see whom you played against.

Before this xml the only way was to ask the server to deliver one page containing one log at a time which puts stress on the server.
This xml puts 200 logs all on the same page thus reducing the amount of pages asked for.

So instead of asking the server for 200 separate pages for game logs, we can now ask for one page which contains all of the data for those logs.

To further increase the speed of downloading these xml files the data is cut down to the bare bones so it's lighter and faster.

This is now the best way for a script or plugin to get finished game data out of the CC database. More info may come later as well, depends on what people ask for.

So really this is for speed and saves stress on the server but also provides a standard interface for code writers to use.
User avatar
Captain chipv
 
Posts: 2750
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Postby whitestazn88 on Tue Sep 23, 2008 8:47 pm

i don't know what this is... but i looked at my page very briefly
Lieutenant whitestazn88
 
Posts: 3128
Joined: Mon Feb 05, 2007 2:59 pm
Location: behind you

Re: Game History XML For Plugins and Script Writers

Postby yeti_c on Wed Sep 24, 2008 3:14 am

Out of curiousity - what happens for games with missing logs?

C.
Image
Highest score : 2297
User avatar
Lieutenant yeti_c
 
Posts: 9624
Joined: Thu Jan 04, 2007 9:02 am

Re: Game History XML For Plugins and Script Writers

Postby chipv on Wed Sep 24, 2008 3:43 am

yeti_c wrote:Out of curiousity - what happens for games with missing logs?

C.


<events></events>

The events tag is blank.
User avatar
Captain chipv
 
Posts: 2750
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Postby pimphawks70 on Wed Sep 24, 2008 8:47 am

chipv wrote:
pimphawks70 wrote:I enjoy using plugins and addons but I have no idea how any of it works or anything. In laymans terms could you please explain what this update does?


Sorry, it was aimed at code writers but does deserve a better explanation.

Scripts and plugins may wish to use data from finished game logs, say printing out the number of players you played against.
To do this you need to look at all of the game logs to see whom you played against.

Before this xml the only way was to ask the server to deliver one page containing one log at a time which puts stress on the server.
This xml puts 200 logs all on the same page thus reducing the amount of pages asked for.

So instead of asking the server for 200 separate pages for game logs, we can now ask for one page which contains all of the data for those logs.

To further increase the speed of downloading these xml files the data is cut down to the bare bones so it's lighter and faster.

This is now the best way for a script or plugin to get finished game data out of the CC database. More info may come later as well, depends on what people ask for.

So really this is for speed and saves stress on the server but also provides a standard interface for code writers to use.

thank you very much
Image
User avatar
Brigadier pimphawks70
 
Posts: 1102
Joined: Fri Aug 31, 2007 10:21 am

Re: Game History XML For Plugins and Script Writers

Postby cspare on Sat Sep 27, 2008 3:22 am

I'm currently developing a plugin that uses this logfile to create a chart that shows a player's score over time.
<event timestamp="1168181913">6 eliminated 1 from the game</event>
<event timestamp="1168181913">4, 5, 6 won the game</event>
<event timestamp="1168181913">1 loses 16 points</event>

Processing this kind of log is rather difficult, because player numbers change each game for a given player. So I first have to lookup the relative position in the <players> element for the player I want to compile the stats for.
I would prefer something like this:
<event timestamp="1168181913">42535 eliminated 16836 from the game</event>
<event timestamp="1168181913">27346, 9063, 42535 won the game</event>
<event timestamp="1168181913">16836 loses 16 points</event>

This would save alot of time processing the log, it would allow me to directly select the right information I need.
I can't imagine a use for knowing the relative player indexes anyways, but if someone wanted to know, they could still retreive it through the <players> element.
Sergeant 1st Class cspare
 
Posts: 101
Joined: Sun Apr 08, 2007 12:56 pm

Re: Game History XML For Plugins and Script Writers

Postby timme10 on Tue Sep 30, 2008 5:48 am

cspare wrote:I'm currently developing a plugin that uses this logfile to create a chart that shows a player's score over time.
<event timestamp="1168181913">6 eliminated 1 from the game</event>
<event timestamp="1168181913">4, 5, 6 won the game</event>
<event timestamp="1168181913">1 loses 16 points</event>

Processing this kind of log is rather difficult, because player numbers change each game for a given player. So I first have to lookup the relative position in the <players> element for the player I want to compile the stats for.
I would prefer something like this:
<event timestamp="1168181913">42535 eliminated 16836 from the game</event>
<event timestamp="1168181913">27346, 9063, 42535 won the game</event>
<event timestamp="1168181913">16836 loses 16 points</event>

This would save alot of time processing the log, it would allow me to directly select the right information I need.
I can't imagine a use for knowing the relative player indexes anyways, but if someone wanted to know, they could still retreive it through the <players> element.


Yeah I agree, it would be a lot easier to select the right information. Good suggestion
Sergeant timme10
 
Posts: 16
Joined: Mon Jul 09, 2007 5:32 am
Location: Australia

Re: Game History XML For Plugins and Script Writers

Postby cspare on Sun Oct 05, 2008 6:28 am

I was wondering if anybody could give an indication whether my suggestion above will be implemented or not? And if so, when it will be released.
I'd like to start developing some new scripts that rely on the new XML.
Sergeant 1st Class cspare
 
Posts: 101
Joined: Sun Apr 08, 2007 12:56 pm

Re: Game History XML For Plugins and Script Writers

Postby Night Strike on Sun Oct 05, 2008 10:10 am

cspare wrote:I was wondering if anybody could give an indication whether my suggestion above will be implemented or not? And if so, when it will be released.
I'd like to start developing some new scripts that rely on the new XML.


More than likely, chipv will have to see this and work with lack on it, but chipv is away for a bit.
Image
User avatar
Major Night Strike
 
Posts: 8512
Joined: Wed Apr 18, 2007 2:52 pm

Re: Game History XML For Plugins and Script Writers

Postby chipv on Mon Oct 06, 2008 12:54 pm

cspare wrote:I was wondering if anybody could give an indication whether my suggestion above will be implemented or not? And if so, when it will be released.
I'd like to start developing some new scripts that rely on the new XML.


Yes, this is do-able, but you'll have to wait as it may not be done straight away, I'm afraid, so you'll have to work around for now.
User avatar
Captain chipv
 
Posts: 2750
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Postby cspare on Tue Oct 07, 2008 3:02 pm

Ok, cool. thanks.
I implemented a work around that works pretty good as well. I improved a Xpath query so that I could eliminate 2 for loops, I'm getting a pretty good performance now. Having to download all xml files still stays the main bottleneck though.

I was also wondering if you guys also keep record of a player's actual score over time, or only the events that cause the score to change (so effectively only storing the score delta)? This could help fill in the gaps of the missing game logs.
Sergeant 1st Class cspare
 
Posts: 101
Joined: Sun Apr 08, 2007 12:56 pm

Re: Game History XML For Plugins and Script Writers

Postby chipv on Tue Oct 07, 2008 3:17 pm

cspare wrote:Ok, cool. thanks.
I implemented a work around that works pretty good as well. I improved a Xpath query so that I could eliminate 2 for loops, I'm getting a pretty good performance now. Having to download all xml files still stays the main bottleneck though.

I was also wondering if you guys also keep record of a player's actual score over time, or only the events that cause the score to change (so effectively only storing the score delta)? This could help fill in the gaps of the missing game logs.


You can make one saving by preventing downloading of all the xml files if the number of games hasn't changed by using persistent data.
Store the data you need as well as the number of games. Read page one of the xmls and compare game numbers to decide whether to use any more of the xmls.

The event tags store all points changes available except for games with missing logs. We've been looking at missing logs for a while now and unfortunately there is no way to accurately recalculate all of the points - the logs are literally gone. Since we can't guarantee finding missing logs sandwiched between visible logs we cannot recalculate all point changes. Pity.
User avatar
Captain chipv
 
Posts: 2750
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Postby cspare on Thu Oct 09, 2008 2:01 pm

While testing ConquerStats I came across some empty user XML pages:
http://www.conquerclub.com/game_history.php?user_id=107287:
Code: Select all
<game_history>
   <user_id>107287</user_id>
   <page>1 of 5</page>
   <games total="1000">
   </games>
</game_history>

http://www.conquerclub.com/game_history.php?user_id=42178
Code: Select all
<game_history>
   <user_id>42178</user_id>
   <page>1 of 4</page>
   <games total="652">
   </games>
</game_history>
Sergeant 1st Class cspare
 
Posts: 101
Joined: Sun Apr 08, 2007 12:56 pm

Re: Game History XML For Plugins and Script Writers

Postby chipv on Thu Oct 09, 2008 3:20 pm

Yes this affects Map Rank also:

viewtopic.php?f=59&t=64678&start=60#p1627523

Problem was reported, awaiting response.
User avatar
Captain chipv
 
Posts: 2750
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Postby chipv on Tue Oct 14, 2008 3:52 pm

RECENT SITE CHANGES REQUIRE CHANGES TO MOST SCRIPTS.

Thread is being locked until they're fixed.
User avatar
Captain chipv
 
Posts: 2750
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Postby chipv on Wed Oct 15, 2008 3:31 pm

XML is fixed now, sorry for the delay.
User avatar
Captain chipv
 
Posts: 2750
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Postby chipv on Wed Oct 15, 2008 5:53 pm

Apologies in advance but the archived logs are about to be cleaned up which means the XML files will be broken again.
This will begin tomorrow morning (EDT) and last approximately 24 hours.

As soon as the logs have been cleaned up, the XMLs should work again and I'll post again to say so.

Apologies for the inconvenience.

For the avoidance of doubt this will currently affect Map Rank GL and Conquerstats charts.
User avatar
Captain chipv
 
Posts: 2750
Joined: Mon Apr 28, 2008 5:30 pm

Re: Game History XML For Plugins and Script Writers

Postby chipv on Fri Oct 17, 2008 3:31 pm

XML is back in business now.
User avatar
Captain chipv
 
Posts: 2750
Joined: Mon Apr 28, 2008 5:30 pm

Next

Return to Tool Archives

Who is online

Users browsing this forum: No registered users