This is topic XML (for a Pokedex) in forum Research Lab at The Azure Heights Forum.


To visit this topic, use this URL:
https://www.math.miami.edu/~jam/azure/forum/buzz/ultimatebb.cgi?ubb=get_topic;f=1;t=001206

Posted by LexyEevee (Member # 2885) on 06-13-2002, 02:10 AM:
 
If you've never programmed, you'll probably have no interest in this...

I have a Pokedex in the works (which I haven't touched in a year and now realize is horribly out of date and wrong in quite a few places etc) which reads data from an XML file using Perl's XML::Parser module and spits out a page. The problem is, the server it's on it already fairly slow, and the script takes a few seconds to run. The only other server I can use is slower in parse time but a little faster in load time... but it's not very reliable.

So! Before I even attempt to finish it, I need to make it horrendously fast. Like, Google fast. Should I convert the script/datafile to:

Perl - I know and love Perl, but some of its modules are fairly complex and/or slow
PHP - I've never done a damn thing with PHP, but it has thorough documentation and is easier to use with both XML and SQL, supposedly

and

XML - XML is far more flexible than a database, however all of my attempts to read it have been quite difficult and quite slow (the file is over 600KB and growing), especially with all the little exceptions and errors I keep having to add
SQL - I could pack all the data into a database without too much trouble; the problem is the time it would take and that I don't know much about SQL, however, it's supposed to be fairly fast

or

All static HTML - I did this for a while; used a script to generate all the Pokedex pages and then uploaded them... this gets really old when you change something every ten seconds, though.
Something else - I'm open to any suggestions o.x

help, please... And sorry for sticking this in this forum, but it didn't seem to fit anywhere else. It's indirectly related to research o.o
 
Posted by MK (Member # 1445) on 06-13-2002, 02:39 AM:
 
First of all, good work on making this project, but, I am not familar with the programming languages you speak of, at best I know a little html.

Maybe someone else can help ya out, I myself am a programmer, but, as I said, I am unfamilar with those languages so I'm not of much help here.

Good Luck
 
Posted by spunman (Member # 1181) on 06-13-2002, 10:51 AM:
 
LexyEevee: Perl - I know and love Perl, but some of its modules are fairly complex and/or slow

i'd use it anyway. if you enjoy perl, it'll show, and you're more likely to get a better result since you'll be motivated to put more into it. i wouldn't worry about the load times--patience is a virtue that people shouldn't have a problem displaying if they really want what they're looking for.

All static HTML - I did this for a while; used a script to generate all the Pokedex pages and then uploaded them... this gets really old when you change something every ten seconds, though.

yes, it does. however, it's preferrable if you're including chain breeding. imagine having a machine trying to figure out every pokémon a breeding move can be acquired from, how each contributor gets it, every pokémon they get it from (if via breeding), how all the new contributors get it... all for each of half a dozen breeding moves... figuring this out locally on my computer took about half an hour for the entire pokédex. no way would i have gotten by having all those calculations done every time someone clicked on a link. i just modified my compiler so i could specify a single entry to update for when i made a little boo-boo (like when i had mime learning only reflect, but not light screen).

where is your current pokédex, btw?
 
Posted by LexyEevee (Member # 2885) on 06-13-2002, 12:34 PM:
 
http://www.nelcorp.com/alex/pokedex/133.html - Old version, all static HTML... most of the links and the search box will all lead to these.
http://www.nelcorp.com/alex/pokedex/pokemon.cgi?133 - New and shiny and dynamic and not done at all.

Both made for IE, since I don't have Netscape and thus can't really test anything with it and irrationally hate it from the depths of my soul. They should work in Mozilla, though.

As for chain breeding... ick x.x; That'd be a horrible mess. I'd almost definitely put that in a seperate script somewhere and link to it... although considering how most of the parse time seems to be just reading the XML, I don't think it'd take much longer than anything else is.
 
Posted by spunman (Member # 1181) on 06-13-2002, 01:08 PM:
 
VERY nice.
 
Posted by LexyEevee (Member # 2885) on 06-13-2002, 03:08 PM:
 
heh, thanks ^_^; Is it too slow or should I not bother trying to speed it up?

hm, all I need to do now is add inheritable Crystal moves, bugs and details on most of the moves, finish the move and type scripts, get ripped location stuff, rewrite my stats calculator... o.O
 
Posted by jshadias (Member # 2710) on 06-14-2002, 01:34 PM:
 
Loads in about .5 secs on my Dial-up. The images take longer, but that doesn't count.
Incredible, only thing is your move order confuses me. You have C-GS-Y-RB, standard is RB-Y-GS-C. It also escapes me how your attacks are ordered.
Also, how do you do alphabetical order? I can't seem to find any way to do it. I'm trying to make a Pokedex myself using ASP, but I'm sure I could convert it over if you were willing to divulge the secret. The only other thing that's escaped me is weaknesses and resistances, but I just haven't devoted enough time to that.
 
Posted by LexyEevee (Member # 2885) on 06-14-2002, 03:38 PM:
 
Odd, I'm sure I've usually seen it C-GS-Y-RB, and somehow mine ended in that order and I never bothered to change it. Wouldn't be too hard, though.

The moves are supposed to be in some semblance of Crystal order, but I need to fix that and the JavaScript sort routine (click on one of the column headers). I had an insanely complicated script that attempted to sort the moves by all six games' levels, listing a move twice if necessary to keep them all in order. It never worked very well for the Pokemon that really needed it like Nidoran, so I got tired of it and ended up sorting just by Crystal level.

Sort in the Perl code? Well, Perl == god so I can go something like this:

@array = sort @array;

Maybe that works in ASP too. If it doesn't or you're masochistic you could also write a recursive algorithm to sort.

I usually use something more complex, like:

@i = sort { $poke{moves}{move}{c}[$a] <=> $poke{moves}{move}{c}[$b] || $poke{moves}{move}{name}[$a] cmp $poke{moves}{move}{name}[$b]; } @i;

That's how I sort the moves ^^;

I've decided to stick with the XML and try to optimize the code that reads it, mainly because I don't want to rewrite everything *again*. Plus XML is the future and blah blah blah.
 
Posted by jshadias (Member # 2710) on 06-15-2002, 12:49 PM:
 
Nope, I can't convert that to ASP. And I thought ASP was the best of all the other scripting languages. Guess I was wrong [Frown] Of course, ASP is the future and blah blah blah.

And what I mean by your attacks order is the individual attack pages. When you choose Drill Peck, the previous one is Peck and the next one is some other move.

[ 06-15-2002, 12:51 PM: Message edited by: jshadias ]
 
Posted by LexyEevee (Member # 2885) on 06-15-2002, 02:39 PM:
 
Oh! The attacks are in (I think) the game's internal order. Just because that's how I have them numbered.
 
Posted by MasterChu19155 (Member # 2101) on 06-16-2002, 03:00 PM:
 
A half yer ago I started together with some people from the german pokemonvillage a great PokeDex project.
We decided to use PHP.
Now are 70 PKMN finished, but our old server is n/a -_-*

I cannot finish the project.
But if you need some inspiration concerning the PHP style I could maybe help you.

Here look at our style we used for the german version.
Here the direct link to the Mareep page.

http://unger.pokemonvillage.gamigo.de/dex.php?pkmn=179.php

[ 06-16-2002, 03:02 PM: Message edited by: MasterChu19155 ]
 
Posted by jshadias (Member # 2710) on 06-18-2002, 12:56 AM:
 
Looks pretty good, even though I can't read most of it. BTW, what are these attacks: Bodycheck, Bodyslam, Bodyguard
Body Slam is obvious, and I'm guessing Bodycheck is Slam. But what's Bodyguard?

[ 06-18-2002, 12:58 AM: Message edited by: jshadias ]
 
Posted by jshadias (Member # 2710) on 06-18-2002, 01:31 AM:
 
Just remembered, if you really want to speed it up more, make every Pokemon have an individual data file if you haven't already. Then pass that data to the sorter. How mine will work (when it's finished) is somewhat different, but it goes like this:
File: 001.asp
<Bulbasaur's data here>
<INCLUDE the sorter script>

Makes it easier for me at least. I think I'm going to just end up leaving it like it is though, or maybe generating includes using variables, which requires writing the command <!-- #INCLUDE FILE="file.asp" --> to a file and then including that file itself. Wait a sec, that won't work [Frown] . I'd have to create a random file for it to work (to avoid people getting their pages mixed up), and since I can't include with variables any other way... Guess I'm stuck with creating individual pages like above or loading all the data at once.
 
Posted by spunman (Member # 1181) on 06-18-2002, 07:22 AM:
 
jshadias: Body Slam is obvious, and I'm guessing Bodycheck is Slam. But what's Bodyguard?

bodycheck is take down. bodyguard is safeguard.
 
Posted by LexyEevee (Member # 2885) on 06-18-2002, 03:06 PM:
 
I've managed to get pokemon.cgi down to less than a second and a half, which is... better, at least. And someone who usually knows what they're talking about told me Perl is three times fast than PHP in terms of raw speed, so I guess my algorithm just sucks o.O

As for having individual datafiles, that kinda defeats the purpose of using XML o.o Plus, I'd need to load and regex all of them anyway as well as data for the attacks and types and TMs and anything else I might end up needing, and that'd end up being a whole lot of files.
 
Posted by jshadias (Member # 2710) on 06-18-2002, 03:52 PM:
 
Ah... I didn't really think a Pokemon would have both Body Slam and Slam anyways.

I'd put mine in individual data files when I'm done, but I'm sticking with the current way because it doesn't have any problems with speed. The layout slows it down, but the printer friendly version loads faster than Google, and yes, I did test this. Of course, I have my preferences set to display 100 results a page.
http://www.pikacenter.com/pokemon/pokedex/pokedex.asp?num=004
For anyone who cares.

BTW, the reason Google is so insanely fast is because they don't use tables. All the other search engines I've seen use tables. Tables slow things down because the browser has to read everything between <TABLE> and </TABLE> before it can display it. That's also why sometimes a site won't load for a while, but when you press the stop button it displays all the data it's downloaded. Using HEIGHT and WIDTH commands in your images can help with this, as well as breaking up tables. Speaking of that, I'm going to have break up this layout because it's too darn slow. Anything short of instant loading just isn't good enough for me [Wink] If I can help it, that is. If I can't, I just have to put up with it.

[ 06-18-2002, 07:39 PM: Message edited by: jshadias ]
 
Posted by jshadias (Member # 2710) on 06-18-2002, 04:00 PM:
 
Also, the reason my Pokedex may have broken images or incomplete data is because it's not finished yet (duh).

For LE: Slowking (Trade with aKing's Roc)
Might want to fix your spelling on that one. It's on Slowpoke's evolutions.

[ 06-18-2002, 04:14 PM: Message edited by: jshadias ]
 
Posted by LexyEevee (Member # 2885) on 06-18-2002, 09:13 PM:
 
Unless Google's on cybercrack, the browser's loading time shouldn't be factored into the parse time at all. It saves the start time, searches, calculates the time it took, and then parses/sends it all. I think.

As for the Slowpoke thing, that was a bug in the last script with trading + items x.x It's fixed in the new version: http://www.nelcorp.com/alex/pokedex/pokemon.cgi?Slowpoke
 
Posted by jshadias (Member # 2710) on 06-18-2002, 09:53 PM:
 
I was talking about parse time + load time.

And Slowbro does not evolve into Slowking, Slowpoke evolves into one or the other. I'm not sure if you got that wrong, but it appears that way. Your Evolution Chain is kind of cryptic.

- Just looked at Eevee's page to see an example of multiple evolutions, and yes, you did get that wrong.
 
Posted by jshadias (Member # 2710) on 06-18-2002, 11:09 PM:
 
Weedle's TCG Pic is broken.
 
Posted by LexyEevee (Member # 2885) on 06-19-2002, 12:09 AM:
 
Slowpoke: mea culpa. fixed.

Weedle: crud. It disappeared. Now I have to... get another one. somehow.

attacks script: still not done, in case anyone's looking at it.
 


Karpe Diem