Errors with processing?

Sianni · Post by **Sianni** » Wed Nov 29, 2006 10:31 pm

Rollie,

Looking at the DB Stats, it looks like the processor is 3 DAYS behind today's date? Have submissions picked up that much? I mean, it's great to be busy and all, but damn!

DM. · Post by **DM.** » Thu Nov 30, 2006 12:45 am

Yea i was thinking the same thing today.... whats up?

Balgair · Post by **Balgair** » Thu Nov 30, 2006 3:11 am

Yeah been noticing that myself for about the last week, seems it's really struggling atm. Good news if it's because of more submissions, but looking at the number of red European servers it's not more submissions from this side of the world!

Post by **Rollie** » Thu Nov 30, 2006 10:14 am

Yeah, I noticed last night that it is WAAAAY behind. I rebooted the DB server and am going to step up my efforts in finishing the big db changes.

Post by **Rollie** » Thu Nov 30, 2006 2:59 pm

So this basically prompted me to do more research into other ways of improving things and I found some really cool things coming down the pipe in the next release version of mySQL( 5.1 for anyone interested). Probably going to hold off on my big changes until the added partitioning features are live as it basically will handle all the work I was working on manually.

For the time being, I am going to be pulling about 10M characters from the primary DB into an archive to help speed things up until the new mysql changes are live. I'll be archiving all characters under lvl 10 that haven't been seen in the past year.

It won't affect stats (other than total characters tracked number) other than removing them from being viewable in character histories. This would also impact anyone using a sig or having claimed on of these chars that I'll be archiving.

Sianni · Post by **Sianni** » Thu Nov 30, 2006 10:22 pm

I think that would be fine Rollie. Thanks for all of your hard work!! XOXO

BWDemor · Post by **BWDemor** » Fri Dec 01, 2006 12:18 pm

Sweet... can't wait to see the changes.

I hate to ask, but it would also help to shrink the file size smaller? I figure there are a few people, myself included that submit close to the 20MB file size? As much as I don't want to lose the large file size, it might help speed up the processing.

BW

Hybuir · Post by **Hybuir** » Mon Dec 04, 2006 9:24 am

I have dreams of WoW, Rollie has dreams of SQL

We live an ominous life.

Post by **Rollie** » Mon Dec 04, 2006 10:18 am

BWDemor wrote:Sweet... can't wait to see the changes.

I hate to ask, but it would also help to shrink the file size smaller? I figure there are a few people, myself included that submit close to the 20MB file size? As much as I don't want to lose the large file size, it might help speed up the processing.

BW

Actually, not really. Larger file sizes do take a little longer to initially parse ( on the order of just a few seconds), but after the file is 'trimmed' to recent data, it really doesn't make that much of a difference.

The bottleneck right now is the comparison of incoming characters against the current data in the system.