Pause allowing submissions?

Post by **Rollie** » Fri Dec 08, 2006 10:24 am

So this archive process is STILL running... that's 3 days. And the process was already 4 days behind. Making the whole thing 7 days behind RIGHT NOW.

Even with some increased speed, I don't know HOW it would be able to catch up.

Should I pause allowing updates at all until the process starts getting caught up? Do you think this would really turn people off?

DM. · Post by **DM.** » Fri Dec 08, 2006 11:10 am

I think people will understand that the database needs some time to catch up so I dont think it will turn people away.

Alanthus · Post by **Alanthus** » Fri Dec 08, 2006 11:29 am

People will understand but it will definately have a permanently detrimental impact on submissions.

Post by **Rollie** » Fri Dec 08, 2006 1:17 pm

Yeah, that's my concern =/

Halaster · Post by **Halaster** » Fri Dec 08, 2006 1:24 pm

It may turn some folks away if they receive no credit for submissions. We all know you do a good job and don't mind things being a few days behind till the system catches up.

Lahaie · Post by **Lahaie** » Fri Dec 08, 2006 2:04 pm

I would understand if you need to pause. Just let us know and I will stop submitting until you give the word to continue...

oiseaux · Post by **oiseaux** » Fri Dec 08, 2006 3:17 pm

Rollie if you do this, I would put something on the FRONT page of the website. Not a lot of people who submit data even look at these boards. I was also wondering how it would work for people who use that uniuploader. I have never used it so I don't know how it works.

Post by **Rollie** » Fri Dec 08, 2006 3:35 pm

Well, if I did it, I'd disable the entire upload area. And I would put a message telling why and such. Uniuploader folks probably wouldn't know it was happening unless they check their response from the site when it uploads.

BWDemor · Post by **BWDemor** » Fri Dec 08, 2006 4:15 pm

Can you edit the message that is generated with UniUploader. I look at the submited document after each submission so I know the submission went up. If you are able to let people know that way also, that would help.

But I do agree a message on the Front page might will help. My last submission was yesterday 2006-12-07 17:28:22 UNPROCESSED and I was going to not submit anymore until the archive was done and then it had a chance to get caught up.

Now were you thinking of stopping submissions until it was all caught up and start anew? or just until the archive was done? My thoughts would be until it was all caught up so it then had a chance to stay a little more current with the submissions.

Thanks for all of the help.

BW

Post by **Rollie** » Fri Dec 08, 2006 4:54 pm

Maybe not caught all the way up, but within 24 hours or so.

Balgair · Post by **Balgair** » Fri Dec 08, 2006 6:08 pm

Sounds a pretty good idea tbh, it's going to be very hard to catch up this much anyway! I guess it'll lead to some new record updates from power censusers too when it's possible to submit again lol, maybe I should get busy censusing while it's down

Balgair · Post by **Balgair** » Sat Dec 09, 2006 1:30 pm

Just noticed the last processed file seems to have moved a little (well, 15 minutes or so), does this mean it's onto the catching up process now? I'm holding off on any new submissions until it's a lot nearer caught up anyway, hope it can catch up soon

Post by **Rollie** » Sat Dec 09, 2006 4:44 pm

Yeah, it's running again, but not as much of an improvement as I was hoping for.

Alanthus · Post by **Alanthus** » Sun Dec 10, 2006 1:14 pm

Hmmm, not 100% sure since I haven't been really tracking the progress but to me it seems to run even slower than before, or is this just me imagining things?

Balgair · Post by **Balgair** » Sun Dec 10, 2006 1:19 pm

That's what I was thinking, it's moved about 6 hours in the 24 hours since I last posted

Hmm, idea that may help it a little - up the minimum time between submissions? I think now if you submit twice in an hour it overwrites the first submission with the 2nd, maybe make that 3-4 hours? Only people it'd really affect negatively would be folk like me who census from more than one computer (everyone else can just submit less often!), but I can adapt

Sianni · Post by **Sianni** » Sun Dec 10, 2006 7:22 pm

I've been holding off on submissions while it plays catch up, and I agree disabling uploads might have a negative overall effect. I still think you might need to do it though.

Balgair · Post by **Balgair** » Mon Dec 11, 2006 4:52 am

Yeah, it's looking like it might be necessary, although frankly it's going slow enough that even if submissions are disabled, once it catches up it'll fall behind again very fast

Btw, problems?

2006-12-05 18:05:23 UNPROCESSED
2006-12-04 16:30:42 UNPROCESSED
2006-12-03 15:52:06 UNPROCESSED
2006-12-02 07:09:04 UNPROCESSED
2006-12-01 01:02:25 26 208

Last processed file was uploaded on 2006-12-02 13:18:24.

Wouldn't be bothered as it's only from one server if I remember right, but it had a level up for one of my characters on it, I think

Post by **Rollie** » Mon Dec 11, 2006 10:27 am

Looks like you were still using version 3.5 when I updated the processor to only do versions 3.5.1 and higher. I'm reverting to still allow 3.5 and will set your file to be redone as well as any other 3.5's in recent history.

And yes, it does appear to be running even slower, but I think I know the cause for that.

I'm in the process of getting a hired gun (mysql guru) to try to help point out other areas of optimization. I think I've reached the limits of my technical knowledge in this area.

Balgair · Post by **Balgair** » Mon Dec 11, 2006 10:36 am

Ahh, oops - hadn't realised I hadn't updated to 3.5.1! Thanks!

Good luck on the optimisation, hope you can find something that helps a bit!

Kerebo · Post by **Kerebo** » Tue Dec 12, 2006 4:40 pm

I would think the pausing would be good.

Either pause it long enough for all the stuff you have to process, or do it stage wise, like pause uploads for a day, then open 1 day and so on.