Web server performance and load

[Rollie]

Seems I have a bit of a problem. The server load around here has been going higher and higher. I am working to optimize things as I can, but it looks like we'll probably be in for a change soon.

For right now, updates are on hold until I can track down the most offending pieces. Hopefully I can have updates restarting again later today. Any data submitted will still be processed, it just probably won't happen for a few hours.

The worst case scenario would mean changing servers. Which could result in some downtime during the transition. I'll keep you posted.

[Linulo]

You write you require everyone accessing the csv data to check the timestamp file first. This makes perfect sense but I doubt many webmasters will follow these rules. Maybe they would if they had the a php code snippet that shows how to do the timestamp checking.

My implementation of it is far from perfect but at least it does not do unnecessary csv downloads. Is it ok to post it here?

[Tyro]

Any word on the status? A guildmate and I each tried uploading different Kel'Thuzad alliance data last night (maybe 18 hours ago), and it's still not up.

[Linulo]

Ok, here is a PHP code snippet you may use in order to comply to Rollies rules:

Code: Select all

$timestamp_url = "http://www.warcraftrealms.com/exports/status.txt";
$timestamp_file = "timestamp.txt";

// Check if cached csv up-to-date.
// Returns TRUE if new data is available, otherwise FALSE.
function new_data_available() {
	global $timestamp_url, $timestamp_file;
	
	$ret = FALSE;
	
  // Get saved timestamp.
  $ts_fp_local = fopen($timestamp_file, "r");
  if(!$ts_fp_local) exit("Error! Cannot open local timestamp file.");
  $timestamp_old = fread($ts_fp_local, filesize($timestamp_file));
  fclose($ts_fp_local);
  
  // Get timestamp from WarcraftRealms.com.
  $ts_fp_census = fopen($timestamp_url, "r");
  if(!$ts_fp_census) exit("Error! Cannot open timestamp from WarcraftRealms.");
  $timestamp_new = fread($ts_fp_census, 10);
  fclose($ts_fp_census);
  
  // Save new timestamp.
  // Theoretically should only be done in case of success
  // but we don't want to hammer the WarcraftRealms-server if anything goes wrong.
  if(($timestamp_new - $timestamp_old) > 120) {
  	if(!is_writable($timestamp_file))
  		exit("Error! Local timestamp file is not writable.");
 		$ts_fp_local = fopen($timestamp_file, 'w');
 		if(!$ts_fp_local)
 			exit("Error! Cannot open local timestamp file.");
   	if(flock($ts_fp_local, LOCK_EX)) {
  		if(fwrite($ts_fp_local, $timestamp_new) === FALSE)
 				exit("Error! Cannot write local timestamp file.");
  		flock($ts_fp_local, LOCK_UN);
  	}
  	else
  		echo "Warning! Could not acquire lock on local timestamp file.";
   	fclose($ts_fp_local);
	 	$ret = TRUE;
  }
	return($ret);
}

Hope that helps. Please help keep Rollie's server alive.

[Rollie]

Unfortunately, the guild and realm exports are not the cause of the resource issues. It's due to the huge amounts of file uploads that I am getting now. The update process was running almost constantly even though I only kick it off every 30 minutes.

I have spent a lot of time rewriting and optimizing the updating code and am testing it as we speak. If everything looks good, I may unleash it on the server tonight, hehe.

[Rollie]

Also, Linlo, thanks for sharing that code snippet =)

You should put a copy in the Developer forum as well so it will be there for safe keeping!