Conditional GET for data dumps?

[Ceto]

Rollie,

Have you considered supporting Conditional GET on the data dumps? The current two-step process works, but it could be trimmed down to just a single step and require less bandwidth and HTTP requests. I found it wasn't very hard to set up, I think I did a test PHP script in around 15 minutes one night.

[Rollie]

Can you explain more of what you are proposing?

[Ceto]

It's something like this HOWTO. The code is pretty simple when you break it down.

1. A client requests a page via Conditional GET, a modification on the normal GET method. The Conditional GET includes a last-modified date.
2. If the file has changed since this date, the server sends down the file normally.
3. If the file has not changed, the server returns 304 Not Modified, so the client gets nothing but a short HTTP header.

So, there's only a single HTTP request involved. Interesting tech, if nothing else. I don't know how many requests you get on these data dumps, or how many people getting these dumps are comfortable enough with PHP to support Conditional GET.