Here's an odd one... Took me ages to figure out there even was a bug!
OK, take a census very early in the morning. Like, say, just now.

Open up the window or right click the mini-map button, takes about 40-50 seconds. Take one with neither the window nor the button showing, takes about 25-30 minutes.
So... what's the deal?