Tax advice for the tax man

Does anyone else find this very ironic: the government's Chancellor paying for tax advice (using our taxes, which he's responsible for..):

http://news.bbc.co.uk/1/hi/uk_politics/8066452.stm

Technical: Insertion Delay Bug in MySQL?

The last week, I've been working for a particular customer who had received data-files on CD, which were to update the records in their website database. There are 38 files, with 29 million records in total. The script which I was provided with to update the database takes (via the CMS) the name of a file, and the database table name into which to insert the data record.

The script runs as follows:

• Validate (check the file, database table exists etc.)
• Empty the database table
• Open the file
• For each line of the file:
  
  • Read the line
  • If its valid input:
    
  • Generate the insertion query
  • Run query and check for errors occured.

(Please note that there are no DELAY parameters in the SQL query, or any other keywords used other than for a standard insert).

In principle this appears to be fine. However, submitting the script on a file with ~1 million entries causes it to run for approximately 10-15 mins afterwhich it just "stops" (left with a blank browser window, with no more loading signs). At this point, checking the number of rows in the database using PHPMyAdmin (or a COUNT SQL statement) reveals a number typically in the region of 600,000. Naturally, one imagines an error has occured, so one inspects the tables and scripts and repeats the exercise. Again, it appears to "fail", with less than the number of records being inserted. Repeat again, and again.

However, by stopping and checking the number of rows again (shortly after the script "fails") it is revealed that the row count has increased (e.g. to 650,000).

Checking a few minutes later shows that it has increased again (e.g. 750,000). Even though the script has apparently ended.

Leaving the system for another 10 mins or so reveals that the correct number of rows have been inserted into the database after all.

I don't really have any conclusions, but I can only think of two explanations: Either there's some form of insert query caching going on - which there shouldn't be.. or, there's some kind of message being sent to the browser which gives it the impression the script has died - even when it has not.

Has anyone come across this before? Anyone have any explanations?

Tempus Fugit

It's come to my attention that I haven't added any blog entries for quite some time (this was reenforced when I came to log in and couldn't remember my logon details!). Having recovered them again I'm now in a position to do some more writing; although not at this exact moment because I'm in work and really have a lot of other things I should be doing at the moment!