Site Upgrade - Wednesday 10th June 3am-6am

[Paul C]

ANNOUNCEMENT

 

There will be scheduled site maintenance on Wednesday 10th June early morning 3am-6am (still Tuesday night for some people) to upgrade the version of MySQL server software on our web server. Since the upgrade is to be done outside of normal UK daytime, I expect the disruption to be minimal. This is part of a series of site changes to try and address an ongoing bunch of issues with site responsiveness.

 

Many thanks for your understanding

[twbm]

Slithering down the list so bumped.

[Dyertribe]

..

[Nightwatch]

Thanks for letting us know.

[Paul C]

Unfortunately, the upgrade threw up other issues which took much of today to resolve. We believe they are resolved now. It is (currently) still very slow but this is due to high demand, please bear with us. I'll give more of an update once I have more information myself. Apologies and thank you all for your patience in this troublesome time.

[nicknorman]

Well done for persevering and giving up so much of your time. These things rarely go as planned!

[MtB]

Is one of the issues double posts?

 

Is one of the issues double posts?

[churchward]

Is one of the issues double posts?

 

Is one of the issues double posts?

dunno!

 

dunno!

[DHutch]

Never

 

Never....?

[Nightwatch]

I thought that.

 

 

I thought that.

[Alan Saunders]

Just waited ~2 mins for this page to re-load after a fatal error. (Allocated ~5GB, tried to allocate ~7K).

 

The home page, gallery, forum and even view new content are fine.

 

"563 users are online (in the past 15 minutes) 62 members, 495 guests, 6 anonymous users."

I guess CWDF is a victim of its success but I would not expect 563 users to cause a problem - only 68 of them can be attempting to post or use search.

Good Luck, and thank you to Paul C and the rest of the crew.

Alan

[DHutch]

...I would not expect 563 users to cause a problem - only 68 of them can be attempting to post or use search.

No, indeed not. Fairly normal traffic for us infact.

 

The reported numbers a higher than they have been, but that's because I have changed the reporting time frame to 15 minutes from 5 minutes at that is the metric one of the potential option for hosting the site else where uses.

 

 

Daniel

...I would not expect 563 users to cause a problem - only 68 of them can be attempting to post or use search.

No, indeed not. Fairly normal traffic for us infact.

 

The reported numbers a higher than they have been, but that's because I have changed the reporting time frame to 15 minutes from 5 minutes at that is the metric one of the potential option for hosting the site else where uses.

 

 

Daniel

[Paul C]

For some reason, after the mySQL server upgrade and the prolonged delay once the site was back running today (Wednesday 10th, technically yesterday....) at 5:20pm the disk utilization went to at/near 100% which basically means something was battering the disk drive. This had the effect of slowing everything else down.

 

Unfortunately the hosts technical support were less than helpful due to the subtleties of the English language meaning the Indian helpdesk person simply couldn't understand the difference between "utilization" (a performance measure) and "usage" (how much free space was on the disk). Anyway, I have now secured a backup of the website manually (that broke too) and have just restarted the server and will monitor the disk utilization through the night. ANOTHER backup will occur earlier than normal, at 1am, to check that the daily automated backup now runs properly too.

 

I know its been a rather unreliable past week or so but both Daniel and I have pressed the hosting team to fix the website responsiveness issue as much as possible. It is being taken seriously, all parties are in agreement that the past week's uptime has been unacceptable and we'll one way or another resolve the issues present.

[larkshall]

Any ideas yet as to what is causing the disk I/O to go exponential? Memory allocation,? Swapping?

[RLWP]

Any ideas yet as to what is causing the disk I/O to go exponential? Memory allocation,? Swapping?

 

I doubt we will get to know. TBH, that's well beyond what someone taking a service from a website hosting company needs to know. 'Just bloody fix it' is all that needs to be said to the hosts

 

Richard

[MtB]

 

I doubt we will get to know. TBH, that's well beyond what someone taking a service from a website hosting company needs to know. 'Just bloody fix it' is all that needs to be said to the hosts

 

Richard

 

 

I suspect this has been tried, and the host disputes it is a hosting issue. I bet they are saying it is a configuration or a traffic issue which CWF must sort out for themselves.

 

I would have thought Invision were the best people to be asking to diagnose/fix it rather than the hosts, as it seems beyond the bounds of knowledge our CWF techies.

[jenlyn]

 

 

I suspect this has been tried, and the host disputes it is a hosting issue. I bet they are saying it is a configuration or a traffic issue which CWF must sort out for themselves.

 

I would have thought Invision were the best people to be asking to diagnose/fix it rather than the hosts, as it seems beyond the bounds of knowledge our CWF techies.

Totally agree with that.

[RLWP]

 

 

I suspect this has been tried, and the host disputes it is a hosting issue. I bet they are saying it is a configuration or a traffic issue which CWF must sort out for themselves.

 

Not based on this they aren't:

 

 

 

at 5:20pm the disk utilization went to at/near 100% which basically means something was battering the disk drive

 

That data isn't available to a website owner

 

 

 

I would have thought Invision were the best people to be asking to diagnose/fix it rather than the hosts, as it seems beyond the bounds of knowledge our CWF techies.

 

Oh yes? You'd ask Microsoft why your computer is running slow?

 

When you take a service from a site host, you get a bit of storage space on a disk array. You don't get a server, or even a whole disk. So, you are sharing a bit of disk space with other websites with their own administrators - all securely walled off from each other. Only the site hosts get to do stuff with the hardware

 

Richard

[MtB]

 

When you take a service from a site host, you get a bit of storage space on a disk array. You don't get a server, or even a whole disk. So, you are sharing a bit of disk space with other websites with their own administrators - all securely walled off from each other. Only the site hosts get to do stuff with the hardware

 

Richard

 

 

That's what I thought too, which is why repeated comments about our techies 're-booting the server' puzzle me. Just a 'virtual reboot' presumably. I'm surprised we are even allowed to do that. I can't do this on the virtual web server hosting my sites.

 

And given that these performance issues carried over from the previous host to the current host, I don't understand why you say it is a hosting issue.

 

I also wonder if there are other Invision forums running on the same physical server, and if they are suffering the same issues. If there are, I suspect they aren't which would point to it not being a hosting issue.

Edited June 11, 2015 by Mike the Boilerman

[RLWP]

Who knows - there could be anything running on the rest of the server. And my guess (WARNING) is that they have restarted the MySQL process - not the whole server.

 

Richard

 

MORE: I'm just about to launch a new boiler repair service based on reading and guessing from website postings. It's not like I have to actually work on the thing is it, I've seen a boiler - we've got one

Edited June 11, 2015 by RLWP

[MtB]

Who knows - there could be anything running on the rest of the server. And my guess (WARNING) is that they have restarted the MySQL process - not the whole server.

 

Richard

 

MORE: I'm just about to launch a new boiler repair service based on reading and guessing from website postings. It's not like I have to actually work on the thing is it, I've seen a boiler - we've got one

 

 

Who knows indeed. My point is that outsiders can sometimes see the wood for the trees whilst those up to their elbows in it miss the big picture. I'm not demanding answers, just posing the questions in case it helps.

 

But I appreciate it probably doesn't.

 

Regarding your new boiler fixing service, make sure the gas is ON. I've spent 20 mins on more than one occasion trying to trace a fault before finding I need to check this. In one case, the whole gas supply had been disconnected because the punter hadn't paid his bill. His face was a picture when he opened the outside cupboard to show me the meter, and there it was, gone!

[RLWP]

Once upon a time, I managed a few database administrators. When things like this happened, they go quiet. If you ask them questions or make helpful suggestions they get irritable and give you short answers. Basically, they are disappearing into their heads to try to hold all the complicated details of a problem together. The last thing they need is some f**king know-nothing distracting them with stupid, pointless questions and suggestions (yes, dear reader, I was that f**king know-nothing)

 

My job in this situation was generally to keep people away from them so they didn't get hurt. The job is quicker that way

 

Sun's shining Mike, it's a lovely day

 

Richard