Octopus Overlords

So here's some learning points:

1. Rip was right. So very right. I feel shame.
2. Database won't convert, I'll open a ticket.. We stay on the old version.
3. First restore took about 30 hours.. Then I screwed it up again (whoops, wrong script).
4. I made some tweaks (!) to the config, and the next restore took exactly 15 minutes.
5. No that's not a joke.. 15 minutes.

Please report any funkiness. I'm going to go feel shame with a glass of red wine.

No funkiness, but I had to look in the Books Read thread to figure out the correct spelling of my username - and I had to guess at my password, too.
(I've written them down now - for the future)

No shame. We're back, we appear to be whole, and you live to fight another day. Lessons learned, and all that.

On the plus side the entire database is in RAM now, and the "write" threads is up to 16 -- so that means restores are WAY faster.

At least now I can reload the backup to another DB and test over there, even on a clean install the phpbb3 script to upgrade the database has a weird failure.. I may need to update our MySQL version to a more modern version. That's for another day.

No shame at all! Thanks so much for all the effort you put in!

Yay, normality is restored, and 10% better than it was before.

Have you patented the methodology that you used to get from 30 hours to 15 minutes, because that seems astoundingly more efficient?

Thanks for all your hard work. Have a drink on me!

gbasden wrote:No shame at all! Thanks so much for all the effort you put in!

Speak for yourselves.






Thanks for all your hard work.

Enjoy the well deserved wine.

As always, thanks for all you guys do with the site! I especially appreciate the updated page and not just coming to a 404 or something.

Smoove_B wrote:Have you patented the methodology that you used to get from 30 hours to 15 minutes, because that seems astoundingly more efficient?

Unpossible. Pied Piper already holds patents on the middle out compression solution that saved us all from internet purgatory.

Bad Demographic wrote:No funkiness, but I had to look in the Books Read thread to figure out the correct spelling of my username - and I had to guess at my password, too.
(I've written them down now - for the future)

Strange as Im still logged in.

Fish thank you so much for it all. Sorry it went badly. umm better luck next time?

Ehh it wasn't that stressful to be honest.. I had solid backups, it didn't involve a lot of my time other than check it once in awhile and update the HTML file for status.. I'm just really pissed off that the performance tweaks I found were something I could have used 10 years ago -- and it was through a forum post that referred to an academic that posted some crazy whitepaper, and a link to a TL;DR version of it with 5 settings for InnoDB tables. I set those (based on RAM) and viola -- fast restore. Crap default values, and obscure articles on how to tune to something reasonable -- frustration!

Anyways now I need to figure out how to create a mirror site, restore the DB there and test the upgrade script there.. I'm wondering if people hitting the site while it was upgrading was the problem (I didn't turn apache off, so I could serve up the status page..).

I only checked every 5 minutes so it wasn't me.

Daehawk wrote:I only checked every 5 minutes so it wasn't me.

Without OO I don't think I need the internet. Just boring Trump crap.

You know we have a core group of people that love this place when Smoove is logged in before I am.. and I'm the guy bringing the site back online.. "Oh look the restore is finished.. let me change this config file.. ok done.. ok lets restart the web server and login.. hey there's smoove!".

Not to mention another 67 "guest users" (we seemed to dump most cached login information on the restore). Sickos! The lot of ya!

What do you expect me to do all day? Work?!

cheeba wrote:As always, thanks for all you guys do with the site! I especially appreciate the updated page and not just coming to a 404 or something.

Or being told you're banned for being a spammer!

Very happy to see the harmony of OO restored to the Universe.

A forced break was good, but there were serious withdrawal symptoms.

Im feeling much better now.

Hooray, tomorrow is going to feel like a vacation day at work now!

What happened to BlackPearl?

Some interesting database issues with mods. All fixed, no more touching for tonight/this morning.

FishPants wrote:Some interesting database issues with mods. All fixed, no more touching for tonight/this morning.

I'm back in Black. Time to hit sack.

Hmm came back just now and it reset me to default silver style. Had to reset it to prosilver 2.

Yeah in short what I did was take the current DB, restore it to another DB (under a fictitious URL/domain).. I accessed it (by editing my hosts file) and ok great I have a mirror.. I read documentation on how to do the upgrade that's been failing (more forum posts).. Hey it says disable all styles and mods.. so I do.. then I realize I am using the same production config so I'm modifying this DB..

Whoops..

it's late.. my bad. restored some mod tables, seems to have fixed it.

It seems that every time there's an extended OOutage, someone goes missing and Daehawk looks well-fed. Not that it's proof of anything, just an observation.

FishPants wrote:All fixed, no more touching for tonight/this morning.

You're not the boss of me.

FishPants wrote:You know we have a core group of people that love this place when Smoove is logged in before I am.. and I'm the guy bringing the site back online.. "Oh look the restore is finished.. let me change this config file.. ok done.. ok lets restart the web server and login.. hey there's smoove!".

The real story here is who told me that it was up and running.

Putin?

So the good news is that I was able to convert the database (after I stopped screwing up this one) without an error. Apparently having you clowns hammering the system while I was converting it was causing problems (I would have thought the conversion script would frigging lock the DB.. I guess not).

So after the holidays I will try again, but will have to disable the web server (so no status page). The good news is that it only took about an hour to do (and if it gets hosed, recovery is 30 mins). That sure beats a couple days of restore time!

Excellent! Are there any user-facing benefits with 3.1?

Zaxxon wrote:Excellent! Are there any user-facing benefits with 3.1?

Take a look at our test instance I really like it, but it's different -- the masses may revolt!

The lack of something with a black background is certainly revolt worthy.

Has black done something wrong?

No. Badhawk.

Rip wrote:The lack of something with a black background is certainly revolt worthy.

Has black done something wrong?

Thank you for the frequent updates during the downtime. It made the withdrawals a lot easier to handle knowing someone was keeping tabs on the situation.