Server move completed. Did I break it?

Post by **FishPants** » Mon Oct 17, 2011 9:37 am

Ok gang, buckle your seatbelts; I am moving us out of this provider. The random lockups are happening again and we can't keep doing this. Board will be unavailable at some point while I backup, move the database, restore and change DNS. I will make sure to update a static web page with progress, and eventually the IP of the new box (in the event your DNS server caches longer than the hour I've set on the records).

Will keep you updated as I try and manage this through my dayjob.

UPDATE: Saturday morning and check my email, new server has arrived. Throughout the day the server will be slow, unavailable or it may be on fire.

I'll do my best to keep you updated, all of this activity will be happening around two mildly hyper girls and a farty Labrador, all who require attention. Hopefully won't take too long!

GreenGoo · Post by **GreenGoo** » Mon Oct 17, 2011 9:43 am

You da Man Fish.

Good luck.

stessier · Post by **stessier** » Mon Oct 17, 2011 9:49 am

May the Schwartz be with us!

Odin · Post by **Odin** » Mon Oct 17, 2011 9:50 am

Thanks as always, FishPants!

Post by **FishPants** » Mon Oct 17, 2011 10:01 am

I've ordered the new server, and we got a smoking deal (Same price as what we pay now through iweb, but it's a Xeon processor w/ 8 gigs of ram). I don't think you clowns can keep crashing this one!

The down side is that they are slammed with new orders, may take 3-5 days to get access to the new server (and I'm sure that's going to cross the billing cycle on the current one.. Murphy's law and all that).

As soon as I have confirmation that it's ready, I'm going to disable the board and start moving files over. Hope the migration from Centos 5.7 to 6.0 is relatively seamless (ya ok, I'm just trying to convince myself of that).

miltonite · Post by **miltonite** » Mon Oct 17, 2011 10:05 am

FishPants wrote:I've ordered the new server, and we got a smoking deal (Same price as what we pay now through iweb, but it's a Xeon processor w/ 8 gigs of ram). I don't think you clowns can keep crashing this one!

GreenGoo · Post by **GreenGoo** » Mon Oct 17, 2011 10:31 am

FishPants wrote:As soon as I have confirmation that it's ready, I'm going to disable the board and start moving files over. Hope the migration from Centos 5.7 to 6.0 is relatively seamless (ya ok, I'm just trying to convince myself of that).

It should be. We've done several boxes here with no real issues. No major changes, it's all minor increments to everything.

Going to do my desktop later this week. Most of the team have already done so with no issues.

Newcastle · Post by **Newcastle** » Mon Oct 17, 2011 12:31 pm

sweet! thanks for all your hardwork!

oh i got dibs on the space at quadrant...19089718972.99928 in vector 8237346.

Kurth · Post by **Kurth** » Mon Oct 17, 2011 1:12 pm

Thanks, FishPants.

Pyperkub · Post by **Pyperkub** » Mon Oct 17, 2011 1:29 pm

Thanks Fish!

gbasden · Post by **gbasden** » Mon Oct 17, 2011 1:38 pm

You rule!

Thanks for all the time you put in on stuff like this.

Daehawk · Post by **Daehawk** » Mon Oct 17, 2011 3:37 pm

Thank you.

$iljanus · Post by **$iljanus** » Mon Oct 17, 2011 4:07 pm

Thanks! See ya on the other side!

KKBlue · Post by **KKBlue** » Mon Oct 17, 2011 4:37 pm

Your care for OO and keeping us happy and computer healthy is admirable.
In other words, THANK YOU FishPants!

As a side note, have to say that I love using your forum name in casual conversation, as if there is nothing strange about saying, "Oh, I saw a note on OO about maintenance, by FishPants" or "FishPants is working on it!" or "Have you heard anything from FishPants?"

Archinerd · Post by **Archinerd** » Mon Oct 17, 2011 9:57 pm

Nice one.

This reminds me, I should remember to use the OO Amazon link more often...

GreenGoo · Post by **GreenGoo** » Wed Oct 19, 2011 9:54 am

Did this happen? I'm on OO a lot, but not ALL the time.

How was the migration, jupe de fruits de mer?

Post by **FishPants** » Wed Oct 19, 2011 10:35 am

Still waiting on Volumedrive to provision the new server (I've paid the moneyz). They've got a bit of a backlog due to the pricing, but I figure by the end of the weekend or early next week the server will be available. We've got until Nov 4th to get off the iweb server.

Blackhawk · Post by **Blackhawk** » Wed Oct 19, 2011 11:08 am

Without asking you to cite actual numbers, how well are our current programs supporting things financially through this move? Is any of this coming out of your FishPocket?

Post by **FishPants** » Wed Oct 19, 2011 12:57 pm

I think a little bit out of pocket, but over the course of the year it usually averages out with big buying splurges around Christmas etc.

LawBeefaroni · Post by **LawBeefaroni** » Wed Oct 19, 2011 1:12 pm

Here's an update from last year.

Use of the affiliate link is huge, please do as often as possible. Server should be mostly covered but that doesn't count the extra costs from FP and the other admins (domain reg (ILB), time, etc).

Like FP said, the holiday bump is crucial.

tgb · Post by **tgb** » Wed Oct 19, 2011 6:40 pm

My bags are packed.

Post by **FishPants** » Sat Oct 22, 2011 9:29 am

Ok took a dump of the database and FTPing it over.. I'm giddy seeing this on the new server:

Code: Select all

Processor information 	Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 8 cores
System uptime 	1 hours, 11 minutes
Running processes 	172
CPU load averages 	0.00 (1 min) 0.01 (5 mins) 0.00 (15 mins)
CPU usage 	0% user, 1% kernel, 0% IO, 99% idle
Real memory 	7.68 GB total, 383.48 MB used
	
Virtual memory 	4 GB total, 0 bytes used
	
Local disk space 	912.96 GB total, 47.97 GB used

stessier · Post by **stessier** » Sat Oct 22, 2011 12:29 pm

Out of idle curiosity, how big is our database?

Thanks again for doing this!

Post by **FishPants** » Sat Oct 22, 2011 12:57 pm

stessier wrote:Out of idle curiosity, how big is our database?

Thanks again for doing this!

Pretty big, I will let you know once I have everything sorted out but I believe last check it was around 6 gigs for all of the OO phpbb3 stuff (uncompressed and in the db).

I'm still mucking around on the new server to get optimal mysql settings -- restore has been taking a lot longer than export (to be expected), but with the extra horsepower I'm pretty sure I can crank up the number of threads and buffer sizes to make it faster.

Smoove_B · Post by **Smoove_B** » Sat Oct 22, 2011 12:59 pm

Post by **FishPants** » Sat Oct 22, 2011 3:46 pm

Still chugging away on the restore.. Looks like when we make the final cutover the site will be offline for 5-8 hours or so when it rebuilds the DB.

Just doing a test run now, and whenever its done I will post a link to the new site for you guys to test out (knowing that we will be wiping that DB out and cutting over at a later date).

There's got to be a faster way to bulk import innodb tables, but I've done everything I've read about speeding the process up and still looking at this kind of time frame.

In the meantime I'm waiting on a staffer who has control of the DNS entries to give me control as well so I can swing us over when ready, so maybe this weekend, maybe next week depending on when I hear from ILB.

Isgrimnur · Post by **Isgrimnur** » Sat Oct 22, 2011 4:11 pm

Do we have an idea on what time frame, if any, of posts that we will lose in the process?

Post by **ImLawBoy** » Sun Oct 23, 2011 12:10 am

Just updated things for you, FP. Hopefully that's not too late for you to do things this weekend.

Post by **FishPants** » Sun Oct 23, 2011 8:18 am

Isgrimnur wrote:Do we have an idea on what time frame, if any, of posts that we will lose in the process?

There wont be any loss of data since the board will be offline while it restores.. The thing ran all night and was still going; I've cancelled the import and working on tweaking a few settings (no way should innodb take that long). Anyhow later on today I may take the server offline here and try FTPing over the raw database files; if that works it's a heck of a lot easier (and faster) to get us up and running over there.

ILB thanks for the info, will let you know when I'm done!

Post by **FishPants** » Sun Oct 23, 2011 2:39 pm

Good news.. I can move the raw database files and have the server up in no time.

For now I need people to test on the new server:

here.

Feedback on speed, if you can break it and so on would be appreciated. My life just got a lot easier figuring out this way of moving the server.

Post by **FishPants** » Sun Oct 23, 2011 6:20 pm

That seemed.. awfully uneventful... Umm let me know if something is broken, and anticipate a few weird things in our shiny new home for a bit until I'm confident everything is 100%. Please report issues in meta.

Isgrimnur · Post by **Isgrimnur** » Sun Oct 23, 2011 6:34 pm

No evident issues at the moment.

Post by **FishPants** » Sun Oct 23, 2011 6:38 pm

Isgrimnur wrote:No evident issues at the moment.

Thanks.. Was getting a little concerned with the number of people logged in and no posting.. I don't like smooth transitions!

Also I think that probably email alerts wont be working right for the time being, I haven't touched the postfix config yet.

Isgrimnur · Post by **Isgrimnur** » Sun Oct 23, 2011 6:42 pm

I tried the search functionality and got what I expected. No issues accessing my stored messages. And, of course, I haven't seen any posting issues.

Thanks for all your hard work.

Smoove_B · Post by **Smoove_B** » Sun Oct 23, 2011 6:46 pm

Seems downright zippy. I am betting it will get a good test during the work week when all the M-F slackers are pressing F5 instead of pushing out TPS reports.

Thanks for figuring this out.

Odin · Post by **Odin** » Sun Oct 23, 2011 7:43 pm

So far so good, FishPants! Thanks a ton!!

theohall · Post by **theohall** » Sun Oct 23, 2011 9:26 pm

Outstanding! Thanks, Fish!!

Hamlet3145 · Post by **Hamlet3145** » Sun Oct 23, 2011 9:28 pm

Yep, definitely seems very zippy!

Nice work!

KKBlue · Post by **KKBlue** » Sun Oct 23, 2011 9:45 pm

Been playing around with it for awhile now on the iPad and all seems to be fine. I did use the "here" link too.
Thanks for the OO love.

Zarathud · Post by **Zarathud** » Sun Oct 23, 2011 10:06 pm

Piece of cake.