Forum Hosting Plans - Update

classic Classic list List threaded Threaded
42 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Forum Hosting Plans - Update

Gary Lewis
Administrator
As I'm sure you are aware, the service we've gotten on the Nabble server hasn't been all that good of late.  And, I've told you about plans to fix that problem.  Here's the latest...

Some time ago a programmer got the open-source software that we use for the forum, called Nabble, to run on his own server.  So the next step has been to secure a commercial server on which to host that software.  With some research we settled on Amazon Web Hosting, and as of this morning we've contracted with AWS for a server that should give us excellent response time and a lot of room to grow.

It will take some time to load up the software, load up one of our forum backups, and check everything out.  I'm sure problems will arise along the way, but I feel pretty confident that this is going to work, and work very well.

Ultimately we will have to cut over to the new server and take down the old one.  And while that may take a few hours, the expectation is that all of our history will be preserved and everyone can log back in as if nothing has happened.

I'll let you know when I know more, but hang in there, relief is on its way.  
Gary, AKA "Gary fellow": Profile

Dad's: '81 F150 Ranger XLT 4x4: Down for restomod: Full-roller "stroked 351M" w/Trick Flow heads & intake, EEC-V SEFI/E4OD/3.50 gears w/Kevlar clutches
Blue: 2015 F150 Platinum 4x4 SuperCrew wearing Blue Jeans & sporting a 3.5L EB & Max Tow
Big Blue: 1985 F250HD 4x4: 460/ZF5/3.55's, D60 w/Ox locker & 10.25 Sterling/Trutrac, Blue Top & Borgeson, & EEC-V MAF/SEFI

Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

Dorsai
Great choice!  I've been working on getting certifications for AWS products, I never realized just how flexible and powerful they are.  I'm sure the forum will be great at its new home.
Matthew
1980 F-150 Custom 2wd longbed, 300-6, C6, 2.75 rear axle
Atlanta GA
Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

Gary Lewis
Administrator
Thanks, Matthew.  That's reassuring to know.

We are going with the Operating system (Linux), Quantity (1), Storage for each EC2 instance (General Purpose SSD (gp2)), Storage amount (20 GB), Instance type (t3a.small).  And that's a really small instance.  But there are plenty beyond that, so there's basically no limit to how much computing we could have if we need it.

And effectively this will be "our" server.  There are a few other really small forums that will be using it, but they are much smaller and have very little traffic - and none in some cases.
Gary, AKA "Gary fellow": Profile

Dad's: '81 F150 Ranger XLT 4x4: Down for restomod: Full-roller "stroked 351M" w/Trick Flow heads & intake, EEC-V SEFI/E4OD/3.50 gears w/Kevlar clutches
Blue: 2015 F150 Platinum 4x4 SuperCrew wearing Blue Jeans & sporting a 3.5L EB & Max Tow
Big Blue: 1985 F250HD 4x4: 460/ZF5/3.55's, D60 w/Ox locker & 10.25 Sterling/Trutrac, Blue Top & Borgeson, & EEC-V MAF/SEFI

Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

Dorsai
Hey, I know what all that means!  I'm sure that will be more than adequate...and if not, it's incredibly easy to scale up.
Matthew
1980 F-150 Custom 2wd longbed, 300-6, C6, 2.75 rear axle
Atlanta GA
Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

Gary Lewis
Administrator
Matthew - I sent you an email, but signed it "Gary guy" instead of "Gary fellow" so you probably think it wasn't me.  

Anyway, take a look.
Gary, AKA "Gary fellow": Profile

Dad's: '81 F150 Ranger XLT 4x4: Down for restomod: Full-roller "stroked 351M" w/Trick Flow heads & intake, EEC-V SEFI/E4OD/3.50 gears w/Kevlar clutches
Blue: 2015 F150 Platinum 4x4 SuperCrew wearing Blue Jeans & sporting a 3.5L EB & Max Tow
Big Blue: 1985 F250HD 4x4: 460/ZF5/3.55's, D60 w/Ox locker & 10.25 Sterling/Trutrac, Blue Top & Borgeson, & EEC-V MAF/SEFI

Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

Dorsai
I'm a little slow, but I did get it figured out.

As I said in my reply, I'm at your service.  Let me know how I can help.
Matthew
1980 F-150 Custom 2wd longbed, 300-6, C6, 2.75 rear axle
Atlanta GA
Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

ArdWrknTrk
Administrator
In reply to this post by Gary Lewis
I'm following with trepidation and hope, that our current hosting problems can be resolved.
 Jim,
Lil'Red is a '87 F250 HD, 4.10's, 1356 4x4, Zf-5, 3G, PMGR, Saginaw PS, desmogged with a Holley 80508 and Performer intake.
Too much other stuff to mention.
Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

Gary Lewis
Administrator
Matthew - I'm not sure how you can help, but the fact that you understood the requirements list tells me that you may be able to.  Thanks!  So, do you know your way around the AWS console?  I don't, but will have to learn and you might be able to help me.

Jim - I have confidence that our problems will be resolved.  We may get some new ones along the way, but the response time and too-busy ones should go away with this change.

The gentleman that is helping, Pedro, logged on yesterday afternoon and started setting up the server to support Nabble.  In fact, he then came back via email and said that AWS is telling him that what he wants to do isn't in the "free" section and asked if he should proceed.  I quickly said "Yes, PLEASE" as I'd already found that the cost is reasonable.  He said that he's going to work on it quite a bit over this holiday, so I'm hoping that he has it up and running for two of Greg's forums by Monday.

Who is Greg, you might ask.  (Jim, I know you know, but the others don't.)  He's my mentor on Nabble and runs a few small forums for things like the church he attends and a sail boat that was made in that area of the UK at one point in time.  A couple of those forums are the test cases for this migration since their backups are quite small.  Pedro got them up on a server a friend of his was renting and that proved the concept.  So I'm sure they are what he's loading up on AWS at the moment.

And when we get this up and running Greg's forums and ours will be the only ones on this server.  So we should have excellent response time and no overloads!
Gary, AKA "Gary fellow": Profile

Dad's: '81 F150 Ranger XLT 4x4: Down for restomod: Full-roller "stroked 351M" w/Trick Flow heads & intake, EEC-V SEFI/E4OD/3.50 gears w/Kevlar clutches
Blue: 2015 F150 Platinum 4x4 SuperCrew wearing Blue Jeans & sporting a 3.5L EB & Max Tow
Big Blue: 1985 F250HD 4x4: 460/ZF5/3.55's, D60 w/Ox locker & 10.25 Sterling/Trutrac, Blue Top & Borgeson, & EEC-V MAF/SEFI

Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

Dorsai
Gary Lewis wrote
Matthew - I'm not sure how you can help, but the fact that you understood the requirements list tells me that you may be able to.  Thanks!  So, do you know your way around the AWS console?  I don't, but will have to learn and you might be able to help me.
I'm learning rapidly, so feel free to shoot me any questions you have and I'll do my best to answer them.
Matthew
1980 F-150 Custom 2wd longbed, 300-6, C6, 2.75 rear axle
Atlanta GA
Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

Gary Lewis
Administrator
Matthew - I'm not smart enough to even ask questions.  

All - The AWS server is now up and Pedro has put up two of Greg's forums on it.  And I've created a new forum, called Bullnose Test Forum, and embedded it on the website.  You can see it in the menu as New Bullnose Test Forum.

You can go look at it and even register if you want, but I'm the only admin on it so it may take a bit for me to let you in.  And, I don't remember what the bog stock registration is, so don't know what you will encounter if you do that.

But, if you do register you'll be free to play around.  However, note that we will probably delete that forum so anything you type is a throw away.

Still, you can check out the speed of the new server, which is FAST!
Gary, AKA "Gary fellow": Profile

Dad's: '81 F150 Ranger XLT 4x4: Down for restomod: Full-roller "stroked 351M" w/Trick Flow heads & intake, EEC-V SEFI/E4OD/3.50 gears w/Kevlar clutches
Blue: 2015 F150 Platinum 4x4 SuperCrew wearing Blue Jeans & sporting a 3.5L EB & Max Tow
Big Blue: 1985 F250HD 4x4: 460/ZF5/3.55's, D60 w/Ox locker & 10.25 Sterling/Trutrac, Blue Top & Borgeson, & EEC-V MAF/SEFI

Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

ArdWrknTrk
Administrator
Does the test forum have the same login as this one?
Or is this something you've just got going with Matthew?

I'm liking that you say the response time is awesome.

I can remember ping commands back in the days of 14/4
 Jim,
Lil'Red is a '87 F250 HD, 4.10's, 1356 4x4, Zf-5, 3G, PMGR, Saginaw PS, desmogged with a Holley 80508 and Performer intake.
Too much other stuff to mention.
Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

Gary Lewis
Administrator
That is a totally new forum and takes registration on it anew.  And I'll caution that it gets REALLY confusing if you use the same email address to register but a different password.  I did it and had a very hard time remembering where I was.

Try it.  I'll approve the request and you can post however you want.
Gary, AKA "Gary fellow": Profile

Dad's: '81 F150 Ranger XLT 4x4: Down for restomod: Full-roller "stroked 351M" w/Trick Flow heads & intake, EEC-V SEFI/E4OD/3.50 gears w/Kevlar clutches
Blue: 2015 F150 Platinum 4x4 SuperCrew wearing Blue Jeans & sporting a 3.5L EB & Max Tow
Big Blue: 1985 F250HD 4x4: 460/ZF5/3.55's, D60 w/Ox locker & 10.25 Sterling/Trutrac, Blue Top & Borgeson, & EEC-V MAF/SEFI

Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

ckuske
Administrator
Hi Gary -

Hopefully I didn't scare you off in our previous attempt to move off of Nabble, but I'm happy to help as well.  Like Matthew, I'm dipping my toes into AWS.  The current project at my job is using it (EC2, RDS, SES, Lambda, and IoT Core), and I'm taking classes at a near-by university to get certified.  Just throwing my hat in the ring.  Feel free to work with Matthew, but just offering another set of hands should you need them.
Chris

'84 F-150 XL 2WD Flareside, 302 w/ AOD transmission, Motorcraft 2150 feedback carb w/ CA Emissions, EEC-IV w/TFI, factory A/C, speed control - Info about me and my truck - My project thread
Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

Gary Lewis
Administrator
Chris - You didn't scare me off.  You helped me understand how hard it would be to move the website itself, meaning the documentation portion.  (Some might not realize that there are two different sets of servers being used here - Nabble's servers for the forum and Weebly's servers for the documentation.  Oh yes, and now a third - AWS servers for the New Bullnose Test Forum, although that will go away once we migrate off Nabble.)  Through that effort I came to realize that while Weebly is not where I would go if I had to do it over again, the migration of the documentation to another site would be extremely painful.  So I thank you for the help.

But, I may well be able to use help from you and Matthew going forward.  I've never managed a server, and find the AWS console to be mind boggling.  I don't want to scare Pedro off so I'm not ready to add others to the access list, but at some point I'd like others to look at the settings and see if they are reasonable or if they need to be tweaked.

Also, is there a way to cause AWS to back up the site?  How often?  Is there a cost?
Gary, AKA "Gary fellow": Profile

Dad's: '81 F150 Ranger XLT 4x4: Down for restomod: Full-roller "stroked 351M" w/Trick Flow heads & intake, EEC-V SEFI/E4OD/3.50 gears w/Kevlar clutches
Blue: 2015 F150 Platinum 4x4 SuperCrew wearing Blue Jeans & sporting a 3.5L EB & Max Tow
Big Blue: 1985 F250HD 4x4: 460/ZF5/3.55's, D60 w/Ox locker & 10.25 Sterling/Trutrac, Blue Top & Borgeson, & EEC-V MAF/SEFI

Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

ckuske
Administrator
Happy to help where I can!

The back up options depend on how Nabble stores data (database vs some sort of file based storage).   Since you didn't mention using RDS (Amazon's database service) above, I'm guessing it's file based.

For backups from an EC2 instance, using Amazon S3 a good fit.  The storage is very cheap, and I doubt you'll pay much of anything for a few GBs of data.  You can also choose different classes of S3 storage (like S3 Glacier) which are very cheap, but depending on the tier of S3 you choose, it can take more time to retrieve your data to restore from.  There is immediate retrieval S3, or stuff like Glacier.  You can take a look at the pricing here:

https://aws.amazon.com/s3/pricing/

You can mount an S3 bucket as an NFS mount (yeah, like NFS on old Unix operating systems) and just copy your backups to the mount point.  Makes backups nice and easy, instead of using a command line tool to put your backups into S3.  

Chris

'84 F-150 XL 2WD Flareside, 302 w/ AOD transmission, Motorcraft 2150 feedback carb w/ CA Emissions, EEC-IV w/TFI, factory A/C, speed control - Info about me and my truck - My project thread
Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

Gary Lewis
Administrator
I got a bit lost in there Chris, but on reading it again I think I have it.  I think you are saying that AWS has cheap storage that would work well for backups.  But what I'm actually asking is if there is a way to cause AWS to automagically back things up periodically.

For instance, every Sunday I download a backup from Nabble for the forum and one from Weebly for the website.  But in both cases I have to tell Nabble and Weebly to generate the backup and then they send me an email with a link and I start the download of the backup to my OneDrive.  Wouldn't it be nice if there was a way for AWS to take periodic backups and keep them X # of days or weeks?  I'd be happy for the backup to be stored on their system instead of my OneDrive.

On a different subject, I just copied the first post in the Big Blue's Transformation thread, which is fairly large, in a new thread on the New Bullnose Test Forum that's hosted on the AWS server.  I wanted to see what the response time would be like.  It was almost instantaneous.  Hang on folks, there's light at the end of the tunnel!!!!  
Gary, AKA "Gary fellow": Profile

Dad's: '81 F150 Ranger XLT 4x4: Down for restomod: Full-roller "stroked 351M" w/Trick Flow heads & intake, EEC-V SEFI/E4OD/3.50 gears w/Kevlar clutches
Blue: 2015 F150 Platinum 4x4 SuperCrew wearing Blue Jeans & sporting a 3.5L EB & Max Tow
Big Blue: 1985 F250HD 4x4: 460/ZF5/3.55's, D60 w/Ox locker & 10.25 Sterling/Trutrac, Blue Top & Borgeson, & EEC-V MAF/SEFI

Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

ckuske
Administrator
Yep, that's what I'm saying!  I don't think we will have a "for sure" answer on the automagic question until we talk with Pedro.  I'm guessing the backup ZIP file is just for convenience.  In a cron job on the machine (scheduled task), we can execute a script to collect all the files that normally go into the ZIP file, ZIP it up, and ship it off to S3.

Glad to hear about the response time!  Amazon's network is pretty fast.
Chris

'84 F-150 XL 2WD Flareside, 302 w/ AOD transmission, Motorcraft 2150 feedback carb w/ CA Emissions, EEC-IV w/TFI, factory A/C, speed control - Info about me and my truck - My project thread
Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

Gary Lewis
Administrator
Time for an update.  We have a backup of this forum up and running on the AWS servers.  You can read about it in the thread called Test The New Forum Hosted On AWS Servers, and there's a link in the first post on that thread so you can go test it and play with it.  But be aware that it is a TEST FORUM!  Anything you post on there WILL be lost.  There will be no attempt made to keep it.

You should be able to log onto that forum with your existing ID and password.  And I think you'll find that the AWS servers are much faster than the Nabble servers.  However, there is one known problem - there is no way for that forum to send email notification of posts to those of us who are subscribed.  But, Pedro knows what the issue is and how to fix it, so that shouldn't be a problem.

Beyond that, we want to fix an issue that Franklin/Mr. Nabble created in the privileges aspect that has broken our joining process.  And once that's done we want to turn on security so we have a green check mark in the browser's URL line.  That apparently needs to be done before we migrate to the new servers.

Then, at some point in the not too distant future we'll have to pick a time and lock down this forum, take a backup, and upload that backup to the AWS servers.  Then we'll be in business there and the old forum on Nabble will initially stay locked and eventually go away when we have the needed confidence in the new forum.
Gary, AKA "Gary fellow": Profile

Dad's: '81 F150 Ranger XLT 4x4: Down for restomod: Full-roller "stroked 351M" w/Trick Flow heads & intake, EEC-V SEFI/E4OD/3.50 gears w/Kevlar clutches
Blue: 2015 F150 Platinum 4x4 SuperCrew wearing Blue Jeans & sporting a 3.5L EB & Max Tow
Big Blue: 1985 F250HD 4x4: 460/ZF5/3.55's, D60 w/Ox locker & 10.25 Sterling/Trutrac, Blue Top & Borgeson, & EEC-V MAF/SEFI

Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

85lebaront2
Administrator
Sounds good sir!
Bill AKA "LOBO" Profile

"Getting old is inevitable, growing up is optional" Darth Vader 1986 F350 460 converted to MAF/SEFI, E4OD 12X3 1/2 rear brakes, traction loc 3:55 gear, 160 amp 3G alternator Wife's 2011 Flex Limited Daily Driver 2009 Flex Limited with factory tow package Project car 1986 Chrysler LeBaron convertible 2.2L Turbo II, modified A413

Reply | Threaded
Open this post in threaded view
|

Re: Forum Hosting Plans - Update

Gary Lewis
Administrator
Time for yet another update.  Especially since I've been seeing slow response times and other issues from Nabble when I post on the forum.

As background, we are running on Nabble software on Nabble servers - for free.  But the powers that be have realized that they aren't covering their costs and are migrating more and more forums onto fewer servers in order to reduce their costs.  And that has had its impact on response time - to the point that sometimes things time out and get dropped.

But the software, called Nabble, was made open source, so a couple of us have been working with a system admin to bring it up on servers we control.  He got really close with one outstanding problem, lack of email notification, when I realized that the service we'd selected, Amazon Web Services, was costing a whole bunch more than expected.

After a call with AWS we turned that instance off and the sys admin has brought the Nabble software up on another Amazon service called Lightsail, which is far less expensive.  At this point we are just about ready to upload one of the 3.6 gig backups of this forum to Lightsail and prove that we are back to where we were and ready to tackle the email problem.

Once the email problem is solved, which will take some time as the sys admin has to find all of the instances in the code that send email and change them, we'll plan the migration - assuming no other problems are found twixt now and then.  But there will be plenty of information shared as we get closer to that.  Hopefully it won't require any changes on your end, but we will let you know.

In the interim:

Gary, AKA "Gary fellow": Profile

Dad's: '81 F150 Ranger XLT 4x4: Down for restomod: Full-roller "stroked 351M" w/Trick Flow heads & intake, EEC-V SEFI/E4OD/3.50 gears w/Kevlar clutches
Blue: 2015 F150 Platinum 4x4 SuperCrew wearing Blue Jeans & sporting a 3.5L EB & Max Tow
Big Blue: 1985 F250HD 4x4: 460/ZF5/3.55's, D60 w/Ox locker & 10.25 Sterling/Trutrac, Blue Top & Borgeson, & EEC-V MAF/SEFI

123