Forum Hosting Plans - Update

[ArdWrknTrk]

Matthew - I'm not smart enough to even ask questions.

All - The AWS server is now up and Pedro has put up two of Greg's forums on it. And I've created a new forum, called Bullnose Test Forum, and embedded it on the website. You can see it in the menu as New Bullnose Test Forum.

You can go look at it and even register if you want, but I'm the only admin on it so it may take a bit for me to let you in. And, I don't remember what the bog stock registration is, so don't know what you will encounter if you do that.

But, if you do register you'll be free to play around. However, note that we will probably delete that forum so anything you type is a throw away.

Still, you can check out the speed of the new server, which is FAST!

Does the test forum have the same login as this one?

Or is this something you've just got going with Matthew?

I'm liking that you say the response time is awesome.

I can remember ping commands back in the days of 14/4

[Gary Lewis]

Does the test forum have the same login as this one?

Or is this something you've just got going with Matthew?

I'm liking that you say the response time is awesome.

I can remember ping commands back in the days of 14/4

That is a totally new forum and takes registration on it anew. And I'll caution that it gets REALLY confusing if you use the same email address to register but a different password. I did it and had a very hard time remembering where I was.

Try it. I'll approve the request and you can post however you want.

[ckuske]

That is a totally new forum and takes registration on it anew. And I'll caution that it gets REALLY confusing if you use the same email address to register but a different password. I did it and had a very hard time remembering where I was.

Try it. I'll approve the request and you can post however you want.

Hi Gary -

Hopefully I didn't scare you off in our previous attempt to move off of Nabble, but I'm happy to help as well. Like Matthew, I'm dipping my toes into AWS. The current project at my job is using it (EC2, RDS, SES, Lambda, and IoT Core), and I'm taking classes at a near-by university to get certified. Just throwing my hat in the ring. Feel free to work with Matthew, but just offering another set of hands should you need them.

[Gary Lewis]

Hi Gary -

Hopefully I didn't scare you off in our previous attempt to move off of Nabble, but I'm happy to help as well. Like Matthew, I'm dipping my toes into AWS. The current project at my job is using it (EC2, RDS, SES, Lambda, and IoT Core), and I'm taking classes at a near-by university to get certified. Just throwing my hat in the ring. Feel free to work with Matthew, but just offering another set of hands should you need them.

Chris - You didn't scare me off. You helped me understand how hard it would be to move the website itself, meaning the documentation portion. (Some might not realize that there are two different sets of servers being used here - Nabble's servers for the forum and Weebly's servers for the documentation. Oh yes, and now a third - AWS servers for the New Bullnose Test Forum, although that will go away once we migrate off Nabble.) Through that effort I came to realize that while Weebly is not where I would go if I had to do it over again, the migration of the documentation to another site would be extremely painful. So I thank you for the help.

But, I may well be able to use help from you and Matthew going forward. I've never managed a server, and find the AWS console to be mind boggling. I don't want to scare Pedro off so I'm not ready to add others to the access list, but at some point I'd like others to look at the settings and see if they are reasonable or if they need to be tweaked.

Also, is there a way to cause AWS to back up the site? How often? Is there a cost?

[ckuske]

Chris - You didn't scare me off. You helped me understand how hard it would be to move the website itself, meaning the documentation portion. (Some might not realize that there are two different sets of servers being used here - Nabble's servers for the forum and Weebly's servers for the documentation. Oh yes, and now a third - AWS servers for the New Bullnose Test Forum, although that will go away once we migrate off Nabble.) Through that effort I came to realize that while Weebly is not where I would go if I had to do it over again, the migration of the documentation to another site would be extremely painful. So I thank you for the help.

But, I may well be able to use help from you and Matthew going forward. I've never managed a server, and find the AWS console to be mind boggling. I don't want to scare Pedro off so I'm not ready to add others to the access list, but at some point I'd like others to look at the settings and see if they are reasonable or if they need to be tweaked.

Also, is there a way to cause AWS to back up the site? How often? Is there a cost?

Happy to help where I can!

The back up options depend on how Nabble stores data (database vs some sort of file based storage). Since you didn't mention using RDS (Amazon's database service) above, I'm guessing it's file based.

For backups from an EC2 instance, using Amazon S3 a good fit. The storage is very cheap, and I doubt you'll pay much of anything for a few GBs of data. You can also choose different classes of S3 storage (like S3 Glacier) which are very cheap, but depending on the tier of S3 you choose, it can take more time to retrieve your data to restore from. There is immediate retrieval S3, or stuff like Glacier. You can take a look at the pricing here:

https://aws.amazon.com/s3/pricing/

You can mount an S3 bucket as an NFS mount (yeah, like NFS on old Unix operating systems) and just copy your backups to the mount point. Makes backups nice and easy, instead of using a command line tool to put your backups into S3.

 

[Gary Lewis]

Happy to help where I can!

The back up options depend on how Nabble stores data (database vs some sort of file based storage). Since you didn't mention using RDS (Amazon's database service) above, I'm guessing it's file based.

For backups from an EC2 instance, using Amazon S3 a good fit. The storage is very cheap, and I doubt you'll pay much of anything for a few GBs of data. You can also choose different classes of S3 storage (like S3 Glacier) which are very cheap, but depending on the tier of S3 you choose, it can take more time to retrieve your data to restore from. There is immediate retrieval S3, or stuff like Glacier. You can take a look at the pricing here:

https://aws.amazon.com/s3/pricing/

You can mount an S3 bucket as an NFS mount (yeah, like NFS on old Unix operating systems) and just copy your backups to the mount point. Makes backups nice and easy, instead of using a command line tool to put your backups into S3.

I got a bit lost in there Chris, but on reading it again I think I have it. I think you are saying that AWS has cheap storage that would work well for backups. But what I'm actually asking is if there is a way to cause AWS to automagically back things up periodically.

For instance, every Sunday I download a backup from Nabble for the forum and one from Weebly for the website. But in both cases I have to tell Nabble and Weebly to generate the backup and then they send me an email with a link and I start the download of the backup to my OneDrive. Wouldn't it be nice if there was a way for AWS to take periodic backups and keep them X # of days or weeks? I'd be happy for the backup to be stored on their system instead of my OneDrive.

On a different subject, I just copied the first post in the Big Blue's Transformation thread, which is fairly large, in a new thread on the New Bullnose Test Forum that's hosted on the AWS server. I wanted to see what the response time would be like. It was almost instantaneous. Hang on folks, there's light at the end of the tunnel!!!!

[ckuske]

I got a bit lost in there Chris, but on reading it again I think I have it. I think you are saying that AWS has cheap storage that would work well for backups. But what I'm actually asking is if there is a way to cause AWS to automagically back things up periodically.

For instance, every Sunday I download a backup from Nabble for the forum and one from Weebly for the website. But in both cases I have to tell Nabble and Weebly to generate the backup and then they send me an email with a link and I start the download of the backup to my OneDrive. Wouldn't it be nice if there was a way for AWS to take periodic backups and keep them X # of days or weeks? I'd be happy for the backup to be stored on their system instead of my OneDrive.

On a different subject, I just copied the first post in the Big Blue's Transformation thread, which is fairly large, in a new thread on the New Bullnose Test Forum that's hosted on the AWS server. I wanted to see what the response time would be like. It was almost instantaneous. Hang on folks, there's light at the end of the tunnel!!!!

Yep, that's what I'm saying! I don't think we will have a "for sure" answer on the automagic question until we talk with Pedro. I'm guessing the backup ZIP file is just for convenience. In a cron job on the machine (scheduled task), we can execute a script to collect all the files that normally go into the ZIP file, ZIP it up, and ship it off to S3.

Glad to hear about the response time! Amazon's network is pretty fast.

[Gary Lewis]

Yep, that's what I'm saying! I don't think we will have a "for sure" answer on the automagic question until we talk with Pedro. I'm guessing the backup ZIP file is just for convenience. In a cron job on the machine (scheduled task), we can execute a script to collect all the files that normally go into the ZIP file, ZIP it up, and ship it off to S3.

Glad to hear about the response time! Amazon's network is pretty fast.

Time for an update. We have a backup of this forum up and running on the AWS servers. You can read about it in the thread called Test The New Forum Hosted On AWS Servers, and there's a link in the first post on that thread so you can go test it and play with it. But be aware that it is a TEST FORUM! Anything you post on there WILL be lost. There will be no attempt made to keep it.

You should be able to log onto that forum with your existing ID and password. And I think you'll find that the AWS servers are much faster than the Nabble servers. However, there is one known problem - there is no way for that forum to send email notification of posts to those of us who are subscribed. But, Pedro knows what the issue is and how to fix it, so that shouldn't be a problem.

Beyond that, we want to fix an issue that Franklin/Mr. Nabble created in the privileges aspect that has broken our joining process. And once that's done we want to turn on security so we have a green check mark in the browser's URL line. That apparently needs to be done before we migrate to the new servers.

Then, at some point in the not too distant future we'll have to pick a time and lock down this forum, take a backup, and upload that backup to the AWS servers. Then we'll be in business there and the old forum on Nabble will initially stay locked and eventually go away when we have the needed confidence in the new forum.

[85lebaront2]

Time for an update. We have a backup of this forum up and running on the AWS servers. You can read about it in the thread called Test The New Forum Hosted On AWS Servers, and there's a link in the first post on that thread so you can go test it and play with it. But be aware that it is a TEST FORUM! Anything you post on there WILL be lost. There will be no attempt made to keep it.

You should be able to log onto that forum with your existing ID and password. And I think you'll find that the AWS servers are much faster than the Nabble servers. However, there is one known problem - there is no way for that forum to send email notification of posts to those of us who are subscribed. But, Pedro knows what the issue is and how to fix it, so that shouldn't be a problem.

Beyond that, we want to fix an issue that Franklin/Mr. Nabble created in the privileges aspect that has broken our joining process. And once that's done we want to turn on security so we have a green check mark in the browser's URL line. That apparently needs to be done before we migrate to the new servers.

Then, at some point in the not too distant future we'll have to pick a time and lock down this forum, take a backup, and upload that backup to the AWS servers. Then we'll be in business there and the old forum on Nabble will initially stay locked and eventually go away when we have the needed confidence in the new forum.

Sounds good sir!

[Gary Lewis]

Sounds good sir!

Time for yet another update. Especially since I've been seeing slow response times and other issues from Nabble when I post on the forum.

As background, we are running on Nabble software on Nabble servers - for free. But the powers that be have realized that they aren't covering their costs and are migrating more and more forums onto fewer servers in order to reduce their costs. And that has had its impact on response time - to the point that sometimes things time out and get dropped.

But the software, called Nabble, was made open source, so a couple of us have been working with a system admin to bring it up on servers we control. He got really close with one outstanding problem, lack of email notification, when I realized that the service we'd selected, Amazon Web Services, was costing a whole bunch more than expected.

After a call with AWS we turned that instance off and the sys admin has brought the Nabble software up on another Amazon service called Lightsail, which is far less expensive. At this point we are just about ready to upload one of the 3.6 gig backups of this forum to Lightsail and prove that we are back to where we were and ready to tackle the email problem.

Once the email problem is solved, which will take some time as the sys admin has to find all of the instances in the code that send email and change them, we'll plan the migration - assuming no other problems are found twixt now and then. But there will be plenty of information shared as we get closer to that. Hopefully it won't require any changes on your end, but we will let you know.

In the interim: