New Discourse site (2022)

computing
, ,
1

:information_source: This is a work-in-progress.

This note goes over how I like to run Discourse sites.

Summary

I enjoy a specific set of network resources for my Discourse sites:

  • VPS for Discourse, currently using Linode
  • Object Storage, currently using Linode
    • Uploads, where uploaded files are stored
    • Assets, for CSS and JS
  • CDN for object storage, currently using KeyCDN
  • Mail delivery, currently using Mailgun
  • Mailbox for email-in, currently using Gandi
  • DNS updates for everything, using Gandi

Dependencies

This process relies on a variety of API calls, some provided by additional software provided by a service vendor.

Decisions

Here are the only decisions to make. By answering these prompts, we are able to construct the commands required to provision our services. :+1:

The project label will be used in various technical contexts, as slugs to identify services. Here are requirements:

  • Can only contain lower-case characters, numbers, periods, and dashes.
  • Must start with a lowercase letter or number.
  • Cannot contain underscores (_), end with a dash (-) or period (.), have consecutive periods (.), or use dashes (-) adjacent to periods (.).

This is the path to an SSH key file that will be authorized to access the VPS. This will be the primary way to access some services.

The default is adequate to create a new Discourse instance. For larger, pre-existing communities a larger type with more resources may be selected.

To see the resources and which ID you should use, view the output of linode-cli linodes types.

Provision

Update tools

Ensure the tools are up-to-date.

pip3 install --upgrade linode-cli

Linode services

The following snippet will generate a root password and then create the following computing resources:

  • a virtual private server where the Discourse app will run, called =project_label=-discourse-vps
  • two object storage buckets for…
    • …uploads to the site, called =project_label=-discourse-uploads
    • …backups from the site, called =project_label=-discourse-backups
  • an access key to the object storage buckets, called =project_label=-object-storage-key

The output will include the access and secret key for accessing the object storage buckets, which will be used when Discourse is installed.

:warning: This is the only time it will be shown, so make sure to note it it before moving forward.

vps_root_password=$(openssl rand -base64 40);
linode-cli linodes create \
  --label "=project_label=-discourse-vps" \
  --type =linode_type= \
  --root_pass "$vps_root_password" \
  --authorized_keys "$(cat =ssh_key_file=)" \
  --region us-west \
  --image linode/ubuntu20.04 \
  --tags "=project_label=" --tags "discourse";
linode-cli obj mb =project_label=-discourse-backups;
linode-cli obj mb =project_label=-discourse-uploads;
linode-cli object-storage keys-create \
  --label "=project_label=-object-storage-key" \
  --bucket_access.cluster "us-east-1" \
  --bucket_access.bucket_name "=project_label=-discourse-backups" \
  --bucket_access.permissions "read_write" \
  --bucket_access.cluster "us-east-1" \
  --bucket_access.bucket_name "=project_label=-discourse-uploads" \
  --bucket_access.permissions "read_write";
unset vps_root_password
2 
vps_root_password=$(openssl rand -base64 40);
linode-cli linodes create \
  --label "example-discourse-vps" \
  --type g6-nanode-1 \
  --root_pass "$vps_root_password" \
  --authorized_keys "$(cat ~/.ssh/id_ed25519.pub)" \
  --region us-west \
  --image linode/ubuntu20.04 \
  --tags "example" --tags "discourse";
linode-cli obj mb example-discourse-backups;
linode-cli obj mb example-discourse-uploads;
linode-cli object-storage keys-create \
  --label "example-object-storage-key" \
  --bucket_access.cluster "us-east-1" \
  --bucket_access.bucket_name "example-discourse-backups" \
  --bucket_access.permissions "read_write" \
  --bucket_access.cluster "us-east-1" \
  --bucket_access.bucket_name "example-discourse-uploads" \
  --bucket_access.permissions "read_write";
unset vps_root_password

That works well, but when I’m looking at the KeyCDN, Gandi, and Mailgun APIs, I think I’ll end up using curl for each of them, so… I’ll probably rewrite the above to be the same. :thinking:

3

There are four companies involved, and each requires a setup prior to this process working.

  • Gandi needs to have a domain registered; after that the API is used to update DNS records
  • Linode needs to have an account; after that any resources there may be provisioned
  • Mailgun needs to have a domain setup, itself requiring usage of Gandi to update DNS; this is only done once per domain, generally, so it may not make sense to automate this part, unless I can incorporate it with updating DNS and getting all the information needed returned via API :thinking:
  • KeyCDN needs to be setup; after that CDN zones may be provisioned via API, but in order to use Let’s Encrypt there is an order to the zone creation and aliases, also requiring Gandi to update DNS

Let’s take a look at the order in which Discourse stuff is configured:

Before you start

  1. Preparing your domain name
  2. Setting up email

Installation

  1. Create new cloud server
  2. Access new cloud server
  3. Install Discourse
  4. Edit Discourse configuration
  5. Start Discourse
  6. Register new account and become admin
  7. Post-install maintenance
  8. (Optional) Add more Discourse features

1. Prep domain name

Gandi, which would have been registered before hand, I don’t register them on a whim, though I suppose their API does allow registering a new domain. :grimacing: Anyhow, I’ll have a domain in place already; that means I will also know the address for the site and all related components.

2. Setting up email

This is a multiple step process, all via API, which means:

  • Create new domain entry in Mailgun (Domains — Mailgun API documentation)
  • Create MX and TXT DNS records in Gandi for email delivery
  • Verify domains are working in Mailgun
  • Grab sending credentials from Mailgun

Note the sending credentials, that will be included in the Discourse config

3. Create new cloud server

This is handled at Linode with that block.:point_up:

Because I’m using object storage from Linode for backups and uploads, I go ahead and create them now, along with a key to access them.

At this point the following information should be notes:

  • IP address of the VPS
  • the access and secret keys for the object storage buckets

4. Access your server

At this point we could just ssh into the server by IP, but instead I’ll create the A record in Gandi so I’ll ssh root@example.org.

5. Install Discourse

I’m considering using a Linode StackScript to update the VPS and run the commands included in the setup:

sudo -s
git clone https://github.com/discourse/discourse_docker.git /var/discourse
cd /var/discourse
chmod 700 containers

This means the server will be ready for me to log in and run discourse-setup. But I wonder if I could include more in the StackScript to pass along configuration info and bootstrap Discourse on VPS startup… :thinking:

6. Edit Discourse Configuration

In this step I would run discourse-setup and answer the following prompts:

Hostname for your Discourse? [discourse.example.com]:
Email address for admin account(s)? [me@example.com,you@example.com]:
SMTP server address? [smtp.example.com]:
SMTP port? [587]:
SMTP user name? [user@example.com]:
SMTP password? [pa$$word]:
Let’s Encrypt account email? (ENTER to skip) [me@example.com]:
Optional Maxmind License key () [xxxxxxxxxxxxxxxx]:

These get written to app.yml, and for the above prompts, I’ll already know all the information. And I think… if I use Gandi and KeyCDN to configure the uploads after I’ve created the object storage bucket for it, I might be able to already have it configured at launch! :tada:

Okay, I’m gonna finish this process, and in the next reply I’ll create my list of operations!

7. Start Discourse

This step involves visiting the site after it’s started. I wonder if I can include a hook somewhere that signals to me it’s complete. :thinking: A simple webhook to a Discourse chat channel might work… and I can even include the URL as configurable! :sunglasses:

8. Register New Account and Become Admin

This is making an account and filling out the wizard, all web browser interactions at that point.

9. Post-Install Maintenance

  • dpkg-reconfigure -plow unattended-upgrades can be added in the StackScript
  • will be using ssh to log in
  • not sure I need a firewall… but might be nice to have configured and ready to go :thinking:

10. (Optional) Add More Discourse Features

Reply by email

I want this to start, and that means having in inbox and those credentials in app.yml. I’ll be provisioning inboxes at Gandi, so this will need to happen before the VPS provision.

Automatic backups

Now this is interesting… as I run the command now the backups bucket is created after the VPS… n order to have the credentials to include in the StackScript I will need to create the buckets (with access keys) before and separately from the VPS. :thinking:

That’s probably doable and worth it, to just have the site ready to go for me. :+1:

Plugins

Ideally I’d have a way to choose which plugins are installed, but if not I’ll just include a good base set in app.yml.

I wonder if I broke the StackScript into multiple scripts called in order I could make a personal template for plugins. :thinking:

CDN

These details will be included in the app.yml at provision, though I’m not sure if it will exists before or after the VPS. Because we’re breaking up the object storage buckets from the VPS provision, we can ensure that is operating and working before the VPS is launched, actually… :thinking:

Okay, a new set up directions is emerging for me, let me think this over. :slight_smile:

4

Okay, here’s a rough outline of the services to call, in order:

  • mailgun - new domain add
  • gandi - create new records for mailgun
  • gandi - since we’re here, let’s make an inbox for reply-by-email as well
  • mailgun - after some amount of time, verify mailgun domain records work
  • mailgun - grab credentials for sending mail
  • linode - create buckets with access keys
  • keycdn - ?, so some stuff that involves Let’s Encrypt, may be another step after…
  • gandi - create records for keycdn
  • linode - create VPS with StackScript filled with details from former steps
  • gandi - create records for VPS
  • visit site and configure :tada:

I’ll need to look more into the order of getting an alias attached to a CDN zone, but I think that’s the shape of it. :sunglasses: