Ultimate Static Site Setup

[pfingy]

Is anyone interested in a step-by-step guide to how I setup static sites, including code and screenshots of me building an example site? I'm considering building the site anyway, so I thought I might share the process.

Hopefully I don't need to sell anyone on why static sites are 1000X better than Wordpress, but if anyone isn't sure why they'd ditch all the bullshit of running a databases and webservers and PHP for the beautiful, inexpensive, reliable simplicity of hosting static sites on a CDN, I'm happy to answer questions about that too.

I use:
- Lektor (created by Armin Ronacher, the guy who created Jinja2 and Flask) to build sites using a "filesystem database" (directories of INI and JSON files)
- Host files on Amazon S3
- Serve files from Amazon CloudFront
- Get free SSL cert from Amazon

Lektor is my favorite static site generator by far, because unlike stuff like Jekyll, it's more of a toolkit for designing your own site generators.

If you are interested in seeing this, let me know.

 

[Ryuzaki]

I'm interested. The one thing I've not been forced to spend a lot of time on is getting more involved in compiling assets from the command line. I can't say I'm a huge fan of JSON or Javascript over PHP but even Wordpress is heading in that direction now. And everyone is using LESS & SASS for CSS pre-processing now too. So for sure I'd like to see your tutorial and get more exposed to all of that. Thanks for taking the time to do it.

 

[Hercules]

I'm interested in the Amazon side of things, I'm always looking for ways to speed up my landers. Each millisecond shaved off of loading time is an increase in ROI and competitive advantage. I've just never learned my way around Amazon Web Services though

 

[pfingy]

I can't say I'm a huge fan of JSON or Javascript over PHP

Lektor is a Python tool, but for there's not much programming involved, just creating the HTML templates, which is done with the Jinja2 templating language (very similar to Django templates).

 

[bernard]

Why are static sites so much better than Wordpress?

Do they work for more dynamic sites (scrapers, calculators, tools)?

 

[Calamari]

Why are static sites so much better than Wordpress?

Do they work for more dynamic sites (scrapers, calculators, tools)?

Static sites can do everything a wordpress site can do and they can do it faster while using less server resources.

OP, I'm definitely interested in learning about lektor from you and your process with it.

 

[SmokeTree]

I've been using Jekyll and Hugo for quite some time now. I prefer Hugo for its speed, hands down. I'm interested in this because I'd like to see how things are with Lektor as I don't have any experience with it and would like to see an example by someone who has spent time with it in production. I'm always open to learning new technologies and talking shop with fellow devs and folks that are into the command line side of things.

 

[pfingy]

I have a little bit of work to do generating content before getting started on building the site. I want to build a site using "bullshit generation" along the lines of many bullshit generators you can find online for tech-speak, marketing phrases, company names, etc. The common way is to pick randomly from word lists, but I'm also looking into a project someone setup to train TensorFlow with tech words scraped from heading tags on tech websites.

I won't spend too much time messing around with the content part. Once I have a proof-of-concept for the content, I'll publish a basic spec for the site, and then start building it in Lektor.

 

[pfingy]

Was just messing around with using Tensorflow to produce marketing gibberish.

Got results like this:

Compliance and infrants and the expects, and worklorms customer insights of the wirely to croud and demo data from appriactive management solutions of provider and througen and enver con the furusion and tradithout to crous to infrastructure. With the business the enterprise and onfive datatee the righatere to continuously transforming to tre through of you how to deliver content virtually and operational times and analytics and that tachines faster on content and publit access to moders enables costomers in a thes tastes to finance and and analysis.

Gonna see if I can get some generation from word lists together without doing too much work. The idea is to make a fake job site for tech jobs. It's just a dumb site to experiment with fake content generation, and I figured since it's not a money site or anything, I could share the process here to help people get into static site generation and hosting on AWS. Hopefully some actual tutorial-type content coming soon.

 

[eliquid]

I think this will be good.

Interested in hosting on AWS in general, followed by the static site stuff

 

[pfingy]

Interested in hosting on AWS in general, followed by the static site stuff

Here's a little AWS stuff to get started, just a tip for anyone that wants to create serverless redirects. This is how you can quickly and easily create a redirect using S3:

What's going on here is I've setup an S3 bucket as a static website and then you can redirection rules to the bucket. Once I setup a DNS record for this domain that points to this bucket, the link will be live at shop.recruiterbros.com, but for now the link is already live at the default S3 URL: http://shop.recruiterbros.com.s3-website.us-east-2.amazonaws.com/affiliate-product-1/

The link is safe to click, it just demonstrates that S3 is redirecting you to a geni.us link.

I will be getting into S3 hosting, CloudFront for CDN and HTTPS, and Route53 for DNS.

 

[pfingy]

Transfer DNS to AWS

The first thing I do is transfer DNS management to AWS Route 53. I want to start with building the site, but the DNS transfer could take awhile, so I'll do it right now and hope that it's complete by the time I'm done with the site building steps.

Once you've created an account at: https://aws.amazon.com/ then click Services -> Route 53 -> Hosted Zones -> Create Hosted Zone. Then enter the domain name you're going to transfer. You'll see a list of nameservers ("Type NS") records for your new zone, go to your registrar and transfer DNS management to these nameservers. So now the nameservers are setup and I'll work on building the first version of the site.

Planning The Site

There's always more to do, but I always start projects by creating the simplest spec I can, then I'll go through the whole process until that project is delivered (in this case, delivery = the site is live). Only then do I worry about adding features, improving styling, etc. I leave some room for incremental improvements via fiddling with small things, but once I have any major updates in mind, I'll create a spec for version two, and so on.

The idea is to generate fake job titles and some bullshit text to use as the job descriptions. According to the wireframe, I'm not even linking job descriptions to a new page with just that job on it, I'm just listing them all on one page. So there's two content types right now: job post and ad.

That's enough to get started demonstrating Lektor, maybe we can add more content types later, especially if anyone offers up some ideas.

Creating The Content

I spent a whole 10 minutes choosing a Python project on Github that generates bullshit, ripping off a job title word list from another project, and writing some code to dump the titles and descriptions to a JSON file. Titles and descriptions for the site can be improved by adding words to the word lists. PM me if you want to help and I'll send the word lists to you for augmentation.

There's not much to see, but for the sake of completeness and because it's only a few lines, here's the code:

import json
import random

import corporate_bullshit

FILENAME = './job_titles.json'
with open(FILENAME, 'r') as fh:
    job_titles = json.load(fh)

jobs = []

for i in range(100):
    title_one = random.choice(job_titles['first'])
    title_two = random.choice(job_titles['last'])
    job_title = f'{title_one} {title_two}'
    job_desc = corporate_bullshit.sentence_guaranteed_amount(random.randint(1,6))

    jobs.append({
        "title": job_title,
        "description": job_desc
    })

FILE_OUT = 'job_data.json'
with open(FILEOUT, 'w') as fh:
    json.dump(jobs, fh)

 

[pfingy]

Setting Up The Static Site Project

We're building the site using Lektor: https://www.getlektor.com/

After installation go to the directory where you want to start your project and type lektor quickstart

Then, after running lektor server, the site will be running on localhost:5000.

The admin site looks like this:

I'm not sure if any other static site builders include an admin interface, that might be one of the unique things about Lektor.

Here's the directory structure for the default project:

We're going to remove the projects directory, ignore about for now, and add a directory called jobs to the content directory. Inside of jobs we'll create a directory for each job post, and inside of that dir will be a file contents.lr which will hold the title and description to be passed into the template. The layout of the contents directory determines the layout of your site, so if you had contents/blog/blog-post-one/contents.lr then you could go to `mysite.com/blog/blog-post-one/`. Other pages in the project can read data from the rest of the site, so for this site, we'll have the homepage loop through all of the content available in the `jobs` directory.

So we'll be adding:

• `contents/jobs` for job data
• `models/job.ini` to specify the schema for job data
• `templates/homepage.html` because our homepage uses the generic page template right now, but we will want a custom template for the homepage.

 

[Steve]

Interesting thread.
Static site generators are great for pure info sites (docs, basic blogs), but the idea they can do anything a dynamic setup can do (as mentioned by another poster) is just plain wrong.

You can shift some dynamic functionality onto the browser, but it has obvious limitations. For example, search functionality - you can have your generator create a big blob of JSON that can be loaded into memory and filtered, but that will require a sizable download for the end user.
Email, e-commerce, user generated content etc, not possible (without integration with some server code somewhere).
Then there is the fact that you can run a dynamic site behind a full page cache, and it will be the exact same speed.

If you can live with these limitations though, static sites do give you excellent security and maintainability, especially compared to a large general purpose CMS (try keeping multi user wp site under version control...)

Tldr: static sites and dynamic sites are different things, with their own stenghts and weaknesses.

Looking forward to see how this thread progresses OP.

 

[pfingy]

I fully agree @Steve, but there is a third option today: single page applications. Most web apps developed today are single page applications, which from a deployment/hosting point of view are exactly like static sites, but they rely heavily on asynchronous API calls to provide the actual content. As you pointed out, if you want user generated content, search, etc. then you need to have server side functionality somewhere. I've already considered that if this tutorial goes far enough, I will add in some SPA-like functionality to show how it's done. The most recognizable example of plugging in server side features into a static site is embedding Disqus comments.

When I build sites today, content sites are static sites and anything else is a single page app. Although it might make sense in some situations, it'd be hard for me to find a reason to build a new site using Django or a traditional CMS.

But yea, @Steve, you bring up a very important point, I don't think it's productive to give people the wrong impression about what a static site can do. (If you're technical and creative, then you can do whatever you want, but for most normal users there are definite limitations).

 

[Calamari]

When I made that statement I did assume server side code to make things like that happen.

I also compared them to wordpress specifically. I have been using wordpress for 12 or so years and a static site builder for less than 2 of those 12 years. I have not come across one instance where I could not get my flat file sites to do what my wordpress sites could. I am however not building anything more complicated than a standard blog style site so I can definitly be missing the bigger picture.

 

[pfingy]

When I made that statement I did assume server side code to make things like that happen.

So much better to have like 5% of the site depend on some server side code rather than 100% of the site needlessly depending on having a database, web server and application code setup correctly. Stuff like email submission forms are super easy to add to a static site.

Once the basic site is setup, we can keep adding features to it and really clarify all of the tradeoffs.

 

[mikey3times]

Any reason that you chose Lektor over another flat CMS like Grav or Hugo?

There are so many to pick from and I’m nervous about them stopping development. Maybe that isn’t a problem.

 

[CCarter]

For example, search functionality - you can have your generator create a big blob of JSON that can be loaded into memory and filtered, but that will require a sizable download for the end user.

Most sophisticated search functions read the content once, then store a cache of the content so when a user is searching it is not doing a “real-time” read of every blogpost but against the indexed content.

The way you are suggesting search work would be the equivalent of Google literally searching every page of the internet everytime a user queries their search engine (unfeashable on mega sites). Instead they are querying against a stored database of “cached” versions of the indexed content on your site. It’s not “real-time” but most on-site search engines can be re-indexed daily or hourly and the search results would be fine.

This type of querying should be communicating server side anyways, otherwise even a small site with lots of content on each page would crash a mobile browser or low-performance computer with restricted or low RAM. Since you cannot control the environment or settings of the device that is requesting the data, you have to run services like search from the server since that is your environment and simply send the end result to the client’s browser.

 

[pfingy]

Any reason that you chose Lektor over another flat CMS like Grav or Hugo?

There are so many to pick from and I’m nervous about them stopping development. Maybe that isn’t a problem.

Several years ago I was building my own static site generators and experimenting with various approaches. Then while checking in on Armin Ronacher one day, I saw he had just released Lektor, a static site generator that was similar to what I was trying to build, only 1000x better. So for me, I chose Lektor because a few years ago it was the only project that matched my needs (I want a general purpose tool, not something that just builds blogs), it was created by a developer whose other projects are heavily used by myself and tons of other programmers (Flask, Jinja2, Werkzeug), I read the code and it made sense to me, and it's in my preferred language (Python).

I wouldn't worry about development stopping... you can take version 1 of Flask and use it for the next 10 years with no problem, it's something you only run locally to compile templates and variables into static HTML, it's not like there are any security issues to patch or 3rd party dependencies to keep up with. Armin has already stepped down as main dev for Lektor and someone else has taken over and is doing great. It's simple code doing a simple job, and doing it well.

I'll let users of other static site generators weigh in with more informed opinions, but I'll try to compare Lektor a bit with Hugo and Jekyll.

Jekyll is a restrictive, single purpose tool. Look at the list of variables: every page has variables like "content", "title", "date", etc. It's totally fine if your goal is to build a blog that sticks to the vision of the Jekyll devs.

Hugo is a heavy tool with a lot more variables and flexibility than Jekyll. Instead of a dozen variables related to blogging, there are several dozen variables that cover a wider range of use cases, and there are also a few dozen built-in functions, a large collection of types of templates and a long list of rules for how Hugo will know which template to use for a piece of content, both partial template support and shortcode support, support for various output formats beyond HTML, and Hugo wants to manage all of your assets for you (compiling SASS, minifying assets, and bundling assets).

If Jekyll and Hugo are like Wordpress, then Lektor is more like a helpful PHP library. To use Hugo, you have to learn the Hugo way of doing things, while Lektor makes it faster and easier to do things your own way.

Instead of defining a bunch of variables that a piece of content can have, Lektor supports some built-in field types (string, date, integer, markdown, etc.) and let's you add more field types via plugins, and then you define what fields a piece of content will have. For example, you might say you have a content type called "recipe" and has fields like "number_of_ingredients", "video_url", "description", whatever. Basically, I could be building special purpose site generators from scratch in Python, but Lektor just makes it a lot faster and easier by doing all of the boilerplate stuff and letting me get right into creating the data models and templates. You could absolutely use Lektor to build a clone of Hugo or Jekyll.

Lektor is a toolset to make it easier to manage content and pump that content into templates.

Another example, Lektor doesn't need to support different types of outputs, because it doesn't dictate any kind of output, templates can be HTML, XML, JSON, INI, SVG, anything, Lektor doesn't know or care.

A final point about Lektor, it was made explicitly to be usable by Armin's non-technical family members that wanted him to setup a website for them. So you can use Lektor to build a website framework, but then the site editors can use the admin panel in the browser to create and publish content.

Someone please let me know if any of the other site generators include a browser-based admin panel.

---

I get a competitive advantage by using more powerful tools, but only if those tools work for me. I don't really care about persuading anyone away from a tool that they've already found works for them, but I'm trying to be helpful and share my knowledge and process, in case someone wants to learn from my experience.

 

[Steve]

Most sophisticated search functions read the content once, then store a cache of the content so when a user is searching it is not doing a “real-time” read of every blogpost but against the indexed content...

Indeed, I'm not sure what your point is. All I way saying is that if you do not have any server side code, then your only option is to precompute this indexed content and load it into the clients memory as a JavaScript object, which isn't practical for anything but very small sites.
It looks like you are agreeing with me?

Anyway, I don't want to derail this thread.

 

[CCarter]

Indeed, I'm not sure what your point is. All I way saying is that if you do not have any server side code, then your only option is to precompute this indexed content and load it into the clients memory as a JavaScript object, which isn't practical for anything but very small sites.
It looks like you are agreeing with me?

Anyway, I don't want to derail this thread.

My point is Static sites do not hinder search functions nor server side processing/requests. Most search functions are already communicating through server side requests. Your example with the JSON/Blob is just odd, cause most search functions would never operate with the workflow you described.

Your comment about search functionality seemed to infer that somehow having a static site would not have practical search functionalities and more broadly there wouldn’t be server side processing. There still is server side processing whether through ajax or other means - meaning you can have ALL the functionality dynamic (wordpress) sites have.

There are no limitations like the ones you state.

 

[Steve]

There still is server side processing whether through ajax or other means - meaning you can have ALL the functionality dynamic (wordpress) sites have.

Ok, so we just have our wires crossed then.
You are talking about using static html and JavaScript in front of an api, I am talking about running a purely static site, hence the weird front end search solution (because that's about the best you can do without **any** server side code).

 

[pfingy]

Don't worry guys, I will build search into this site and everyone can see how it works!

 

[pfingy]

Previously we transferred the DNS to AWS, wire-framed the site and created the initial content, then we setup the basic Lektor project. Now I'm going to add the data model and template for listing jobs on the site, and then use a simple script to populate the filesystem-database with the fake job posts.

Lektor has two different ways of storing data, as content type that you define (blog post, job post, dating profile, etc.) or in a "databag" which is a JSON or INI file containing site-wide data that can be used for configuration variables or any other data that makes more sense as global data rather than as a specific type of content.

So the first thing I'll do is create a "jobs" path to hold all the individual job posts, even though I won't be showing individual pages per post in my first pass through the site, it will be ready to go for the next round of development.

I create a "jobs" model and a "job" model inside of models/ in the Lektor project.

job.ini

[model]
name = Job Post
label = {{ this.title }}
hidden = no

[fields.id]
label = ID
type = integer

[fields.title]
label = Title
type = string
size = large

[fields.description]
label = Description
type = markdown

jobs.ini

[model]
name = Jobs
label = Jobs
hidden = yes

[children]
model = job
order_by = -id

[pagination]
enabled = yes
per_page = 10

So a "job" (a job post) has a ID, a title, and a description. And "jobs" is hidden, it will not show any content, but it is the parent of all of the job posts.

In a minute I'll create content for the job posts, but first I want to update the homepage template. The default homepage has a model of "page" so it uses the "page.html" template, but here I've set it to use the "homepage.html" template instead.

I have a Jinja2 template macro (like a "partial" in other templating languages) to render a single job, then I loop through the jobs in the homepage template and call the macro for each job.

{% macro render_job_post(post) %}
<div>
  <h2>{{ post.title }}</h2>
  <p>{{ post.description }}</p>
</div>
{% endmacro %}

{% extends "layout.html" %}
{% from "macros/job-post.html" import render_job_post %}
{% block title %}{{ this.title }}{% endblock %}
{% block body %}
{% set jobs = site.get('/jobs') %}
{% for job in jobs.children %}
{{ render_job_post(job) }}
{% endfor %}
{% endblock %}

I won't go too much into how Jinja2 templates work, it's pretty standard stuff and the docs are easy to read: http://jinja.pocoo.org/docs/2.10/templates/

Now I can use the admin interface to add a new job post:

And then the posts show up on the homepage: