Introducing MakeStaticSite for website deployment

(from a technical perspective)

Launch of MakeStaticSite.sh Website

November 2022: A website dedicated to MakeStaticSite is now available at https://makestaticsite.sh/. It provides the latest version of the software, released under AGPL, along with various documentation to help getting started.

Motivation

The process of developing this website has evolved considerably. On setting it up a few years ago in the conventional manner using WordPress, I duly familiarised myself with routines for maintaining security and improving performance on a hosted platform. However, there were several aspects that weren’t ideal: the live content was hosted remotely, albeit with backups; updating the site in situ, particularly for major redesigns, was inconvenient; and performance wasn’t always as good as expected. Observing that most of the content was not dynamic in nature, I resolved to develop the site locally and then, as a separate process, deliver it as a largely static site.

The rationale for generating static sites has been around a long-time; I first tackled this more than 20 years ago, when I needed to generate a CD ROM from a CGI/Perl multimedia website. Nowadays, the field is mature; a Netlify blog post (Netlify is a cloud-based service where many static outputs end up) gives many valid reasons, rooted in the need for more responsive work processes. Typically underpinned by a simple language called markdown, there has been a surge in solutions; hence, options for Jamstack abound. However, the processes tend to be heavily oriented to developers at the expense of content authors, issues I know first-hand from enabling web access to museum collections.

I wanted instead to support existing content management processes, i.e. to continue with creating and managing content in WordPress, and then ‘at the press of a button’ generate and publish a static copy online. There are already some solutions, including dedicated plugins such as Simply Static. Tim Nash, having observed increasingly complexity with the Jamstack options, has described how he used WP2Static via WP-CLI, the WordPress command-line interface, for a (relatively) simple and compact solution. But I hesitate over the use of a WordPress plugin to generate the static site. Apart from the ongoing issue of maintenance and compatibility with WP Core, plugins are not really necessary since a public-facing website should be de facto crawlable by a web spider and not require internal system information.

And so I turned to general-purpose tools I could run from the command line. Enter Wget, a ubiquitous utility for retrieving and packaging any number of web pages for subsequent viewing. I needed to configure it, add a few additional inputs, and then choose where to store the outputs. This led naturally to delving into (Bash) shell scripting to bring it all together and I continued the process, adding deployment options and invoking more commands, particularly WP-CLI to prep the WordPress output (and to include a version of the plugin, WP Static Search, revised to work offline). If you examine the source code of this page, you will see the result.

Pipeline

Shell scripting became a means to orchestrate the overall process, which is readily seen as a pipeline comprising a sequence of phases. For example, one phases runs wget for the initial capture, whilst another re-runs wget to more completely capture a site. In fact, many core web development processes can be incorporated, such as replacing certain internal-facing pages or sections with alternative content; and invoking command-line tools for validation and streamlining output. As of version 0.21, there are ten phases altogether; and command-line options allow you to control the start phase and end phase.

The flowchart below illustrates an emerging pipeline (for the latest perspective, please refer to pages on workflow and phases).

Extending the usefulness

MakeStaticSite is a prototype that was initially developed for individuals who are creating and maintaining websites on a personal computer. However, the question soon arose: how might a team of content authors and editors be supported? (The answer is: yes!)

For example, what if this site is to have multiple contributors? In this case, the most straightforward approach is to set up or continue hosting with a third-party hosting provider, in the usual way. This means those who are creating and managing content carry on as normal through the dashboard. The main change is to separate the editing environment from the public site, which can be achieved by restricting access by some means of web authentication. Then, run MakeStaticSite to deploy the static site to the public, whether on the server or remotely. This approach ensures continuity; the only additional consideration is how to initiate MakeStaticSite once changes have been made — from within the CMS or external to it?

An initial test has shown that once set up, the process of maintenance and deployment can be carried out entirely from within WordPress. This can be achieved by running the script under the terminal emulator WPTerm (MakeStaticSite has an option to run ‘unattended’, which makes default selections). This depends on appropriate support for MakeStaticSite on the server; for shared hosting, the versions of required components, particularly Wget, may be somewhat old and, of course, for any serious use with a third party, access to WPTerm should be properly secured as instructed. It’s preferable to keep the running of MakeStaticSite on a personal computer, an arrangement that has been used for fuengsin.org.

However, hosting with a third party means that the data is no longer immediately under our control; although a mixture of database replication and file synchronization can compensate, there’s some loss of independence. Perhaps a better solution lies in p2p networks, but that’s an area I have yet to explore …

For now, this tool serves my needs well and I hope it is helpful to others. Please refer to the MakeStaticSite project site for further information.

This page was published on 21 September 2022 and last updated on December 13, 2022 .