Contents
Author / Developer
This project is being built by @jakintosh.
You can contact him via the `developer` email user at this domain, or follow via the social link above.
Site Status
Last Update 10/21/23
There are still known gaps in content througout the site, mostly waiting for a spare moment to be filled in.
Remaining work:
- Fill in the backlog of missing devlogs from Feb '21 to Dec '22
- Wrangle all the existing documentation for project compenents from my personal notes and publish them on the relevant project pages
Influences
This project was born out of a specific combination of my personal exploration into and observartions on computing, information theory, complex systems, and philosophy.
Along the way, there were a handful of technical projects that deepened my knowledge or changed my perspective of what is possible. These are those projects:
Colophon
A colophon is an old publishing term that refers to the last page of a book that would give the reader information about how the book was made. I think it's important to explain and document as much of our work as we can to make the most of the time we spent learning from our mistakes. So in that spirit and in the tradition of the colophon, this section documents the tools and workflows that went into actually building this website.
- collect - a plain text programming language
- sam - human writable XML markup
- Building - the "build pipeline"
- Deployment - the "deployment process"
- Links - link types
collect
Collect is a little language that I wrote for structuring and rendering text in general, but especially for something like building a website. Essentially, it has a syntax that lets you define a tree of objects that represent both data and instructions, and then that tree is executed by walking it. It's only external capability is writing text to the console or a file.
Syntax Overview
The language has a set number of object types, and each object can have a predetermined number of string parameters. Declaring an object starts by naming the object beginning with the @ symbol, followed by the string parameters which are started and ended with the " symbol, and separated by white space. Multiline string parameters can be used by placing """ on its own line at the beginning and end of the string. If you are adding children to an object that allows children, you close the object with the @end object. Every token is whitespace delimited, so indentation is unneccesary, but I use it heavily as a convention for readability.
Data Structures
For data, you can define:
- @string : a block of text with a name
-
@map
: an object containing
@group
,
@value
, or
@list
children
- @group : an object that adds a @map to a queryable group
- @value : exactly the same as a @string, but as a @map child
-
@list
: an object containing
@map
or
@query
children
- @query : an object containing query commands that resolve to a @list
Here is what defining these data structures look like in collect:
@string "name" "jakintosh"
@string "multiline-string"
"""
This is an example
of a multiline string.
"""
@map "my-cat"
@value "name" "mochi"
@value "color" "grey"
@list "favorite-toys"
@item "rainbow-toy"
@item "raccoon-toy"
@end
@end
@map "rainbow-toy"
@group "catnip" "1"
@value "name" "rainbow"
@value "color" "rainbow"
@end
@map "raccoon-toy"
@value "name" "bean paste"
@value "color" "grey"
@end
@list "catnip-toys"
@query
@include "catnip"
@end
@end
You'll notice how the @query object uses the names defined by the @group objects inside a @map. All @groups objects added to a @map must also have a unique integer ID that is used for sorting. There is a special @date object that allows a date to be added to a @map, which can also be used for sorting. Queries also have matching commands such as @exclude and @filter, as well as ordering commands like @reverse, @take, and @sort-by-date.
Logic Objects
For logic, there are two primary objects: @scope, which establishes a local context for object names and execution, and @switch, which provides for conditional logic.
Here's an example of what this can look like in practice:
@string "blog-byline"
"""
x-byline: $exec($(this.author))(author-handle-renderer) | $(date)
"""
@scope "blog-post-stub-renderer"
@switch "spacing"
@has-obj "iter.next" "\n"
@default ""
@render
"""
section:(class="blog stub") {$(this.title)}(a|href="./blog-post-$(this.slug).html")
$>(blog-byline)$(spacing)
"""
@end
Here, we first define a @string named "blog-byline" to use in the broader context, and then we create a new @scope for rendering a blog post stub. Inside that @scope, we define a @switch named "spacing", with two conditions: if it has an object (@has-obj) in the context at 'iter.next', it renders to "n", otherwise it is blank. Finally, we define a @render object, which can be thought of as the return value of the @scope. Whenever you try to render a scope, the string that it returns is whatever the @render object resolves to.
Text Rendering
The last major thing to know is that inside any of the string parameters in a collect tree, you can render values from the tree itself by using the $() syntax and placing the named path of the value you want to render between the parenthesis. You can see this above when we render $(this.title) inside the @render object.
There are also a few special forms of the $() syntax. The $>() form will maintain the current indentation on newlines in the rendered text. The $exec()() form takes two parameters, the name of a @map and the name of a @scope, then passes that @map into the @scope and executes its @render object. Finally, the $iter()() works the same as @exec()() except it passes in a @list to the @scope. The active @map during an execution of a @scope can be accessed using the 'this' object, and the entire @list can be accessed via the 'iter' object, and navigated with chains of '.prev' and '.next' specifiers.
Further Reading
At some point, I will write up a more complete and easily digestable guide to the language. In the meantime, you can explore its source code and some real examples of how it is used below. To give you an idea of its capability, this entire site was built by hand in plain text using collect.
sam
"Semantic Authoring Markdown (SAM) is a simplified markup for semantic authoring." The specification for the format was outlined by Mark Baker (detailed at the link just above). Its core insight is that "XML was designed to be human readable, but not human writable". It defines an alternative syntax for XML (and thus, HTML) that is much easier to write.
SAM makes writing HTML look like this:
html:
head:
meta:(charset="utf-8")
link:(rel="stylesheet",type="text/css",href="styles.css")
body:
header: My Website
main: About Sam
section: Paragraphs
This is a paragraph. This is a {link}(a|href="http://coalescent.computer").
This is another paragraph with {strong, emphasized text}(strong)(em).
section: Lists
1. This is an ordered list (item)
* This is an unordered list (item)
footer:
Webpage built with SAM
I wrote a (loosely faithful) parser for Mark Baker's SAM in Rust (in April 2022), and I use it to write all the "raw HTML" for this (and other) website(s).
Building
With these two tools, the majority of the build process is quite streamlined, composed mostly of ingesting all of the collect files, rendering them into SAM files, and then transforming the SAM into HTML.
However, the build script also copies over Linux executable downloads, generates their SHA256 hash, and updates the download areas of the site. Since I make a lot of assumptions, this is also quite minimal; the build script just grabs the currently installed binaries using 'which', then generates "collect" files that expose variables for the version number and SHA hash to be used during the render process.
This script is the entirety of the build process:
#!/bin/zsh
set -e # exits if any commands fail
echo "Updating Binaries"
co=`which co`
cohost=`which cohost`
co_sum=`shasum -a 256 $co`
cohost_sum=`shasum -a 256 $cohost`
mkdir -p site/downloads
cp "$co" site/downloads/co
cp "$cohost" site/downloads/cohost
echo "@string \"co-bin-version\" \"$(co --version)\"" > source/objects/co-version.collect
echo "@string \"co-bin-sha\" \"${co_sum%% *}\"" >> source/objects/co-version.collect
echo "@string \"cohost-bin-version\" \"$(cohost --version)\"" > source/objects/cohost-version.collect
echo "@string \"cohost-bin-sha\" \"${cohost_sum%% *}\"" >> source/objects/cohost-version.collect
echo "Rendering Templates"
collect source .sam
echo "Transforming SAM -> HTML"
sam .sam site --batch
echo "Cleaning Temporary Files"
rm -rf .sam
Deployment
Finally, to keep things pointed all the way through, deployment is done through 'rsync'. Using rsync lets me quickly upload the entire site via delta changes, and I just exclude the hidden directories, .sh scripts, and the source files.
#!/bin/zsh
set -e # exits if any commands fail
# first build
./build.sh
# deploy
rsync -avxP
--exclude '.*/'
--exclude '*.sh'
--exclude 'source/'
site/ {user}@{server}:{server-directory}
I use namecheap.com as my domain registrar. This site is hosted on a VPS (virtual private server) from 1984.hosting, whose data centers run on geothermal and hydro power in Iceland. On my VPS I run vanilla Ubuntu, serve websites using Nginx, and use Certbot to provide SSL for HTTPS. This site–and my others–are available over both HTTP and HTTPS.
Links
This is what an internal site link looks like, and this is what and external link looks like.