Speaking at the CUNY Publishing Institute

Today I spoke for the 2014 class at the CUNY Publishing Institute run by John Oakes of OR Books. Here’s a great article John wrote about the Institute for the Journal of Electronic Publishing: “By the Book: A New Approach to Teaching the Art of Book Publishing”.


Here are the slides from my talk:


And, while they weren’t part of the slides, we also talked about using Markdown when writing for the web and static site generators. Here are my two posts about those topics:

“The Basics of Markdown Tutorial”

“Want to Speed up Your Website? Skip the Database”

László Krasznahorkai and Colm Tóibín

I wrote a brief piece on seeing Colm Tóibín interview László Krasznahorkai at a PEN World Voices Festival event. It was a truly fascinating conversation. Here’s an excerpt:

I found one of the most interesting facts about Mr. Krasznahorkai is the way he combines his love of travel (or more likely, love of being in different places) with his love of music. He said that he plays several instruments and that while traveling he seeks out music, but will then, the very next day, try to find the instrument that he heard and play it himself. That insatiable curiousity and desire for empathy seem to me to be the hallmark of a great writer, living a life of curiosity rather than some notion of a writer’s life.

Read the rest at Words Without Borders

The No-db Landscape (part 1)

It’s no coincidence that as I’ve become obsessed with website performance I’ve also become enamored with building sites with no database. They’re unequivocally faster, simpler to maintain and build, and more secure and stable than database-driven websites. There are tradeoffs of course, but in my view well worth it. I don’t understand the talk I see about static sites being for “hackers” or small or simple sites. I’ve seen some large, complex and highly trafficked sites built this way. See my earlier post in this series for more information.

For the most part I’ve been using two applications. For purely static sites I use Jekyll and for “flat-file” CMS sites I’ve used Statamic. Each is excellent in its own right, yet serves a different purpose than the other.

This article is aimed squarely at the designer/developer building websites for clients. Choosing the right technology is an important part of a developer’s job so I’ll go into my thinking here about when I why I might choose either.

So what is the difference between a “Flat File CMS” and a “Static Site Generator”?

Both flat-file CMSs (or file-based CMSs) and statically generated websites have no database at all. That alone is significant. All of your site’s content is merely a collection of text files.

This is what a document might look like in a flat-file site (or see this post as a raw text file):

---
title: Title  
category: Category  
layout: article  
date: November 10, 2013  
author: authorname  
---  
content goes here



Generally speaking templates and layout schemes tend to be similar in both types of sites in that they use a central default or base layout and some easy way to include partials in your templates. While this layout scheme is not unique to no-db websites, it’s a natural fit. In contrast to some of the popular, “traditional” content management systems, these apps assume nothing about your design, thankfully.

All of the no-db applications organize content similarly in pages that can be organized around collections or categories, though folder structures and the way URL schemes vary from one app to another.

The similarities end there.

Static site generators do little more than the name implies. They compile text-based content files (when I say text file that may mean any sort of text file, depending on the app, including markdown or text) into HTML pages using templates to determine the layout of the page. Depending on the app, they may process Sass/Less or other files to help make development more efficient.

Once the site is compiled you get a folder with your website in it. None of these are currently what you would call a CMS because there is no control panel and no inherent asset/content organizing principle or interface. And not a single bell or whistle to be found.

Most, though not all, of these apps run on a local computer via command line and once the site is compiled it can be moved to pretty much any server. There are more direct methods of deployment, including, prominently, Jekyll’s integration with Github’s Pages hosting service, which builds the site for you whenever you update your site’s Github repo.

The beauty of purely static sites, besides the significant performance gain, is that you can create all the complex queries you want in your templates and the end user will not be impacted because they’re not run when the page loads, but at build time. Personally, this is a revelation. The value of a well organized website is that it should keep content creators from repeating themselves while simultaneously creating rich relationships and context around the content of a site. Static sites give me the freedom to create all the complex relationships I like, using content from wherever appropriate, with near total impunity in terms of performance.

Keep in mind, that since the site is purely static, calendars or other dynamic elements will need to be done outside of the site. I’ve been using javascript for calendaring in Jekyll, so this so far has not proven to be a problem.

While both types of software could be called flat-file, what we mean when we say “flat-file CMS” is likely to be a dynamic application that creates HTML pages on the fly when the page loads. These are a terrific go-between the traditional CMS and static site generators because there are no issues with deployment and no problems with dynamic content like calendars while still maintaining the benefits of storing your content in text files. Optionally, you can offer content creators a control panel and WYSIWYG editor, which is very often necessary for client work.

Flat-file CMS

Among the flat-file CMSs there are only a handful that have held my attention. Pico is a very lightweight flat-file CMS created by Gilbert Pellegrom from Dev7studios (interestingly, a Wordpress plugin shop). Kirby is the creation of Bastian Allgeier, which at $39.00 per site appears to be a sophisticated system at a low cost.

Statamic, which is also commercial and priced at $99 per site for developers, is an elegant CMS that is my go-to when I need to have a control panel for clients. You can give them the option of writing in Markdown, text, or WYSIWG. The feedback I’ve gotten from clients is that it’s extremely easy to use, though some people get confused about what is a page and what is a post and how to create them.

Statamic uses a templating language that is simple yet fully featured. Organizing complex sites is doable but there’s a learning curve. I’ve tripped up here and there figuring out how to create field groups for a particular content collection, as well as figuring out how URL structures and navigation work, but I’ve never felt that there was anything I could not create.

I really put Statamic through its paces last year when I had–under a very tight deadline–to put together a conference website where I had to bring together schedules for events from various disparate parts of the conference (in addition to the main conference there was a film festival and a children’s festival and an art festival). For all the events there were related participants, and related venues. You’ll see if you visit the page that it loads fairly quickly given its size (the content is greyed out because I wanted to show past events but make them grey as the event’s time past). Each event, participant and venue page listed out all the other relationships as well.

Statamic enjoys an active developer community and has a growing number of very polished add-ons. A Statamic site can get fairly expensive, given that integrated search and contact forms will cost you $49 each, bringing a fully featured site to about $200, but the use-case here is for when you need to have a control panel that is confidently designed and a broad feature set.

Here are some additional examples of sites I’ve built in Statamic: 1, 2, 3.

Static Site Generators

I have a feeling that the best of the static site generators will surface as “web architectures” rather than just static site generators. That’s the phrase Docpad uses to describe itself and even though what it does is precisely generate a website, Docpad is indeed an extensible tool set for building any sort of website, even one with a database, or with dynamic characteristics.

Likewise, Harp bills itself as a web server, making the distinction between generating a website and all the other processes that go into a modern website, like Sass, or Jade templates, or coffeeScript. Similarly, Middleman describes itself as simply a collection of tools.

Ingeniously, the developers behind Harp have also built a commercial platform for deploying sites (the server itself is open source) that appears to be as simple as placing files in a Dropbox folder. I like to have a sense that a project will have longevity, so open source projects, in my view, should have a business model close at hand so that the core developers are able to continue on with the project.

Many of the static site generators are open source. There are three exceptions that I’m aware of. Both Cactus and Mixture.io are focused on making the developer’s life easier, and though Mixture defines itself as a prototyping tool, you could easily deploy sites generated with it in production.

On the other side of the coin, there’s Siteleaf, also a commercial product, which, uniquely, provides a control panel for content entry. With Siteleaf it appears that you can manage multiple sites from one control panel and deploy to any number of servers.

One reason I love static site generators is that you can define custom fields on the fly just by writing in the field in the front matter of a page, or defining it in a YAML file and then referencing it in your template. I’m not sure yet how this would work in Siteleaf because of the control panel, but I imagine for straight page-based sites where you have to give clients a control panel, it’s a good choice. I should note that the control panel is nicely designed and seems extremely friendly, but that very friendliness makes it simplistic enough that I would hesitate to use it for larger clients who might prefer something more serious.

With Jekyll sites–the only static site generator I’ve used in production–I’ve been using the prose.io content editor. Prose is terrific and works hand-in-hand with GitHub, allowing you to easily traverse all your repositories and files within. For each repository you can define files to exclude so content creators can only see files they’d want to work on. Prose also allows you to define your front-matter in a config file so the user can easily edit an entry’s metadata.

Prose screenshot


It’s important to note that Prose’s development isn’t terribly active as of this writing and it has some quirky behavior that makes it not entirely suitable for client use, but I don’t feel it’s that far away from being a great minimalist content editor. In fact, I’ve edited some of this post using it on my iPhone.

Jekyll is a Ruby app created by the founder of Github. It’s the house-app for serving Github Pages, that company’s free static website hosting service, and that alone gives it a leg up on most of these apps, even if just making deployment a bit easier than otherwise.

As of this writing, when used with GitHub Pages, Jekyll doesn’t allow any plugins because of restrictions on the hosting platform. If deployment isn’t an issue there are plugins put out there to overcome some of Jekyll’s–born a “blogging platform for hackers”–lacunae, and there will be a whitelist of plugins allowed in a future release and I’ve managed to get by fine without them.

Jekyll is also a bit more structured than many of the apps, using a post and page model where posts get taxonomies and chronological sorting whereas pages do not. The lack of taxonomies and sorting for pages is a big shortcoming, but I know that the project owners are working toward a more flexible, collections-based, model. All in all, I’ve enjoyed having to find creative solutions to Jekyll’s constraints and have managed to create some fairly complex relationships between pieces of content, even if most everything has to be nested in little _posts folders scattered throughout the site.

I use Jekyll’s data model liberally and have enjoyed the freedom of just dumping lists in a Yaml file to use anywhere on a site, like turning that data into JSON files for use in, say, a calendar, or tabular data, or relationships like authors on a multi-author site.

Here’s an example of a data file for a list of articles I’ve used in Jekyll:


-
  title: "Biometric Privacy Trade-off Exposed in Missing Jet's Passports"
  date: 2014-03-10
  publication: "Bloomberg Businessweek"
  link_to_original: "http://www.businessweek.com/"
  category: Finance

-
  title: "New Details Emerge in Mystery of Bronze Apollo Held by Hamas"
  date: 2014-02-10
  link_to_original: "http://www.businessweek.com/"
  publication: "Bloomberg Businessweek"
  category: Ancient


To my mind there’s nothing simpler than dumping some info in a text file, which compared to copying out each line of these into separate fields in a control panel is cumbersome.

One thing to be aware of for more complex sites is that for a large site the time to build the site can get pretty long. It’s page generation that takes so long (though we’re mostly talking about a matter of seconds) so I’ve moved anything I can into data files, which has proven to dramatically help build-times.

The guys running and contributing to the project are avid and thoughtful about where Jekyll is going and it’s integration with Github make it the best place to start for anyone new to static site generators, particularly given its ease of deployment and use of the easy, if somewhat limited, Liquid templating language.

My use case for Jekyll is for when I need a simple deployment and when I need to know the app will have a long life, which its Github backing would seem to ensure. Despite any shortcomings mention above, I really enjoy working with it, find it quite easy to use. The site this article was published on, budparr.com was created on Jekyll as well as several others, including a project I have in beta that has quite an involved content-set.

While Jekyll is my go-to app (and the app of choice for the firm that introduced me to static site generators, Development Seed) I’m actively looking at Middleman, also a Ruby app; Docpad, Harp, Assemblejs, and Siteleaf for use in a variety of projects. There are myriad choices to suit your needs.

With no-db websites in general, it’s proven not difficult to move content and even site structure between apps, so it should be easy to get to know several of them so that you can always choose the right set of tools for the task at hand. I’ve never liked the way CMSs develop fan-bases rather than user-bases and welcome the change to choosing toolsets to get a job done.

If you want to get your feet wet without worry about setting up the proper environment for Jekyll, give Mixture a try. While it’s meant to be a prototyping tool, Mixture is well designed and easy to use so you can get a static site up an running pretty quickly, with a vareity of boilerplates to choose from to help you start as well.

I’ll go into these more as I’ve gotten a chance to build some sites with them and report back.

The Basics of Markdown Tutorial

An abreviated form of this article is available as a Gist on Github

What is Markdown?

Markdown is a formatting syntax that allows you to write in plain text in any editing program, but still format your document like you would with a word processor. It was created to convert plain text into HTML for publishing on the web and happens to be super easy to write and just as easy to read even without being converted.

Markdown ensures that a writer’s intended formatting never gets lost. If you’ve ever labored on a Word document only to have its formatting scrambled when you sent it to be published to your website or when a colleague opened it in another program, you’ll appreciate the simplicity and stability of Markdown.

While you can edit Markdown files in any text editor, there are are some, like iA Writer for Macs, that are specially built to make this easy way of writing even easier.

How do I write in Markdown?

Markdown is extremely simple. In fact, it’s mostly plain text, so writing a paragraph would be the same as with any text editor (there’s a line between each paragraph).

If you want to put in a line break
you merely put two spaces after a line
and it shows up like this.

Emphasis

If you want to make a word bold, you merely put two asterisks around it:

like **this** is like this

To emphasize a word with italics you put one asterisk around it:

like *this* is like this

or

use an _underscore_ to get the same results.

You can even use these same characters to emphasize complete sentences.

Links are simple. Just put the text to be linked in [] brackets, and the link in () parenthesis.

[Links are simple](https://www.google.com/search?q=markdown)

Blockquotes

When typing an extended quote, just put the “greater than” sign before the paragraph.


> "I have dipped into books of aesthetics, but I had an uncomfortable feeling that I was reading the works of astronomers who had never looked at the stars. They were writing about poetry as if poetry were a task, and not what it really is, a passion, and a joy."

“I have dipped into books of aesthetics, but I had an uncomfortable feeling that I was reading the works of astronomers who had never looked at the stars. They were writing about poetry as if poetry were a task, and not what it really is, a passion, and a joy.”

Lists

  • Using Lists is Easy
  • Just use a dash with a space before each line
- Using Lists is Easy
- Just use a dash with a space before each line

Headings


# The largest heading  
## The second largest heading  
…
###### The 6th largest heading

Images

Images are just like links, but they have an exclamation point in front of them (and of course the link should be to where the image is!). The text between the brackets becomes the title, but it’s not required.

Images are easy


![Images are easy](http://link-to-image.jpg)

Resources

See also “What is Markdown” for a list of good Markdown editors.

If you’re editing content in a Jekyll-based site, you can use the prose editor, if your site is configured for it. If you want more advanced formatting, check out the original Markdown site, or the Github Markdown page.

Want to Speed up Your Website? Skip the database

The Very Brief History

In 2012 developers for the Obama re-election campaign moved from the ExpressionEngine-based website of the prior campaign to a practically retro HTML site with no content management system. Only this wasn’t 1999 and the website was actually quite modern, having been generated by the popular open source software called Jekyll1.

A hugely succesful site 2, it had over 81 million page views and was a model of performance and stability. It was a milestone in the movement for web developers to utilize a less bloated technology stack than had become the norm over the past decade3.

Static site generators, like Jekyll, convert text documents into HTML using one or more templates for laying out the pages. There is no content management system and no database. While once the realm of “hackers” only, there’s a growing list use-cases for sites created this way because they’re incredibly fast, simple and flexible.

The Problem

The heart of a “traditional” CMS driven website is a SQL database where all the site’s content and settings are rigidly organized into tables with rows and columns, something like a very complicated spreadsheet. Each CMS has its own scheme of organizing this information and on top of the database there is an application layer that converts the data into pages, route those pages into web addresses to serve to end users, and manages various user functions, whether or not we need them.

Querying the database every time a page loads takes time, which, even measured in milliseconds has shown to impact how people perceive and interact with your website4. Many CMSs cache (save pre-rendered web pages in memory) to speed up the process, though that merely adds a another layer of complexity rather than solve the original problem.5

SQL databases are not only potential sources for performance bottlenecks but are targets for hacking. What’s more, they can be difficult to maintain in a version control system, and act as a barrier–when content is organized around them–to moving content from one system to another. Even free open source systems may come at a great cost of maintenance and hosting at scale as well as reliance on a variety of add-on developers that becomes expensive over time.

This is not to suggest that database driven websites are inherently bad in all cases, but for a vast number of independent websites a database driven content management system may very well be overkill, like using a canon when a rifle will do. As websites get fatter and slower6 it becomes ever more important to question how they are built and look for new efficiencies. While static websites aren’t a panacea, they very well may be a gateway toward a new way of thinking.

The Post-CMS World

Ben Balter, Github Gov 2.0 Evangelist and former Presidential Innovation Fellow, calls this the “Post-CMS” world7, and I believe he’s right. Solutions such as Jekyll or other static site generators (there are many) are not content management systems because they only process files (i.e.convert documents and templates into HTML pages). However, it’s only the database and system of application layers that is missing. Developers and website owners are beginning to use a variety of loosely connected tools that are taking the place of formerly integrated content management systems, choosing only the bits they need and only the ones that work best for their needs.

Docpad is an excellent embodiment of this paradigm, billing itself as a “next generation” web architecture where one can choose pretty much any feature they need, whether that be a database, a particular templating system or editor, or other function to suit their needs. No assumptions and nothing you don’t need.

Documents and Data

Content management for static site generator is nothing more than a set of files in folders, much like what we are all used to using on a day-to-day basis. In the real world we organize our little bits of data into spreadsheets and our text in documents, yet in the world of CMS driven websites we lump all of our documents into spreadsheet-like cells, which in turn creates a separation between us and our content.

I’ve worked with writers and editors over the years to help them get accustomed to the act of writing in a word processor (very sticky Word!) to copying their work into one or more little boxes with a new set of formatting buttons so each piece can be injected into a database. Some get it, many don’t and everyone seems relatively uncomfortable with it.

This is what a document might look like in a static site:


---
title: Title  
category: Category  
layout: article  
date: November 10, 2013  
author: authorname  
---  
content goes here




This is a text document that can be moved to any system or shared with ease. In a database driven website the information above may be separated into as many as three or four different tables. While a highly organized database isn’t inherently wrong, document-based templating systems maintain the connection between the creator of the content and the resulting document.

Cost Effectiveness & Faster Iterations

Static sites don’t require as complex web server setups so are much cheaper to host. In fact, depending on the circumstances, they can be hosted for free on Github’s Pages service, which tightly ties their version control with the site’s hosting.

Healthcare.gov (not the problematic back-end application) was originally planned to be on a commercial CMS but was later moved to static. This website has over a thousand pages organized for various users and was built with the same tools as the Obama campaign website. As a result of the move from a CMS to static, the site’s hosting requirements went from 32 servers down to one.8

The lack of rigidity in developing sites without a CMS make changes much faster to put into effect. The ease of version control also makes it such that a developer or developers anywhere can contribute to the site’s development and a site can easily be “forked,” that is copied for use on another site, in a matter of minutes. The entire website can be treated as an open source project (it can also be kept private in this set up).

The Future

Static site generation is quickly moving out of the realm of “blogs for hackers” and being used for a variety of use-cases. There are still gaps in the usability of these systems for content creators and the software can only be used by developers comfortable in the realm of working without the interface that many rely on.

While there are hurdles, the landscape is changing quickly. Siteleaf is an excellent example where some smart developers understand the need for an interface for content creators while still maintaing the essence of static site generation. There are also platforms like Harp that facilitate publishing static sites through Dropbox.

There’s no shortage of innovation and I believe these tools will soon be in the mainstream. Both Cactus and Mixture.io are tools that make the development side as easy as downloading software, and soon we will likely see a new form of hybrid content management systems that are more like a loose collection of tools curated rather than developed by one person.

I’d highly recommend you read through some of the articles listed below, particularly Development Seed’s “How We Build CMS Free Websites,” Rob Muhlestein’s “The Static Web Returns,” and Ben Balter’s “Welcome to the Post-CMS World” where he has recently updated his performance analysis.