SAAS or Spacemacs as a service


Introduction

For people who don't know what Spacemacs is, it is an Emacs configuration that is meant to be be Discoverable, Consistent, Crowd-configured and with an emphasis on Mnemonic. It supports vim's bindings out of the box (thanks to evil) and provides tons of functionalities through a module configuration system. If you haven't tested it yet, you may give it a try, even if you already have an emacs configuration. This post is oriented towards Spacemacs, but should be easily adapted to any Emacs version.

In Spacemacs, despite the packages being lazy loaded, reading the configurations and setting everything up can take some time. For instance right now on my (quite decent) computer:

[425 packages loaded in 4.756s]

It takes nearly 5 seconds to start. This is not too bad taking into consideration the number of packages, but this is still annoying while working. There are two schools to address this problem:

  • Run one emacs instance, do everything in it and never close it.
  • Run a daemon in the background, and open client frames when needed.

I'm not really fond of having always an emacs instance that should never be closed. I'm still used to the vim's way of working with files, i.e. opening a new editor whenever I need to do something different. The second approach then fits well my workflow.

Running as a daemon

The Emacs daemon can simply be started with emacs --daemon. This will setup Emacs, and when done the daemon will be forked into a background process. To open client frames on this daemon there is an emacsclient executable taking some arguments, like -c to ask it to create a new GUI frame. Note that -t may be used to create a new frame within a terminal.

You can test running emacs as a daemon right away with these commands:

$ emacs --daemon
$ emacsclient -c

This should open instantaneously an Emacs frame. You can then close the newly created frame by pressing SPC q z. You can also try to open several frames in parallel by running several more time emacsclient -c. It is possible to kill the server by pressing SPC q q.

All the Emacs runtime is shared between clients as they are on the same Emacs instance. This mean you can access any buffer or layout from any client. You can try this by opening a file in a first client, then pressing SPC b b in a second: you will see the buffer opened on the first client.

Using a service

Starting the daemon is a repetitive and boring task that can be automated at boot/login time. In Linux distributions using it, systemd can be used to setup user services. User services are conceptually the same as system services, the only difference is that they are at the user level and thus placed on the home folder. This characteristic allows to easily add user services to some version-control tool and share the configuration among several machines. The format of user services is the same as system services. On Archlinux, user service should be placed in ~/.local/share/systemd/user/ and ends with .service.

For running emacs as a daemon, the service file is pretty simple:

[Unit]
Description=Emacs: the extensible, self-documenting text editor

[Service]
Type=forking
ExecStart=/usr/bin/emacs --daemon
ExecStop=/usr/bin/emacsclient --eval "(kill-emacs)"
Restart=always
Environment="DISPLAY=:0" "SSH_AUTH_SOCK=/home/fabien/.gnupg/S.gpg-agent.ssh"

[Install]
WantedBy=default.target

This can be placed in ~/.local/share/systemd/user/emacs.service. It can be enabled at user login time with:

$ systemctl --user enable emacs

And it can be started right away with:

$ systemctl --user start emacs

Stopping it and disabling it from user login time is as easy as:

$ systemctl --user stop emacs
$ systemctl --user disable emacs

Note: For an unknown reason this is not working right away on my computer:

Failed to connect to bus: No such file or directory

I solved this by adding XDG_RUNTIME_DIR=/run/user/$UID in front of commands, using an alias to make it simpler:

$ alias scu="XDG_RUNTIME_DIR=/run/user/$UID systemctl --user"
$ scu enable emacs
$ scu start emacs
$ scu stop emacs
$ scu disable emacs

Extra: using WIN + s to start new clients in i3

If like me you are using the i3 window manager, you can bind emacsclient -c to a key in order to start Spacemacs even more quickly in all situations, without having to start a terminal first to enter the command. For instance you can bind it to WIN + s by adding the following line in your ~/.i3/config file:

bindsym $win+e exec "emacsclient -c"

Blogging in org-mode with Nikola


Introduction

The last trend for blogging is to use a static site generator. This has some advantages over dynamic websites (wordpress) and online blog services (blogger):

  • This is more easy to do than deploying a application server, as there is no special installation/configuration needed (especially if you use GitHub for hosting).
  • This prevents attacks on the server as there are no interactions with dynamic parts.
  • The posts and associated contents can be versioned in a git repository as it is using only text files, not databases.
  • The content of your blog is not locked to an online blog engine, so you are free to move easily whenever you want all your content to a new system.

Thanks to Spacemacs, I entered the Emacs' world recently and started to enjoy org-mode. When I decided (again) to start a blog, I looked around for a static site generation supporting this file format. The problem of org-mode is that it is only fully supported within emacs, as most of its power come from all the extensions that will probably not be supported by any parser. I looked at different static site generators, among which only a few where supporting more than Markdown/RestrucuredText. My choice was directed to Nikola which has a support of a lot of format thanks to pandoc. But I was pleasantly surprised when I read the description of the orgmode plugin for Nikola:

This plugin implements an Emacs Org-mode based compiler for Nikola.

If your emacs does not ship with org-mode (>=8.x), you will have to edit the init.el file supplied with this plugin, and load a newer version of org-mode.

This meant that the plugin is using the real Emacs org-mode to render the posts of the website. This finished to convinced me starting my blog using this engine, and to write about this experience (in org-mode of course) as a first post. You are reading it. In this article I will detail the installation procedure to setup a website powered by Nikola and hosted on GitHub pages.

Setting up the system

This first part will describe how to set up the whole system, from GitHub pages to Nikola's org-mode support.

GitHub pages

GitHub is offering a service called GitHub pages that allows people to serve static websites through their GitHub repositories. There is the possibility to serve one website per repository and also to create one website linked to the username. Thanks to this, people can own their associate their blog with their GitHub's usernames. One restriction being that GitHub is only serving static content, i.e. .html pages with some javascript or css, but this should be far enough for running most personal blogs.

The main user website is be accessible at http(s)://<username>.github.io/. The website will be served from the master branch of a repository that should be called <username>.github.io. You can have a look at the GitHub's documentation for more details about this.

The first step to install the blog is to go on GitHub, and create a repository called <username>.github.io. You can give a description if you want, but don't check the "Initialize this repository" checkbox, nor add a .gitignore or a license: we want the repository to be empty. Once the repository has been created, you can clone it on your computer with:

$ git clone git@github.com:<username>.github.io.git ~/blog

As the master branch will be used for the rendered website, let us create a branch called sources to put the… sources of your blog:

$ cd ~/blog
$ git checkout -b sources

You can already add a .gitignore to the folder:

*.py[cod]
__pycache__
cache
output
.doit.db
_env/

and commit it (be sure to be on the sources branch):

$ git add .gitignore
$ git commit -m "Add .gitignore"

We should be ready to continue.

Installing Nikola

There are different ways of installing Nikola:

  • You can install it from your package manager, if your distribution provides a package for it. This is the best solution if the package exists.
  • You can install it globally on your computer with pip, but this has the tendency to "pollute" the filesystem, as it is not managed by the package manager.
  • You can install it within a Python virtual environment. This is the method I will present you here.
  • If you have your own preferred way to install it, please do so.

I choose to use a virtual environment because Archlinux is not providing an official package for Nikola. There is an AUR package for it, but this implies to build several of its dependencies from AUR too, and I don't went to spend my time checking their PKGBUILD at each update. As I also don't like using pip (or alternative tools) to install globally on my system, I decided to use a virtual environment.

The pyvenv virtual environment is provided directly by the python package on Archlinux, so I decided to go for it. You can of course use another virtual environment tool if you prefer. In order to make the installation of Nikola in a virtual environment the simplest as possible, I created a simple Makefile that will do the most of the job.

Let us start by defining a variable for the name of the environment folder:

ENVIRONMENT := _env/

I also added a default target as first one in the Makefile to select what should be called when no target is specified. It is a good practice from my point of view, as it uncouples the logic from the target ordering.

default: help

Then a target to create and clean the environment:

$(ENVIRONMENT):
  pyvenv "$@"

clean:
  rm -rf "$(ENVIRONMENT)"

If you are not used to Makefiles, it can look a little bit complicated. As we are using a variable, the target name is replaced by its value, same for $@ that is a special variable that as for value the target's name. So it translates to:

_env/:
  pyvenv "_env/"

clean:
  nikola clean
  rm -rf "_env/"

In order to use a virtual environment, one as to manually source from his terminal the $(ENVIRONMENT)/bin/activate file. Once the environment has been activated, the user need to install Nikola. To make this simpler for the user, I have done a setup target printing all required commands:

setup:
  @echo "make $(ENVIRONMENT);"
  @echo "source $(ENVIRONMENT)bin/activate;"
  @echo "pip install --upgrade pip 'Nikola[extras]';"

When this target is executed, it will only print this to the command line:

make _env/;
source _env/bin/activate;
pip install --upgrade pip 'Nikola[extras]';

This contains all steps required to setup the system:

  • Call the Makefile to create the environment
  • Source the activate file to enable it
  • Use pip within the environment to upgrade pip and install/upgrade Nikola

Instead of asking the user to copy/paste this in their terminal, it is possible to evaluate it directly with:

$ eval $(make setup)

This will do all the work, but let us add an help target that explains this literally to the user:

help:
  @echo "To setup Nikola environment, please use 'eval \$$(make setup)'."

You can now initialize the environment and install Nikola with:

$ eval $(make setup)

It will takes time the first call, but later ones will be much faster. Try to call nikola -h to be sure everything was installed correctly. The resulting Makefile can be found on my GitHub repository.

Site creation

The skeleton of the website can be created with these commands. Note you will have to give some information to create the site:

$ nikola init my_first_site
# Fill the asked information
$ mv my_first_site/* .
$ rm -r my_first_site

Nikola will create the site in a sub-directory so we have to move it to our current directory. You will see that for now there is only empty folders and a conf.py file. It is a pretty long configuration file, but you have to go through it if you want to see all possibilities to customize Nikola.

You can now commit the Makefile and the conf.py files in the sources branch (if not already done):

$ git add Makefile conf.py
$ git commit -m "Add Makefile and conf.py"

org-mode support

Here comes the interesting parts: making Nikola play with org-mode! There is an "official" Nikola's plugin for this, so is quite simple to install:

$ nikola plugin -i orgmode

The prerequisite being to have Emacs installed with org-mode > 8.x.

You will also have to edit the conf.py file so that Nikola will recognize the org files. For this, edit the file and add the following lines:

# Add the orgmode compiler to your COMPILERS dict.
COMPILERS["orgmode"] = ('.org',)

# Add org files to your POSTS, PAGES
POSTS = POSTS + (("posts/*.org", "posts", "post.tmpl"),)
PAGES = PAGES + (("stories/*.org", "stories", "story.tmpl"),)

You can commit the added files:

$ git add plugins/orgmode conf.py
$ git commit -m "Add org-mode support"

Testing the system

Everything should be set up now. In order to test it, you can use the following commands:

$ nikola build
$ nikola serve

And open your browser at http://localhost:8000. If you have a web page, it is working, congratulations. The server can be stopped by pressing C-c.

Blogging

Now that the system is set up correctly, let's start blogging.

Writing posts

To write a new post, simply use the following command:

$ nikola new_post -e -f orgmode

The -e flag tell Nikola to open the file directly for editing, and the -f orgmode tell it to use the org-mode format. Then simply save the file and close your editor when this has been done.

Building the website

As we already saw previously, the site can be built and tested with the following commands:

$ nikola build
$ nikola serve

The build command create the static content of the site, and the serve command run a development server to see what it looks like locally. This can be done whenever you want to see the changes. Note there is also this command that can be used advantageously as it will detect changes and rebuild the site automatically:

$ nikola auto

Deploying on GitHub

In order to deploy on GitHub easily, only one simple step has to be done: edit the conf.py file and change these variables as in this snippet:

GITHUB_SOURCE_BRANCH = 'sources'
GITHUB_DEPLOY_BRANCH = 'master'

It is all. Now you can deploy your website on GitHub by simply calling:

$ nikola github_deploy

Go to your http(s)://<username>.github.io/, your website should be there :-)

Extras

Separate metadata from posts

It is possible to extract the metadata out of the posts files. For this simply change this variable in the conf.py file:

ONE_FILE_POSTS = False

Themes

The theme of the website can be changed easily. Nikola provides some predefined themes that you can easily use. For this use the following command:

$ nikola install_theme <theme_name>

And change the following variable in the `conf.py` file:

THEME = "<theme_name>"

Some theme requires more configurations, read the information provided on their pages.

Code syntax coloration with org-mode

The pygmentize tool is needed in order to have syntactic coloration with org-mode. It should have been installed within the virtual environment already, so no step should be needed.

You need to generate a custom css file in order to make the coloration being displayed on the html page. For this run the following commands:

$ mkdir -p files/assets/css/
$ pygmentize -S friendly -a .highlight -f html >> files/assets/css/custom.css

Then rebuild the website: the source codes should be highlighted now!

$ nikola build
$ nikola serve

Conclusion

It was a really interesting work to setup this project. Nikola is a simple and nice static site generator, and it is plugin for working with org-mode is the better that I have seen.

I should mention that I have not tested the org-mode really far, so there may be some surprises. For now the only problem that I have seen is the inner links: I did not find a way to make it work for inserting a table of contents. Also, including images work, but for an unknown reason I have to set links to one level above, i.e. ../../images/logo.png instead of ../images/logo.png. All this are relatively minor problems, and having a tool offering org-mode, Markdown, RestructuredText, and many other format is a nice to have.