Taking notes with MkDocs
I’ve been using TiddlyWiki for note-taking for a few years. I use them to keep track of my technical notes and checklists. TiddlyWiki is a brilliant piece of software. It is a single HTML file with a note-taking interface, where the notes you take are stored directly in the HTML file itself, so you can easily carry (copy) the file around and easily deploy it online for sharing. However, most modern browsers don’t allow web pages to access the filesystem, so in order to let TiddlyWiki save the notes, you need to rely on browser extensions or Dropbox integration service like TiddlyWiki in the Sky. But they still have some frictions.
So recently I started to look for other alternatives for note-taking. Here are some of the requirements I desire:
- Notes are stored in non-proprietary format. In case the service/software is discontinued, I can easily migrate them to other tool.
- Has some form of formatting, e.g. write in Markdown and render to HTML.
- Auto-generated table of contents and search functionality.
- Can be used offline and data is stored locally.
- Can be easily shared with other people.
- Can be version-controlled, and you won’t lose data if there is a conflict.
TiddlyWiki excels at 1 to 5, and I can easily sync it with Dropbox. However, I often forgot to click “save” in TiddlyWiki, and in some case when I accidentally create some conflict while syncing, Dropbox simply creates two copies and I have to manually merge them. There is also no version history so it’s hard to merge and look into the history.
Then I suddenly had an epiphany during shower: all I need is a bunch of Markdown files, version controlled by git, then I can use a static site generator to render them as HTML, with table of contents and client-side search. After some quick search I found MkDocs, which is a Python-based static-site generator for project documentations. It also have a live-reloading development server, which I really love.
MkDocs is really straight-forward to setup and use (under Linux). To install, simply use
pip (assuming you have Python and pip installed):
sudo pip install mkdocs
Then you can create your document folder using
mkdocs new <project name>
The generated folder will have the following structure:
<project name> ├── docs │ └── index.md └── mkdocs.yml
You can then start to write documents in the
docs folder. You can create subfolders to organize the Markdown files. To view the generated HTML,
cd into the project folder, then run
mkdocs serve, the development server will start to serve on port 8000. Opening
127.0.0.1:8000 in your browser then you can see the document. You can also run
mkdocs build to generate the static HTML into the
sites folder. The folder can then be hosted using any server.
mkdocs.yml file contains some configuration. For example, you can use the ReadTheDocs theme by adding the line:
If you wish to open the generated site locally using the
file:// protocol, you can add this line:
When you have a note named
foo.md, the generated file will be
use_directory_urls is set to
true, the URL for linking to the
foo.md page will be
/foo/, which is a more modern URL naming convention. But for this routing to work, you must host the files in a web server. If you want to open it locally, you need to set the config to
false and all the link will be
Migrating from TiddlyWiki
Moving all the notes from TiddlyWiki to MkDocs is very easy. If you are using TiddlyWiki 5.x+, you can go to “Tools” right under the search box, there is a “export all” button. Export the tiddlers (notes) to CSV. Then you can use the
tiddly2md script to convert the CSV to individual
.md files. If your tiddler has UTF-8 titles, you need to add a parameter
encoding='utf-8' to the
pd.read_csv() call in the script for it to work.
The exported Markdown files will lose the tag information, so if you are using tag as folder structure like me, you’ll have to manually create folders to arrange them. Some tiddlers using the old WikiText format will also be empty (I use Markdown in my TiddlyWiki, but there are some old ones from the old versions). You can use
ls -lS to see which file has no content and manually fix them. After the
.md files are in place, run
mkdocs as usual.
MkDocs is a pretty simple but powerful tool for note-taking. You get all the benefit of editing the notes using plain text editor, and have them version controlled by git. But you also get a nicely rendered HTML version with search functionality. One thing I miss from TiddlyWiki is the ability to generate a single HTML file containing all the notes. (MkDocs generates a folder of separate HTML, CSS and JS files.) There are some scripts like mkdocs-combine that claims to do this using Pandoc, but I haven’t tested them.