Introducing tenjinreader.com for Haskellers

Posted on February 27, 2018

I created this application based on my own experiences and requirements for Japanese language learning.

This post is mainly for Haskell developers, so I will not discuss anything about the Japanese language learning part of this application here. Checkout my other post for details on that.


The Big Picture

The app is written end-to-end in Haskell.

Moreover, I tried to use a bunch of new (experimental) stuff, so this blog post is about my experience

I started web programming just one year back. I have used Haskell for more than 4 years, and it is easier for me to use complex Haskell stuff than learn javascript.

In a period of almost 5 months I was able to make this app from an idea to a beta release.

The total code base is approx (lines of Haskell code). Source code on Github

and about 1k more for some extra libraries I created for this project, but pulled in a separate project.

Reflex FRP

Frontend is written entirely using Reflex-DOM FRP library.

The good

I am sure there is a lot more good, but I think it would be obvious in a comparison to other technologies. I hardly have any experience of frontend development, so I will now start with the pain points…

Difficulties

Haskey for persistent DB

For the DB I have taken an even more experimental approach by not using any conventional DB or even acid libraries. The reason for this was I wanted simplicity in usage, and the data should not be completely in memory.

The haskey library released in late last year fortunately had both these features.

This was a very important for me to do fast development, as I re-modelled the schema dozens of times based on the requirements in Frontend, and incremental addition of small features here and there.

So haskey is a new library and definitely not yet ready for production.

While still developing the application I found a major bug in its code, it was fixed fast thanks to the author!

But the real scary stuff was; after putting the site on production my own data got corrupted after a few days. I dont know yet what happened; fortunately I had backups to recover it from the previous day.

So there are definitely more bugs lurking around in the code, and the only safe way to use this library is to always take backups / snapshots (like hourly)

One good thing I did with haskey is to keep the data of every user in a separate DB (filesystem folder). This greatly simplified my code base and doing recovery from backups.

Migrations in Haskey

For migrations I used a method inspired by Trees that Grow paper. The idea is to use type family to have different variations of a tree data structure. In my case I have just two variations of the main schema: a CurrentDB and an OldDB. The application code always works on the CurrentDB and the OldDB is used to for migrating old schema to the current one.

Maintaining many versions of schema will be a much more complicated task, perhaps it can be done using this approach but I am not too sure.

There were a few issues in getting the code to compile with the use of type family, specifically in the instance / deriving instance declarations. I have used some hit and trial to get it working. But it would be good to have a better understanding / proper way of doing it. (ie dont refer to my code as a correct way to do it, there might be bugs)

Yesod

The backend is using yesod, it is very simple to set up a web site with authentication using the existing libraries. The major work is done via websocket, so there was no need for a library like servant.

Dev workflow

Final thoughts