Why Another Pollen Editor

Pollen, Beautiful Pollen

Anyone who has walked through the first tutorial of Pollen document will see that Pollen is by far the most advanced yet simplest framework for document processing, a superior to all ad-hoc, ugly markup languages like Markdown, AsciiDoc, RestructuredText, etc.

Pollen is not easy for beginners, but it is simple. Its simplicity lies on the unification of document structure and S expression, and that semantic tags correspond to functions and thus can transform any piece of a document.

Matthew, the author of Pollen, says in Pollen document that "The Pollen language is based on Scribble." Actually, the idea behind scribble has long been used in a tool called GNU TeXmacs, which has pushed to extreme the idea of mapping complex document elements to S expression.

Pollen and TeXmacs both are beautiful systems that have elegant, and solid foundations.

What’s Wrong Then?

While Pollen and TeXmacs share the same root, unlike TeXmacs, which itself is an editor with a special UX to manipulate the underlying data, Pollen as a document processing framework doesn’t provide any editing tool. For the end users, us, editing and debugging pollen source is quite a pain in a normal text editor:

1. The normal edit-run-debug process is distributed to different windows: your editor for writing, racket source editor for coding, terminal for rendering server, a browser for debugging and proofreading.
2. Pollen gives no warnings to undefined tags in Pollen source. It takes unnecessary long time to trace down rendering errors back to the output file.
3. Frequently inserting the default command char in an unconfigured editor is uneasy. Also if you have different command chars for different projects, it is you, not the editor, that need to remember which command char is used in which project, which is awkward.
4. Working with different machines means that you need to synchronize your editor dot files. Even though we have tools like stow, synchronizing configurations in different environments has all kinds of unexpected problems (take this emacs configuration issue).
5. A missing bracket or brace is hard to catch in some editors.
6. Racket runtime error sometimes is tricky to trace down. Bugs in Pollen source are harder to locate.
7. Just like coding in other languages, coding in Pollen frequently needs refer to function documents, or at least function signatures. It takes surprisingly long time to locate my tag functions to find out keyword arguments and positional arguments.

I want to solve these problems.

Emacs to Rescue? Not This Time

My first attempt was an Emacs Pollen Mode.

Using Emacs helps simplify the edit-run-debug cycle: writing, coding, launching rendering server, etc. all can be done in the Pollen mode. pollen-company provides auto-completion for tag names, which helps reduce undefined tags. Moreover, Emacs has built-in bracket matching facility, so It’s unlikely to get unbalanced brackets.

Unfortunately, the facility to preview the rendered content is missing in Emacs, and preview is indispensable in the edit-run-debug cycle. Emacs also has to push all the complexity of auto-completion to external Racket code, which introduces complexity in maintenance.

The more difficult part is to make sure a consistent user experiences among all users in different environments. Lots of emacs configurations are totally out of the control of mode writers. Want to launch the server? Make sure your Emacs can read your system PATH. Can’t get the auto-completion to show tag candidates? You need to first check this and this, and then ...

Furthermore, as an Emacs user with quite some experience with Elisp, I believe it will take some amount of effort for new users to appreciate the power of Emacs. Most of experienced Pollen users don’t use Emacs now, and I don’t see why they would in the future. Let alone beginners.

Some of these problems also apply to other editors. Atom, Sublime Text, you name it. No matter how that editor has been made for hacking, as long as it doesn’t understand Racket, it is a gauche solution for long run.

A New Editor?

To offer a real editing assistant, the tool must understand everything about Pollen. To understand Pollen, it needs to understand Racket. So the core component of this tool can only be in Racket. If the core component is in Racket, why not just write it all in Racket?

So here is my second attempt, Pollen Rock, a Pollen editor in the browser that can

• Reduce edit-run-debug cycle, with writing, coding, preview, debugging all in a browser window.
• Reduce undesired undefined tags by offering auto-completion for defined tags, and warning for undefined ones.
• Understand everything of a Pollen project.
• Offer consistent cross platform user experience with web technology.
• Help you focus on writing when you write, and coding when you typeset your prose.
• Help you locate references of function document and signatures.

What makes it more interesting than the Emacs solution?

Pollen Rock consists of two components: an in-browser editor, let’s call it the front-end, for end users, and a server that speaks JSON and responds to various HTTP requests. As you may have realized, the editor is dispensable; the core is the server.

With the help of Pollen Rock, all modern, extensible editors can be an alternative of the in-browser front-end, because they all have the ability to talk to Pollen Rock. They can talk to Pollen Rock to find out the correct command char for the opening Pollen source. They can also request rendered source file, or snippet of it, which is going to be an important piece for an editor find out the actual semantics of a tag (in runtime! Yeah!). Sometimes you want to load the Pollen source and evaluate Racket expressions? The server makes that possible too.

In sum, Pollen Rock gives the power back to developers, and thus editors again.

Even though I said “the editor is dispensable,” the in-browser editor is still the most convenient way to edit Pollen source across various desktop environments and devices. I sincerely hope Pollen users would find this new editor interesting and convenient to work with. Pollen deserves an editor exclusively made for its own beauty.

Have a new editor feature in mind? I would be happy to hear from you.