How I built forlater.email

A technical breakdown of my first big side-project

Ever since I began browsing sites like Hacker News and Lobsters, coming across new and exciting links to check out every day, I found it hard to keep up. On most days, I just didn’t. And that’s fine—good, even. But oftentimes, I’d come across a genuinely interesting link but no time to actually read it.

I began using Pocket. It was alright—the article view was very good; but it stopped there. I didn’t like nor use the other junk baked into the app: discover, following/friends thing, etc. It’s also proprietary, and that irked me—more so than the other “features”.

Thus, somewhat inspired by rss2email, I began building forlater.email—a bookmarking/read-later service that works via email. Email is the perfect tool for this use-case: works offline; you can organize it however you like; you own your data.

Pictured above is how forlater works. Each component is explained below.

OpenSMTPD

Mail containing links to be saved arrive here. OpenSMTPD is beautiful software, and its configuration is stupid simple (smtpd.conf(5)):

table blocklist file:/etc/smtpd/blocklist

action webhook mda "/home/icy/forlater/mdawh/mdawh"
match mail-from <blocklist> for any reject
match from any for rcpt-to "save@forlater.email" action webhook

The filter and listen directives have been snipped for brevity. The rest, in essence, simply sends all mail to save@forlater.email to an MDA program, via stdin. Any mail from an address in the blocklist file get rejected.

rspamd is used to prevent spam.

mdawh

mdawh, or the MDA webhook tool. A small Go program that processes mail coming from stdin and generates a JSON payload that looks like so:

{
  "from": "foo@bar.com",
  "date": "Fri, 1 Jan 2010 00:00:00 UTC",
  "replyto": "...",
  "body": "...",
  "parts": {
    "text/plain": "...",
    "text/html": "...",
  }
}

This is POSTed to a configured HTTP endpoint—which in this case, is navani.

navani

navani is forlater’s primary mail processing service¹. Listens for webhooks from mdawh, processes them, and sends mail using a configured SMTP server. URLs are cached in Redis along with the HTML content.

For the readable HTML, go-readability is used; the output of which is rendered into a minimal HTML email template -- something that I never want to write again.

The plaintext part is currently generated using lynx -image_links -dump -stdin. The -image_links flag is handy because it generates footnote links for images as well, instead of simply ignoring images altogether. I plan to rewrite this; possibly using a blend of HTML-to-plaintext libraries and handwritten rules.

future improvements

I plan to implement some kind of settings@ address to configure and store user settings (dark theme? fonts?). However, this introduces state in an otherwise mostly stateless system.

The other thing I’ve been thinking of is making your own newsletter of sorts. For example: save a bunch of links during the week, and have them all delivered over the weekend.

Neither of these “features” are confirmed to happen, primarily because forlater is feature-complete for my use. That said, I’m happy to consider any improvements or suggestions that you might have—please email me.

Finally, thanks to everyone who tossed a few bucks my way—mighty kind of you.

Questions or comments? Send an email.