IMAP IDLE: Making mbsync Feel Instant

A post by Yousef Akbar

· 4 min read

tech · email · imap · mbsync

For the two people out there who still manage their mail offline in the terminal with tools like mbsync and notmuch, this is a useful tip to make your synchronization with IMAP servers more seamless and efficient, without needing to poll and pull so frequently. Enter: IDLE on IMAP and goimapnotify.

For context, this is my current setup to get my email into a Maildir directory and access it offline:

    ┌──────────────┐        
    │ IMAP Server  │        
    │ (e.g. Gmail) │        
    └──────┬───────┘        
           │                
         mbsync             
        Push/Pull           
           │                
           ▼                
    ┌──────────────┐        
    │ Local Machine│        
    │   Maildir    │        
    └──────┬───────┘        
           │                
       notmuch new          
           │                
           ▼                
    ┌──────────────┐        
    │   Indexed    │        
    └──────────────┘        
                            
Accessible with any frontend
                            
  e.g. - Emacs              
       - *notmuch.nvim*
       - neomutt

The mbsync and notmuch stack is pretty canonical at this point and most setups use the same. It works fine on its own but requires frequent polling and it’s still not as fresh as I would like.

Note: Check out my Neovim plugin notmuch.nvim, which acts as a frontend for reading and writing email inside your favorite text editor :-).

IMAP IDLE command

As per RFC 2177, IMAP servers have a useful capability called IDLE. The IDLE command allows clients to receive “push notifications” from the server upon an update (new/updated/removed mail) from a watched mailbox.

With a specialized tool (see below for goimapnotify), you can setup hooks that pull/synchronize your Maildir with the server whenever a new message arrives in that mailbox.

To check whether your mail server supports this capability, run the following:

# Replace "imap.example.com:993" with your provider's IMAP details
openssl s_client -connect imap.example.com:993 -crlf -quiet
* OK IMAP server ready

# Type this once you are connected
a1 CAPABILITY

* CAPABILITY IMAP4rev1 IDLE NAMESPACE UIDPLUS MOVE AUTH=PLAIN

For more information on IDLE and how it works, check out the RFC reference.

goimapnotify

goimapnotify is a Go utility (improved version of the older imapnotify) that executes scripts on IMAP mailbox changes (new/deleted/updated messages) using the IDLE command on capable servers.

It’s configured with a YAML file that allows for control over multiple accounts, and configurable per-mailbox behavior.

Installation

It’s a Go program so you can use go install to install it:

go install gitlab.com/shackra/goimapnotify/cmd/goimapnotify@latest

Configuration

Here’s my sample configuration that allows me to get notifications on new/updated/removed mail (no sensitive information):

# ~/.config/imapnotify/gmail.yaml

configurations:
  -
    host: imap.gmail.com
    port: 993
    tls: true
    tlsOptions:
      rejectUnauthorized: true
      starttls: false
    idleLogoutTimeout: 15
    username: '<YOUR-EMAIL>'
    # password: '<YOUR-PASSWORD>'
    passwordCMD: '<COMMAND-TO-PASSWORD-MANAGER>'
    xoAuth2: false
    boxes:
      -
        mailbox: INBOX
        onNewMail: 'mbsync gmail:INBOX'
        onChangedMail: 'mbsync gmail:INBOX'
        onNewMailPost: 'notmuch new'
        onChangedMailPost: 'notmuch new'

The boxes section defines the behavior. onNewMail and onChangedMail trigger mbsync <account>:INBOX to start synchronizing to my Maildir. onNewMailPost and onChangedMailPost run notmuch new to index any updates in notmuch where the tagging and searching capabilities work.

Persistent systemd service

You could run the goimapnotify program with this configuration manually:

goimapnotify -conf ~/.config/imapnotify/gmail.yaml

But you would be better served by defining a small systemd service for this so that it runs in the background automatically:

; ~/.config/systemd/user/goimapnotify.service

[Unit]
Description=IMAP IDLE Notifier
Documentation=https://gitlab.com/shackra/goimapnotify

[Service]
Type=simple
; Replace this path with `command -v goimapnotify`
ExecStart=/home/yha/go/bin/goimapnotify -conf %h/.config/imapnotify/gmail.yaml
Restart=on-failure
RestartSec=5

[Install]
WantedBy=default.target

Don’t forget to reload your systemd config and enable and start the service:

systemctl --user daemon-reload
systemctl --user enable --now goimapnotify.service

Conclusion

Now, whenever I receive new emails, I am not beholden to whenever my whim compels me to refresh and update my sync flow. I can safely rely on goimapnotify to receive those update notifications from Gmail and accordingly run mbsync and notmuch to sync my Maildir, which is particularly useful when I need to reach for an email OTP or a confirmation email quickly.

For any thoughts or suggestions, feel free to contact me.