meli-website/posts/2019-06-15-pre-alpha.markdown

7.9 KiB

title ogTitle author date date_iso_8601 bees left right updates
pre-alpha release pre-alpha release epilys 2019-06-15 00:00:00 2019-06-15T00:00:00+02:00 What if bees are contagious?
url desc
/images/sample-config.png Sample generated config on first run.
url desc
/images/thread-view.png Viewing a thread.
url desc class
/images/compact-listing.png Compact mail listing. dull-red bold
url desc class
/images/plain-listing.png Plain mail listing. cyan bold
url desc class
/images/threaded-listing.png Threaded mail listing. green bold
url desc
/images/shortcuts-view.png View currently applicable shortcuts.
date content
2019-07-08 Added mastodon account link, discussion link.

An early release with basic features and only Maildir support has been published in meli's git repositories. meli is a new experimental mail client for the terminal. It's a from-scratch implementation in order to experiment with ideas I had about a client's design.

Contributions and development will be hosted on meli-devel@, as on the long run I would like to make mailing-list driven development comfortable and newbie-friendly by eliminating effort on the side of the contributor; the mailing list software should lift most of the burden by acting as a bug tracker, patch archive, maintainer tool, build tool. Note: no contribution guide has been drafted yet.

You can follow meli-announce@, my mastodon account @epilys@chaos.social and the RSS feed for updates.

General discussion, comments, etc is welcome on meli-general@.

meli aims to be a modal and hopefully a very configurable client. An API for plugins is in the current plans, allowing sufficient control of the client through scripting.

what is usable now

While this stuff is on the roadmap, the pre-alpha supports Maildir folder structures, and three ways to render every folder: [one row per thread]{.dull-red .bold}, [one row per message]{.cyan .bold} and [one row per message]{.green .bold} but with the thread structure visible. Browsing a thread is done in the same view as reading the email, where you can simultaneously browse the thread and read an mail's content. Replying to an email can also come with the same thread view in order to have the whole conversation available while composing; composing is done in a different tab, so you can switch tabs anytime.

Commands can be executed via configurable shortcuts (bindings for the current view can be listed with the ? shortcut) in NORMAL mode or commands in EXECUTE mode. Documentation is provided in a man page located in the root folder of the repository. You can view it with man -l meli.1{.shell} or rendered with mandoc on the documentation page.

what basic functionality is coming later

For the moment there's no attachment ability in new e-mails, no IMAP support, no full text search ability or integration with notmuch. I suspect notmuch integration will coincide with the start of the API.

what I am doing generally

MUAs are a lot of work, it turns out. I've been trying to scale it horizontally from the beginning, and I believe this helped as I was more focused on the overall architecture than individual functionalities. I have also tried to avoid adding dependencies when I could. I find it scary to install something and get hundreds of packages pulled.

I hope I can make it into a decent tool :)

technical stuff

meli is written in Rust, with an Entity-Component-System design. The mail specific functions are done in a separate library crate, which means it can be reused for other mail applications.

entity component system

A description of the ECS logic can be found here https://en.wikipedia.org/wiki/Entity_component_system Every component of the application represents a single entity that can draw on the terminal, access user data or spawn processes. This is the entity & component part of the ECS architecture merged together.

All components implement the Component{.rust} trait:

pub trait Component {
    fn draw(&mut self, grid: &mut CellBuffer, area: Area, context: &mut Context);
    fn process_event(&mut self, event: &mut UIEvent, context: &mut Context) -> bool;
    fn is_dirty(&self) -> bool;
    fn set_dirty(&mut self);

		/* ... */
}

The program's data state is held in a System object called Context{.rust}. The State{.rust} object holds the necessary bits to run the basic event loop.

struct Context {
    pub accounts: Vec<Account>,
    pub mailbox_hashes: FnvHashMap<FolderHash, usize>,
    pub settings: Settings,

    pub runtime_settings: Settings,
    /// Areas of the screen that must be redrawn in the next render
    pub dirty_areas: VecDeque<Area>,

    /// Events queue that components send back to the state
    pub replies: VecDeque<UIEvent>,
    sender: Sender<ThreadEvent>,
    receiver: Receiver<ThreadEvent>,
    input: InputHandler,

    pub temp_files: Vec<File>,
}

struct State {
    cols: usize,
    rows: usize,

    grid: CellBuffer, /* The terminal as a grid */
    stdout: Option<StateStdout>,
    child: Option<ForkType>,
    pub mode: UIMode, /* Normal, Input, Execute input, Fork.. */
    components: Vec<Box<Component>>, /* UI components */
    pub context: Context,
    threads: FnvHashMap<thread::ThreadId, (chan::Sender<bool>, thread::JoinHandle<()>)>,
    work_controller: WorkController, /* async jobs */
}

drawing output

An event loop looks roughly like this:

  • see if Context{.rust} contains events to be received. These events are added by Components in previous loops.
  • if any event existed, the grid might be modified. So redraw all dirty bits of the terminal grid.
  • block on multiple writers-single reader channel that receives from the input thread, the mail update polling threads, timers, and anything else that accesses Context.sender{.rust} field.
  • State{.rust} passes down the events by calling component.process_event(&mut event, &mut self.context){.rust}. Components recursively pass the event down to their children.

UI updates are supposed to be incremental for speed. Single envelope updates should only cause the minimally required UI updates.

When drawing, Components update the grid and push the area coordinates of their updates to Context{.rust}. When State finally redraws in the next event loop, it does a horizontal sweep in all the dirty areas to avoid redrawing common parts. Areas are sorted by x coordinate and common segments of all dirty areas are drawn once.

data handling

To represent relationships between different objects without references, vector indexes and hashmap keys are used. This way most data reading or modification is done inside the Context{.rust} object. Caution must be exercised to keep all these indirect references up to date when something is modified or deleted.

A great difficulty I face in debugging is visualising all these connections. I've started a library to provide access to Debug{.rust} implementations for gdb's pretty printing python API but I haven't reached any result yet.

Mail parsing is done asynchronously in separate threads. Mail backends such as IMAP, Maildir have to implement a Backend{.rust} trait. So far I've used the trait only with Maildir, which means it was written with its needs and abilities in mind. If all goes well, IMAP should start in the nearish future.

Discussions: