Hanging when loading maildir? #96

New Issue

jmct · 2020-09-10T10:31:07+03:00

jmct commented

2020-09-10 10:31:07 +03:00

Hello,

When I run meli it seems unable to load my maildir emails. The main window just shows loading..., for as long as I let it run (and I've let it run for hours). The behavior is only slightly different between HEAD and alpha-0.6.1 (I'll explain more below).

Relevant section of my config:

# Setting up a Maildir account
[accounts.galois]
root_mailbox = "/home/jmct/.mail/example.com/"
format = "Maildir"
index_style = "plain" # or [plain, threaded, compact]
identity="jmct@example.com"
display_name = "José Manuel Calderón Trilla"
subscribed_mailboxes = ["inbox"]

I've tried to minimize the maildir itself, so the output of tree .mail/example.com is below:

[jmct@flip ~]$ tree .mail/example.com      
.mail/example.com
└── inbox
    ├── cur
    │   ├── 1599696809.9819_43489.flip,U=67:2,S
    │   └── 1599696809.9819_43510.flip,U=88:2,S
    ├── new
    └── tmp

Lastly, when using the bcc tool, opensnoop I can see that meli is opening files that seem to make sense: things like .mail/example.com/inbox/cur, etc. (EDIT: It will not let me upload the log file. I will past the whole thing as a comment). One thing that's interesting is that meli attempts to open libgmime-3.0.so.0 a few times (looking in different places) before succeeding:

89458  meli               -1   2 /usr/lib/tls/x86_64/x86_64/libgmime-3.0.so.0
89458  meli               -1   2 /usr/lib/tls/x86_64/libgmime-3.0.so.0
89458  meli               -1   2 /usr/lib/tls/x86_64/libgmime-3.0.so.0
89458  meli               -1   2 /usr/lib/tls/libgmime-3.0.so.0
89458  meli               -1   2 /usr/lib/x86_64/x86_64/libgmime-3.0.so.0
89458  meli               -1   2 /usr/lib/x86_64/libgmime-3.0.so.0
89458  meli               -1   2 /usr/lib/x86_64/libgmime-3.0.so.0
89458  meli                9   0 /usr/lib/libgmime-3.0.so.0

The only difference (that I observed) between the HEAD and the alpha-0.6.1 versions is that the HEAD version also outputs a long UUID looking thing followed by JobRequest::Watch below the loading... prompt in the main window.

I am happy to continue debugging this and helping out, but I'm not sure where to look next.

I would also believe that this is an issue with my environment, but as far as I can tell I have all the dependencies installed.

I think this is a very cool project and I'd like to contribute, so if I've started on the wrong foot, please correct me!

Hello, When I run `meli` it seems unable to load my `maildir` emails. The main window just shows `loading...`, for as long as I let it run (and I've let it run for hours). The behavior is only slightly different between `HEAD` and `alpha-0.6.1` (I'll explain more below). Relevant section of my config: ``` # Setting up a Maildir account [accounts.galois] root_mailbox = "/home/jmct/.mail/example.com/" format = "Maildir" index_style = "plain" # or [plain, threaded, compact] identity="jmct@example.com" display_name = "José Manuel Calderón Trilla" subscribed_mailboxes = ["inbox"] ``` I've tried to minimize the `maildir` itself, so the output of `tree .mail/example.com` is below: ``` [jmct@flip ~]$ tree .mail/example.com .mail/example.com └── inbox ├── cur │ ├── 1599696809.9819_43489.flip,U=67:2,S │ └── 1599696809.9819_43510.flip,U=88:2,S ├── new └── tmp ``` Lastly, when using the `bcc` tool, `opensnoop` I can see that `meli` is opening files that seem to make sense: things like `.mail/example.com/inbox/cur`, etc. (EDIT: It will not let me upload the log file. I will past the whole thing as a comment). One thing that's interesting is that `meli` attempts to open `libgmime-3.0.so.0` a few times (looking in different places) before succeeding: ``` 89458 meli -1 2 /usr/lib/tls/x86_64/x86_64/libgmime-3.0.so.0 89458 meli -1 2 /usr/lib/tls/x86_64/libgmime-3.0.so.0 89458 meli -1 2 /usr/lib/tls/x86_64/libgmime-3.0.so.0 89458 meli -1 2 /usr/lib/tls/libgmime-3.0.so.0 89458 meli -1 2 /usr/lib/x86_64/x86_64/libgmime-3.0.so.0 89458 meli -1 2 /usr/lib/x86_64/libgmime-3.0.so.0 89458 meli -1 2 /usr/lib/x86_64/libgmime-3.0.so.0 89458 meli 9 0 /usr/lib/libgmime-3.0.so.0 ``` The only difference (that I observed) between the `HEAD` and the `alpha-0.6.1` versions is that the `HEAD` version also outputs a long UUID looking thing followed by `JobRequest::Watch` below the `loading...` prompt in the main window. I am happy to continue debugging this and helping out, but I'm not sure where to look next. I would also believe that this is an issue with my environment, but as far as I can tell I have all the dependencies installed. I think this is a very cool project and I'd like to contribute, so if I've started on the wrong foot, please correct me!

jmct commented

2020-09-10 10:34:17 +03:00

Poster

The log file:

[jmct@flip ~]$ cat meli-open.txt 
89458  meli                3   0 /etc/ld.so.cache
89458  meli                3   0 /usr/lib/librt.so.1
89458  meli                3   0 /usr/lib/libdl.so.2
89458  meli                3   0 /usr/lib/libsqlite3.so.0
89458  meli                3   0 /usr/lib/libssl.so.1.1
89458  meli                3   0 /usr/lib/libcrypto.so.1.1
89458  meli                3   0 /usr/lib/libdbus-1.so.3
89458  meli                3   0 /usr/lib/libpthread.so.0
89458  meli                3   0 /usr/lib/libgcc_s.so.1
89458  meli                3   0 /usr/lib/libc.so.6
89458  meli                3   0 /usr/lib/libm.so.6
89458  meli                3   0 /usr/lib/libz.so.1
89458  meli                3   0 /usr/lib/libsystemd.so.0
89458  meli                3   0 /usr/lib/liblzma.so.5
89458  meli                3   0 /usr/lib/libzstd.so.1
89458  meli                3   0 /usr/lib/liblz4.so.1
89458  meli                3   0 
89458  meli                3   0 /usr/lib/libgpg-error.so.0
89458  meli                3   0 /proc/self/maps
89458  meli                9   0 /etc/ld.so.cache
89458  meli                9   0 /usr/lib/libnotmuch.so.5
89458  meli               -1   2 /usr/lib/tls/x86_64/x86_64/libgmime-3.0.so.0
89458  meli               -1   2 /usr/lib/tls/x86_64/libgmime-3.0.so.0
89458  meli               -1   2 /usr/lib/tls/x86_64/libgmime-3.0.so.0
89458  meli               -1   2 /usr/lib/tls/libgmime-3.0.so.0
89458  meli               -1   2 /usr/lib/x86_64/x86_64/libgmime-3.0.so.0
89458  meli               -1   2 /usr/lib/x86_64/libgmime-3.0.so.0
89458  meli               -1   2 /usr/lib/x86_64/libgmime-3.0.so.0
89458  meli                9   0 /usr/lib/libgmime-3.0.so.0
89458  meli                9   0 /usr/lib/libgobject-2.0.so.0
89458  meli                9   0 /usr/lib/libglib-2.0.so.0
89458  meli                9   0 /usr/lib/libtalloc.so.2
89458  meli                9   0 /usr/lib/libxapian.so.30
89458  meli                9   0 /usr/lib/libstdc++.so.6
89458  meli                9   0 /usr/lib/libgio-2.0.so.0
89458  meli                9   0 /usr/lib/libgpgme.so.11
89458  meli                9   0 /usr/lib/libidn2.so.0
89458  meli                9   0 /usr/lib/libffi.so.7
89458  meli                9   0 /usr/lib/libpcre.so.1
89458  meli                9   0 /usr/lib/libuuid.so.1
89458  meli                9   0 /usr/lib/libgmodule-2.0.so.0
89458  meli                9   0 /usr/lib/libmount.so.1
89458  meli                9   0 /usr/lib/libresolv.so.2
89458  meli                9   0 /usr/lib/libassuan.so.0
89458  meli                9   0 /usr/lib/libunistring.so.2
89458  meli                9   0 /usr/lib/libblkid.so.1
89458  meli                9   0 /home/jmct/.config/meli/config.toml
89458  meli               10   0 /proc/self/cgroup
89458  meli               10   0 /proc/self/mountinfo
89458  meli               10   0 /sys/fs/cgroup/cpu,cpuacct/user.slice/cpu.cfs_quota_us
89458  meli-executor-1    12   0 /sys/devices/system/cpu/online
89458  meli               12   0 /home/jmct/.mail/example.com
89458  meli               13   0 /home/jmct/.mail/example.com/inbox
89458  meli               12   0 /home/jmct/.local/share/meli/galois/addressbook
89458  meli               16   0 /home/jmct/.mail/example.com
89458  meli               17   0 /home/jmct/.mail/example.com/inbox
89458  meli               18   0 /home/jmct/.mail/example.com/inbox/new
89458  meli               18   0 /home/jmct/.mail/example.com/inbox/tmp
89458  meli               18   0 /home/jmct/.mail/example.com/inbox/cur
89458  meli               16   0 /home/jmct/.local/share/meli/cmd_history
89458  meli               17   0 /home/jmct/.local/share/meli/galois/addressbook

The log file: ``` [jmct@flip ~]$ cat meli-open.txt 89458 meli 3 0 /etc/ld.so.cache 89458 meli 3 0 /usr/lib/librt.so.1 89458 meli 3 0 /usr/lib/libdl.so.2 89458 meli 3 0 /usr/lib/libsqlite3.so.0 89458 meli 3 0 /usr/lib/libssl.so.1.1 89458 meli 3 0 /usr/lib/libcrypto.so.1.1 89458 meli 3 0 /usr/lib/libdbus-1.so.3 89458 meli 3 0 /usr/lib/libpthread.so.0 89458 meli 3 0 /usr/lib/libgcc_s.so.1 89458 meli 3 0 /usr/lib/libc.so.6 89458 meli 3 0 /usr/lib/libm.so.6 89458 meli 3 0 /usr/lib/libz.so.1 89458 meli 3 0 /usr/lib/libsystemd.so.0 89458 meli 3 0 /usr/lib/liblzma.so.5 89458 meli 3 0 /usr/lib/libzstd.so.1 89458 meli 3 0 /usr/lib/liblz4.so.1 89458 meli 3 0 89458 meli 3 0 /usr/lib/libgpg-error.so.0 89458 meli 3 0 /proc/self/maps 89458 meli 9 0 /etc/ld.so.cache 89458 meli 9 0 /usr/lib/libnotmuch.so.5 89458 meli -1 2 /usr/lib/tls/x86_64/x86_64/libgmime-3.0.so.0 89458 meli -1 2 /usr/lib/tls/x86_64/libgmime-3.0.so.0 89458 meli -1 2 /usr/lib/tls/x86_64/libgmime-3.0.so.0 89458 meli -1 2 /usr/lib/tls/libgmime-3.0.so.0 89458 meli -1 2 /usr/lib/x86_64/x86_64/libgmime-3.0.so.0 89458 meli -1 2 /usr/lib/x86_64/libgmime-3.0.so.0 89458 meli -1 2 /usr/lib/x86_64/libgmime-3.0.so.0 89458 meli 9 0 /usr/lib/libgmime-3.0.so.0 89458 meli 9 0 /usr/lib/libgobject-2.0.so.0 89458 meli 9 0 /usr/lib/libglib-2.0.so.0 89458 meli 9 0 /usr/lib/libtalloc.so.2 89458 meli 9 0 /usr/lib/libxapian.so.30 89458 meli 9 0 /usr/lib/libstdc++.so.6 89458 meli 9 0 /usr/lib/libgio-2.0.so.0 89458 meli 9 0 /usr/lib/libgpgme.so.11 89458 meli 9 0 /usr/lib/libidn2.so.0 89458 meli 9 0 /usr/lib/libffi.so.7 89458 meli 9 0 /usr/lib/libpcre.so.1 89458 meli 9 0 /usr/lib/libuuid.so.1 89458 meli 9 0 /usr/lib/libgmodule-2.0.so.0 89458 meli 9 0 /usr/lib/libmount.so.1 89458 meli 9 0 /usr/lib/libresolv.so.2 89458 meli 9 0 /usr/lib/libassuan.so.0 89458 meli 9 0 /usr/lib/libunistring.so.2 89458 meli 9 0 /usr/lib/libblkid.so.1 89458 meli 9 0 /home/jmct/.config/meli/config.toml 89458 meli 10 0 /proc/self/cgroup 89458 meli 10 0 /proc/self/mountinfo 89458 meli 10 0 /sys/fs/cgroup/cpu,cpuacct/user.slice/cpu.cfs_quota_us 89458 meli-executor-1 12 0 /sys/devices/system/cpu/online 89458 meli 12 0 /home/jmct/.mail/example.com 89458 meli 13 0 /home/jmct/.mail/example.com/inbox 89458 meli 12 0 /home/jmct/.local/share/meli/galois/addressbook 89458 meli 16 0 /home/jmct/.mail/example.com 89458 meli 17 0 /home/jmct/.mail/example.com/inbox 89458 meli 18 0 /home/jmct/.mail/example.com/inbox/new 89458 meli 18 0 /home/jmct/.mail/example.com/inbox/tmp 89458 meli 18 0 /home/jmct/.mail/example.com/inbox/cur 89458 meli 16 0 /home/jmct/.local/share/meli/cmd_history 89458 meli 17 0 /home/jmct/.local/share/meli/galois/addressbook ```

Manos Pitsidianakis commented

2020-09-10 15:53:40 +03:00

Thank you very much for taking the time for this bug report!

You probably have a notmuch account configured as well, this is why libgmime is called: it's called by libnotmuch. As for the hang, nothing comes to mind at the moment. Would you be willing to compile current HEAD with the feature debug-tracing enabled? Then you can run meli as follows:

RUST_BACKTRACE=1 cargo run 2> trace.log

The file trace.log will be all stderr output from meli. I suspect there's a panic or deadlock somewhere in the maildir backend. There will probably be sensitive info in the log that you want to remove first. You might want to comment out other accounts in the config so that they don't pollute the log with irrelevant info.

Thank you very much for taking the time for this bug report! You probably have a notmuch account configured as well, this is why libgmime is called: it's called by libnotmuch. As for the hang, nothing comes to mind at the moment. Would you be willing to compile current HEAD with the feature `debug-tracing` enabled? Then you can run meli as follows: ```shell RUST_BACKTRACE=1 cargo run 2> trace.log ``` The file trace.log will be all `stderr` output from meli. I suspect there's a panic or deadlock somewhere in the maildir backend. There will probably be sensitive info in the log that you want to remove first. You might want to comment out other accounts in the config so that they don't pollute the log with irrelevant info.

Manos Pitsidianakis added the

bug

label 2020-09-10 15:53:54 +03:00

jmct commented

2020-09-12 03:12:39 +03:00

Poster

Hello again,

It looks like almost no debugging information is being printed. This is after I've run the following:

cargo build --features="debug-trace"

and then

RUST_BACKTRACE=1 cargo run 2> trace.log

The following is my trace.log:

    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/meli`

Which seems... incomplete. I'm definitely currently on HEAD (commit 4829e13c).

Also, to be very clear, the application is responsive, I can bring up the help menu with ? or bring up the mail compose screen, etc. It just never loads the emails in the maildir.

I'm poking around to see if I can figure out where it's going wrong.
This is probably all good practice for me anyway as it's forcing me to take a tour of the codebase :)

And just so I'm certain, the docs here do not reflect the organization at HEAD, correct? I should build the rustdocs myself for that?

Hello again, It looks like almost no debugging information is being printed. This is after I've run the following: ``` cargo build --features="debug-trace" ``` and then ``` RUST_BACKTRACE=1 cargo run 2> trace.log ``` The following is my `trace.log`: ``` Finished dev [unoptimized + debuginfo] target(s) in 0.07s Running `target/debug/meli` ``` Which seems... incomplete. I'm definitely currently on `HEAD` (commit `4829e13c`). Also, to be very clear, the application is responsive, I can bring up the help menu with `?` or bring up the mail compose screen, etc. It just never loads the emails in the maildir. I'm poking around to see if I can figure out where it's going wrong. This is probably all good practice for me anyway as it's forcing me to take a tour of the codebase :) And just so I'm certain, the docs [here](https://meli.delivery/rustdoc/meli/index.html) do not reflect the organization at `HEAD`, correct? I should build the rustdocs myself for that?

Manos Pitsidianakis commented

2020-09-12 16:24:16 +03:00

RUST_BACKTRACE=1 cargo run 2> trace.log

This should build and run the project with the default features. Try

RUST_BACKTRACE=1 cargo run --features=debug-tracing 2> trace.log

(note that the feature is debug-tracing not debug-trace)

I'm poking around to see if I can figure out where it's going wrong.
This is probably all good practice for me anyway as it's forcing me to take a tour of the codebase :)

And just so I'm certain, the docs here do not reflect the organization at HEAD, correct? I should build the rustdocs myself for that?

Yeah these are out of date, I think. If you build the docs yourself remember to include the flag --document-private-items. If you need any more help you can also join #meli on freenode!

> ``` > RUST_BACKTRACE=1 cargo run 2> trace.log > ``` This should build and run the project with the default features. Try > ``` > RUST_BACKTRACE=1 cargo run --features=debug-tracing 2> trace.log > ``` (note that the feature is `debug-tracing` not `debug-trace`) > > I'm poking around to see if I can figure out where it's going wrong. > This is probably all good practice for me anyway as it's forcing me to take a tour of the codebase :) > > And just so I'm certain, the docs [here](https://meli.delivery/rustdoc/meli/index.html) do not reflect the organization at `HEAD`, correct? I should build the rustdocs myself for that? Yeah these are out of date, I think. If you build the docs yourself remember to include the flag `--document-private-items`. If you need any more help you can also join #meli on freenode!

Manos Pitsidianakis commented

2020-09-12 17:54:39 +03:00

Ok, I just re-read your first post and noticed something.

root_mailbox = "/home/jmct/.mail/example.com/"

This folder includes only the folder "inbox" and is not a maildir folder itself. I think what happens is that you don't endup with any subscribed folders since inbox =/= example.com/inbox but there's no error/warning shown to you, but there should be. I assume there's a line in ~/.local/meli/meli.log telling you that inbox was configured but not found.

In this case, the problem here is not showing a notice to the user and to stop showing "loading..." when there's nothing to load.

Ok, I just re-read your first post and noticed something. > root_mailbox = "/home/jmct/.mail/example.com/" This folder includes only the folder "inbox" and is not a maildir folder itself. I think what happens is that you don't endup with any subscribed folders since `inbox` =/= `example.com/inbox` but there's no error/warning shown to you, but there should be. I assume there's a line in `~/.local/meli/meli.log` telling you that `inbox` was configured but not found. In this case, the problem here is not showing a notice to the user and to stop showing "loading..." when there's nothing to load.

jmct commented

2020-09-12 22:36:59 +03:00

Poster

Hey again,

So I can confirm that changing subscribed_mailboxes = ["inbox"] to subscribed_mailboxes = ["example.com/inbox"] does the trick.

This is a bit different to how I had mutt configured, which is I think what led to the confusion. e.g.:

set mbox_type=Maildir
set folder=~/.mail/example.com/
set spoolfile=+/inbox
set postponed=+/drafts

So first of all, thanks for your help!

Secondly, I will continue doing my exploration of the codebase in order to see if I can submit a patch that addresses the issue as you now describe it: that meli should state that it has nothing to load.

Thanks again for your patience and guidance!

Hey again, So I can confirm that changing `subscribed_mailboxes = ["inbox"]` to `subscribed_mailboxes = ["example.com/inbox"]` does the trick. This is a bit different to how I had `mutt` configured, which is I think what led to the confusion. e.g.: ``` set mbox_type=Maildir set folder=~/.mail/example.com/ set spoolfile=+/inbox set postponed=+/drafts ``` So first of all, thanks for your help! Secondly, I will continue doing my exploration of the codebase in order to see if I can submit a patch that addresses the issue as you now describe it: that meli should state that it has nothing to load. Thanks again for your patience and guidance!

Manos Pitsidianakis commented

2020-09-12 23:33:48 +03:00

Thank you as well for spending time with alpha software :)

The relevant code that checks the subscribed mailboxes is here:

https://git.meli.delivery/meli/meli/src/branch/master/src/conf/accounts.rs#L527

There can be a single check there if mailbox_entries is empty, and in that case log/show an error.

Thank you as well for spending time with alpha software :) The relevant code that checks the subscribed mailboxes is here: https://git.meli.delivery/meli/meli/src/branch/master/src/conf/accounts.rs#L527 There can be a single check there if `mailbox_entries` is empty, and in that case log/show an error.

Manos Pitsidianakis closed this issue

2020-10-16 15:24:03 +03:00

Sign in to join this conversation.