Can't build on macOS #74

New Issue

JD · 2020-07-07T06:37:05+03:00

JD commented

2020-07-07 06:37:05 +03:00

macOS: 10.11
Rust: 1.44.1

Fails while building like this:

Compiling rmpv v0.4.4
warning: unused import: `std::os::unix::io::AsRawFd`
   --> melib/src/lib.rs:148:9
    |
148 |     use std::os::unix::io::AsRawFd;
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: `#[warn(unused_imports)]` on by default

warning: 1 warning emitted

error[E0432]: unresolved import `libc::itimerspec`
  --> src/unix.rs:43:16
   |
43 |     use libc::{itimerspec, timespec};
   |                ^^^^^^^^^^
   |                |
   |                no `itimerspec` in the root
   |                help: a similar name exists in the module: `timespec`

error[E0432]: unresolved import `libc::itimerspec`
  --> src/unix.rs:43:16
   |
43 |     use libc::{itimerspec, timespec};
   |                ^^^^^^^^^^
   |                |
   |                no `itimerspec` in the root
   |                help: a similar name exists in the module: `timespec`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0432`.
error: could not compile `meli`.

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: aborting due to previous error

For more information about this error, try `rustc --explain E0432`.
error: could not compile `meli`.

To learn more, run the command again with --verbose.
make: *** [meli] Error 101

macOS: 10.11 Rust: 1.44.1 Fails while building like this: Compiling rmpv v0.4.4 warning: unused import: `std::os::unix::io::AsRawFd` --> melib/src/lib.rs:148:9 | 148 | use std::os::unix::io::AsRawFd; | ^^^^^^^^^^^^^^^^^^^^^^^^^^ | = note: `#[warn(unused_imports)]` on by default warning: 1 warning emitted error[E0432]: unresolved import `libc::itimerspec` --> src/unix.rs:43:16 | 43 | use libc::{itimerspec, timespec}; | ^^^^^^^^^^ | | | no `itimerspec` in the root | help: a similar name exists in the module: `timespec` error[E0432]: unresolved import `libc::itimerspec` --> src/unix.rs:43:16 | 43 | use libc::{itimerspec, timespec}; | ^^^^^^^^^^ | | | no `itimerspec` in the root | help: a similar name exists in the module: `timespec` error: aborting due to previous error For more information about this error, try `rustc --explain E0432`. error: could not compile `meli`. To learn more, run the command again with --verbose. warning: build failed, waiting for other jobs to finish... error: aborting due to previous error For more information about this error, try `rustc --explain E0432`. error: could not compile `meli`. To learn more, run the command again with --verbose. make: *** [meli] Error 101

Manos Pitsidianakis commented

2020-07-09 09:28:04 +03:00

Thank you for your post! Unfortunately I don't have any access in any OSX machine in order to fix these things :/ If I manage to get an OSX vm running on qemu, I will look into it.

benjaminfjones commented

2020-08-01 21:00:14 +03:00

I also ran into this same exact problem trying to build meli from source on macOS. I have access to the platform and I have some rust experience. If you can give me some hints on where to start, I can try to look into it.

Here is my toolchain:

$ rustup show
Default host: x86_64-apple-darwin
rustup home:  /Users/bfj/.rustup

stable-x86_64-apple-darwin (default)
rustc 1.45.1 (c367798cf 2020-07-26)

I also ran into this same exact problem trying to build meli from source on macOS. I have access to the platform and I have some rust experience. If you can give me some hints on where to start, I can try to look into it. Here is my toolchain: ``` $ rustup show Default host: x86_64-apple-darwin rustup home: /Users/bfj/.rustup stable-x86_64-apple-darwin (default) rustc 1.45.1 (c367798cf 2020-07-26) ```

👍 1

Manos Pitsidianakis commented

2020-08-01 21:36:51 +03:00

Hello,

The problem is that OSX lacks the POSIX timer extension. This can be bypassed by using async I/O timers which use OSX's kqueue mechanism

meli uses the smol runtime, which has an async timer type that from what it says works on OSX.

The POSIX timers issue a signal with the timer's id in the signal data. This is read in the signal handler in src/bin.rs. The signal handler uses the self-pipe trick: it writes a byte (the timer's id) in a pipe to inform a thread in a signal-safe way that a timer fired. That thread's job is firing a wakeup event every X milliseconds and also send timer/signal events to the main process. This is all necessary because UNIX signals are not a very safe interface, they can interrupt a process's execution at any time more or less.

Async tasks are spawned by the spawn_ functions in the JobExecutor struct. An async timer could work by spawning it with a special spawn_timer function that spawns an async task that instead of sending a JobFinished event, sends a Timer event with the timer's id:

This is how a regular job is spawned:

        let (task, handle) = async_task::spawn(
            async move {
                let r = future.await;
                finished_sender
                    .send(ThreadEvent::JobFinished(__job_id))
                    .unwrap();
                r
            },
            move |task| injector.push(MeliTask { task, id: _job_id }),
            (),
        );

This is how a timer could be spawned:

        let (task, handle) = async_task::spawn(
            async move {
                let r = timer.await;
                finished_sender
                    .send(ThreadEvent::UIEvent(UIEvent::Timer(timer.id)))
                    .unwrap();
                /* IMPORTANT: here, the timer must be re-armed if it has a non-zero
                   interval, which means it is a periodic timer and not oneshot*/
            },
            move |task| injector.push(MeliTask { task, id: _job_id }),
            (),
        );

The only unclear issue is re-arming the timer. Feel free to ask more questions, there are a lot of details that must be considered, but I think this is the only difficulty.

Hello, The problem is that OSX lacks the POSIX timer extension. This can be bypassed by using async I/O timers which use OSX's kqueue mechanism meli uses the smol runtime, which has an async timer type that from what it says works on OSX. The POSIX timers issue a signal with the timer's id in the signal data. This is read in the signal handler in [src/bin.rs](https://git.meli.delivery/meli/meli/src/commit/522f6673501b5f5edd89bfea1dde64ef53927579/src/bin.rs#L98). The signal handler uses the self-pipe trick: it writes a byte (the timer's id) in a pipe to [inform a thread](https://git.meli.delivery/meli/meli/src/commit/522f6673501b5f5edd89bfea1dde64ef53927579/src/bin.rs#L125) in a signal-safe way that a timer fired. That thread's job is firing a wakeup event every X milliseconds and also send timer/signal events to the main process. This is all necessary because UNIX signals are not a very safe interface, they can interrupt a process's execution at any time more or less. Async tasks are spawned [by the `spawn_` functions in the JobExecutor struct](https://git.meli.delivery/meli/meli/src/commit/522f6673501b5f5edd89bfea1dde64ef53927579/src/jobs.rs#L155). An async timer could work by spawning it with a special `spawn_timer` function that spawns an async task that instead of sending a `JobFinished` event, sends a `Timer` event with the timer's id: This is how a regular job is spawned: ```rust let (task, handle) = async_task::spawn( async move { let r = future.await; finished_sender .send(ThreadEvent::JobFinished(__job_id)) .unwrap(); r }, move |task| injector.push(MeliTask { task, id: _job_id }), (), ); ``` This is how a timer could be spawned: ```rust let (task, handle) = async_task::spawn( async move { let r = timer.await; finished_sender .send(ThreadEvent::UIEvent(UIEvent::Timer(timer.id))) .unwrap(); /* IMPORTANT: here, the timer must be re-armed if it has a non-zero interval, which means it is a periodic timer and not oneshot*/ }, move |task| injector.push(MeliTask { task, id: _job_id }), (), ); ``` The only unclear issue is re-arming the timer. Feel free to ask more questions, there are a lot of details that must be considered, but I think this is the only difficulty.

Manos Pitsidianakis commented

2020-09-18 17:06:54 +03:00

@benjaminfjones I was looking to day at timers again, and found this:

https://stackoverflow.com/a/44814430 I believe the dispatch interface can be called from rust, indeed there's a crate that does just that but doesn't implement the timer interfaces: https://docs.rs/dispatch/0.1.2/src/dispatch/ffi.rs.html

This means there's no need for async timers, the POSIX real time signal timer can be replaced with a timer event with Dispatch. There are two possible options here, implement it in meli (which will require defining the FFI function signatures in a rust module and call the API with unsafe) or implement it in the dispatch crate and contribute a patch there. Unfortunatelly I still don't have access to an OSX machine, but I'm 100% willing to help/mentor/assist anyone willing to look into this. Either post here, send an e-mail or come to #meli on freenode

@benjaminfjones I was looking to day at timers again, and found this: https://stackoverflow.com/a/44814430 I believe the dispatch interface can be called from rust, indeed there's a crate that does just that but doesn't implement the timer interfaces: https://docs.rs/dispatch/0.1.2/src/dispatch/ffi.rs.html This means there's no need for async timers, the POSIX real time signal timer can be replaced with a timer event with Dispatch. There are two possible options here, implement it in meli (which will require defining the FFI function signatures in a rust module and call the API with unsafe) or implement it in the [`dispatch`](https://github.com/SSheldon/rust-dispatch) crate and contribute a patch there. Unfortunatelly I still don't have access to an OSX machine, but I'm 100% willing to help/mentor/assist anyone willing to look into this. Either post here, send an e-mail or come to `#meli` on freenode

Manos Pitsidianakis commented

2020-09-18 17:29:31 +03:00

Oh, and I guess a least effort solution is to spawn a new thread for each timer and do a loop { sleep(); send_event(); }

Resource:

https://medium.com/programming-servo/programming-servo-the-incredibly-shrinking-timer-7283ae2a2669

Oh, and I guess a least effort solution is to spawn a new thread for each timer and do a `loop { sleep(); send_event(); }` Resource: https://medium.com/programming-servo/programming-servo-the-incredibly-shrinking-timer-7283ae2a2669

Manos Pitsidianakis commented

2020-10-29 14:19:34 +02:00

Posix timers were removed in 6392904047 , I'm closing this issue but if OSX build still fails please reopen the issue.

Posix timers were removed in 6392904047919a8f6f4a3bbae3d371037f840300 , I'm closing this issue but if OSX build still fails please reopen the issue.

Manos Pitsidianakis closed this issue

2020-10-29 14:19:35 +02:00

Sign in to join this conversation.