Can't build on macOS #46

Closed
opened 1 year ago by JD · 6 comments
JD commented 1 year ago

macOS: 10.11
Rust: 1.44.1

Fails while building like this:

Compiling rmpv v0.4.4
warning: unused import: `std::os::unix::io::AsRawFd`
   --> melib/src/lib.rs:148:9
    |
148 |     use std::os::unix::io::AsRawFd;
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: `#[warn(unused_imports)]` on by default

warning: 1 warning emitted

error[E0432]: unresolved import `libc::itimerspec`
  --> src/unix.rs:43:16
   |
43 |     use libc::{itimerspec, timespec};
   |                ^^^^^^^^^^
   |                |
   |                no `itimerspec` in the root
   |                help: a similar name exists in the module: `timespec`

error[E0432]: unresolved import `libc::itimerspec`
  --> src/unix.rs:43:16
   |
43 |     use libc::{itimerspec, timespec};
   |                ^^^^^^^^^^
   |                |
   |                no `itimerspec` in the root
   |                help: a similar name exists in the module: `timespec`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0432`.
error: could not compile `meli`.

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: aborting due to previous error

For more information about this error, try `rustc --explain E0432`.
error: could not compile `meli`.

To learn more, run the command again with --verbose.
make: *** [meli] Error 101
macOS: 10.11 Rust: 1.44.1 Fails while building like this: Compiling rmpv v0.4.4 warning: unused import: `std::os::unix::io::AsRawFd` --> melib/src/lib.rs:148:9 | 148 | use std::os::unix::io::AsRawFd; | ^^^^^^^^^^^^^^^^^^^^^^^^^^ | = note: `#[warn(unused_imports)]` on by default warning: 1 warning emitted error[E0432]: unresolved import `libc::itimerspec` --> src/unix.rs:43:16 | 43 | use libc::{itimerspec, timespec}; | ^^^^^^^^^^ | | | no `itimerspec` in the root | help: a similar name exists in the module: `timespec` error[E0432]: unresolved import `libc::itimerspec` --> src/unix.rs:43:16 | 43 | use libc::{itimerspec, timespec}; | ^^^^^^^^^^ | | | no `itimerspec` in the root | help: a similar name exists in the module: `timespec` error: aborting due to previous error For more information about this error, try `rustc --explain E0432`. error: could not compile `meli`. To learn more, run the command again with --verbose. warning: build failed, waiting for other jobs to finish... error: aborting due to previous error For more information about this error, try `rustc --explain E0432`. error: could not compile `meli`. To learn more, run the command again with --verbose. make: *** [meli] Error 101
Owner

Thank you for your post! Unfortunately I don't have any access in any OSX machine in order to fix these things :/ If I manage to get an OSX vm running on qemu, I will look into it.

Thank you for your post! Unfortunately I don't have any access in any OSX machine in order to fix these things :/ If I manage to get an OSX vm running on qemu, I will look into it.

I also ran into this same exact problem trying to build meli from source on macOS. I have access to the platform and I have some rust experience. If you can give me some hints on where to start, I can try to look into it.

Here is my toolchain:

$ rustup show
Default host: x86_64-apple-darwin
rustup home:  /Users/bfj/.rustup

stable-x86_64-apple-darwin (default)
rustc 1.45.1 (c367798cf 2020-07-26)
I also ran into this same exact problem trying to build meli from source on macOS. I have access to the platform and I have some rust experience. If you can give me some hints on where to start, I can try to look into it. Here is my toolchain: ``` $ rustup show Default host: x86_64-apple-darwin rustup home: /Users/bfj/.rustup stable-x86_64-apple-darwin (default) rustc 1.45.1 (c367798cf 2020-07-26) ```
Owner

Hello,

The problem is that OSX lacks the POSIX timer extension. This can be bypassed by using async I/O timers which use OSX's kqueue mechanism

meli uses the smol runtime, which has an async timer type that from what it says works on OSX.

The POSIX timers issue a signal with the timer's id in the signal data. This is read in the signal handler in src/bin.rs. The signal handler uses the self-pipe trick: it writes a byte (the timer's id) in a pipe to inform a thread in a signal-safe way that a timer fired. That thread's job is firing a wakeup event every X milliseconds and also send timer/signal events to the main process. This is all necessary because UNIX signals are not a very safe interface, they can interrupt a process's execution at any time more or less.

Async tasks are spawned by the spawn_ functions in the JobExecutor struct. An async timer could work by spawning it with a special spawn_timer function that spawns an async task that instead of sending a JobFinished event, sends a Timer event with the timer's id:

This is how a regular job is spawned:

        let (task, handle) = async_task::spawn(
            async move {
                let r = future.await;
                finished_sender
                    .send(ThreadEvent::JobFinished(__job_id))
                    .unwrap();
                r
            },
            move |task| injector.push(MeliTask { task, id: _job_id }),
            (),
        );

This is how a timer could be spawned:

        let (task, handle) = async_task::spawn(
            async move {
                let r = timer.await;
                finished_sender
                    .send(ThreadEvent::UIEvent(UIEvent::Timer(timer.id)))
                    .unwrap();
                /* IMPORTANT: here, the timer must be re-armed if it has a non-zero
                   interval, which means it is a periodic timer and not oneshot*/
            },
            move |task| injector.push(MeliTask { task, id: _job_id }),
            (),
        );

The only unclear issue is re-arming the timer. Feel free to ask more questions, there are a lot of details that must be considered, but I think this is the only difficulty.

Hello, The problem is that OSX lacks the POSIX timer extension. This can be bypassed by using async I/O timers which use OSX's kqueue mechanism meli uses the smol runtime, which has an async timer type that from what it says works on OSX. The POSIX timers issue a signal with the timer's id in the signal data. This is read in the signal handler in [src/bin.rs](https://git.meli.delivery/meli/meli/src/commit/522f6673501b5f5edd89bfea1dde64ef53927579/src/bin.rs#L98). The signal handler uses the self-pipe trick: it writes a byte (the timer's id) in a pipe to [inform a thread](https://git.meli.delivery/meli/meli/src/commit/522f6673501b5f5edd89bfea1dde64ef53927579/src/bin.rs#L125) in a signal-safe way that a timer fired. That thread's job is firing a wakeup event every X milliseconds and also send timer/signal events to the main process. This is all necessary because UNIX signals are not a very safe interface, they can interrupt a process's execution at any time more or less. Async tasks are spawned [by the `spawn_` functions in the JobExecutor struct](https://git.meli.delivery/meli/meli/src/commit/522f6673501b5f5edd89bfea1dde64ef53927579/src/jobs.rs#L155). An async timer could work by spawning it with a special `spawn_timer` function that spawns an async task that instead of sending a `JobFinished` event, sends a `Timer` event with the timer's id: This is how a regular job is spawned: ```rust let (task, handle) = async_task::spawn( async move { let r = future.await; finished_sender .send(ThreadEvent::JobFinished(__job_id)) .unwrap(); r }, move |task| injector.push(MeliTask { task, id: _job_id }), (), ); ``` This is how a timer could be spawned: ```rust let (task, handle) = async_task::spawn( async move { let r = timer.await; finished_sender .send(ThreadEvent::UIEvent(UIEvent::Timer(timer.id))) .unwrap(); /* IMPORTANT: here, the timer must be re-armed if it has a non-zero interval, which means it is a periodic timer and not oneshot*/ }, move |task| injector.push(MeliTask { task, id: _job_id }), (), ); ``` The only unclear issue is re-arming the timer. Feel free to ask more questions, there are a lot of details that must be considered, but I think this is the only difficulty.
Owner

@benjaminfjones I was looking to day at timers again, and found this:

https://stackoverflow.com/a/44814430 I believe the dispatch interface can be called from rust, indeed there's a crate that does just that but doesn't implement the timer interfaces: https://docs.rs/dispatch/0.1.2/src/dispatch/ffi.rs.html

This means there's no need for async timers, the POSIX real time signal timer can be replaced with a timer event with Dispatch. There are two possible options here, implement it in meli (which will require defining the FFI function signatures in a rust module and call the API with unsafe) or implement it in the dispatch crate and contribute a patch there. Unfortunatelly I still don't have access to an OSX machine, but I'm 100% willing to help/mentor/assist anyone willing to look into this. Either post here, send an e-mail or come to #meli on freenode

@benjaminfjones I was looking to day at timers again, and found this: https://stackoverflow.com/a/44814430 I believe the dispatch interface can be called from rust, indeed there's a crate that does just that but doesn't implement the timer interfaces: https://docs.rs/dispatch/0.1.2/src/dispatch/ffi.rs.html This means there's no need for async timers, the POSIX real time signal timer can be replaced with a timer event with Dispatch. There are two possible options here, implement it in meli (which will require defining the FFI function signatures in a rust module and call the API with unsafe) or implement it in the [`dispatch`](https://github.com/SSheldon/rust-dispatch) crate and contribute a patch there. Unfortunatelly I still don't have access to an OSX machine, but I'm 100% willing to help/mentor/assist anyone willing to look into this. Either post here, send an e-mail or come to `#meli` on freenode
Owner

Oh, and I guess a least effort solution is to spawn a new thread for each timer and do a loop { sleep(); send_event(); }

Resource:

https://medium.com/programming-servo/programming-servo-the-incredibly-shrinking-timer-7283ae2a2669

Oh, and I guess a least effort solution is to spawn a new thread for each timer and do a `loop { sleep(); send_event(); }` Resource: https://medium.com/programming-servo/programming-servo-the-incredibly-shrinking-timer-7283ae2a2669
Owner

Posix timers were removed in 6392904047 , I'm closing this issue but if OSX build still fails please reopen the issue.

Posix timers were removed in 6392904047919a8f6f4a3bbae3d371037f840300 , I'm closing this issue but if OSX build still fails please reopen the issue.
epilys closed this issue 12 months ago
Sign in to join this conversation.
No Milestone
No project
No Assignees
3 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Loading…
There is no content yet.