[Barrelfish-users] [Barrelfish] Messages in Barrelfish

[Andrew Baumann]

Hi Zeus,

> I made a very simple example where core 0 is sending n messages to core 1
> using the UMP backend. Core 1 is just waiting and dispatching messages.
> 
> So the problem is that I can not manage to send more than 31 messages to
> core 1. The message 32 is sent but core 1 doesn't receive it. The error I get is:
> 
> Failure: (       flounder) Cannot queue message for transmit: queue is full
> [FLOUNDER_ERR_TX_BUSY]
> 
> In the meantime all other 31 messages have been dispatched and received
> by core 1 but the sending error keeps on, so no more messages can be sent.
> Message 32 seems to be on queue but never dispatched.

Are you processing the incoming event queue on core 0, by dispatching the waitset associated with the channel?

If your program is single-threaded, then a loop like while (1) send_message(); won't work, because you also need to dispatch events on the incoming waitset in order for the UMP channel implementation to see the acks which tell it that the receiver has consumed the messages. The TX_BUSY error you're seeing is the channel implementation telling you (on core 0) that there is no space in the outgoing message queue.

The correct thing to do in this case depends on your program, but assuming you want to use the single-threaded state-machine model, the right thing to do is to register a callback for the ability to send further messages (e.g. with b->register_send()), and then return to your event processing loop which should free up space in the outgoing message queue and then run your callback. You might want to have a look at the idctest program in the tree for some examples of how to use callbacks and continuations for sending.

If this all sounds very painful and low-level, that's because it is :) Have you considered using a higher-level abstraction like THC?

Cheers,
Andrew