[Barrelfish-users] Flux control in Barrelfish

[Baumann Andrew]

Hi Zeus,

The first answer I would give is to make sure you handle the incoming messages on core A, but I understand that you may not have control over this.

It sounds like you've run into the problem that the two halves of a UMP channel are not independent: you cannot keep sending messages only in one direction without handling messages in the reverse direction, because the ACKs get piggy-backed on messages in the same channel. This is a nice optimisation for many use-cases, but it's unfortunate that it gets exposed to the user in this way, and it should really be fixed in the UMP implementation. The workaround THC currently uses is to have two underlying channels for each connection: one for each direction. This isolates sending and receiving and gives you true independent flow control, at the expense of some resource overhead for the extra channel.

BTW, Note that using more threads doesn't change anything: the queuing problem is on the channel structure itself.

Cheers,
Andrew

From: Zeus Gómez Marmolejo [mailto:zeus.gomez at bsc.es]
Sent: Tuesday, 20 December, 2011 10:29
To: barrelfish-users at lists.inf.ethz.ch
Subject: [Barrelfish-users] Flux control in Barrelfish

Hi,

I'd like to ask you a question. Suppose we have core A always sending messages to core B. And core B is replying all messages back to core A. Core A never sends a new message before the previous one has not been sent (via continuation closure). But, reply message that B sends to A is much bigger than the one A sends to B. So when the 2nd reply message is to be sent from core B it always gets the FLOUNDER_ERR_TX_BUSY.

So my question is if there is a way to do a message flux control between the two cores. I understand that core B cannot send back a message if the previous reply has not been completely sent. But on the other side, the message handler cannot block and wait, as all the message processing stalls. I also tried to queue all replies using 2 threads to not block the message processing thread, sending them back when the previous one has been sent, but finally my reply queue ends up filling completely.

So what I want is that core A stop sending so many messages controlling the sending flux. Is there a way to do it with flounder? Which ways do you recommend me to do this?


Many thanks!!

--
Zeus Gómez Marmolejo
Barcelona Supercomputing Center
PhD student
http://www.bsc.es

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.inf.ethz.ch/pipermail/barrelfish-users/attachments/20111220/55e8b4ff/attachment.html