Hi Andrew,<div><br></div><div>Thank you for your suggestion. Two channels per connection is a very useful idea. I was thinking how to solve it, and I came up that it could be solved if the two channels are served using different waitsets. Then the receiving waitset on B can block if the transmitting channel on B is busy. This way, ACKs could be served anyway on the transmitting channel. But to do that, you need 2 threads per core. Because you need to block the handler executed on B so the ACK is not sent back to A and prevent him from sending more messages.</div>
<div><br></div><div>This has a drawback. What happens if while the receiving waitset on B is blocked, there is another node named C that wants to send a message to B. It's not being served because the receiving waitset on B is blocked to avoid the ACK back to A. So no more nodes can communicate to B.</div>
<div><br></div><div>Is there a way to send back an event to the waitset if you cannot process it in this moment?</div><div><br></div><div><br></div><div>Thanks again for your advise,</div><div><br></div><div>Zeus.</div><div>
<br><br><div class="gmail_quote">El 20 de diciembre de 2011 19:44, Baumann Andrew <span dir="ltr"><<a href="mailto:andrewb@inf.ethz.ch">andrewb@inf.ethz.ch</a>></span> escribió:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-AU" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi Zeus,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">The first answer I would give is to make sure you handle the incoming messages on core A, but I understand that you may not have control over this.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">It sounds like you’ve run into the problem that the two halves of a UMP channel are not independent: you cannot keep sending messages only in one direction
without handling messages in the reverse direction, because the ACKs get piggy-backed on messages in the same channel. This is a nice optimisation for many use-cases, but it’s unfortunate that it gets exposed to the user in this way, and it should really be
fixed in the UMP implementation. The workaround THC currently uses is to have two underlying channels for each connection: one for each direction. This isolates sending and receiving and gives you true independent flow control, at the expense of some resource
overhead for the extra channel.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">BTW, Note that using more threads doesn’t change anything: the queuing problem is on the channel structure itself.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Cheers,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Andrew<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Zeus Gómez Marmolejo [mailto:<a href="mailto:zeus.gomez@bsc.es" target="_blank">zeus.gomez@bsc.es</a>]
<br>
<b>Sent:</b> Tuesday, 20 December, 2011 10:29<br>
<b>To:</b> <a href="mailto:barrelfish-users@lists.inf.ethz.ch" target="_blank">barrelfish-users@lists.inf.ethz.ch</a><br>
<b>Subject:</b> [Barrelfish-users] Flux control in Barrelfish<u></u><u></u></span></p><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Hi,<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I'd like to ask you a question. Suppose we have core A always sending messages to core B. And core B is replying all messages back to core A. Core A never sends a new message before the previous one has not been sent (via continuation closure).
But, reply message that B sends to A is much bigger than the one A sends to B. So when the 2nd reply message is to be sent from core B it always gets the FLOUNDER_ERR_TX_BUSY.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">So my question is if there is a way to do a message flux control between the two cores. I understand that core B cannot send back a message if the previous reply has not been completely sent. But on the other side, the message handler cannot
block and wait, as all the message processing stalls. I also tried to queue all replies using 2 threads to not block the message processing thread, sending them back when the previous one has been sent, but finally my reply queue ends up filling completely.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">So what I want is that core A stop sending so many messages controlling the sending flux. Is there a way to do it with flounder? Which ways do you recommend me to do this?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Many thanks!!<u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">-- <br>
Zeus Gómez Marmolejo<br>
Barcelona Supercomputing Center<br>
PhD student<br>
<a href="http://www.bsc.es" target="_blank">http://www.bsc.es</a><br>
<br>
<u></u><u></u></p>
</div>
</div></div></div>
</div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>Zeus Gómez Marmolejo<br>Barcelona Supercomputing Center<br>PhD student<br><a href="http://www.bsc.es" target="_blank">http://www.bsc.es</a><br><br><br>
</div>