[Barrelfish-users] UNSUBSCRIBE
Markku Porvari
markku.porvari at mit.fi
Mon Jul 18 12:06:42 CEST 2011
UNSUBSCRIBE
------------------------------------------------------------------
Markku Porvari, Advisor, manage IT, markku.porvari at mit.fi
Kauppamiehentie 4, H. 412, FIN-02100 Espoo, +358 40 725 2547
------------------------------------------------------------------
-----Original Message-----
From: barrelfish-users-request at lists.inf.ethz.ch
[mailto:barrelfish-users-request at lists.inf.ethz.ch]
Sent: 18. heinäkuuta 2011 1:00
To: barrelfish-users at lists.inf.ethz.ch
Subject: Barrelfish-users Digest, Vol 18, Issue 9
Send Barrelfish-users mailing list submissions to
barrelfish-users at lists.inf.ethz.ch
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users
or, via email, send a message with subject or body 'help' to
barrelfish-users-request at lists.inf.ethz.ch
You can reach the person managing the list at
barrelfish-users-owner at lists.inf.ethz.ch
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Barrelfish-users digest..."
Today's Topics:
1. Messages in Barrelfish (Zeus G?mez Marmolejo)
2. Re: [Barrelfish] Messages in Barrelfish (Andrew Baumann)
3. Re: [Barrelfish] Messages in Barrelfish (Zeus G?mez Marmolejo)
4. Re: [Barrelfish] Messages in Barrelfish (Andrew Baumann)
----------------------------------------------------------------------
Message: 1
Date: Sun, 17 Jul 2011 19:02:29 +0200
From: Zeus G?mez Marmolejo <zeus.gomez at bsc.es>
Subject: [Barrelfish-users] Messages in Barrelfish
To: barrelfish-users at lists.inf.ethz.ch
Message-ID:
<CAPgvYs7ze8FymZpCP956Mcan4qtY=pyw7PHSUS3pn23hZ2YdVA at mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi all,
I need some help with the message infrastructure.
I made a very simple example where core 0 is sending n messages to core 1
using the UMP backend. Core 1 is just waiting and dispatching messages.
So the problem is that I can not manage to send more than 31 messages to
core 1. The message 32 is sent but core 1 doesn't receive it. The error I
get is:
Failure: ( flounder) Cannot queue message for transmit: queue is full
[FLOUNDER_ERR_TX_BUSY]
In the meantime all other 31 messages have been dispatched and received by
core 1 but the sending error keeps on, so no more messages can be sent.
Message 32 seems to be on queue but never dispatched.
Do you have any idea about this?? Is there any way to flush the queue?
Thank you very much for your help!!
--
Zeus G?mez Marmolejo
Barcelona Supercomputing Center
PhD student
http://www.bsc.es
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
https://lists.inf.ethz.ch/pipermail/barrelfish-users/attachments/20110717/45
c93150/attachment-0001.html
------------------------------
Message: 2
Date: Sun, 17 Jul 2011 17:54:43 +0000
From: Andrew Baumann <Andrew.Baumann at microsoft.com>
Subject: Re: [Barrelfish-users] [Barrelfish] Messages in Barrelfish
To: Zeus G?mez Marmolejo <zeus.gomez at bsc.es>
Cc: "barrelfish-users at lists.inf.ethz.ch"
<barrelfish-users at lists.inf.ethz.ch>
Message-ID:
<0BCCB4EE8EE68342833E0863DBB35A940A80A19D at TK5EX14MBXC262.redmond.corp.micros
oft.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi Zeus,
> I made a very simple example where core 0 is sending n messages to core 1
> using the UMP backend. Core 1 is just waiting and dispatching messages.
>
> So the problem is that I can not manage to send more than 31 messages to
> core 1. The message 32 is sent but core 1 doesn't receive it. The error I
get is:
>
> Failure: ( flounder) Cannot queue message for transmit: queue is
full
> [FLOUNDER_ERR_TX_BUSY]
>
> In the meantime all other 31 messages have been dispatched and received
> by core 1 but the sending error keeps on, so no more messages can be sent.
> Message 32 seems to be on queue but never dispatched.
Are you processing the incoming event queue on core 0, by dispatching the
waitset associated with the channel?
If your program is single-threaded, then a loop like while (1)
send_message(); won't work, because you also need to dispatch events on the
incoming waitset in order for the UMP channel implementation to see the acks
which tell it that the receiver has consumed the messages. The TX_BUSY error
you're seeing is the channel implementation telling you (on core 0) that
there is no space in the outgoing message queue.
The correct thing to do in this case depends on your program, but assuming
you want to use the single-threaded state-machine model, the right thing to
do is to register a callback for the ability to send further messages (e.g.
with b->register_send()), and then return to your event processing loop
which should free up space in the outgoing message queue and then run your
callback. You might want to have a look at the idctest program in the tree
for some examples of how to use callbacks and continuations for sending.
If this all sounds very painful and low-level, that's because it is :) Have
you considered using a higher-level abstraction like THC?
Cheers,
Andrew
------------------------------
Message: 3
Date: Mon, 18 Jul 2011 03:10:41 +0200
From: Zeus G?mez Marmolejo <zeus.gomez at bsc.es>
Subject: Re: [Barrelfish-users] [Barrelfish] Messages in Barrelfish
To: Andrew Baumann <Andrew.Baumann at microsoft.com>
Cc: "barrelfish-users at lists.inf.ethz.ch"
<barrelfish-users at lists.inf.ethz.ch>
Message-ID:
<CAPgvYs7iUhNPTczK7wYnFd9PxVZ_7eqfBs3Wuu+Y_Z6Q4-Kfeg at mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi Andrew,
Thank you very much for your response. Now everything is much clearer. What
was happening was exactly that.
So I tried the two variants. First, in the single-threaded one, I used the
continuation closure associated to the message to trigger the callback when
the message is sent. Just like the idctest example you were telling. In that
case it's working perfectly. So this may solve my problem, although things
get a bit more complicated.
I would better try the multi-threaded version, so I created one extra thread
per core just dispatching messages from the default waitset, as you
suggested. However, in this case the problem doesn't get solved. The
transmit queue fills and there is no more space after 32 messages. I'm
wondering if I'm waiting in the correct waitset. You said there is an
incoming waitset? Is it different from the default one?
The "dispatcher" thread looks like this:
static int dispatch_thread(void *p)
{
while (1) event_dispatch(get_default_waitset());
return 0;
}
Thank you again for your help!
zeus.
El 17 de julio de 2011 19:54, Andrew Baumann
<Andrew.Baumann at microsoft.com>escribi?:
> Hi Zeus,
>
> > I made a very simple example where core 0 is sending n messages to core
1
> > using the UMP backend. Core 1 is just waiting and dispatching messages.
> >
> > So the problem is that I can not manage to send more than 31 messages to
> > core 1. The message 32 is sent but core 1 doesn't receive it. The error
I
> get is:
> >
> > Failure: ( flounder) Cannot queue message for transmit: queue is
> full
> > [FLOUNDER_ERR_TX_BUSY]
> >
> > In the meantime all other 31 messages have been dispatched and received
> > by core 1 but the sending error keeps on, so no more messages can be
> sent.
> > Message 32 seems to be on queue but never dispatched.
>
> Are you processing the incoming event queue on core 0, by dispatching the
> waitset associated with the channel?
>
> If your program is single-threaded, then a loop like while (1)
> send_message(); won't work, because you also need to dispatch events on
the
> incoming waitset in order for the UMP channel implementation to see the
acks
> which tell it that the receiver has consumed the messages. The TX_BUSY
error
> you're seeing is the channel implementation telling you (on core 0) that
> there is no space in the outgoing message queue.
>
> The correct thing to do in this case depends on your program, but assuming
> you want to use the single-threaded state-machine model, the right thing
to
> do is to register a callback for the ability to send further messages
(e.g.
> with b->register_send()), and then return to your event processing loop
> which should free up space in the outgoing message queue and then run your
> callback. You might want to have a look at the idctest program in the tree
> for some examples of how to use callbacks and continuations for sending.
>
> If this all sounds very painful and low-level, that's because it is :)
Have
> you considered using a higher-level abstraction like THC?
>
> Cheers,
> Andrew
>
--
Zeus G?mez Marmolejo
Barcelona Supercomputing Center
PhD student
http://www.bsc.es
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
https://lists.inf.ethz.ch/pipermail/barrelfish-users/attachments/20110718/a4
36df88/attachment-0001.html
------------------------------
Message: 4
Date: Mon, 18 Jul 2011 04:09:52 +0000
From: Andrew Baumann <Andrew.Baumann at microsoft.com>
Subject: Re: [Barrelfish-users] [Barrelfish] Messages in Barrelfish
To: Zeus G?mez Marmolejo <zeus.gomez at bsc.es>
Cc: "barrelfish-users at lists.inf.ethz.ch"
<barrelfish-users at lists.inf.ethz.ch>
Message-ID:
<0BCCB4EE8EE68342833E0863DBB35A940A80A210 at TK5EX14MBXC262.redmond.corp.micros
oft.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi again,
> I would better try the multi-threaded version, so I created one extra
thread
> per core just dispatching messages from the default waitset, as you
> suggested. However, in this case the problem doesn't get solved. The
> transmit queue fills and there is no more space after 32 messages. I'm
> wondering if I'm waiting in the correct waitset. You said there is an
incoming
> waitset? Is it different from the default one?
Unless you're doing anything special to create your own waitset, the default
waitset is the correct one. However, the support for using the low-level IDC
primitives in multi-threaded domains is a bit lacking, so there are two
places you can trip up:
1. The IDC binding objects are not thread-safe. You need to ensure that you
do not race between the thread trying to send messages and the thread that
is handling waitset events (some of which will call into the binding for
message delivery). For concurrency control on bindings, you could use
anything like mutexes, semaphores, etc. Note that there is also something
unusual called an event_mutex (lib/barrelfish/event_mutex.c) which is
designed for mutual exclusion between both threads (with a blocking lock
acquire) and event handlers (which takes a closure to be fired when the
mutex is held).
2. There is no blocking send primitive: a TX_BUSY error is still possible,
and must be handled by blocking the sending thread until there is space. You
could do this by having it register a continuation (with register_send) and
then block on a condition variable which will be signalled by the
continuation.
Again, this is all very fiddly and easy to get wrong, so I'd encourage you
to consider using THC instead.
Cheers,
Andrew
------------------------------
_______________________________________________
Barrelfish-users mailing list
Barrelfish-users at lists.inf.ethz.ch
https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users
End of Barrelfish-users Digest, Vol 18, Issue 9
***********************************************
More information about the Barrelfish-users
mailing list