Hi, Andrew.<div><br></div><div>Thanks for your helpful reply! Yes indeed, I thought the underlying send function would keep a separate copy of the message, so I just allocate the payload on stack. I guess that's why the message content is corrupted.</div>
<div><br></div><div>I've fixed this bug and now got not message corruption! :-)</div><div><br></div><div>FYI, I also composed a bulk transfer version of my comm module. I found he bulk_bench (/usr/bench/bulk_bench) example is very helpful. Also, for those who want to try bulk_transfer on SCC, I'd like to remind that we must map the shared mem as VREGION_FLAGS_READ_WRITE_MPB since there are no cache coherence, otherwise, we may read some stale data.</div>
<div><br></div><div>I also find the variable size array transfer you mentioned very interesting and seems exactly what I'm looking for. I'll also try that later.</div><div><br></div><div>Thanks,</div><div>Jinghao<br>
<br><div class="gmail_quote">On Thu, May 10, 2012 at 1:14 AM, Baumann Andrew <span dir="ltr"><<a href="mailto:andrewb@inf.ethz.ch" target="_blank">andrewb@inf.ethz.ch</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-AU" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">There are no size limits to direct array transfer, however you need to ensure that the memory containing the array remains live until the message is sent. Payload
is declared on the sender’s stack, and my suspicion is that your send function returns shortly after test_msg__tx returns. You need to wait for the send continuation to run before you can release the memory for payload (e.g. make it static or put it on the
heap).<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">It’s also not clear to me why you need the casts to unsigned char *.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">BTW, if you are sending large or variable arrays of bytes, you’ll probably find it’s more efficient to use variable-length arrays in flounder; i.e.:<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">message msg(uint8 buf[len]);<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Cheers,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Andrew<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Shi Jinghao (</span><span lang="ZH-CN" style="font-size:10.0pt">史经浩）</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
[mailto:<a href="mailto:jhshi@cs.hku.hk" target="_blank">jhshi@cs.hku.hk</a>] <br>
<b>Sent:</b> Wednesday, 09 May, 2012 4:58<br>
<b>To:</b> <a href="mailto:barrelfish-users@lists.inf.ethz.ch" target="_blank">barrelfish-users@lists.inf.ethz.ch</a><br>
<b>Subject:</b> [Barrelfish-users] Message Content Corrupted on SCC<u></u><u></u></span></p><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Hi,<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">When I'm trying to send a relatively large byte array (~800B) between core 0 and 1, I found that starting from the 3rd message, the message content is corrupted. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Here is the flounder spec I use:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">interface test {</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""> typedef uint8 payload[800];</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""> message msg(payload p);</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">}</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">And here is the code snippet that I send and receive the message:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">// sender side (core 0)</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">test_payload_t payload;</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">// fill payload with some magic numbers</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">for (int i = 0; i < sizeof(payload); i++) {</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""> ((unsigned char*)payload)[i] = 0xaa;</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">}</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">test_msg__tx(b, NOP_CONT, payload);</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">// receiver side, check the message</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">for (int i = 0; i < sizeof(payload); i++) {</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""> assert(((unsigned char*)payload)[i] == 0xaa);</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">}</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">So I was wondering whether there are any size limits of direct byte array transfer. I don't know if I should switch to bulk_transfer since about 80% of my message are of small size (< 100B), but other 20% message are about 4K size. Any
suggestions about what kind of IDC I should choose in Barrelfish?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Jinghao<u></u><u></u></p>
</div>
</div></div></div>
</div>
</blockquote></div><br></div>