<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Much as we’d like to help, it’s tough when it requires an SCC to reproduce. Can you perhaps restructure your code to avoid the page fault handler (e.g. pre-allocate/pre-fault
the memory)?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Andrew<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> Shi Jinghao [mailto:jhshi@cs.hku.hk]
<br>
<b>Sent:</b> Thursday, 6 December 2012 01:33<br>
<b>To:</b> barrelfish-users@lists.inf.ethz.ch<br>
<b>Subject:</b> Re: [Barrelfish-users] A Weird Bug about Page Fault<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hi,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Are there any progresses on this issue? I've been tracing kernel's page fault path for a while but found no clue yet. It's quite frustrating since this bug has prevented me doing any kind of benchmarks that has double/float data type...<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Jinghao<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Wed, Dec 5, 2012 at 5:52 AM, Andrew Baumann <<a href="mailto:Andrew.Baumann@microsoft.com" target="_blank">Andrew.Baumann@microsoft.com</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">That’s interesting… to my knowledge there’s very little code that’s unique to SCC and not shared
with x86_32. </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">One likely the culprit is fpu_save and fpu_restore (from /include/arch/x86_32/barrelfish_kpi/asm_inlines_arch.h)
which do fxsave and fxrstor on x86_32 but fnsave and frstror on SCC. Are we sure the two are equivalent?</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">It might also be helpful if someone could test on real x86_32 hardware, just to rule out qemu.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Andrew</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">
<a href="mailto:jhshi89@gmail.com" target="_blank">jhshi89@gmail.com</a> [mailto:<a href="mailto:jhshi89@gmail.com" target="_blank">jhshi89@gmail.com</a>]
<b>On Behalf Of </b>Shi Jinghao<br>
<b>Sent:</b> Tuesday, 4 December 2012 02:16<br>
<b>To:</b> Simon Peter<br>
<b>Cc:</b> Andrew Baumann; <a href="mailto:barrelfish-users@lists.inf.ethz.ch" target="_blank">
barrelfish-users@lists.inf.ethz.ch</a><br>
<b>Subject:</b> Re: [Barrelfish-users] A Weird Bug about Page Fault</span><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi Simon,<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Yes, I think so. But this bug didn't occur on sccLinux running on SCC (see write_fault.c). So I suspect that some code in Barrelfish that deals with exception don't behave right.
But I really have no idea where to debug...<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Can someone in the community who has access to SCC test the code? Many thanks.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt">Jinghao<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Tue, Dec 4, 2012 at 4:45 PM, Simon Peter <<a href="mailto:speter@inf.ethz.ch" target="_blank">speter@inf.ethz.ch</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi Jinghao,<br>
<br>
It seems this is SCC specific. I just ran your test-case on QEMU on both x86-64 and -32 platforms and it seems to work just fine (i.e. I get the "all good" output).<br>
<br>
Simon<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
<br>
On 12/03/2012 12:47 AM, Shi Jinghao wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi Andrew,<br>
<br>
Thanks for your reply. The two different exceptions you mentioned is<br>
insightful I tried your suggestion. But that does not help. The NaN<br>
errors still occur. I also tried to put extra dummy float point<br>
operations in page fault handler. And that does not help, either.<br>
<br>
Thanks,<br>
Jinghao<br>
<br>
On Sun, Dec 2, 2012 at 2:06 AM, Andrew Baumann<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><<a href="mailto:Andrew.Baumann@microsoft.com" target="_blank">Andrew.Baumann@microsoft.com</a> <mailto:<a href="mailto:Andrew.Baumann@microsoft.com" target="_blank">Andrew.Baumann@microsoft.com</a>>>
wrote:<br>
<br>
Hi Jinghao,____<br>
<br>
__ __<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
<br>
I notice that the first time you use floating point in this program<br>
is when writing to the array. There should be two different<br>
exceptions raised and handled here: one for the page fault, and one<br>
for the first use of the floating point hardware (which we lazily<br>
context-switch). My guess is that the page-fault path, which is not<br>
heavily exercised, does not interact well with the floating point<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> save/restore code.____<br>
<br>
__ __<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
<br>
If you initialise the floating point hardware by doing some other<br>
floating point operations (or writing to a statically allocated<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> variable) beforehand, does the problem go away?____<br>
<br>
__ __<br>
<br>
Andrew____<br>
<br>
__ __<br>
<br>
*From:* Shi Jinghao [mailto:<a href="mailto:jhshi@cs.hku.hk" target="_blank">jhshi@cs.hku.hk</a> <mailto:<a href="mailto:jhshi@cs.hku.hk" target="_blank">jhshi@cs.hku.hk</a>>]<br>
*Sent:* Saturday, 1 December 2012 02:20<br>
*To:* <a href="mailto:barrelfish-users@lists.inf.ethz.ch" target="_blank">barrelfish-users@lists.inf.ethz.ch</a><br>
<mailto:<a href="mailto:barrelfish-users@lists.inf.ethz.ch" target="_blank">barrelfish-users@lists.inf.ethz.ch</a>><br>
*Subject:* [Barrelfish-users] A Weird Bug about Page Fault____<br>
<br>
__ __<br>
<br>
Hi,____<br>
<br>
__ __<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
<br>
I've been developing a memory management library on Barrelfish<br>
(SCC). Recently I bumped into a very weird bug about page fault. I<br>
attached a minimal case (pgfault_test.tgz) that can reproduce this<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> bug.____<br>
<br>
__ __<br>
<br>
The work flow of the test case is as simple as following:____<br>
<br>
__ __<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
<br>
1) Allocate an array of doubles as read-only, using frame_alloc and<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> vspace_map_one_frame_attr (or pmap->f.map, this doesn't matter)____<br>
<br>
__ __<br>
<br>
2) Initiate the array, this will generate page fault____<br>
<br>
__ __<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
<br>
3) In page fault handler, remap the faulted page as read-write,<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> using pmap->f.modify_flags____<br>
<br>
__ __<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
<br>
The weird thing is: the first touch of this array will not result in<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> a proper value, but just NaN!____<br>
<br>
__ __<br>
<br>
I've conducted several runs and found the following:____<br>
<br>
__ __<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
<br>
1) This bug will occur when the array type is double or float.<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> Everything is fine if it's a integer array.____<br>
<br>
__ __<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
<br>
2) Only the item that caused the page fault will end in a NaN value,<br>
others items are just fine. And this applies when the faulted be<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> anywhere within that page, not just the page start.____<br>
<br>
__ __<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
<br>
3) If you assign each array value with a constant value (say 1.0),<br>
or a int/double variable, then all items will end up with a right<br>
value. It seems only when we assign a[i] with i (or any expression<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> contains i) will produce this bug.____<br>
<br>
__ __<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
<br>
I tested the attached code in release2012-05-25 (the revision I work<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> on) and the latest revision (release2012-10-03).____<br>
<br>
__ __<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
<br>
I've also composed a minimal test case in sccLinux (write_fault.c).<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> It turns out that everything is all good. No annoying NaN values.____<br>
<br>
__ __<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
<br>
This bug has bothered me for quite a few days. Really appreciate if<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> someone can give a hint on this.____<br>
<br>
__ __<br>
<br>
Thanks,____<br>
<br>
Jinghao____<br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
Barrelfish-users mailing list<br>
<a href="mailto:Barrelfish-users@lists.inf.ethz.ch" target="_blank">Barrelfish-users@lists.inf.ethz.ch</a><br>
<a href="https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users" target="_blank">https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users</a><o:p></o:p></p>
</blockquote>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>