Hi, guys.<div><br></div><div>Apology for the huge delay on this problem! The SCC data center has been down for the past few days...</div><div><br></div><div>I tested the patch provided by Simon, it works! Many thanks.</div>
<div><br></div><div>Jinghao<br><br><div class="gmail_quote">On Thu, Dec 13, 2012 at 4:01 AM, Simon Peter <span dir="ltr"><<a href="mailto:speter@inf.ethz.ch" target="_blank">speter@inf.ethz.ch</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Apparently, FNSAVE also resets the FPU
after storing its state to memory, while FXSAVE does not. I've
implemented fpu_save() on the assumption that it just stores the
FPU state, not modify it. So this is likely where the problem is.
One quick workaround would be to reload the FPU state immediately
after storing it in fpu_save(). This is slow, but should work well
in your situation.<br>
<br>
In include/arch/x86_32/barrelfish_kpi, replace line 68:<br>
<br>
__asm volatile("fnsave %0; fwait" : "=m" (*regs));<br>
<br>
with:<br>
<br>
__asm volatile("fnsave %0; fwait; frstor %0" : "=m" (*regs));<br>
<br>
This is untested, but should work.<span class="HOEnZb"><font color="#888888"><br>
<br>
Simon</font></span><div><div class="h5"><br>
<br>
On 12-12-09 09:52 PM, Shi Jinghao wrote:<br>
</div></div></div><div><div class="h5">
<blockquote type="cite">
Hi Andrew,
<div><br>
</div>
<div>Yes, understood. But unfortunately, page fault is inevitable
in my program. No only because of demand-paging, but I also use
page fault to monitor the access status of each page, and
trigger responding library routine as needed.</div>
<div><br>
</div>
<div>I'll keep working on this issue, and keep the community
updated if got any progress.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Jinghao<br>
<br>
<div class="gmail_quote">On Sat, Dec 8, 2012 at 7:38 AM, Andrew
Baumann <span dir="ltr"><<a href="mailto:Andrew.Baumann@microsoft.com" target="_blank">Andrew.Baumann@microsoft.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div link="blue" vlink="purple" lang="EN-US">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi,</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Much
as we’d like to help, it’s tough when it requires an
SCC to reproduce. Can you perhaps restructure your
code to avoid the page fault handler (e.g.
pre-allocate/pre-fault the memory)?</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Andrew</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">
Shi Jinghao [mailto:<a href="mailto:jhshi@cs.hku.hk" target="_blank">jhshi@cs.hku.hk</a>]
<br>
<b>Sent:</b> Thursday, 6 December 2012 01:33<br>
<b>To:</b> <a href="mailto:barrelfish-users@lists.inf.ethz.ch" target="_blank">barrelfish-users@lists.inf.ethz.ch</a></span></p>
<div>
<div><br>
<b>Subject:</b> Re: [Barrelfish-users] A Weird Bug
about Page Fault</div>
</div>
<div>
<div>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Hi,</p>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">Are there any progresses on
this issue? I've been tracing kernel's page
fault path for a while but found no clue yet.
It's quite frustrating since this bug has
prevented me doing any kind of benchmarks that
has double/float data type...</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">Jinghao</p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"> </p>
<div>
<p class="MsoNormal">On Wed, Dec 5, 2012 at 5:52
AM, Andrew Baumann <<a href="mailto:Andrew.Baumann@microsoft.com" target="_blank">Andrew.Baumann@microsoft.com</a>>
wrote:</p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">That’s
interesting… to my knowledge there’s
very little code that’s unique to SCC
and not shared with x86_32. </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">One
likely the culprit is fpu_save and
fpu_restore (from
/include/arch/x86_32/barrelfish_kpi/asm_inlines_arch.h)
which do fxsave and fxrstor on x86_32
but fnsave and frstror on SCC. Are we
sure the two are equivalent?</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">It
might also be helpful if someone could
test on real x86_32 hardware, just to
rule out qemu.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Andrew</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">
<a href="mailto:jhshi89@gmail.com" target="_blank">jhshi89@gmail.com</a>
[mailto:<a href="mailto:jhshi89@gmail.com" target="_blank">jhshi89@gmail.com</a>]
<b>On Behalf Of </b>Shi Jinghao<br>
<b>Sent:</b> Tuesday, 4 December 2012
02:16<br>
<b>To:</b> Simon Peter<br>
<b>Cc:</b> Andrew Baumann; <a href="mailto:barrelfish-users@lists.inf.ethz.ch" target="_blank">
barrelfish-users@lists.inf.ethz.ch</a><br>
<b>Subject:</b> Re: [Barrelfish-users]
A Weird Bug about Page Fault</span></p>
<div>
<div>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Hi Simon,</p>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">Yes, I think
so. But this bug didn't occur on
sccLinux running on SCC (see
write_fault.c). So I suspect that
some code in Barrelfish that deals
with exception don't behave right.
But I really have no idea where to
debug...</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">Can someone in
the community who has access to
SCC test the code? Many thanks.</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Jinghao</p>
<div>
<p class="MsoNormal">On Tue, Dec
4, 2012 at 4:45 PM, Simon Peter
<<a href="mailto:speter@inf.ethz.ch" target="_blank">speter@inf.ethz.ch</a>>
wrote:</p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal">Hi Jinghao,<br>
<br>
It seems this is SCC specific.
I just ran your test-case on
QEMU on both x86-64 and -32
platforms and it seems to work
just fine (i.e. I get the "all
good" output).<br>
<br>
Simon</p>
<div>
<p class="MsoNormal"><br>
<br>
On 12/03/2012 12:47 AM, Shi
Jinghao wrote:</p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">Hi
Andrew,<br>
<br>
Thanks for your reply. The
two different exceptions
you mentioned is<br>
insightful I tried your
suggestion. But that does
not help. The NaN<br>
errors still occur. I also
tried to put extra dummy
float point<br>
operations in page fault
handler. And that does not
help, either.<br>
<br>
Thanks,<br>
Jinghao<br>
<br>
On Sun, Dec 2, 2012 at
2:06 AM, Andrew Baumann</p>
</div>
<p class="MsoNormal"><<a href="mailto:Andrew.Baumann@microsoft.com" target="_blank">Andrew.Baumann@microsoft.com</a>
<mailto:<a href="mailto:Andrew.Baumann@microsoft.com" target="_blank">Andrew.Baumann@microsoft.com</a>>>
wrote:<br>
<br>
Hi Jinghao,____<br>
<br>
__ __</p>
<div>
<p class="MsoNormal"><br>
<br>
I notice that the
first time you use
floating point in this
program<br>
is when writing to the
array. There should be two
different<br>
exceptions raised and
handled here: one for the
page fault, and one<br>
for the first use of
the floating point
hardware (which we lazily<br>
context-switch). My
guess is that the
page-fault path, which is
not<br>
heavily exercised,
does not interact well
with the floating point</p>
</div>
<p class="MsoNormal">
save/restore code.____<br>
<br>
__ __</p>
<div>
<p class="MsoNormal"><br>
<br>
If you initialise the
floating point hardware by
doing some other<br>
floating point
operations (or writing to
a statically allocated</p>
</div>
<p class="MsoNormal">
variable) beforehand, does
the problem go away?____<br>
<br>
__ __<br>
<br>
Andrew____<br>
<br>
__ __<br>
<br>
*From:* Shi Jinghao
[mailto:<a href="mailto:jhshi@cs.hku.hk" target="_blank">jhshi@cs.hku.hk</a>
<mailto:<a href="mailto:jhshi@cs.hku.hk" target="_blank">jhshi@cs.hku.hk</a>>]<br>
*Sent:* Saturday, 1
December 2012 02:20<br>
*To:* <a href="mailto:barrelfish-users@lists.inf.ethz.ch" target="_blank">barrelfish-users@lists.inf.ethz.ch</a><br>
<mailto:<a href="mailto:barrelfish-users@lists.inf.ethz.ch" target="_blank">barrelfish-users@lists.inf.ethz.ch</a>><br>
*Subject:*
[Barrelfish-users] A Weird
Bug about Page Fault____<br>
<br>
__ __<br>
<br>
Hi,____<br>
<br>
__ __</p>
<div>
<p class="MsoNormal"><br>
<br>
I've been developing a
memory management library
on Barrelfish<br>
(SCC). Recently I
bumped into a very weird
bug about page fault. I<br>
attached a minimal
case (pgfault_test.tgz)
that can reproduce this</p>
</div>
<p class="MsoNormal">
bug.____<br>
<br>
__ __<br>
<br>
The work flow of the
test case is as simple as
following:____<br>
<br>
__ __</p>
<div>
<p class="MsoNormal"><br>
<br>
1) Allocate an array
of doubles as read-only,
using frame_alloc and</p>
</div>
<p class="MsoNormal">
vspace_map_one_frame_attr
(or pmap->f.map, this
doesn't matter)____<br>
<br>
__ __<br>
<br>
2) Initiate the array,
this will generate page
fault____<br>
<br>
__ __</p>
<div>
<p class="MsoNormal"><br>
<br>
3) In page fault
handler, remap the faulted
page as read-write,</p>
</div>
<p class="MsoNormal"> using
pmap->f.modify_flags____<br>
<br>
__ __</p>
<div>
<p class="MsoNormal"><br>
<br>
The weird thing is:
the first touch of this
array will not result in</p>
</div>
<p class="MsoNormal"> a
proper value, but just
NaN!____<br>
<br>
__ __<br>
<br>
I've conducted several
runs and found the
following:____<br>
<br>
__ __</p>
<div>
<p class="MsoNormal"><br>
<br>
1) This bug will occur
when the array type is
double or float.</p>
</div>
<p class="MsoNormal">
Everything is fine if it's a
integer array.____<br>
<br>
__ __</p>
<div>
<p class="MsoNormal"><br>
<br>
2) Only the item that
caused the page fault will
end in a NaN value,<br>
others items are just
fine. And this applies
when the faulted be</p>
</div>
<p class="MsoNormal">
anywhere within that page,
not just the page start.____<br>
<br>
__ __</p>
<div>
<p class="MsoNormal"><br>
<br>
3) If you assign each
array value with a
constant value (say 1.0),<br>
or a int/double
variable, then all items
will end up with a right<br>
value. It seems only
when we assign a[i] with i
(or any expression</p>
</div>
<p class="MsoNormal">
contains i) will produce
this bug.____<br>
<br>
__ __</p>
<div>
<p class="MsoNormal"><br>
<br>
I tested the attached
code in release2012-05-25
(the revision I work</p>
</div>
<p class="MsoNormal"> on)
and the latest revision
(release2012-10-03).____<br>
<br>
__ __</p>
<div>
<p class="MsoNormal"><br>
<br>
I've also composed a
minimal test case in
sccLinux (write_fault.c).</p>
</div>
<p class="MsoNormal"> It
turns out that everything is
all good. No annoying NaN
values.____<br>
<br>
__ __</p>
<div>
<p class="MsoNormal"><br>
<br>
This bug has bothered
me for quite a few days.
Really appreciate if</p>
</div>
<p class="MsoNormal">
someone can give a hint on
this.____<br>
<br>
__ __<br>
<br>
Thanks,____<br>
<br>
Jinghao____<br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
Barrelfish-users mailing
list<br>
<a href="mailto:Barrelfish-users@lists.inf.ethz.ch" target="_blank">Barrelfish-users@lists.inf.ethz.ch</a><br>
<a href="https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users" target="_blank">https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users</a></p>
</blockquote>
<p class="MsoNormal"> </p>
</blockquote>
</div>
<p class="MsoNormal"> </p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"> </p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Barrelfish-users mailing list
<a href="mailto:Barrelfish-users@lists.inf.ethz.ch" target="_blank">Barrelfish-users@lists.inf.ethz.ch</a>
<a href="https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users" target="_blank">https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users</a>
</pre>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br></div>