[Barrelfish-users] various boot issues (was: skb problems booting on x86_32: can't find Eclipse?)
Simon Gerber
simon.gerber at inf.ethz.ch
Thu Aug 15 10:36:51 CEST 2013
Hi Malte,
can you try to boot with the following lines for the kernel (in menu.lst):
kernel /x86_64/sbin/elver loglevel=5 logmask=1
module /x86/64/sbin/cpu loglevel=5 logmask=1
This should turn more debugging output on (all the
debug(SUBSYS_STARTUP, ...) lines to be precise). I'm mostly interested
in the output from create_phys_caps() which should tell us if there is
a RAM region that causes the issue.
This should allow you to see what memory regions get put into the
top-level cnode that holds RAM capabilities. The cause for the bug
you're experiencing is that every RAM capability needs to have a size
that is a power of two, and if some RAM region starts at a not very
well aligned address you're using a lot of capability slots to create
smaller RAM caps (e.g. if there is a region that is only aligned to
4k, you're going to create a 4k RAM cap, then an 8k RAM cap and so on
until you hit some address that's better aligned, e.g. to a gigabyte
and then you can start creating large RAM caps).
As Kornilios said though, we don't have a proper fix for the issue
(that being to make the top-level cnode bigger) because there are
assumptions in the code about the size of that cnode that aren't
expressed as #defines. If you look in
kernel/arch/x86_64/startup_arch.c:688 you'll find the band-aid fix
that we implemented when we were running into the same issue. What we
do there is to make sure that the RAM region that was used in the
early boot allocator (alloc_phys()) is aligned to a 1M boundary. You
might want to use the same band-aid fix for any RAM regions that are
not aligned to at least a 1M boundary to make the problem less
pronounced.
Hope this sheds some more light on the issue,
-- Simon
On 14 August 2013 15:53, Malte Schwarzkopf
<malte.schwarzkopf at cl.cam.ac.uk> wrote:
> Hi Manuel, Simon and others,
>
> Thanks a lot for getting back to me! I'll dig a little deeper on the
> 64-bit side of things; the use of the x86_32 machine was mainly due to a
> shortage of suitable hardware, and it's not blocking anything if that
> one doesn't work (I'll dig into it nonetheless if I get a chance, though).
>
> I tried booting (with the debug flags turned on) on one of our latest
> Sandy Bridge Intel machines, and got the below output, crashing
> somewhere in startup_arch.c:
>
> --------------8<--------------------
> Kernel starting at address 0xffffff8005a65000
> Barrelfish CPU driver starting on x86_64 apic_id 0
> kernel 0: Measured 99758001 APIC timer counts in one RTC second, 210701
> data points.
> kernel 0: Measured 3292025 TSC counts per ms, 99 data points. Average
> jitter 13 TSC ticks.
> spawn module: x86_64/sbin/init
> aligning free memory start to 0x6900000 (was 0x6880000): wasting 512 kB
> Raw MMAP from BIOS
> 0x00000000 - 0x00090000 Type: 1 Length: 0x90000
> 0x00090000 - 0x000a0000 Type: 2 Length: 0x10000
> 0x000e0000 - 0x00100000 Type: 2 Length: 0x20000
> 0x00100000 - 0xb897a000 Type: 1 Length: 0xb887a000
> 0xb897a000 - 0xb8a4e000 Type: 3 Length: 0xd4000
> 0xb8a4e000 - 0xba9ac000 Type: 1 Length: 0x1f5e000
> 0xba9ac000 - 0xba9bd000 Type: 2 Length: 0x11000
> 0xba9bd000 - 0xba9c9000 Type: 1 Length: 0xc000
> 0xba9c9000 - 0xba9de000 Type: 2 Length: 0x15000
> 0xba9de000 - 0xbaadc000 Type: 1 Length: 0xfe000
> 0xbaadc000 - 0xbaadd000 Type: 2 Length: 0x1000
> 0xbaadd000 - 0xbaae7000 Type: 1 Length: 0xa000
> 0xbaae7000 - 0xbaaea000 Type: 2 Length: 0x3000
> 0xbaaea000 - 0xbab3a000 Type: 1 Length: 0x50000
> 0xbab3a000 - 0xbab57000 Type: 2 Length: 0x1d000
> 0xbab57000 - 0xbab5c000 Type: 1 Length: 0x5000
> 0xbab5c000 - 0xbab60000 Type: 2 Length: 0x4000
> 0xbab60000 - 0xbab72000 Type: 1 Length: 0x12000
> 0xbab72000 - 0xbab82000 Type: 2 Length: 0x10000
> 0xbab82000 - 0xbabbd000 Type: 1 Length: 0x3b000
> 0xbabbd000 - 0xbabc1000 Type: 2 Length: 0x4000
> 0xbabc1000 - 0xbacd2000 Type: 1 Length: 0x111000
> 0xbacd2000 - 0xbacd9000 Type: 2 Length: 0x7000
> 0xbacd9000 - 0xbad44000 Type: 1 Length: 0x6b000
> 0xbad44000 - 0xbad49000 Type: 2 Length: 0x5000
> 0xbad49000 - 0xbad60000 Type: 1 Length: 0x17000
> 0xbad60000 - 0xbad6f000 Type: 2 Length: 0xf000
> 0xbad6f000 - 0xbadb6000 Type: 1 Length: 0x47000
> 0xbadb6000 - 0xbadde000 Type: 2 Length: 0x28000
> 0xbadde000 - 0xbae00000 Type: 1 Length: 0x22000
> 0xbae00000 - 0xbb138000 Type: 2 Length: 0x338000
> 0xbb138000 - 0xbd9f7000 Type: 1 Length: 0x28bf000
> 0xbd9f7000 - 0xbdbf7000 Type: 2 Length: 0x200000
> 0xbdbf7000 - 0xbdcdd000 Type: 1 Length: 0xe6000
> 0xbdcdd000 - 0xbdde7000 Type: 2 Length: 0x10a000
> 0xbdde7000 - 0xbde2a000 Type: 4 Length: 0x43000
> 0xbde2a000 - 0xbde2b000 Type: 3 Length: 0x1000
> 0xbde2b000 - 0xbde2d000 Type: 4 Length: 0x2000
> 0xbde2d000 - 0xbde2e000 Type: 3 Length: 0x1000
> 0xbde2e000 - 0xbdefe000 Type: 4 Length: 0xd0000
> 0xbdefe000 - 0xbdf00000 Type: 3 Length: 0x2000
> 0xbdf00000 - 0xbdf03000 Type: 4 Length: 0x3000
> 0xbdf03000 - 0xbdf1f000 Type: 3 Length: 0x1c000
> 0xbdf1f000 - 0xbdfac000 Type: 4 Length: 0x8d000
> 0xbdfac000 - 0xbe000000 Type: 1 Length: 0x54000
> 0x100000000 - 0x2040000000 Type: 1 Length: 0x1f40000000
> 0xbe000000 - 0xd0000000 Type: 2 Length: 0x12000000
> 0xfec00000 - 0xfec01000 Type: 2 Length: 0x1000
> 0xfed19000 - 0xfed1a000 Type: 2 Length: 0x1000
> 0xfed1c000 - 0xfed20000 Type: 2 Length: 0x4000
> 0xfee00000 - 0xfee01000 Type: 2 Length: 0x1000
> 0xffa20000 - 0x100000000 Type: 2 Length: 0x5e0000
> Sorted MMAP
> 0x00000000 - 0x00090000 Type: 1 Length: 0x90000
> 0x00090000 - 0x000a0000 Type: 2 Length: 0x10000
> 0x000e0000 - 0x00100000 Type: 2 Length: 0x20000
> 0x00100000 - 0xb897a000 Type: 1 Length: 0xb887a000
> 0xb897a000 - 0xb8a4e000 Type: 3 Length: 0xd4000
> 0xb8a4e000 - 0xba9ac000 Type: 1 Length: 0x1f5e000
> 0xba9ac000 - 0xba9bd000 Type: 2 Length: 0x11000
> 0xba9bd000 - 0xba9c9000 Type: 1 Length: 0xc000
> 0xba9c9000 - 0xba9de000 Type: 2 Length: 0x15000
> 0xba9de000 - 0xbaadc000 Type: 1 Length: 0xfe000
> 0xbaadc000 - 0xbaadd000 Type: 2 Length: 0x1000
> 0xbaadd000 - 0xbaae7000 Type: 1 Length: 0xa000
> 0xbaae7000 - 0xbaaea000 Type: 2 Length: 0x3000
> 0xbaaea000 - 0xbab3a000 Type: 1 Length: 0x50000
> 0xbab3a000 - 0xbab57000 Type: 2 Length: 0x1d000
> 0xbab57000 - 0xbab5c000 Type: 1 Length: 0x5000
> 0xbab5c000 - 0xbab60000 Type: 2 Length: 0x4000
> 0xbab60000 - 0xbab72000 Type: 1 Length: 0x12000
> 0xbab72000 - 0xbab82000 Type: 2 Length: 0x10000
> 0xbab82000 - 0xbabbd000 Type: 1 Length: 0x3b000
> 0xbabbd000 - 0xbabc1000 Type: 2 Length: 0x4000
> 0xbabc1000 - 0xbacd2000 Type: 1 Length: 0x111000
> 0xbacd2000 - 0xbacd9000 Type: 2 Length: 0x7000
> 0xbacd9000 - 0xbad44000 Type: 1 Length: 0x6b000
> 0xbad44000 - 0xbad49000 Type: 2 Length: 0x5000
> 0xbad49000 - 0xbad60000 Type: 1 Length: 0x17000
> 0xbad60000 - 0xbad6f000 Type: 2 Length: 0xf000
> 0xbad6f000 - 0xbadb6000 Type: 1 Length: 0x47000
> 0xbadb6000 - 0xbadde000 Type: 2 Length: 0x28000
> 0xbadde000 - 0xbae00000 Type: 1 Length: 0x22000
> 0xbae00000 - 0xbb138000 Type: 2 Length: 0x338000
> 0xbb138000 - 0xbd9f7000 Type: 1 Length: 0x28bf000
> 0xbd9f7000 - 0xbdbf7000 Type: 2 Length: 0x200000
> 0xbdbf7000 - 0xbdcdd000 Type: 1 Length: 0xe6000
> 0xbdcdd000 - 0xbdde7000 Type: 2 Length: 0x10a000
> 0xbdde7000 - 0xbde2a000 Type: 4 Length: 0x43000
> 0xbde2a000 - 0xbde2b000 Type: 3 Length: 0x1000
> 0xbde2b000 - 0xbde2d000 Type: 4 Length: 0x2000
> 0xbde2d000 - 0xbde2e000 Type: 3 Length: 0x1000
> 0xbde2e000 - 0xbdefe000 Type: 4 Length: 0xd0000
> 0xbdefe000 - 0xbdf00000 Type: 3 Length: 0x2000
> 0xbdf00000 - 0xbdf03000 Type: 4 Length: 0x3000
> 0xbdf03000 - 0xbdf1f000 Type: 3 Length: 0x1c000
> 0xbdf1f000 - 0xbdfac000 Type: 4 Length: 0x8d000
> 0xbdfac000 - 0xbe000000 Type: 1 Length: 0x54000
> 0xbe000000 - 0xd0000000 Type: 2 Length: 0x12000000
> 0xfec00000 - 0xfec01000 Type: 2 Length: 0x1000
> 0xfed19000 - 0xfed1a000 Type: 2 Length: 0x1000
> 0xfed1c000 - 0xfed20000 Type: 2 Length: 0x4000
> 0xfee00000 - 0xfee01000 Type: 2 Length: 0x1000
> 0xffa20000 - 0x100000000 Type: 2 Length: 0x5e0000
> 0x100000000 - 0x2040000000 Type: 1 Length: 0x1f40000000
> Preprocessed MMAP
> 0x00000000 - 0x00090000 Type: 1 Length: 0x90000
> 0x00090000 - 0x000a0000 Type: 2 Length: 0x10000
> 0x000e0000 - 0x00100000 Type: 2 Length: 0x20000
> 0x00100000 - 0xb897a000 Type: 1 Length: 0xb887a000
> 0xb897a000 - 0xb8a4e000 Type: 3 Length: 0xd4000
> 0xb8a4e000 - 0xba9ac000 Type: 1 Length: 0x1f5e000
> 0xba9ac000 - 0xba9bd000 Type: 2 Length: 0x11000
> 0xba9bd000 - 0xba9c9000 Type: 1 Length: 0xc000
> 0xba9c9000 - 0xba9de000 Type: 2 Length: 0x15000
> 0xba9de000 - 0xbaadc000 Type: 1 Length: 0xfe000
> 0xbaadc000 - 0xbaadd000 Type: 2 Length: 0x1000
> 0xbaadd000 - 0xbaae7000 Type: 1 Length: 0xa000
> 0xbaae7000 - 0xbaaea000 Type: 2 Length: 0x3000
> 0xbaaea000 - 0xbab3a000 Type: 1 Length: 0x50000
> 0xbab3a000 - 0xbab57000 Type: 2 Length: 0x1d000
> 0xbab57000 - 0xbab5c000 Type: 1 Length: 0x5000
> 0xbab5c000 - 0xbab60000 Type: 2 Length: 0x4000
> 0xbab60000 - 0xbab72000 Type: 1 Length: 0x12000
> 0xbab72000 - 0xbab82000 Type: 2 Length: 0x10000
> 0xbab82000 - 0xbabbd000 Type: 1 Length: 0x3b000
> 0xbabbd000 - 0xbabc1000 Type: 2 Length: 0x4000
> 0xbabc1000 - 0xbacd2000 Type: 1 Length: 0x111000
> 0xbacd2000 - 0xbacd9000 Type: 2 Length: 0x7000
> 0xbacd9000 - 0xbad44000 Type: 1 Length: 0x6b000
> 0xbad44000 - 0xbad49000 Type: 2 Length: 0x5000
> 0xbad49000 - 0xbad60000 Type: 1 Length: 0x17000
> 0xbad60000 - 0xbad6f000 Type: 2 Length: 0xf000
> 0xbad6f000 - 0xbadb6000 Type: 1 Length: 0x47000
> 0xbadb6000 - 0xbadde000 Type: 2 Length: 0x28000
> 0xbadde000 - 0xbae00000 Type: 1 Length: 0x22000
> 0xbae00000 - 0xbb138000 Type: 2 Length: 0x338000
> 0xbb138000 - 0xbd9f7000 Type: 1 Length: 0x28bf000
> 0xbd9f7000 - 0xbdbf7000 Type: 2 Length: 0x200000
> 0xbdbf7000 - 0xbdcdd000 Type: 1 Length: 0xe6000
> 0xbdcdd000 - 0xbdde7000 Type: 2 Length: 0x10a000
> 0xbdde7000 - 0xbde2a000 Type: 4 Length: 0x43000
> 0xbde2a000 - 0xbde2b000 Type: 3 Length: 0x1000
> 0xbde2b000 - 0xbde2d000 Type: 4 Length: 0x2000
> 0xbde2d000 - 0xbde2e000 Type: 3 Length: 0x1000
> 0xbde2e000 - 0xbdefe000 Type: 4 Length: 0xd0000
> 0xbdefe000 - 0xbdf00000 Type: 3 Length: 0x2000
> 0xbdf00000 - 0xbdf03000 Type: 4 Length: 0x3000
> 0xbdf03000 - 0xbdf1f000 Type: 3 Length: 0x1c000
> 0xbdf1f000 - 0xbdfac000 Type: 4 Length: 0x8d000
> 0xbdfac000 - 0xbe000000 Type: 1 Length: 0x54000
> 0xbe000000 - 0xd0000000 Type: 2 Length: 0x12000000
> 0xfec00000 - 0xfec01000 Type: 2 Length: 0x1000
> 0xfed19000 - 0xfed1a000 Type: 2 Length: 0x1000
> 0xfed1c000 - 0xfed20000 Type: 2 Length: 0x4000
> 0xfee00000 - 0xfee01000 Type: 2 Length: 0x1000
> 0xffa20000 - 0x100000000 Type: 2 Length: 0x5e0000
> 0x100000000 - 0x2040000000 Type: 1 Length: 0x1f40000000
> Pagealigned MMAP
> 0x00000000 - 0x00090000 Type: 1 Length: 0x90000
> 0x00090000 - 0x000a0000 Type: 2 Length: 0x10000
> 0x000e0000 - 0x00100000 Type: 2 Length: 0x20000
> 0x00100000 - 0xb897a000 Type: 1 Length: 0xb887a000
> 0xb897a000 - 0xb8a4e000 Type: 3 Length: 0xd4000
> 0xb8a4e000 - 0xba9ac000 Type: 1 Length: 0x1f5e000
> 0xba9ac000 - 0xba9bd000 Type: 2 Length: 0x11000
> 0xba9bd000 - 0xba9c9000 Type: 1 Length: 0xc000
> 0xba9c9000 - 0xba9de000 Type: 2 Length: 0x15000
> 0xba9de000 - 0xbaadc000 Type: 1 Length: 0xfe000
> 0xbaadc000 - 0xbaadd000 Type: 2 Length: 0x1000
> 0xbaadd000 - 0xbaae7000 Type: 1 Length: 0xa000
> 0xbaae7000 - 0xbaaea000 Type: 2 Length: 0x3000
> 0xbaaea000 - 0xbab3a000 Type: 1 Length: 0x50000
> 0xbab3a000 - 0xbab57000 Type: 2 Length: 0x1d000
> 0xbab57000 - 0xbab5c000 Type: 1 Length: 0x5000
> 0xbab5c000 - 0xbab60000 Type: 2 Length: 0x4000
> 0xbab60000 - 0xbab72000 Type: 1 Length: 0x12000
> 0xbab72000 - 0xbab82000 Type: 2 Length: 0x10000
> 0xbab82000 - 0xbabbd000 Type: 1 Length: 0x3b000
> 0xbabbd000 - 0xbabc1000 Type: 2 Length: 0x4000
> 0xbabc1000 - 0xbacd2000 Type: 1 Length: 0x111000
> 0xbacd2000 - 0xbacd9000 Type: 2 Length: 0x7000
> 0xbacd9000 - 0xbad44000 Type: 1 Length: 0x6b000
> 0xbad44000 - 0xbad49000 Type: 2 Length: 0x5000
> 0xbad49000 - 0xbad60000 Type: 1 Length: 0x17000
> 0xbad60000 - 0xbad6f000 Type: 2 Length: 0xf000
> 0xbad6f000 - 0xbadb6000 Type: 1 Length: 0x47000
> 0xbadb6000 - 0xbadde000 Type: 2 Length: 0x28000
> 0xbadde000 - 0xbae00000 Type: 1 Length: 0x22000
> 0xbae00000 - 0xbb138000 Type: 2 Length: 0x338000
> 0xbb138000 - 0xbd9f7000 Type: 1 Length: 0x28bf000
> 0xbd9f7000 - 0xbdbf7000 Type: 2 Length: 0x200000
> 0xbdbf7000 - 0xbdcdd000 Type: 1 Length: 0xe6000
> 0xbdcdd000 - 0xbdde7000 Type: 2 Length: 0x10a000
> 0xbdde7000 - 0xbde2a000 Type: 4 Length: 0x43000
> 0xbde2a000 - 0xbde2b000 Type: 3 Length: 0x1000
> 0xbde2b000 - 0xbde2d000 Type: 4 Length: 0x2000
> 0xbde2d000 - 0xbde2e000 Type: 3 Length: 0x1000
> 0xbde2e000 - 0xbdefe000 Type: 4 Length: 0xd0000
> 0xbdefe000 - 0xbdf00000 Type: 3 Length: 0x2000
> 0xbdf00000 - 0xbdf03000 Type: 4 Length: 0x3000
> 0xbdf03000 - 0xbdf1f000 Type: 3 Length: 0x1c000
> 0xbdf1f000 - 0xbdfac000 Type: 4 Length: 0x8d000
> 0xbdfac000 - 0xbe000000 Type: 1 Length: 0x54000
> 0xbe000000 - 0xd0000000 Type: 2 Length: 0x12000000
> 0xfec00000 - 0xfec01000 Type: 2 Length: 0x1000
> 0xfed19000 - 0xfed1a000 Type: 2 Length: 0x1000
> 0xfed1c000 - 0xfed20000 Type: 2 Length: 0x4000
> 0xfee00000 - 0xfee01000 Type: 2 Length: 0x1000
> 0xffa20000 - 0x100000000 Type: 2 Length: 0x5e0000
> 0x100000000 - 0x2040000000 Type: 1 Length: 0x1f40000000
> kernel 0: create_caps_to_cnode: Cannot create more caps in CNode
> kernel 0 PANIC! kernel assertion "err_is_ok(err)" failed at
> ../kernel/arch/x86_64/startup_arch.c:421
> kernel 0: exception 3 (error code 0x0): vec_bp: breakpoint
> Faulting instruction pointer (or next instruction): 0xffffff8005a7417d
> => i.e. unrelocated kernel address 0x10f17d
> Registers:
> rax: 0xffffff8005a92000 r8 : 0x0000000000000000
> rbx: 0xffffff80069000a8 r9 : 0x0000000000000000
> rcx: 0x00000000000003f8 r10: 0x0000000000000000
> rdx: 0xffffff8005b272e0 r11: 0xffffffffffffffff
> rsp: 0xffffff800634cc18 r12: 0x00000000ba9c9000
> rdi: 0x0000000000000000 r13: 0xffffff8005a8c600
> rsi: 0x000000000000000a r14: 0xffffff80069004e0
> rip: 0xffffff8005a7417d r15: 0x00000000ba9bd000
> Top o' stack:
> 0 0xffffff8005a77f2b (18446743524048600875)
> 1 0xffffff8006351180 (18446743524057878912)
> 2 0xffffff8000000020 (18446743523953737760)
> 3 0xffffff800634cc80 (18446743524057861248)
> 4 0xffffff800634cc40 (18446743524057861184)
> 5 0x0000000000200010 (2097168)
> 6 0xffffff8005a8abf3 (18446743524048677875)
> 7 0xffffff8005a88900 (18446743524048668928)
> 8 0x00000000000001a5 (421)
> 9 0x0000000000000000 (0)
> kernel 0: Waiting for GDB connection...
> --------------8<--------------------
>
> Having glanced over this, all seems good: all regions are page-aligned,
> and correctly sorted after preprocessing.
>
> The failure appears to be in the code creating caps for the region that
> was figured out to be RAM; and specifically, space for caps seems to be
> running out (?). This might be plausible: the machine I ran this on has
> 128 GB of RAM. Is there maybe some size limitation on a data structure
> holding the caps? I would have thought that the top-level RAM capability
> should only use up (and only massive fragmentation would lead to
> problems on machines with lots of RAM), but my understanding of the
> Barrelfish capability system is, euphemistically put, limited ;-)
>
> Any ideas?
>
> Cheers,
> M.
>
>
> On 24/07/13 08:52, Manuel Stocker wrote:
>> Hi Malte,
>>
>> As far as I remember we had similar symptoms when trying to boot
>> Barrelfish on a DELL Optiplex machine for our lab project. It has been a
>> while, so I might be wrong here. The issues were caused by a bad memory
>> map given by the DELL BIOS, so I added some fixup code in startup_arch.c
>> (only for x86_64).
>>
>> The problem is described in section 10.2.3 in TN-015 [1]
>> The fixup code is in create_phys_caps for x64 [2]
>> And missing in 32bit [3]
>>
>> You might want to check the debug output it should generate for x64 and
>> maybe adapt the code for 32bit.
>>
>> Cheers,
>> Manuel
>>
>> [1] http://www.barrelfish.org/TN-015-DiskDriverArchitecture.pdf
>> [2]
>> http://hg.barrelfish.org/barrelfish/file/tip/kernel/arch/x86_64/startup_arch.c#l145
>> [3]
>> http://hg.barrelfish.org/barrelfish/file/tip/kernel/arch/x86_32/startup_arch.c#l151
>>
>>
>
>
> _______________________________________________
> Barrelfish-users mailing list
> Barrelfish-users at lists.inf.ethz.ch
> https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users
More information about the Barrelfish-users
mailing list