[Barrelfish-users] [Barrelfish] Hagfish kernel image structure and latest version

David Cock david.cock at inf.ethz.ch
Thu Apr 15 19:20:50 CEST 2021 

That's a bit odd.  It ought to be as simple as increasing the size of 
the table - I don't think there's anything fundamental about it.  I'm 
pretty sure I just wanted to avoid dynamic allocation at that point in 
the code. Let me have a look at it.

David

On 15/4/21 6:08 pm, sebastian.scherbel at fau.de wrote:
> Hi,
>
> >The memory map is 17952B, but MEM_MAP_SIZE is 8192.
> > This is compile-time limit in Hagfish - please report this overflow, 
> it's a bug.
>
> I have stumbled across a limitation in Hagfish. 8192B is not enough on 
> a HPE Apollo 70 with 256 cores and 256 GB RAM and 374 entries of 48B 
> each. I tried to just increase the value, but now I crash somewhere 
> between exiting the UEFI boot services and jumping to my ELFs 
> entrypoint. Without any output.
>
> Do you happen to have an educated guess what the problem could 
> possibly be? I don't have a hardware debugger, unfortunately, which 
> makes it somewhat challenging. And limits me to printk debugging.
>
> >We'd be happy to integrate a patch if you wanted to modify it.
> I'll do that after i have finished my project.
>
> Regards Sebastian
>
>
> Am Freitag, 12. März 2021, 12:08:39 MEZ hat David Cock 
> <david.cock at inf.ethz.ch> Folgendes geschrieben:
>
>
> Sebastian
>
> It looks like what's going on is as Daniel described.  Hagfish seems 
> to use an API call (get_file_size) that's supported by EDK2, but not 
> by many commercial offerings.  I think it'd almost certainly be 
> possible to update Hagfish such that it doesn't depend on it - 
> something like preallocating a "reasonably-sized" buffer, and if it 
> turns out it wasn't big enough trying again with a larger one. It's 
> been a few years since I wrote this stuff, but I think you'll get back 
> an EFI_BUFFER_TOO_SMALL if it was too small, and *BufferSize will be 
> filled with the actual size so that you can reallocate the buffer. 
> We'd be happy to integrate a patch if you wanted to modify it.
>
> David
>
> On 11/3/21 2:19 pm, Schwyn Daniel wrote:
>
> Hi Sebastian,
>
> David and I are both part of the Barrelfish core team at ETH so our 
> setup is the same😉We do load Hagfish over PXE but as far as I know 
> always directly with the PXE implementation in EDK2. Hagfish is 
> theoretically designed to be flexible and cover as many EFI based boot 
> environments as possible. In practice it’s only tested on the machines 
> we have. Those are EDK2 based and the PXE implementation in there 
> supports the two MTFTP opcodes we use. It should be possible to either 
> implement support in iPXE or make Hagfish not use the get_file_size 
> e.g. by allocating a large enough buffer and make sure the read_file 
> doesn’t overflow it.
>
> Regards,
>
> Daniel
>
> *Von:*sebastian.scherbel at fau.de <mailto:sebastian.scherbel at fau.de> 
> <sebastian.scherbel at fau.de> <mailto:sebastian.scherbel at fau.de>
> *Gesendet:* Mittwoch, 10. März 2021 16:38
> *An:* Schwyn Daniel <daniel.schwyn at inf.ethz.ch> 
> <mailto:daniel.schwyn at inf.ethz.ch>
> *Cc:* barrelfish-users at lists.inf.ethz.ch 
> <mailto:barrelfish-users at lists.inf.ethz.ch>
> *Betreff:* Re: AW: [Barrelfish-users] [Barrelfish] Hagfish kernel 
> image structure and latest version
>
> Hi Daniel,
>
> thank you for your reply. That's what I already suspected.
>
> I'm waiting to hear back from David about his setup. He seemed quite 
> sure that chainloading works, so I was a bit surprised that neither 
> iPXE nor Grub did work for me on EDK2.
>
> Otherwise, I'll fix iPXE myself.
>
> Regards Sebastian
>
> Am Mittwoch, 10. März 2021, 10:59:48 MEZ hat Schwyn Daniel 
> <daniel.schwyn at inf.ethz.ch <mailto:daniel.schwyn at inf.ethz.ch>> 
> Folgendes geschrieben:
>
> Hi Sebastian,
>
> I have seen this error before with iPXE when I tried chainloading 
> Hagfish with iPXE on top of U-Boot with EFI support. U-Boot’s EFI 
> support is rather minimalistic so I’m not sure the cause was the same 
> in my case. Looking at the iPXE source code though it looks to me as 
> if it only supports the EFI_PXE_BASE_CODE_MTFTP_READ_FILE opcode while 
> Hagfish also needs the EFI_PXE_BASE_CODE_MTFTP_GET_FILE_SIZE operation 
> to allocate appropriately sized buffers for the files it wants to load.
>
> Regards,
>
> Daniel
>
> *Von:*Barrelfish-users <barrelfish-users-bounces at lists.inf.ethz.ch 
> <mailto:barrelfish-users-bounces at lists.inf.ethz.ch>> *Im Auftrag von 
> *sebastian.scherbel at fau.de <mailto:sebastian.scherbel at fau.de>
> *Gesendet:* Dienstag, 2. März 2021 22:56
> *An:* Cock David <david.cock at inf.ethz.ch <mailto:david.cock at inf.ethz.ch>>
> *Cc:* barrelfish-users at lists.inf.ethz.ch 
> <mailto:barrelfish-users at lists.inf.ethz.ch>
> *Betreff:* Re: [Barrelfish-users] [Barrelfish] Hagfish kernel image 
> structure and latest version
>
> Hello,
>
> >Hagfish definitely supports chainloading over PXE - that's exactly what it was designed for.
>
> Which bootloaders have you tested? What does your setup look like?
>
> I have tried:
>
> EDK2 -> iPXE -> Hagfish
>
> /iPXE> chain tftp://x.x.x.x/Hagfish_ETHZ.efi/
>
> /...
> Hagfish UEFI loader starting
> UEFI vendor: EDK II
> Hagfish loaded at 402D9000, size 147456B, by handle 41839B98
> Could not connect to shell or not enough parameters, assuming PXE boot.
> Connecting to the PXE service that loaded me.
> PXE loader at 403ED660, revision 10000, running
> Loading "hagfish.cfg.10.0.0.10"
> *Mtftp: Unsupported,
> file size: Unsupported*/
>
> EDK2 -> Grub2 -> Hagfish
>
> /grub> boot
> Status: 0x0000000E
> Failed to initialize ShellLib, aborting.
> Hagfish UEFI loader starting
> UEFI vendor: EDK II
> Hagfish loaded at 402DA000, size 147456B, by handle 41862A18
> Could not connect to shell or not enough parameters, assuming PXE boot.
> Connecting to the PXE service that loaded me.
> *OpenProtocol: Unsupported
> Failed to initialize loader: Load Error*/
>
> > The error there seems to be the UEFI implementation not supporting TFTP - did you 
> compile it in?
>
> When I load Hagfish directly, everything works fine. On American 
> Megatrends Aptio V and EDK2.
>
> Regards Sebastian
>
> Am Donnerstag, 28. Januar 2021, 16:36:02 MEZ hat David Cock 
> <david.cock at inf.ethz.ch <mailto:david.cock at inf.ethz.ch>> Folgendes 
> geschrieben:
>
> What you're seeing there is Hagfish trying to load its configuration 
> file over TFTP as it's figured out that it was itself loaded over 
> PXE.  The README might be inconsistent wrt filenames - if in doubt, 
> trust the code. Hagfish definitely supports chainloading over PXE - 
> that's exactly what it was designed for.
>
> The error there seems to be the UEFI implementation not supporting 
> TFTP - did you compile it in?
>
> David
>
> On 28/1/21 3:53 pm, sebastian.scherbel at fau.de 
> <mailto:sebastian.scherbel at fau.de> wrote:
>
> Hello,
>
> sounds good. I have tried to chainload Hagfish after iPXE. However, 
> loading of the cfg fails on QEMU and on the real hardware with the 
> same error.
>
> /Hagfish UEFI loader starting
> UEFI vendor: American Megatrends
> Hagfish loaded at F951D000, size 143360B, by handle F6EA1D18
> Could not connect to shell or not enough parameters, assuming PXE boot.
> Connecting to the PXE service that loaded me.
> PXE loader at F5EE1840, revision 10000, running
> Loading "hagfish.cfg.A.B.C.D"*(shouldn't the cfg name be 
> hafish.A.B.C.D.cfg according to the readme?)*
> *Mtftp: Unsupported, */
>
> */file size: Unsupported/*
>
> Is this a known bug? Is chainloading unsupported? Your setup is 
> directly booting into Hagfish?
>
> Regards Sebastian
>
> Am Donnerstag, 28. Januar 2021, 14:03:36 MEZ hat David Cock 
> <david.cock at inf.ethz.ch> <mailto:david.cock at inf.ethz.ch> Folgendes 
> geschrieben:
>
> Sebastian,
>
> Hagfish is a custom loader for the Barrelfish CPU driver, which 
> operates under a few unusual assumptions: Mainly, the CPU driver 
> (kernel) itself is not self-relocating, but relies on the bootloader 
> (Hagfish) to do the relocation for it.
>
> It would definitely be possible to modify Hagfish to load an arbitrary 
> ELF, however.  You'd have to make a few changes in the relocation code 
> to not look for Barrelfish-specific segments and so on, but there's 
> nothing particularly weird or non-standard about it.
>
> Hagfish is pretty static, and doesn't change much from version to 
> version.  We'll update the repository shortly (which we should have 
> done together with the release). Feel free to ask more specific 
> questions if you decide to use it - we're happy to help.
>
> David
>
> On 27/1/21 5:41 pm, sebastian.scherbel at fau.de 
> <mailto:sebastian.scherbel at fau.de> wrote:
>
> Hello,
>
> I'm a CS student trying to port an research operating system from 
> x86_64 to ARMv8 and stumbled across Hagfish.
>
> Hagfish should basically be able to load any ELF with its sections 
> into memory with minor modifications? Or is there certain requirement 
> for the kernel image structure that I didn't notice while reading your 
> publication [1] and skimming the source code?
>
> The last commit in the public repository is from March 2017. But in 
> the changelog of your last Barrelfish release from 2020 states "Update 
> Hagfish to latest version". Have there been any bigger changes or 
> improvements in the meantime?
>
> Regards Sebastian
>
> [0] 
> https://lists.inf.ethz.ch/pipermail/barrelfish-users/2020-March/001597.html 
> <https://lists.inf.ethz.ch/pipermail/barrelfish-users/2020-March/001597.html>
>
> [1] http://www.barrelfish.org/publications/TN-022-ARMv8.pdf 
> <http://www.barrelfish.org/publications/TN-022-ARMv8.pdf>
>
> _______________________________________________
>
> Barrelfish-users mailing list
>
> Barrelfish-users at lists.inf.ethz.ch 
> <mailto:Barrelfish-users at lists.inf.ethz.ch>
>
> https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users 
> <https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users>
>
> _______________________________________________
> Barrelfish-users mailing list
> Barrelfish-users at lists.inf.ethz.ch 
> <mailto:Barrelfish-users at lists.inf.ethz.ch>
> https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users 
> <https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users>
>
> _______________________________________________
>
> Barrelfish-users mailing list
>
> Barrelfish-users at lists.inf.ethz.ch 
> <mailto:Barrelfish-users at lists.inf.ethz.ch>
>
> https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users 
> <https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users>
>
> _______________________________________________
> Barrelfish-users mailing list
> Barrelfish-users at lists.inf.ethz.ch 
> <mailto:Barrelfish-users at lists.inf.ethz.ch>
> https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users 
> <https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users>
>
>
> _______________________________________________
> Barrelfish-users mailing list
> Barrelfish-users at lists.inf.ethz.ch  <mailto:Barrelfish-users at lists.inf.ethz.ch>
> https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users  <https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users>
>
> _______________________________________________
> Barrelfish-users mailing list
> Barrelfish-users at lists.inf.ethz.ch
> https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.inf.ethz.ch/pipermail/barrelfish-users/attachments/20210415/8d64d3f1/attachment-0001.html>

More information about the Barrelfish-users mailing list