[Barrelfish-users] [Barrelfish] Hagfish kernel image structure and latest version

David Cock david.cock at inf.ethz.ch
Fri Mar 12 12:08:27 CET 2021 

Sebastian

It looks like what's going on is as Daniel described.  Hagfish seems to 
use an API call (get_file_size) that's supported by EDK2, but not by 
many commercial offerings.  I think it'd almost certainly be possible to 
update Hagfish such that it doesn't depend on it - something like 
preallocating a "reasonably-sized" buffer, and if it turns out it wasn't 
big enough trying again with a larger one.  It's been a few years since 
I wrote this stuff, but I think you'll get back an EFI_BUFFER_TOO_SMALL 
if it was too small, and *BufferSize will be filled with the actual size 
so that you can reallocate the buffer.  We'd be happy to integrate a 
patch if you wanted to modify it.

David

On 11/3/21 2:19 pm, Schwyn Daniel wrote:
>
> Hi Sebastian,
>
> David and I are both part of the Barrelfish core team at ETH so our 
> setup is the same😉We do load Hagfish over PXE but as far as I know 
> always directly with the PXE implementation in EDK2. Hagfish is 
> theoretically designed to be flexible and cover as many EFI based boot 
> environments as possible. In practice it’s only tested on the machines 
> we have. Those are EDK2 based and the PXE implementation in there 
> supports the two MTFTP opcodes we use. It should be possible to either 
> implement support in iPXE or make Hagfish not use the get_file_size 
> e.g. by allocating a large enough buffer and make sure the read_file 
> doesn’t overflow it.
>
> Regards,
>
> Daniel
>
> *Von:*sebastian.scherbel at fau.de <sebastian.scherbel at fau.de>
> *Gesendet:* Mittwoch, 10. März 2021 16:38
> *An:* Schwyn Daniel <daniel.schwyn at inf.ethz.ch>
> *Cc:* barrelfish-users at lists.inf.ethz.ch
> *Betreff:* Re: AW: [Barrelfish-users] [Barrelfish] Hagfish kernel 
> image structure and latest version
>
> Hi Daniel,
>
> thank you for your reply. That's what I already suspected.
>
> I'm waiting to hear back from David about his setup. He seemed quite 
> sure that chainloading works, so I was a bit surprised that neither 
> iPXE nor Grub did work for me on EDK2.
>
> Otherwise, I'll fix iPXE myself.
>
> Regards Sebastian
>
> Am Mittwoch, 10. März 2021, 10:59:48 MEZ hat Schwyn Daniel 
> <daniel.schwyn at inf.ethz.ch <mailto:daniel.schwyn at inf.ethz.ch>> 
> Folgendes geschrieben:
>
> Hi Sebastian,
>
> I have seen this error before with iPXE when I tried chainloading 
> Hagfish with iPXE on top of U-Boot with EFI support. U-Boot’s EFI 
> support is rather minimalistic so I’m not sure the cause was the same 
> in my case. Looking at the iPXE source code though it looks to me as 
> if it only supports the EFI_PXE_BASE_CODE_MTFTP_READ_FILE opcode while 
> Hagfish also needs the EFI_PXE_BASE_CODE_MTFTP_GET_FILE_SIZE operation 
> to allocate appropriately sized buffers for the files it wants to load.
>
> Regards,
>
> Daniel
>
> *Von:*Barrelfish-users <barrelfish-users-bounces at lists.inf.ethz.ch 
> <mailto:barrelfish-users-bounces at lists.inf.ethz.ch>> *Im Auftrag von 
> *sebastian.scherbel at fau.de <mailto:sebastian.scherbel at fau.de>
> *Gesendet:* Dienstag, 2. März 2021 22:56
> *An:* Cock David <david.cock at inf.ethz.ch <mailto:david.cock at inf.ethz.ch>>
> *Cc:* barrelfish-users at lists.inf.ethz.ch 
> <mailto:barrelfish-users at lists.inf.ethz.ch>
> *Betreff:* Re: [Barrelfish-users] [Barrelfish] Hagfish kernel image 
> structure and latest version
>
> Hello,
>
> >Hagfish definitely supports chainloading over PXE - that's exactly what it 
> was designed for.
>
> Which bootloaders have you tested? What does your setup look like?
>
> I have tried:
>
> EDK2 -> iPXE -> Hagfish
>
> /iPXE> chain tftp://x.x.x.x/Hagfish_ETHZ.efi/
>
> /...
> Hagfish UEFI loader starting
> UEFI vendor: EDK II
> Hagfish loaded at 402D9000, size 147456B, by handle 41839B98
> Could not connect to shell or not enough parameters, assuming PXE boot.
> Connecting to the PXE service that loaded me.
> PXE loader at 403ED660, revision 10000, running
> Loading "hagfish.cfg.10.0.0.10"
> *Mtftp: Unsupported,
> file size: Unsupported*/
>
> EDK2 -> Grub2 -> Hagfish
>
> /grub> boot
> Status: 0x0000000E
> Failed to initialize ShellLib, aborting.
> Hagfish UEFI loader starting
> UEFI vendor: EDK II
> Hagfish loaded at 402DA000, size 147456B, by handle 41862A18
> Could not connect to shell or not enough parameters, assuming PXE boot.
> Connecting to the PXE service that loaded me.
> *OpenProtocol: Unsupported
> Failed to initialize loader: Load Error*/
>
> > The error there seems to be the UEFI implementation not supporting 
> TFTP - did you compile it in?
>
> When I load Hagfish directly, everything works fine. On American 
> Megatrends Aptio V and EDK2.
>
> Regards Sebastian
>
> Am Donnerstag, 28. Januar 2021, 16:36:02 MEZ hat David Cock 
> <david.cock at inf.ethz.ch <mailto:david.cock at inf.ethz.ch>> Folgendes 
> geschrieben:
>
> What you're seeing there is Hagfish trying to load its configuration 
> file over TFTP as it's figured out that it was itself loaded over 
> PXE.  The README might be inconsistent wrt filenames - if in doubt, 
> trust the code.  Hagfish definitely supports chainloading over PXE - 
> that's exactly what it was designed for.
>
> The error there seems to be the UEFI implementation not supporting 
> TFTP - did you compile it in?
>
> David
>
> On 28/1/21 3:53 pm, sebastian.scherbel at fau.de 
> <mailto:sebastian.scherbel at fau.de> wrote:
>
> Hello,
>
> sounds good. I have tried to chainload Hagfish after iPXE. However, 
> loading of the cfg fails on QEMU and on the real hardware with the 
> same error.
>
> /Hagfish UEFI loader starting
> UEFI vendor: American Megatrends
> Hagfish loaded at F951D000, size 143360B, by handle F6EA1D18
> Could not connect to shell or not enough parameters, assuming PXE boot.
> Connecting to the PXE service that loaded me.
> PXE loader at F5EE1840, revision 10000, running
> Loading "hagfish.cfg.A.B.C.D"*(shouldn't the cfg name be 
> hafish.A.B.C.D.cfg according to the readme?)*
> *Mtftp: Unsupported, */
>
> */file size: Unsupported/*
>
> Is this a known bug? Is chainloading unsupported? Your setup is 
> directly booting into Hagfish?
>
> Regards Sebastian
>
> Am Donnerstag, 28. Januar 2021, 14:03:36 MEZ hat David Cock 
> <david.cock at inf.ethz.ch> <mailto:david.cock at inf.ethz.ch> Folgendes 
> geschrieben:
>
> Sebastian,
>
> Hagfish is a custom loader for the Barrelfish CPU driver, which 
> operates under a few unusual assumptions: Mainly, the CPU driver 
> (kernel) itself is not self-relocating, but relies on the bootloader 
> (Hagfish) to do the relocation for it.
>
> It would definitely be possible to modify Hagfish to load an arbitrary 
> ELF, however.  You'd have to make a few changes in the relocation code 
> to not look for Barrelfish-specific segments and so on, but there's 
> nothing particularly weird or non-standard about it.
>
> Hagfish is pretty static, and doesn't change much from version to 
> version.  We'll update the repository shortly (which we should have 
> done together with the release).  Feel free to ask more specific 
> questions if you decide to use it - we're happy to help.
>
> David
>
> On 27/1/21 5:41 pm, sebastian.scherbel at fau.de 
> <mailto:sebastian.scherbel at fau.de> wrote:
>
> Hello,
>
> I'm a CS student trying to port an research operating system from 
> x86_64 to ARMv8 and stumbled across Hagfish.
>
> Hagfish should basically be able to load any ELF with its sections 
> into memory with minor modifications? Or is there certain requirement 
> for the kernel image structure that I didn't notice while reading your 
> publication [1] and skimming the source code?
>
> The last commit in the public repository is from March 2017. But in 
> the changelog of your last Barrelfish release from 2020 states "Update 
> Hagfish to latest version". Have there been any bigger changes or 
> improvements in the meantime?
>
> Regards Sebastian
>
> [0] 
> https://lists.inf.ethz.ch/pipermail/barrelfish-users/2020-March/001597.html 
> <https://lists.inf.ethz.ch/pipermail/barrelfish-users/2020-March/001597.html>
>
> [1] http://www.barrelfish.org/publications/TN-022-ARMv8.pdf 
> <http://www.barrelfish.org/publications/TN-022-ARMv8.pdf>
>
> _______________________________________________
>
> Barrelfish-users mailing list
>
> Barrelfish-users at lists.inf.ethz.ch 
> <mailto:Barrelfish-users at lists.inf.ethz.ch>
>
> https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users 
> <https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users>
>
> _______________________________________________
> Barrelfish-users mailing list
> Barrelfish-users at lists.inf.ethz.ch 
> <mailto:Barrelfish-users at lists.inf.ethz.ch>
> https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users 
> <https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users>
>
> _______________________________________________
>
> Barrelfish-users mailing list
>
> Barrelfish-users at lists.inf.ethz.ch 
> <mailto:Barrelfish-users at lists.inf.ethz.ch>
>
> https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users 
> <https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users>
>
> _______________________________________________
> Barrelfish-users mailing list
> Barrelfish-users at lists.inf.ethz.ch 
> <mailto:Barrelfish-users at lists.inf.ethz.ch>
> https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users 
> <https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users>
>
>
> _______________________________________________
> Barrelfish-users mailing list
> Barrelfish-users at lists.inf.ethz.ch
> https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.inf.ethz.ch/pipermail/barrelfish-users/attachments/20210312/cf58657a/attachment-0001.html>

More information about the Barrelfish-users mailing list