[Barrelfish-users] Hake building

[Tim Kelly]

Dear Timothy,
Thank you for the detailed response.  I appreciate the time you took to 
write it.  It's been my unfortunate experience working solo that if I 
can't control the toolchain, I will spend a lot of time fixing things 
that other people break in the name of progress.  I had hoped that 
perhaps the C code in Barrelfish could be built by handwritten Makefiles 
(as many as necessary to separate components), but if there is embedded 
DSL code and Prolog code, either of which being dependent on Haskell, 
then this creates quite a challenge.

 From my reading of GHC, every option for getting GHC working on a new 
OS starts with "use a working copy of GHC from another platform."  At 
this stage of the project, my invoking another OS like Ubuntu would be 
an admission of being wrong (should have just used it from the start, 
what's the point in using this other piece), and it does impinge on my 
belief BSD/MIT licensed tools should use BSD/MIT licensed tools to build.

I think Barrelfish is an exciting project, and I've been a fan of 
exokernels since reading about MIT's exopc/XOK.  I need Barrelfish or 
something like it because the OS I'm using is not multiprocessor aware. 
  I'll keep looking at the code, and any pointers on how to isolate the 
DSL and Prolog components are appreciated.  I'll have to figure out if 
it will be easier to build something to compile those parts or just make 
the OS run on four different cores (most likely the latter).

Did Lua not meet the needs that GHC filled?  Perhaps that's naive on my 
part, but I've certainly used it to generate C code and even disassemble 
(PowerPC) object code.  Lua can also bootstrap itself (with an embedded 
machine code file), so all one needs is a working C compiler.

tim

Timothy Roscoe wrote:
> Dear Tim,
> 
> What you're trying to do (running different kernels on different
> cores) makes sense - Barrelfish is designed so that the kernel
> makes no assumptions about having the whole machine to itself, and we
> regularly run different kernels alongside each other in the same
> system.
> 
> However, if you want use a completely different for building
> Barrelfish, you might have some work to do. 
> 
> Given the vagaries of compilers and utilities, it's really hard to
> support building a complete OS using multiple build environments, and
> we're just a university research group.  We support Ubuntu LTS
> (currently 16.04), and the getting started guide has a list of the
> main Ubuntu packages we require to build Barrelfish.  In practice, we
> had to pick a toolchain to support, and so we chose standard Ubuntu.
> Even the Microsoft folks would build Barrelfish with Ubuntu inside
> HyperV.
> 
> Note that we take care not to use any GPLed code in the OS itself
> (hence the MIT-style licence), but as you point out we employ many GPL
> tools in the toolchain.
> 
> Barrelfish uses a number of domain-specific languages (of which Hake
> is one); these are all implemented using Haskell, either as embedded
> DSLs or, more commonly, a compiler from the DSL to C.  Haskell turns
> out to be a highly convenient language for implementing little
> compilers.
> 
> No Haskell code is executed at runtime.  When the OS boots, it's all
> hand-written C, C generated by DSLs implemented in Haskell, or Prolog.
> 
> Hake is used to solve the problem of simultaneously building both
> parts of the toolchain, and the OS and libraries themselves for
> multiple architectures all at once (since Barrelfish images can
> contain multiple different architecture targets).
> 
> The way Hake works is to take the specifications in all the Hakefiles
> in the source tree, and generate a single, very large but very simple
> Makefile for the whole OS.  We are, in fact, big fans of make - it is
> incredibly fast at building source, particularly when there are hardly
> any symbolic targets in the Makefile, and so we consciously avoid
> replacing it with anything.  
> 
> The "source directory" that Hake (or rather, its bootstrap script
> hake.sh) needs is simply the top of the Barrelfish source directory.
> It's a good idea to have your build directory (the cwd when you run
> hake.sh) as separate from this, due to the way Hake searches the
> source tree for Hakefiles.  It looks from your example like you're
> including your build directory inside the source tree, which works,
> but can be inconvenient and/or slow. 
> 
> Hope this helps,
> 
>  -- Timothy Roscoe
> 
> At Sat, 24 Mar 2018 16:55:39 -0400, Tim Kelly <gtkelly at dialectronics.com> wrote:
>> Hello all,
>>
>> I have a project in which Barrelfish looks to be a great solution.  I'm
>> working with an unsupported BSD-licensed single processor (unthreaded)
>> OS (obviously not a *BSD).  I've got a BSD-licensed toolchain (not
>> clang).  The toolchain builds itself and the OS.  I have a particular
>> executable to run on SBC hardware, which will be dedicated for running
>> that executable (no, I'm not bitcoin mining, it's systems of ODEs).  The
>> SoC is multi-core x86, and I have code that can be parallelized
>> (ensembles with different initial conditions).  I'd like to run the OS I
>> have on CPU0 and then run Barrelfish on CPU[1-3], using shared memory
>> (2-4G total on-chip).  In theory, perhaps the OS could run on top of
>> Barrelfish and I could run my executable on all four CPUs.
>>
>> However, even getting started is proving a bit difficult.  Barrelfish
>> builds with Hake, but building Hake requires the Glasgow-Haskell
>> compiler (GHC).
>>
>> # ../hake/hake.sh -s ../
>> Install directory defaulting to '.'
>> Source directory is ../
>> Architectures to build: "x86_64"
>> Setting up hake build directory...
>> You already have Config.hs, leaving it as-is.
>> Building hake...
>> ghc: not found
>>
>> (It's a little unclear to me from TN-003-Hake what /home/barrelfish/src
>> is supposed to be.  There's no src/ directory in the download.  I am
>> using /home/barrelfish as the source directory.  I ran the hake.sh file
>> from the build/ directory.)
>>
>> GHC requires several non-BSD-licensed tools to build (perl, python, git,
>> gnu binutils, etc).  Other than nano and gunzip, I'm not using any GNU
>> tools (that I know of, I replace them when I find them with BSD-licensed
>> alternatives).  It's bad enough I have to use configure, but a lot of
>> times I can run the stock Makefile or use the .in template with a few
>> tweaks.  I'm not a fan of make, but replacing it with something that
>> requires GHC to be built is a bit of a tall order for my project.
>>
>> Is the Haskell language used in any of the Barrelfish OS code itself?
>>
>> Thank you,
>> tim
>>
>> --
>> Every gun that is made, every warship launched, every rocket fired,
>> signifies, in the final sense, a theft from those who hunger and are not
>> fed, those who are cold and are not clothed.
>>
>> 		-- Dwight D. Eisenhower, 34th U.S. President
>> _______________________________________________
>> Barrelfish-users mailing list
>> Barrelfish-users at lists.inf.ethz.ch
>> https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users
>>
>>
>>

-- 
A problem solver is someone that makes the problem fit the solution.