[Barrelfish-users] New Barrelfish Release

[Simon Gerber]

Dear all,

We are pleased to announce a new release of the Barrelfish OS. Along
with various bugfixes, this release includes a new implementation for 
distributed capability management based on work by Mark Nevill.

The system works roughly as follows:
* Each capability has an owner which is responsible to synchronize 
operations that need synchronization in order to preserve system-wide 
consistency.
* The operations that need synchronization are implemented in the 
monitors with synchronization being implemented as broadcasts on the 
intermon bindings and called whenever a local capability management 
invocation returns with the error message SYS_ERR_RETRY_THROUGH_MONITOR.
* These calls are done as blocking calls to the local monitor.

For a more detailed explanation, please have a look at Mark Nevill's 
master's thesis available on the Barrelfish website [1].

As always, you can get the latest version by cloning or pulling from our 
git repository [2].

Cheers,
-- Simon

[1] http://www.barrelfish.org/nevill-master-capabilities.pdf
[2] git://git.barrelfish.org/git/barrelfish

---
Gerd Zellweger (2):
       Add regex parsing for skb_simple get_names call.
       Cleaning up the fish mess.

Kornilios Kourtis (1):
       vfs_fat: quick fix for potential off-by-one

Mark Nevill (117):
       Pseudo-implemented "copy" operation.
       Implemented move, put "transport" functions in own .c file.
       Implemented delete op, added broadcast helpers.
       Added some code for revoke&delete, does not compile.
       Added retype, made things more compilable.
       Renamed capops/transport to capops/capsend.
       Suitably prefixed all "capsend" functions.
       Split over-generic "send to all" with "to copies" and "to 
descendants".
       Reordered some things in revoke.c for clarity.
       Reindented intermon.if.
       Added capop messages to intermon.if.
       Moved find_descendants into capsend.c.
       Made capops state type more opaque in intermon.if.
       Added distcap state to struct cte in kernel.
       Added invocations for distcap_get_state.
       Integrated error codes into errno.fugu.
       Fixed missing/implicit includes in libbarrelfish headers.
       Added basic cap locking system to capops.
       Migrated capops to clearer "locking" semantics.
       Integrated capops with monitor_cap_create.
       Split up and moved around monitor's private includes.
       Implemented get/set owner monitor invocations.
       Added support for cap types that do not need locality.
       Prefix remaining unimplemented invocatios with "monitor_".
       Implemented monitor_(un)lock_cap.
       Changed API to better support caps from other domains.
       Added monitor_create_caps invocation.
       Integrated delete & revoke handling.
       Fixed cap lock/unlock calls to allow lookup from custom root cap.
       Fixed handling of root cap lookup in some monitor invocations.
       Disable revoke on dispatcher exit as this currently breaks the 
monitor.
       Fixed copypasta error in monitor invocations.
       Re-enabled "remote_relations" in mdb and fixed over-eager memory 
reclamation.
       Removed leftover printk from cleanup_copy.
       Fixed direct comparison of errvals without err_no.
       Properly set cap ownership when creating new caps in the kernel.
       Fixed get/set owner invocation argument handling in kernel.
       Manually set cap ownership for memory used by newly spawned core.
       Catch NULL monitor rpc client in cap monitor rpcs.
       Added a hack in mem_serv to initialize its monitor rpc client.
       Disabled mem_serv monitor_rpc_client hack again.
       Disabled cap_set_remote monitor rpc.
       Fixed whitespace in monitor.if.
       Added valid bits param to cap copy result.
       Moved capops headers into monitor's include dir, added function 
prefix.
       Export capop rx handlers so they can be used in vtbls.
       Changed copy result to cnode & slot.
       First stage of integrating "copy" into ump cap transfer.
       Removed unused and unimplemented "monitor_revoke" stub.
       Implemented monitor_copy_existing invocation.
       Made caps_try_delete static in cap_delete.c as it has no external 
references.
       Fixed not copying over cap owner in caps_copy_to_cte.
       Added basic framework for tracing in-kernel cap operations 
covering a particular memory region.
       Added and fixed various debug printfs related to cap operations.
       Fixed "copy" to properly clean up temporary caps and set copied 
cap to remote.
       Fixed recreating bad RAM caps when reclaiming memory.
       Handle NULL cte in TRACE_CAP macro.
       Added message argument to cap tracing to ease debugging.
       Fixed condition in is_retypeable check to handle foreign caps.
       Completed implementation of "give away" cap support.
       Fixed inter-monitor memory allocation.
       Disabled kernel mem cap tracing.
       Use (now functioning) lmp give_away_cap to implement ump 
give_away_cap.
       Fixed some wrongly aligned cptrs in invocations from capops.
       Fixed uninitialized state pointer in retype request message state.
       Fixed some unshifted cptrs in monitor invocations.
       Handle tx_busy errors in queued send continuations.
       Properly cleanup caps during alloc through monitor_mem.
       Revert "Properly cleanup caps during alloc through monitor_mem."
       Properly cleanup caps during alloc through monitor_mem, take 2.
       Workaround for multi-retype hack.
       Implemented "has_descendants" for retype.
       Fixed calling find_descendants result handler multiple times.
       Fixed some cap locking and other issues in capops. It boots!
       Removed some debugging prints.
       Properly use lock queueing when trying to delete a locked cap.
       Do not engage monitors in distributed cap ops until mem alloc is 
setup.
       Fixed inconsistent rx handler names in capops.
       Set remote relations for new foreign caps that have no local copies.
       Copy remote relations attribute when copying caps between slots.
       Fixed cptr alignment in various places.
       Use give_away_cap to implicitly delete caps being sent to local 
domains for a channel.
       Do not assert for no remote relations in caps_delete_last as 
monitor ensures this anyway.
       Set remote_relations when deserializing caps.
       Added individual remote relations bits for 
copies/ancestors/descendants.
       Catch null rpc client in barrelfish/debug.c.
       Fixed double slot free in update_owner__rx_handler.
       Fixed spurious cptr shift in delete when move fails.
       Fixed incorrect parameter ordering in capops_move.
       Fixed some missing includes.
       Added layer to encapsulate transferring caps across intermon.
       Added function to initialize intermon rx_vtbl capop entries.
       Handle null capref in aborted captx recv.
       Removed redundant coreid definition from monitor_mem.if.
       Implemented support for "give_away" transfers in capops_copy.
       Fixed cap locking in revoke.
       Removed some remaining printfs from capability operations.
       Improved naming & cleanup in copy capop.
       Consistently use conts to avoid duplicate error handling code in 
copy op.
       Removed dead code from monitor_rpc_server.
       Refactored retype in monitor.
       Reinsert minted IO caps into MDB.
       Improve captrace output for minting.
       Fixed not setting "to" field of copy capop state.
       Include local and remote relations when xferring ownership.
       Incomplete cleanup of delete capop to use conts.
       Added "retrieve" operation for revoke.
       Rewrote distributed revoke and delete handling.
       Removed trailing spaces from xcorecap test.
       Fixed alloc sizes in xcorecapbench so it runs.
       Keep cptrs right-shifted throughout capops.
       Minor fixes to error handling.
       Removed unused NYI "monitor_set_cap_deleted" function.
       Fixed reversed condition in delete_remote handling.
       Implemented monitor_delete_foreign invocation.
       Do not send capops-related broadcasts to self.
       Removed rcap_db from monitor.

Moritz Hoffmann (2):
       vfs_fat: fix another potential off-by-one bug in vfs, also 
reported by gcc 4.9
       vfs_fat: correctly handle error cases when dealing with long file 
names.

Reto Achermann (1):
       moved apic init() before the printf of the barrelfish to get the 
correct apic_id
git shortlog of changes since the last release:


Simon Gerber (114):
       Merge master into distops.
       Fixed kernel Hakefile that got messed up in merge.
       Fixed merge errors in kernel/include/capabilities.h
       Fixed capability system merge errors. Builds, boots and promptly 
crashes.
       Fixed some issues in revoke machinery.
       Added explanatory printf in revoke machinery; fixed DEBUG_ERR in 
spawnd:cleanup_cap().
       monitor: Fixed missing list cleanup in capops/revoke.
       Added comment in spawnd.
       Added configurable toggles for libmdb printf tracing and strict 
checking.
       Added debug syscall & Config.hs variable for physmem capability 
tracing.
       More fixes in capability machinery.
       Fixed missing check for incomplete vnode chain in paging_generic.c
       Added call to unmap_capability in cleanup_copy().
       Added missing cap types to debug_print_cap in lib/barrelfish and 
to sprint_cap in the kernel.
       Added exception for CHAN_ALREADY_REGISTERED error in 
event_queue_add() trigger step.
       Use get_address() for vnodes instead of assuming identical layout 
in cleanup after deleting a cap.
       Make monitor_identify_cap return a fabricated null cap when 
called with a capref to an empty slot.
       Added some new trace output in cap revocation and deletion code.
       Added new mdb test for operations on root node.
       Added missing ifdefs in TRACE_PMEM_CTRL syscall.
       Fixed mdb_skew and mdb_split to properly update root node if 
necessary.
       Merge branch 'master' into master-into-distops
       Added missing includes after merge.
       Merge branch 'master' into master-into-distops
       Mark's capabilities merged with upstream.
       flounder: fixed UMP capability receive stub to properly call 
receive handler under all circumstances.
       monitor debug stuff + other debug msgs.
       Added forwarding of free'd ram to core 0 + more debug messages
       fixed stuff + more output.
       Revert "flounder: fixed UMP capability receive stub to properly 
call receive handler under all circumstances."
       Hacked x-core cap transfer to short-circuit on null caps.
       turn off monitor and spawn client debugging.
       flounder: Fixed wrong DEBUG_ERR in UMP.
       changed some debug prints in monitor and removed non-necessary 
debug output in fish and flounder stubs.
       capops: captx_get_capref understands null cap msgs.
       fixed missing perfcount_type in brie machine data.
       Fixed assertion that fails to compile with assertions disabled.
       Make x86_32 compile; some of the monitor invocations are disabled 
right now.
       monitor: removed unused monitor_identify_cnode_get_cap.
       x86_32: simple test works.
       armv7: compiles.
       asmoffsets: fixed missing size-specifier for some ints.
       armv7: fixed stuff and added missing invocation handlers.
       Add race prevention code back in for architectures that boot 
cores through spawnd.
       Fixed corner case in capsend broadcast when we only have one core.
       temporarily disable argc check in invocation macro.
       Trying to debug why fish can't connect to spawnd.
       Debugging pandaboard lockups.
       TEMP: Merge branch 'master' into master-into-distops
       arm_molly: reformatted and squashed build errors.
       tmp
       arm_gem5: fixed calls to caps_create_new to supply core_id.
       armv7: added cap ownership transfer for cpu and monitor regions 
for non-BSP cores.
       Fixed monitor arm code.
       Fixed argument mixup in memcpy in monitor ram_alloc code.
       monitor: disabled debugging output.
       Merge branch 'master' into master-into-distops
       fixed merge errors in usr/spawnd/service.c
       Trace all mdb inserts and deletes.
       Revert "Debugging pandaboard lockups."
       Added new mdb tracing option and some assertions.
       kernel: fixed clear list in cap delete code to be stack instead 
of queue.
       pmem tracing: make default enable types to trace all types.
       Revert "Trying to debug why fish can't connect to spawnd."
       Resurrected blocking in pandaboard version of Kaluga until 
spawnd.0 up.
       Removed yet another debug printf.
       Workaround for perfmon_amd_supported() causing page fault.
       x86_32: Added missing monitor invocation: monitor_get_arch_id.
       monitor: wrapped debug print in #ifdef
       tmp: debugging pandaboard fish not able to spawn domains.
       Merge branch 'master' into master-into-distops
       Removed a bunch of debugging output.
       monitor: fixed revoke and delete paths for single-core execution.
       Changed fish for arm to give session cap to spawned programs.
       Cleaned up remaining debug printfs.
       armv5: fixed build errors.
       arm: unified and deduplicated kernel-side syscall handling.
       Added noreturn attribute to mdb_dump_and_fail().
       Merge branch 'master' into distops
       Implemented simple octopus-based num_monitors_online() query.
       Removed last traces of rcap_db stuff.
       Added non-recursive invariant checks in libmdb.
       Added missing flounder bindings to octopus in monitor.
       Fixed double-init of delete stepping machinery.
       Catch broadcast targets that are not ready for distops yet.
       capsend broadcast: can now remember set of destination cores for 
operations involving >1 broadcast.
       Fixed vmkit harness test to point to new NFS export.
       libvfs: Added explanatory comments on LFN handling in vfs_fat.
       Added park/unpark operations in corectrl to not have problems 
with domain cleanup when a KCB is not scheduled anywhere.
       Added deferred-event based monitor heartbeat.
       Merge branch 'master' into distops
       Added waiting for periodicprint in some coreboot tests.
       kernel: arm: removed left-over arm-specific sys_monitor_spawn_core().
       Fixed libbarrelfish spawn_client() and fish_arm to use standard 
functionality.
       Fixed bogus return value in num_monitors_online() and added 
warning printf.
       armv5: Added condition when adding free phys regions.
       armv5: added memtest in menu.lst.
       libmdb: added config flags and macros for toplevel invariant 
checking.
       armv5: fixed printf format specifier.
       Merge Mark Nevill's distributed capability system.
       Disabled mdb invariant checking by default, added new harness 
build that enables invariants.
       harness: Added possibility to set test timeout on per-machine basis.
       Added missing code to bind to octopus in monitor.
       Fixed typo in libmdb Hakefile and removed duplicate invariant 
checking.
       monitor: Return sensible #monitors online on octopus error.
       libmdb: Improved top-level assertion checking preprocessor switch.
       libmdb: Fixed broken CHECK_INVARIANTS macro when 
MDB_RECHECK_INVARIANTS defined.
       libmdb: fixed CHECK_INVARIANTS() macro (again).
       libmdb: fixed potention NULL deref in mdb_is_reachable().
       Fixed potential NULL deref in eclipse lexer.
       Added missing include for assert() in eclipse lex.c.
       harness: Added per-machine test timeout.
       harness: Fixed interactive test not picking up right default timeout.
       harness: Removed superfluous debug print.