From roni.haecki at inf.ethz.ch Thu Oct 12 15:34:29 2017 From: roni.haecki at inf.ethz.ch (haeckir) Date: Thu, 12 Oct 2017 15:34:29 +0200 Subject: [Barrelfish-users] [Barrelfish] New Barrelfish Release Message-ID: <9c3895d7-afd7-6f36-3180-91ac85b54ee4@inf.ethz.ch> Dear all, We are pleased to announce a new release of the Barrelfish OS. Along with various bug fixes, this release includes a new process manager, driver support for the Mellanox ConnectX-3 NIC, and a new DSL called Sockeye for describing hardware. As always, you can get the latest version by cloning or pulling from our git repository [1]. Cheers, -- Roni [1] git://git.barrelfish.org/git/barrelfish --- Adam Turowski (4): flounder: proper token reading when a buffer is sent first Fixup of some headers. flounder: moving receiving cap slots from a thread to a dispatcher mlx4: import of Vlad's Mellanox ConnectX InfiniBand driver Daniel Schwyn (194): First take on Sockeye AST First take at Sockeye Parser Small refactoring Better error message for addresses Add compiler main function, just runs parser and prints the AST for now Some refactoring in main file Add hakefile Some cleanup Support parsing singleton address blocks Add support for omitting accept or translate block Add parser support for overlay. !!Will be parsed but ignored atm!! Add support for adding multiple nodes at once with the 'are' keyword Ensure parser reaches end of file Refactor to match abstract syntax in paper more closely Improve parse error messages for node specs Add support for specifying blocks via prefix/length Add check for duplicate identifiers Refactor AST Add check for references to undefined nodes Improve error message for failed check Add support for ommitting destination address in map Add support for multiple destinations in map Clean up Add support for plugin in different backends Improve option handling Allow '_' in identifiers Add option to specify output file Add stub for prolog backend Use unlines to print lines First take on Prolog backend decoding net predicates (need to be moved to SKB at some point) Make sure atoms start with lower case Fix order in some predicates to make search tree (considerably) smaller Fix translate predicate Make sure the @none for null-overlays is treated as an atom Start using constraints for decoding net predicates Predicates: Use constraints for addresses and backtracking for nodes make net predicate dynamic Refactor predicates Add predicates to print all origin/target ranges of a node Add first versions of predicates for configuration queries Rename new sockeye binary to sockeye2 Add Sockeye file for OMAP4460 Move Prolog predicates to usr/skb/programs Do not generate preamble when compiling Sockeye files to Prolog Merge remote-tracking branch 'upstream/skate' into sockeye Rename sockey2 -> sockeye Integrate sockeye with hake Add types to node Use node type for shared memory query Start more complete spec of OMAP4460 Add support for '-' in node IDs Make checker case insensitive OMAP4460 spec with all interconnects Group devices by modules Improve error message on syntax errors with 'is'/'are' Ensure all node identifers are compiled to Prolog atoms Use device type for device frame query First take on complete OMAP4460 specs Add power/clock management details to OMAP specs Add device name finder query Make Sockeye case sensitive again Add CORTEXA9 node to OMAP spec Refactor decoding net skb programs Fix CORTEXA9 -> L3 mapping Start writing technote Add section about checks to technote Remove omap4460_paper.soc Remove naive decoding net implementation Merge remote-tracking branch 'upstream/master' into sockeye Merge remote-tracking branch 'upstream/master' into sockeye Separate node IDs by whitespace for 'are' instead of commas Technote: Add syntax description Technote: Clean up Remove test .soc file from Hakefile Technote: Add hake chapter Technote: Fix bibliography Sockeye: Generate hex addresses for Prolog Technote: Describe Prolog mapping Technote: clean up OMAP4460 spec: Fix address mapping for CKGEN_CM1 OMAP4460 spec: Fix address mapping for L3_OCM_RAM in L3 SKB: Refactor decodingNet modules Backport OMAP44xx spec changes from page-tables branch Technote: Add more syntax examples Technote: Compile Sockeye examples as part of build process Technote: Add TODOs for missing parts Sockeye: New AST for Frontend Sockeye: Complete new AST for Frontend Sockeye: Correct imports Sockeye: WIP: Parser for new syntax Sockeye: Parser for new syntax Sockeye: Better error messages for parser Sockeye: Correct file headers for ASTs Sockeye: Only one list for port mappings Sockeye: Introduce for loops Sockeye: Fix Identifier parsing Sockeye: Allow to define ports with inline range in identifier Sockeye: Allow do define port mappings with inline range in identifier Sockeye: Refactoring Sockeye: Allow to use inlined for in node decls Sockeye: Allow to instantiate modules with inlined range in name space Sockeye: fix typo Sockeye: Start implementation of checker Sockeye: Checker almost finished Sockeye: Finish checker for NodeSpec Sockeye: Checker now checks everything Sockeye: Cleanup checker Sockeye: Better error messages for checker Sockeye: Rename 'int' type to 'nat' Sockeye: Add stub for net builder Sockeye: Refactor ASTs Sockeye: Checker code improvements Sockeye: Add check for self instantiation Sockeye: Start implementation of net builder Sockeye: Start implementation of net builder Sockeye: Implement nodeSpec conversion Sockeye: Better naming for some AST types Sockeye: Implement checks in net builder Sockeye: Adapt prolog backend to new AST Sockeye: Start implementing port mappings Sockeye: Start implementing new simplification approach for net builder Sockeye: Finish AST simplification Sockeye: Start implementing decoding net transformation Sockeye: Implement proper module instantiation check Sockeye: Implement port mappings Sockeye: Remove debug imports Sockeye: Represent NodeId as name + namespace list in Prolog Update omap44xx spec to new Sockeye syntax Sockeye: Implement import support Sockeye: Switch default mapping to 0x0 if no base address given Sockeye: Start implementing overley to map translation Sockeye: Finish implementation of overlay to map translation DecodingNet: Adapt resolution and queries to nodes without overlay Sockeye: Handle arbitrary large numbers Sockeye: clean up Adapt OMAP44xx spec to new sockeye Sockeye: Add possibility to add reserved blocks Sockeye: Fix import system Sockeye: Enable generation of dependency file Start modularizing omap44xx spec Sockeye: Enable generating make depend file during compilation Hake: Ddd dependency tracking for sockeye Merge branch 'sockeye-modules' into sockeye Merge remote-tracking branch 'upstream/master' into sockeye Sockeye TN: adapt example to syntax changes Sockeye: TN remove limitations chapter, update hake chapter Sockeye TN: Add short introduction about decoding nets Sockeye: Improve import handling Sockeye TN: Update usage info Sockeye TN: More updates Sockeye TN: Add description of template indentifiers Sockeye TN: Complete syntax chapter Sockeye TN: Update hake section Sockeye TN: Fix some typos Sockeye TN: Start updating checks section Sockeye: Refactor arguments type check Sockeye: Start implementing port checks Sockeye: Implement module instantiator Sockeye: Start reimplementing net builder on top of instantiator Sockeye: Reimplement port translation Sockeye: Rename Checker to TypeChecker Sockeye: Clean up ASTs Sockeye: Collect errors in type checker instead of failing at first one Sockeye: Fix duplicate variable check for nested for loops Sockeye: Show global and root module fails first Sockeye: Consier input mappings and output ports in duplicate identifier check Sockeye: implement reference check inside node declarations Sockeye: Implement reference check for port mappings Sockeye: Implement reference check in input port declarations Sockeye: Implement check for undefined ports Sockeye: Change default dest base for mapping to 0 (again...) Sockeye: Fix empty top level namespace being compiled to empty atom in Prolog Sockeye: Code cleanup Sockeye TN: Update chapter on checks Sockeye: Generate decimal addresses Revert "Sockeye: Generate decimal addresses" Sockeye: Generate addresses with ISO hex prefix Kaluga: Fix bug with new driver model code Sockeye: Add node type for cores Sockeye: Use ECLiPSe struct notation in generated code SKB: Adapt decoding net scripts to use struct notation Sockeye TN: Update info about node types Sockeye TN: Update info about Prolog mapping Sockeye TN: Use examples from OMAP4460 Merge remote-tracking branch 'upstream/master' into sockeye Prolog code style fixes Sockeye TN: Adapt Prolog mapping to snake case Decoding net: Refactor Decoding net: Cleanup core module Sockeye TN: Use example environment for examples Decoding net: Fail on region resolution with non contiuous domain for address Merge remote-tracking branch 'remotes/upstream/master' into sockeye Merge branch 'sockeye' into master Lukas Humbel (1): Removing unused declarations Razvan Damachi (23): First outline of a process management service server. Refactor process manager to enable spawnd discovery. Create "ProcessManager" and "Domain" capabilities. Add spawn, spawn_with_caps and span calls to the Process Manager API. Implement kill_request_handler in spawnd, using domain caps. Implement kill() and exit() in the process manager. Implement "wait" in the process manager + fix bug in kill reply handler. Extract a through-monitor-only cap_identify call from debug_cap_identify. Have the spawn interface for the process manager require cap_procmng. Break the spawnd kill API into kill + cleanup. Add per-spawnd message queues to the process manager. Fix a bug where pending cleanup clients wouldn't be added on exit calls. Remove the mutex-based event queuing in the process manager. Add extra layer of queuing above the Flounder UMP one. Change proc_mgmt/pending_clients.h to use reply queues instead of hashtables. DIRTY benchmark commit Fix bug in libdomain/domain_cap_hash() Fix typo in if/proc_mgmt.if Preallocate an L2 cnode and fill it with domain caps in proc_mgmt. Change 'exit' from RPC to message in if/proc_mgmt.if. Clean up and document the process management code. Tidy up the process management test. Merge branch 'master' of ssh://code.systems.ethz.ch:8006/source/barrelfish Reto Achermann (3): fixing compilation for release mode remove unsave uses of __builtin_return_address() Revert "remove unsave uses of __builtin_return_address()" Roni H?cki (78): devif: devif_debug test fixed libnet: dhcp header file removed unneeded include harness: fixing devif test solarflare: conversion of IP for filter no longer required harness: fixed devif test processing bug e10k: added missing dependency harness: devif test fix parsing destination IP on emmentaler2 harness: devif test simplified getting IP devif: devif test disable debugging output libnet: added small library to get a hardware queue devif: prototype of UDP queue libnet: libnet now uses queue creation library libnet: queue creation small fix devif: refactored udp queue devif: adding test for udp queue harness: added udp test to devif tests devif: "null" backend which simply forwards everything devif: added benchmarking code to test overhead of stacking queues devif: upd backend bugfix in valid_length computation libnet: added possiblity to query own IP address devif: udp queue simplified creation harness: devif udp test adapted arguments to simplified creation devif: added uncommited files devif: debug fixed case when IDC backend us used devif: destroy no longer queue specific devif: IDC backend changed destroy devif: networking backends to destroy called from queue interface devif: changed rest of the queues to library destroy devif: solarflare backend deallocated some memory twice devif: fixed some printfs that seem to cause compilation errors with automated tests harness: vmkit test added net_socket module libnet: added functionality to request ARP lookup sent from default q devif: added dst MAC lookup to UDP queue init harness: devif test gethostip sometimes fails, fallback added. kaluga: waiting for all spawnds to be up before starting drivers/apps libnet: ARP resending more robust with periodic events libnet: HW filter disable/enable on queue creation libet: HW filter flag fixed libnet: also show cardname when dhcp is done webserver: check error when initalizing net_sockets client devif: solarflare reading out all error codes from RX events devif: library fixed compilation with debugging flags enabled libnet: bugfix when using interrupts libnet: avoid failing setting up filter with e1000 like cardnames e10k/solarflare: no longer start network stack for default queue net_sockets: client library checks all available net_socket services net_sockets: change service so it can be used for all the NICS. kaluga: starting net_socket server for each of the cards. harness: added nfscat tests that use e10k and solarflare NICs harness: devif test only spawn required net_socket server octopus: skb queries increased buffer size libnet: fixed debugging mode harness: webserver tests added test using e10k/solarflare devif: solarflare backend IP_FRAG error should not be treated as an error ... nfs: reduced time until retransmit e10k: added missing dependency harness: xeon phi test did not contain proc_mgmt module schedsim: fixed building of test proc_mgmt: make sure that exit message is sent before things are cleaned up proc_mgmt: adding tests proc_mgmt: Hake file simplified proc_mgmt: bugfix for domains spawned from startd harness: small additions to proc_mgmt test examples: spawntest changed to spawn using proc_mgmt startd: fixed dependency issue proc_mgmt: fixed several dependency issues libbarrelfish: started merging proc_mgmt_client and spawn_client proc_mgmt: merged proc management client with spawnd client proc_mgmt: implementation spawn_wait() with nohang flag proc_mgmt: getting domain list implemented proc_mgmt: get status query implemented collections: added comment to hash table traversal reset proc_mgmt: xeon phi translation did to cap implemented xeonphi: adapted to new spawn calls xeonphi: wait for spawnd 0 to be up before trying to spawn on core 0 usb manager: converted spawn_program call to new cap based interface harness: devif test load only e10k driver on babybel2 Merge remote-tracking branch 'origin/proc_mgmt_unify' Simon Gerber (26): harness: memtest: properly fail memtest_multicore when no memtest spawned Fix various memory leaks and uninitialized variable uses as reported by cppcheck hake: armv7_gem5_vexpressemm: Fix armv7 gem5 menu.lst to supply a device db to kaluga stdio: add __printflike() to ANSI C functions T381: x86_64: properly handle racy page table allocations in pmap code libbarrelfish: pinned: refill slabs in 64 page chunks instead of page by page libbarrelfish: pinned: reserve 128MB for pinned slabs, up from 4MB monitor: capops: ownership xfer, delete, retype, revoke: delete temp capabilities when operations done monitor: capops: copy: properly zero copy state structs on allocation monitor: capops: copy: add DEBUG_CAPOPS and clearer messages in forwarding path monitor: capops: move: reformat some function definitions Use min/max macros from bitmacros.h instead of including sys/param.h libmdb: wrap function that is only used with assertions enabled in #ifndef NDEBUG omap44xx: sdma: remove redundant redefinitions of MIN() x86_64: kernel: do *not* include machine/param.h hake: x86_64: re-enable -Werror for cpu driver vmkitmon/arrakismon: do not include sys/param.h libskb: include sys/types.h (instead of sys/param.h) to get u_char and co. kernel: caps_mark_revoke: fix marking of cap copies for revoke harness: fix release build to ignore unused variables harness: machines: define custom buildall targets for armv7/armv8 machines kernel: capabilities: wrap check_caps_create_arguments() in #ifndef NDEBUG because it is only called from assert() kernel: caps_mark_revoke: fix assertions after marking copies kernel: capabilities: treat full object retypes with identical source and dest types as copy Merge Razvan's process management code. armv7,armv8: Add process manager to targets and menu.lsts