[Barrelfish-users] Threads, domains and their confusing API.

[Georgios Varisteas]

Hi,

I'm doing research on shared memory user level scheduling for barrelfish and I'm trying to figure out the "correct way" of working with threads and domains in the latest release. So, I'm going to present what I have figured out thus far, expecting some further insight compensating for the lack of documentation (sorry if there actually is documentation lying around somewhere and I've missed it).

Taking it from the top, there is of course the threads library that replicates some basic functionality of pthreads. However, barrelfish has the extra notion of domains; hence the domain.c library contains wrappers for most but not all functions (call me join). Also, I have found a nice set of higher level wrapper functions in bomp that inspired some ideas.

What I require at this point is to replicate the exact API of Linux's nptl, as I have Linux code that I need to migrate. This means that I require pthread_create() and pthread_join() functions, with automatic scheduling of the created threads on the physical cores. I have written a layer that interfaces the pthread-based code with the bf threads library, also spanning the domain to use all cores. What I'm missing is a single function which will create a new thread on a specific core (dispatcher) and also return that thread structure. From what I've seen so far I can achieve this in two imperfect ways:

1. Use thread_create_varstack() (threads.c) to create the thread on the same dispatcher and then move it with domain_thread_move_to() (domain.c). This method is simply one function call too much.
2. Use domain_thread_create_on() (domain.c) and somehow get the thread to return an instance of its descriptor structure with thread_self() (threads.c). This method is far too complicated and requires me to change the legacy Linux code which I desperately want to avoid.

I know that there has to be a straightforward way to achieving this with one simple function call and I am simply missing it. I could always alter the domain.c API to return the thread structure (as a return value or a void * parameter like in pthreads), but I thought of asking before changing anything. Any ideas? Thanks in advance.

On an other matter, are there any already-developed tools to get simple execution measurements like time, memory usage and core utilization?

Best regards,
Georgios Varisteas