[Barrelfish-users] 3-way Handshake IDC

[Stefan Kaestle]

Hi Dominic,

Unfortunately, we do not have access to SCC hardware any more so it is 
hard for us to reproduce SCC related problems.

One thing to keep in mind is that the x86-default flounder backend for 
cross-core message passing (UMP) does not work on the SCC. This is due 
to a lack of cache coherence. Instead, we use UMP-IPI, an extension to 
UMP that sends a notification (i.e. an inter-processor interrupt) to 
indicate message availability to the receiver. The receiver can then 
invalidate its cache to make sure that the next read is done from memory 
(and not the outdated cache).

 From looking at your code, I am a bit suspicious about the way you send 
messages (although I might be wrong and that is actually not the 
problem). I am assuming that export and connect work?
In general, a good starting point for developing IPC applications is to 
look at some of our examples in usr/examples.

I suggest the following:

  1. Can you please try to run usr/examples/xmpl-call-response on your 
SCC and tell me if that works?
  2. Convert xmpl-call-response to do the 3-way handshake
  3. Test on x86_32 in qemu
  4. Test on SCC

Let me know if this helps!

Also, can you please attach the following additional information to your 
next email:

  1. Your menu.lst
  2. A list of changes to hake/Config.hs
  3. Output when you run the program (e.g. does connect/export work)
  4. Your Barrelfish release, gcc and ghc version


Hope this helps,
     Stefan

On 11/19/2014 06:54 AM, Dominic Hung wrote:
> Dear Team Barrelfish,
>
> I am currently having a trouble with doing a 3-way handshake like IDC. 
> What I would like to achieve is
>
> Core 0 -- Send Syn -> Core 1
> Core 0 <- Send Syn/Ack -- Core 1
> Core 0 -- Send Ack --> Core 1
>
> I got no luck doing this on my original system, the message can be 
> sent from core 0 with no error code but core 1 keep looping waiting 
> for event. Having troubleshooting for quite some time, then I turned 
> to wrote a smaller scale testing program to verify that 3-way 
> handshake can be done. And I still got the same result, i.e., core 0 
> sent ack with no error code while core 1 spin waiting for event with 
> no arrival and delivery.
>
> My platform is again... I am sorry, Intel SCC. The latest tag I can 
> compile with Intel SCC is release2012-11-03 and the real tag I am now 
> using is release2012-06-06. Hope you may help.
>
> The micro program is as attached.
>
> #include "stdio.h"
> #include "stdbool.h" // FOR BOOL
> #include <errors/errno.h> // FOR ERRVAL_T
> #include "arch/x86_32/barrelfish_kpi/eflags_arch.h"
> #include "arch/x86_32/barrelfish_kpi/paging_arch.h"
> #include "barrelfish_kpi/syscalls.h" // FOR STRUCT SYSRET
> #include "barrelfish/capabilities.h"
> #include "if/pingpong_defs.h"
> #include "barrelfish/domain.h"
> #include "barrelfish/spawn_client.h"
> #include "barrelfish/waitset.h"
> #include "barrelfish/nameservice_client.h"
> #include "rck_dev.h"
>
> struct pingpong_binding* binding;
> int flag_connect = 0;
> int flag_ack = 0;
>
> void syn_handler(struct pingpong_binding *b){
> debug_printf("arrived at syn\n");
>
> b->tx_vtbl.syn_ack(b, NOP_CONT);
>
>     do{
>         printf("wait ack\n");
> event_dispatch(get_default_waitset());
>     } while(!flag_ack);
> }
>
> void syn_ack_handler(struct pingpong_binding *b){
> debug_printf("arrived at syn_ack\n");
> }
>
> void ack_handler(struct pingpong_binding *b){
> debug_printf("arrived at ack\n");
>
>     flag_ack = 1;
> }
>
> struct pingpong_rx_vtbl pingpong_rx_vtbl = {
>     .syn     = syn_handler,
>     .syn_ack = syn_ack_handler,
>     .ack     = ack_handler,
> };
>
> static void bind_cb(void *st, errval_t err, struct pingpong_binding *b)
> {
>     if (err_is_fail(err)) {
> USER_PANIC_ERR(err, "bind failed");
>     }
>
>     printf("client bound!\n");
>
>     // copy my message receive handler vtable to the binding
>     b->rx_vtbl = pingpong_rx_vtbl;
>
>     binding = b;
> }
>
> static void start_client(void)
> {
>     iref_t iref;
>     errval_t err;
>
>     printf("client looking up '%s' in name service...\n", 
> "pingpong_service");
>     err = nameservice_blocking_lookup("pingpong_service", &iref);
>     if (err_is_fail(err)) {
> USER_PANIC_ERR(err, "nameservice_blocking_lookup failed");
>     }
>
>     printf("client binding to %"PRIuIREF"...\n", iref);
>     err = pingpong_bind(iref, bind_cb, NULL, get_default_waitset(), 
> IDC_BIND_FLAGS_DEFAULT);
>     if (err_is_fail(err)) {
> USER_PANIC_ERR(err, "bind failed");
>     }
>
>     while(binding == NULL)
> event_dispatch(get_default_waitset());
> }
>
> /* ------------------------------ SERVER ------------------------------ */
>
> static void export_cb(void *st, errval_t err, iref_t iref)
> {
>     if (err_is_fail(err)) {
> USER_PANIC_ERR(err, "export failed");
>     }
>
>     printf("service exported at iref %"PRIuIREF"\n", iref);
>
>     // register this iref with the name service
>     err = nameservice_register("pingpong_service", iref);
>     printf("service exported at iref %"PRIuIREF"\n", iref);
>     if (err_is_fail(err)) {
> USER_PANIC_ERR(err, "nameservice_register failed");
>     }
>     printf("service exported at iref %"PRIuIREF"\n", iref);
> }
>
> static errval_t connect_cb(void *st, struct pingpong_binding *b)
> {
>     printf("service got a connection!\n");
>
>     // copy my message receive handler vtable to the binding
>     b->rx_vtbl = pingpong_rx_vtbl;
>
>     flag_connect = 1;
>
>     // accept the connection (we could return an error to refuse it)
>     return SYS_ERR_OK;
> }
>
> static void start_server(void)
> {
>     errval_t err;
>
>     err = pingpong_export(NULL, export_cb, connect_cb, 
> get_default_waitset(), IDC_EXPORT_FLAGS_DEFAULT);
>     if (err_is_fail(err)) {
> USER_PANIC_ERR(err, "export failed");
>     }
>
>     do{
> event_dispatch(get_default_waitset());
>     } while(!flag_connect);
> }
>
> int main(int argc, char* argv[]){
> debug_printf("%d\n", disp_get_core_id());
>
> if(disp_get_core_id() == 0){
> spawn_program(1, "/scc/sbin/pingpong", argv, NULL, SPAWN_NEW_DOMAIN, 
> NULL);
>
> start_server();
>     }
>     else start_client();
>
> if(disp_get_core_id() == 1){
> binding->tx_vtbl.syn(binding, NOP_CONT);
> event_dispatch(get_default_waitset());
> binding->tx_vtbl.ack(binding, NOP_CONT);
> printf("sent ack\n");
>     }
>     else{
> event_dispatch(get_default_waitset());
>     }
>
>     return 0;
> }
>
> Thanks again!
>
> Cheers,
> Dominic Hung
>
> --------------------------------------------------
> C H Dominic Hung, B.Eng. (CE) HK
> M. Phil. Student, Dept. of CS., Faculty of Engg.
>
> Mobile: +852-9819-9360
> Email: domchdh at hku.hk <mailto:domchdh at hku.hk>
>
>
> _______________________________________________
> Barrelfish-users mailing list
> Barrelfish-users at lists.inf.ethz.ch
> https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users

-- 
Stefan
http://people.inf.ethz.ch/skaestle/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.inf.ethz.ch/pipermail/barrelfish-users/attachments/20141119/1dae8ba7/attachment-0001.html