[Barrelfish-users] FW: Memorry allocation weirdness

Wed Dec 7 12:27:19 CET 2011

Well now I do know that it is expected behavior. Sometimes we spend so much time on the really complicated issues that the most fundamental details evade us. Sorry for all the fuss.

cheers,
Georgios
-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

"Tim Harris (RESEARCH)" <tharris at microsoft.com> wrote:

Hi,

This sounds like it could be a double-free problem, or a memory trampling problem.  That would explain a lot of odd behaviour.

If you've not already then its worth checking:

- The sizes passed to malloc are correct, particularly making sure that you allocate sizeof(foo) where needed and not sizeof(foo*).

- None of the accesses through computed pointers or array indexes are out of bounds.

- Check the addresses passed to free are identical to those passed to malloc, and there is no double-freeing going on.

At a high level, what you're doing sounds fine -- so it looks like a low level C problem rather than something specific to Barrelfish.

How big is the actual code?  Happy to take a look, in case a second pair of eyes would help. 

--Tim 

Sent from my Windows Phone

_____________________________________________
From: Georgios Varisteas
Sent: 07/12/2011 00:05
To: barrelfish-users at lists.inf.ethz.ch
Subject: [Barrelfish-users] FW: Memorry allocation weirdness

So, given the scenario I described in the first email, but disregarding the crashes I mention, assume the following structures:

typedef struct bdis_client_tbl {
        bdis_core_usage_t *core_usg;    // ARRAY of core usages, one entry per core.
        bdis_num_cores_t num_cores;             // number of cores in the System
        hash_table **process_state;             //
} *bdis_client_tbl_t;

typedef struct bdis_process_state {
...
        bdis_coreid_t *cores;                   // collection of allotted cores
        bdis_num_cores_t num_cores;             // number of allotted cores
...
} *bdis_process_state_t;

typedef struct bdis_core_usage {
        bdis_core_usage_p_t *pids;              // array of processes using this core
        bdis_sz_t count;                                // counter
} *bdis_core_usage_t;

The first is the main strctures used by the  server to keep all information.
The second stores the state of each client. These are stored in the hash table of the first structure.
The first field of the first structure (core_usg) is meant to be an array of the third structure.

Now cleaning up after an application disconnects, the server might free the "pids" field of each instance of the third structure. In my example I'm using single-threaded applications so that would be only one.

Then the second application comes, I allocate a new process state and put it in the hash table. Then the server terminates with a page fault while trying to access the "cores" field of the process state. This is again manually allocated memory meant to be used as an array.
By the way I do check each allocation and they all succeed.

However, if "pids" is not freed, the server wont crash. It will continue running normally accepting more clients.

Normally these two (client_tbl->core_usg[0]->pids and an entry in the process_state hash table) would be totally unrelated data structures and should not affect each other. In any case I cannot understand what is wrong here. Please help.

cheers,
Georgios Varisteas

________________________________________
From: Georgios Varisteas [yorgos at kth.se]
Sent: Wednesday, December 07, 2011 00:46
To: barrelfish-users at lists.inf.ethz.ch
Subject: [Barrelfish-users] FW: Memorry allocation weirdness

Actually what I wrote in the previous email is not totally true. After 12 hours of debugging I got very confused. Please disregard it until I write a more correct and technical version.

--Georgios

________________________________________
From: Georgios Varisteas [yorgos at kth.se]
Sent: Wednesday, December 07, 2011 00:31
To: barrelfish-users at lists.inf.ethz.ch
Subject: [Barrelfish-users] Memorry allocation weirdness

Hello,

What guarantees does barrelfish provide for memory allocated by applications? Does the kernel clean up the memory when the application terminates? If yes does that extend to all of its domain? And what kind of consistency checks does malloc() provide? Is the allocated memory fragmented?

I'm only asking because of some very weird behavior I witness. Let me give the most recent example:

I have a server client design. I'm also interested for the server to handle multiple instances of applications simultaneously or not.
Let's assume server S on core 0. After it starts I execute client application A1 on core 0 and I wait for it to finish. S of course allocates memory to keep the status of A1. After A1 terminates it signals S which cleans up that memory, aka it calls free() on the related structures.
Everything looks fine to this point.
Then I start a new instance of the same application, lets call it A2.
A2 will crash with a page fault. If I do not free the memory that S allocates, A2 will NOT crash.
A2 will not crash if I span it to more cores and/or if I start it on a different core.

I have checked my code character by character and I cannot understand how freeing memory on the Server can cause the client to crash. Especially since the client executes after the free() command on S has finished. My only guess is that I assume free() to work in a very different way than it does.

Please shed some light to all of this...

Best regards,
Georgios

_______________________________________________
Barrelfish-users mailing list
Barrelfish-users at lists.inf.ethz.ch
https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users

_______________________________________________
Barrelfish-users mailing list
Barrelfish-users at lists.inf.ethz.ch
https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users

_______________________________________________
Barrelfish-users mailing list
Barrelfish-users at lists.inf.ethz.ch
https://lists.inf.ethz.ch/mailman/listinfo/barrelfish-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.inf.ethz.ch/pipermail/barrelfish-users/attachments/20111207/0d962257/attachment.html 