AnsweredAssumed Answered

vrf Memory Eater...dll problems

Question asked by VRFuser on Jan 5, 2005

Like • Show 0 Likes0
Comment • 6

> How is the DLL compiled?

Don't know. I suspect it's probably MSVC or GNU. At any rate, simple problem
easily solved, but quite difficult to explain!
-SHAWN-

---
You are currently subscribed to vrf as: rsb@soco.agilent.com
To subscribe send a blank email to "join-vrf@it.lists.it.agilent.com".
To unsubscribe send a blank email to "leave-vrf@it.lists.it.agilent.com".
To send messages to this mailing list, email "vrf@agilent.com".
If you need help with the mailing list send a message to "owner-vrf@it.lists.it.agilent.com".

No one else has this question

Outcomes

6 Replies

VRFuser Jan 5, 2005 1:42 PM
Mark Correct
Correct Answer
> I can view the source code for the dll.

At least that's helpful. One easy thing to do is count up all the malloc and
free calls, or GlobalAlloc and GlobalFree, or whatever calls it's using to
do memory management... at any rate, the total should be even. More
specifically for each alloc there should be a free. If not, then it *may* be
a problem.

> Headerfile:

K. I don't see anything that is immediately indicative of a problem except
pRxStr. This is an out pointer - a pointer to a string pointer - and
actually I'm impressed you can use this.

The usual scenario with stuff like this is that the dll allocates enough
memory for the returned string and stuffs the pointer to it in the passed
address - a pointer to a string pointer. The dll assumes that the caller
frees the memory.

If that's the case, then you're right. That's where your problem is. VEE
will not free externally allocated memory. At least I'm pretty sure it
won't. It might, but that kind of attention to detail would be over and
above the call on the part of the CFI designers and to depend on it would
probably be foolish.

In fact, it will interpret this as a pointer to a string array. That it
works is somewhat surprising - if the returned string is a multiple of 16
bytes long you might run into problems with this (VEE might not see proper
termination). If you don't have to use array notation to get at the returned
string I'm amazed: somebody is performing some kind of magic and I'd love to
know what it is.

At any rate, to determine if this is a problem, check the source code for
something like this:

*pRxStr = malloc(lstrlen(pSomething));
or
*pRxStr = malloc(iRxStrLen);

The malloc may be GlobalAlloc, LocalAlloc or HeapAlloc. If you find it, then
that is definitely a problem. The solution is relatively simple though it
will take a bit of programming hassle. It can be done either with straight
VEE or a wrapper dll.

Of course the dll documentation should say exactly how to handle it. It
could be that it was designed specifically for use with VEE and it does
interpret pRxStr as a previously allocated string array, but that doesn't
seem likely. It's also possible that the function expects a pointer to a
previously allocated single buffer (as is common in the COM API with
notation such as LPVOID *ppv - see CoCreateInstance). That doesn't seem very
likely either.
-SHAWN-

---
You are currently subscribed to vrf as: rsb@soco.agilent.com
To subscribe send a blank email to "join-vrf@it.lists.it.agilent.com".
To unsubscribe send a blank email to "leave-vrf@it.lists.it.agilent.com".
To send messages to this mailing list, email "vrf@agilent.com".
If you need help with the mailing list send a message to "owner-vrf@it.lists.it.agilent.com".
Like • Show 0 Likes0
Actions
VRFuser Jan 5, 2005 2:58 PM
Mark Correct
Correct Answer
> Is it possible to make memory alloc for pRxStr as shown
> in this example?

Kind of

I have to install VEE 7 again on my recently catastrophically crashed
computer & take a look at it.

Anyway, there are two ways to go about fixing a possible problem. First, the
dll source is essential to confirm that there actually is a problem. Then
either modify your current VEE code to correct the problem (which will be
kind of traumatic I'm afraid - it always is when you have to deal with
memory management in VEE) or create a dll specifically to release possibly
hanging allocations & give VEE something it can deal with in it's automatic
memory allocation scheme (which is not nearly as traumatic).

Either way, the dll source is still necessary because the free has to match
the alloc. If transmitting the source is out of the question, just let me
know which allocator is being used. If it's "new" then it's malloc, but it's
probably either GlobalAlloc or HeapAlloc. If it's HeapAlloc then
modification of the dll source is almost a must. HeapFree needs a heap
handle that the dll keeps to itself. It's somewhat feasible to have another
dll or VEE hijack this heap, but it would take way to long to figure it out
and hijacking a heap isn't a nice thing to do anyway.

I'm installing VEE right now and will send further details shortly.
-SHAWN-

---
You are currently subscribed to vrf as: rsb@soco.agilent.com
To subscribe send a blank email to "join-vrf@it.lists.it.agilent.com".
To unsubscribe send a blank email to "leave-vrf@it.lists.it.agilent.com".
To send messages to this mailing list, email "vrf@agilent.com".
If you need help with the mailing list send a message to "owner-vrf@it.lists.it.agilent.com".
Like • Show 0 Likes0
Actions
VRFuser Jan 5, 2005 4:04 PM
Mark Correct
Correct Answer
Hmmm bummer.

I have to restart and I can't do that for another three hours.
-SHAWN-

---
You are currently subscribed to vrf as: rsb@soco.agilent.com
To subscribe send a blank email to "join-vrf@it.lists.it.agilent.com".
To unsubscribe send a blank email to "leave-vrf@it.lists.it.agilent.com".
To send messages to this mailing list, email "vrf@agilent.com".
If you need help with the mailing list send a message to "owner-vrf@it.lists.it.agilent.com".
Like • Show 0 Likes0
Actions
VRFuser Jan 5, 2005 7:59 PM
Mark Correct
Correct Answer
I can view the source code for the dll. But I haven't got acces to debug it...

Headerfile:
int __stdcall MsgClientInit(void);
int __stdcall MsgClientTxStr( char *TxStr );
int __stdcall MsgClientQueryStr( char *pTxStr, int Primitive, char **pRxStr );
int __stdcall MsgClientQueryStrEx( char *pTxStr, int Primitive, char **pRxStr, int Timeout );

Br Susanne

-----Original Message-----
From: Shawn Fessenden [mailto:shawn@testech-ltd.com]
Sent: 6. januar 2005 11:47
To: Susanne Ejby Sndergaard Jrgensen; vrf@it.lists.it.agilent.com
Subject: RE: [vrf] RE: Memory Eater

> Hello Shawn,

Greetings Susanne!

> I think I might have some problems with memory leaks.

Yuck! These are never fun to try to find.

> But I think my problem is related to a dll.

That could be fortunate - do you have access to the source code?

> I guess I can use the windows task manager to monitor
> the memory usage. Or do you have other suggestions?

[Oh boy, this gets very long and boring The short answer is use a debugger - the task manager can't possibly give you a clear idea of what's going on.]

Thank you *very* much for bringing that up Susanne. The Performance tab of the task manager can give you a general feel for what's going on, but there are at least two reasons you can't use it to seriously track memory usage.

First and foremost, both versions of Windows allocate memory in pages. 8K is also known as a "granule" (i.e., member dwAllocationGranularity of SYSTEM_INFO structure as used with GetSystemInfo). One page is 8K (for Win32 on a Pentium. Other processors sometimes use other sizes). Whenever a process allocates or frees memory, it does so in units of 8K. It is the responsibility of the virtual memory manager to keep track of process memory. Units of memory are handled within this chunk by the C runtime library or it's equivalent for programs that do not link the CRT. The Win32 API functions GlobalXXX, LocalXXX and HeapXXX all use a thread safe version of the CRT malloc and free functions.

The task manager tries to estimate the actual memory usage of all processes by walking their default heaps, but skips auxiliary heaps (those created with HeapCreate). In any case, it does not actually break down each heap pool and analyze exactly what's going on with each one - that would take far too long. So the task manager's memory display is a best guess estimate that is sometimes on the low side.

And the second reason is because it displays a summary of *all* processes. It can't zero in on one process alone. And no matter what you do, there are always at the very least a dozen or more processes running. Right now the laptop that I use for mail is running 32, only a half dozen or so of which are superfluous.

Memory usage details can change surprisingly when a context switch occurs (when Windows "task switches"). As little as possible is done to facilitate performance, but sometimes some pages can be swapped none the less - especially with resource hogs. The task manager is a normal process. By the time it gets to do it's thing, VEE memory usage probably has already changed and it will not see these changes.

The only way to get an accurate picture of a single process's memory usage is to debug that process with a *good* debugger. It can be done with windbg - the debugger that ships with the Windows SDK but there are much better choices available. A debugger sinks it's teeth into the process being debugged and monitors nearly every aspect of the process. In particular, each and every time memory is allocated and freed a record is generated. Most conveniently, when the target process terminates the debugger can give you a complete report on exactly what memory is left hanging around, exactly when it was allocated and exactly what code at what address allocated it.

VEE's memory leaking habits are by no means critical. In some situations they can accumulate to become so however. In fact, I'm willing to bet that they are a lot less significant than a good deal of commonly used software - even that from Microsoft. Don't kid yourself into thinking that this is unusual - it takes a very carefully crafted piece of software to prevent memory leaks of one kind or another, and as RAD tools became more and more prevalent that proportion of software that did leak went way, way up. The reason this is not generally known is because nobody cares.

Nobody except us that is. We have to create software that will run for days, weeks, and months on end flawlessly. Day in and day out, without rebooting the machine. Well, GenApp development doesn't have to do that. You use a program for a few hours and then close it and shut off the machine and go home. It's a matter of perspective. Crappy, leaky software abounds because, quite simply, it can.
-SHAWN-

---
You are currently subscribed to vrf as: rsb@soco.agilent.com
To subscribe send a blank email to "join-vrf@it.lists.it.agilent.com".
To unsubscribe send a blank email to "leave-vrf@it.lists.it.agilent.com".
To send messages to this mailing list, email "vrf@agilent.com".
If you need help with the mailing list send a message to "owner-vrf@it.lists.it.agilent.com".
Like • Show 0 Likes0
Actions
VRFuser Jan 5, 2005 9:01 PM
Mark Correct
Correct Answer
Shawn,

How is the DLL compiled? If using MS Windows VS, then there exist some
things that can be added, else, ' might want to recompile and include some
useful "stuff" from .NET.

Rufus

-----Original Message-----
From: Shawn Fessenden [mailto:shawn@testech-ltd.com]
Sent: Thursday, January 06, 2005 6:42 AM
To: VRF
Subject: [vrf] RE: Memory Eater...dll problems

> I can view the source code for the dll.

At least that's helpful. One easy thing to do is count up all the malloc and
free calls, or GlobalAlloc and GlobalFree, or whatever calls it's using to
do memory management... at any rate, the total should be even. More
specifically for each alloc there should be a free. If not, then it *may* be
a problem.

> Headerfile:

K. I don't see anything that is immediately indicative of a problem except
pRxStr. This is an out pointer - a pointer to a string pointer - and
actually I'm impressed you can use this.

The usual scenario with stuff like this is that the dll allocates enough
memory for the returned string and stuffs the pointer to it in the passed
address - a pointer to a string pointer. The dll assumes that the caller
frees the memory.

If that's the case, then you're right. That's where your problem is. VEE
will not free externally allocated memory. At least I'm pretty sure it
won't. It might, but that kind of attention to detail would be over and
above the call on the part of the CFI designers and to depend on it would
probably be foolish.

In fact, it will interpret this as a pointer to a string array. That it
works is somewhat surprising - if the returned string is a multiple of 16
bytes long you might run into problems with this (VEE might not see proper
termination). If you don't have to use array notation to get at the returned
string I'm amazed: somebody is performing some kind of magic and I'd love to
know what it is.

At any rate, to determine if this is a problem, check the source code for
something like this:

*pRxStr = malloc(lstrlen(pSomething));
or
*pRxStr = malloc(iRxStrLen);

The malloc may be GlobalAlloc, LocalAlloc or HeapAlloc. If you find it, then
that is definitely a problem. The solution is relatively simple though it
will take a bit of programming hassle. It can be done either with straight
VEE or a wrapper dll.

Of course the dll documentation should say exactly how to handle it. It
could be that it was designed specifically for use with VEE and it does
interpret pRxStr as a previously allocated string array, but that doesn't
seem likely. It's also possible that the function expects a pointer to a
previously allocated single buffer (as is common in the COM API with
notation such as LPVOID *ppv - see CoCreateInstance). That doesn't seem very
likely either.
-SHAWN-

---
You are currently subscribed to vrf as: rwarren@amti.net To subscribe send a
blank email to "join-vrf@it.lists.it.agilent.com".
To unsubscribe send a blank email to "leave-vrf@it.lists.it.agilent.com".
To send messages to this mailing list, email "vrf@agilent.com".
If you need help with the mailing list send a message to
"owner-vrf@it.lists.it.agilent.com".

---
You are currently subscribed to vrf as: rsb@soco.agilent.com
To subscribe send a blank email to "join-vrf@it.lists.it.agilent.com".
To unsubscribe send a blank email to "leave-vrf@it.lists.it.agilent.com".
To send messages to this mailing list, email "vrf@agilent.com".
If you need help with the mailing list send a message to "owner-vrf@it.lists.it.agilent.com".
Like • Show 0 Likes0
Actions
VRFuser Jan 5, 2005 9:19 PM
Mark Correct
Correct Answer
> How is the DLL compiled?

Don't know. I suspect it's probably MSVC or GNU. At any rate, simple problem
easily solved, but quite difficult to explain!
-SHAWN-

---
You are currently subscribed to vrf as: rsb@soco.agilent.com
To subscribe send a blank email to "join-vrf@it.lists.it.agilent.com".
To unsubscribe send a blank email to "leave-vrf@it.lists.it.agilent.com".
To send messages to this mailing list, email "vrf@agilent.com".
If you need help with the mailing list send a message to "owner-vrf@it.lists.it.agilent.com".
Like • Show 0 Likes0
Actions

vrf Memory Eater...dll problems

Outcomes

Related Content

Recommended Content