Hi Guys,

Yet another set of problems (sorry) . . .

After working with another portfolio file, I did a File-->"Restore from backup" to reload my previous portfolio file. I had created a backup, including price history, before working in the new portfolio file.

When the restore was complete, many tickers no longer displayed intra-day data in the charts.

So, I went into Options-->Preferences-->Historical Data and clicked the "Import history data files into database" button. I have just under 1200 files in both, the end-of-day and intra-day folders.

After about 20 minutes of no progress indication from PSM, with only intermittent disk activity, and the inability to select/display the PSM window by clicking on it's taskbar icon . . . I opened the Windows Task Manager to find out if PSM was running or not. It was shown as "Not Responding", but when I looked at the disk I/O numbers, they were increasing, so I figured it was in fact running.

When I looked at the Working Set Memory value, it too was INCREASING continually, until it reached over "1,500,000 K".

When the import finally finished (30+ minutes total running time), I did a File-->Backup to create a new backup file of this portfolio.

When the backup finished, I attempted to do a File-->Save. However, after a couple of minutes, an error box popped up with the message "Out of memory".

Going back into Task Manager, the Working Set Memory value was now "1,767,964 K" (1.7 GB), the Commit Size was "1,733,936 K", and the Paged Pool was "510 K".

My system has 12 GB of RAM and just under 8 GB of physical memory was available at the time this error was displayed. So I don't know why PSM thought it was "Out of memory", unless it has the Windows 2 GB memory limit?

At this point, I didn't know if the new backup file was any good or not, OR if the File-->Save function had actually saved the portfolio file or not. But my only option was to Exit the program.

After exiting and re-running PSM, the current Working Set Memory value was only "123,012 K", which is but a tiny fraction of the earlier "1,767,964 K" value. It appears that the portfolio file IS intact and the missing intra-day IS restored.

My psm.exe file is currently set to Windows XP SP3 compatibility mode, on a Win 7 Pro SP1 system.

So . . .

1) Why is PSM dropping intra-day data when switching between different portfolio files via the File-->"Restore from backup" function?

2) It would appear that the "Import history data files into database" function has a bad memory leak, which is not being taken care of when the function completes (via garbage collection).

3) Why does PSM take so long to import the history data on a super-fast (CPU and disk) system?

4) Why is there no progress display for this function?

5) Why did PSM think it was "Out of memory" while performing a Save function even when my system still had nearly 8 GB of Physical RAM remaining? The Windows 2 GB memory limit?

I sure hope you guys are able to find some of the problems I've been reporting lately, by using the portfolio file backup I sent you a link for. This is getting very old, and you guys are probably just as tired of reading all the problems I've been reporting, as I am of writing them all.

Thank you,

-Don

Hey Don,

Just a quick item -- why are you running in compatibility? I have Win 7 Pro SP1 64-bit installed, and I just checked -- not running PSM in compatibilty mode, and I haven't had any of the problems you've been seeing. You do much more with charts than I do, but otherwise I think we have comparable set ups. HAve you tried unchecking that, and seeing what happens?

Howdy Randi,

Yes, when I first moved PSM to my new system I ran it normally, under Win 7. But I kept getting errors while running scripts along with PSM hanging and freezing one or more times every day.

So, I changed the psm.exe file to run in Win XP SP2 compatibility mode but still had some problems. So, I upped it to XP SP3, which has provided the fewest number of errors and problems of all three settings.

I'm starting to get the feeling that no other PSM user has ever leaned on the program like I am, taking FULL advantage of practically every bell and whistle that's built into it, including using the Notes box to store lots of text in just about every ticker, many custom columns, many different Views, several custom scripts, alerts everywhere, multiple kinds of annotations on practically every chart, etc., etc.

So, I basically spent the entire day Saturday creating a backup file of our current portfolio, minus our currently active securities and past transactions. All in the hope that Anatoly can use it to duplicate the problems I'm having, and then be able to zero-in on the cause(s) -- which might be one or more of the things listed above that I'm doing in PSM. I hope it helps him.

A 32-bit program such as PSM has a 2GB memory limit, regardless of how much memory your system has. And it's not a memory leak, but it is loading everything into memory, that's just how it was originally written, so yes, you're running into some system limits. I would have to do significant work on the historical data import routine to not load everything into memory before writing it to the historical database. As to why it's not restoring historical data from a backup, it should, but again that was written years ago, so I would have to do some investigation to figure out what's going on there.

Thank you for your reply Anatoly.

I remembered the 2GB memory limit while I was writing my message. Thanks for confirming this.

Could you possibly add a progress indicator, such as the simple one in the Backup function?

Out of curiosity, I don't suppose you guys would have any history file maintenance scripts / programs for PSM/PSS lying around, do you? So that users could shrink the total size of ticker history storage by:

  * Creating a list of the ticker history files that no longer have associated tickers in the user-selected list of their active portfolio files (if they use more than one file), from which the user could then select which history files to delete.

  * Ask for a "Start Date" and then the program would remove all individual entries from BEFORE this date, from within all of the ticker history files that reside in both the intraday and endofday folders.

Yes, I remember that PSM currently uses TWO history systems: 1) Physical history files in folders, and 2) Internal database. Thus, the maintenance program would need to make the changes to the physical files first and then synchronize the changes to the internal database. Or this step could then be done via the Import history function.

Thanks again Anatoly,

-Don

Follow-up: This problem does not occur in Windows XP Mode (Virtual PC).