Notes from the Trenches

Mystery Solved!

Recently, I noticed on my Windows XP system that the SnapshotCM GUI process was nearly 1 GB in size, about 100x what it should have been. It was also taking a long time and much CPU time to exit after the window closed. I then tried the wls CLI and found that a "quick" recursive listing would run quickly, except the process would grow to 170MB, and the command would take another 18 CPU seconds to exit, slowing shrinking as it went. Certainly not the behavior I expected. I tried installing older releases and the large process size and slow exit persisted. Yet on Windows NT, the problem did not exist, even with identical bits. Hmm. Time to search the net.

I found lots of interesting things in my search, but no solution to this problem. The breakthrough came when I noticed that wll did not have the problem, even though wls and wll are identical .exe files, except for their name.

On a hunch, I searched the registry for "wls.exe" and I found a hit in the "HKLM\Software\Microsoft\Windows NT\CurrentVersion\Image File Execution Options" key. In fact, there was a hit there for the GUI process as well. But what were "Image File Execution Options" and how did they get set?

While I never did find a comprehensive listing of this registry key, I did discover a reference to the GFlags.exe program located in the "Debugging Tools for Windows" directory. And once I ran it, I remembered reading about and playing with a program that could enable memory debugging in running programs.

In short, I had enabled the page heap, which causes every allocation to be followed by an inaccessible page of memory, to help test for buffer overruns. And I had forgotten that I had set it :-(. Debugging Tools for Windows contains lots of useful utilities of benefit to Windows developers.

The good news is that I've been reminded of these tools, and I'm not likely to forget about it again!

From the Mail Bag

Q: How can I really delete a file? I find it likely that there is a snapshot somewhere where it still exists, and it is likely that some user will promote the file and it will reappear. How can I get rid of all copies of a file I really do not want?

A: To permanently remove a file from all snapshots, one must identify all the snapshots that reference the file and delete (or unlink) the file from each one. The history report (and whist CLI) have an option to show all snapshots where a revision is used. The Active History also shows all referenced snapshots in a data tip when you hover over the revision field, indicating by an asterisk those revisions which are referenced. Once the file is removed from all snapshots referencing any revision of the file, the file will be removed.

We recommend that you use the remove operation rather than the unlink operation, as the remove operation schedules the permanent delete for later, allowing time to recover the delete should you change your mind. Once the last reference is unlinked, however, the file and history is gone.

You might also consider whether just removing the file from your initial snapshot might not be sufficient. As the delete is propagated up and down the development hierarchy, it may reach all snapshots and be scheduled for removal without any extra effort on your part.

Q: How do I start using the SnapshotCM Proxy Server?

A: You have two steps to start using a proxy server. First, you must configure the proxy server. Once that is done, you can change your workspaces to reference the proxy server rather than the repository server.

For directions for configuring a SnapshotCM Proxy Server, see the Administration Guide's discussion on the Proxy Server (for windows, for unix).

Once the proxy server is configured and running, edit your workspaces to reference the proxy server's host name rather than the repository server's host name. You can do this directly using the wmap CLI, or by editing a workspace's properties and replacing the repository server's host name with the proxy server's host name.

Two other items you may want to address. The first is trusted login access. In short, trusted access through a proxy server is distinct from trusted access directly to the repository server. For details, see the SnapshotCM GUI's Dialog Help on Login Patterns.

The second item is your network query settings (Project->Options, Network menu in GUI).

Once your workspaces are changed, access should be the same as before, except that most check out operations will be faster as they use the proxy server's cache and eliminate the network copy of the file content from the repository server.

Extreme Check Out Performance

I was browsing a CM forum recently, when I noticed a user asking for suggestions with check out performance with their CM installation. Their situation was a bit extreme, with 160 GB (yes gigabytes) of data in 50,000 files. They claimed to have the latest hardware, yet their check outs were taking about 6 hours to complete.

A few back of envelop calculations shows that on a 1000 Base-T network, this is only about 6% of the network bandwidth. It got me wondering how SnapshotCM would hold up.

I did not do a full 50,000 file test, but I did take a 6.6 GB backup file and check it into and out from a couple of different servers. With 3-4 year old hardware on both ends, it wasn't hard to saturate our 100 Base-T LAN. Even when the server pre-compressed the file and the client had to do the file expansion, the LAN remained saturated, with an effective throughput of 20 MB/s and under 10% CPU. At this rate, the 160 GB checkout would take only 2 hours, 15 minutes, though handling 50,000 files would add some additional overhead.

The conclusion is that, with anything like a recent vintage client and server, a 100 Base-T network is going to limit your check out performance. And yes, I've added a gigabit switch to my wish list.

A side note: This testing discovered that the background compression of large files has not been occuring in recent releases. Release 1.77.02 fixes this.

E-mail Us

Send your comments, questions and suggestions to support@truebluesoftware.com.

To see SnapshotCM in operation, get a free SnapshotCM evaluation at http://www.truebluesoftware.com/.