OI Crashing for remote users

AusMarkB

We have been experiencing frequent crashes of OI for remote users accessing the application on a terminal server.
They were reporting the crashes as occurring in a variety of situations and processes. What they weren't to know was every process involved an SRP control. Indeed I could replicate on demand by clicking specific buttons.

I insisted the IT guys unregister and re-register the SRP control on the terminal server as it is very possible that had not been done after an update of the control had taken place. That in itself didn't resolve the issue. If they logged into the server as the administrator, the issue didn't occur. A number of options were tried in an attempt to resolve the issue when (to cut a long long story short), it was identified that "ntdll" was the source of the crash.

Replaced ntdll from a backup, rebooted the server. Problem resolved.

Thought I'd post here because all symptoms pointed at srpcontrols but whilst there may have been a correlation, in the end they were symptoms.

DonBakke

Mark - I am most interested in your report. We have noticed increased occurrences of OI crashes ourselves, specifically on terminal servers. As you indicated, the Windows error message suggested that SRPControls.ocx was involved (although I occasionally see SRPUtilities.dll get the blame as well).

I've always been suspicious that our products are the victims rather than the perpetrators, but the problem is very difficult to pin down. When we have attempted to catch the crash in the act using a debug build of our control, the problem never occurs. When Kevin runs his development environment on the server (which gives him complete ability to identify the source of the crash), the problem never occurs.

I have seen NTDLL also blamed in some crashes. I am somewhat surprised that a restore of that component fixed the problem. It suggests the file was corrupted somehow. Any ideas as to how or why?

What version of Windows Server is this?

AusMarkB

Sorry. Not much more I can add.
After providing a way to replicate the crash and thus test any changes it was kind of out of my hands.

I don't believe they know how or why either, just that it was crashing so restoring it was worth a shot.
The next alternative was to log in as each individual user, give them administrator rights, re-register the srpcontrol and test, then change everything back and try again. It seemed like a long road ahead so the restore and suck it and see approach was more inviting.
Seemed to work so they can wash their hands and move along.

DonBakke

Thanks again, Mark. It's a shame they don't have the NTDLL file that had to be replaced. It think it would have been interesting to see if there were any byte changes. Part of me wonders if it isn't so much a corrupted file but rather some process that is flagging the file (such as identifying it as malware). Restoring the NTDLL may have reset the flag and allowed it to run without incident. I'm guessing you'll be notified if the problem occurs again. If so, see if you can get more information out of the IT people along these lines.

Thanks for the server information. In our case, the problem seems to be more prevalent with Win2012. I have not had many problems with Win2008.

Chris_Meyer

I have a client have several OI crashes in the last 3 weeks on their terminal server (Win2012R2). Today the OI crash occurred while accessing some of the SRP functions on the terminal server. Accessing the same functions on the SBS server (Via rdp) without a problem. Restarting the terminal server fixed the problem previously.
Getting the IT department looking into the problem. Will report back if any further details.