OI Licencing part 2

Opto_Will

Sorry to revisit with more licencing questions but I am still having issues
The site in question is a bit complicated to trace but here is background.

It has 3 OI Systems, 2 of which have the same DNumber Rev licence.
(for SRP. 2 of the systems are an iTMS and Opto which share data. The other is standalone Opto)

All 3 systems serve Web/API via OEngines (Opto using OECGI 4.0.1.0, iTMS using OECGI3 because its old web app is hardcoded for it.)
They have a mix of Network and local RDS users.
All instances are Rev 9.4.6 running correct 9.4.6 version licence files.

Universal driver is now installed (4.7.2). Because of this I have been able to validate actually running users. my main problem now is that one of the systems (OECGI4) spins up a crazy number of OEngines that shouldnt be possible with the eserver config, nor does it seem to kill them off. We have to stop the service when the client complains.
This Opto system is one of the shared licences/shared data but there are no actual users. It only exists to serve APIs via SRP's HTTP Framework 4.0.9.7.

Below is the eserver snippet (we have made them fairly aggressive) as well as the running processes.

//Engine parameters //Number of connections to engine MaxConnections=0 //Number of engines allowed MaxEngines=3 //Max time in minutes allowed active in memory MaxUpTime=2 //Max time in minutes allowed run a request before 'hung' and terminated MaxRunTime=2 //Time in minutes can be idle IdleTimeout=1 //Time in seconds between checks for idle and uptime/runtime IdleCheck=30



Main question. Why wont these die off?

Related questions:
- How can OEngine spin up more instances than MaxEngines in eserver.cfg (running for long past MaxActive/Hung with a very short Idle/Check time)
-
- Does each OEngine.exe take up a licence or is it the actual OECGI calls queued? (i beleive it is the Engine but happy to be wrong)

Side question:
- In a normal OI system is Rev counting Oengine.dll or Oinsight.exe as the licence use (as I have seen seen some systems with multiple oinsight processes and only one OEngine lock for a network user)

BarryStevens

>>is that one of the systems (OECGI4) spins up a crazy number of OEngines that shouldnt be possible with the eserver config, nor does it seem to kill them off.

I am also seeing this, so, look forward to the responses.
Not all the time. On my PC testing ( and also remote testing server). I then go to open OI and get user count msg. Look at taskmanager and see 2 x oengine.exe in processes.

also using SRP's HTTP Framework

BarryStevens

@Opto_Will how did you get that status type list (the black background one)

Opto_Will

@BarryStevens

I wrote a little powershell script.

I am not really a Windows programmer by trade so it took me a tick to make what I was looking for. Its also not optimized and hobbled together.

Basically used variations of this (with $FILEPATH ponitng to my OI executable location)
For network: Get-SmbOpenFile | Where-Object Path -eq $FilePath | Sort-Object ClientUserName For Local: Get-WmiObject win32_process | Select-Object ExecutablePath | ? {$_.ExecutablePath -eq $FilePath} For Engines $processes = ps | ? {$_.Name -eq "OENGINE"} I recently tacked on a handy for another local-oncly client so I could tell them they were lying to me!: Write-Host "" $processList = Get-Process -Name OINSIGHT -IncludeUserName | Sort-Object Username $newline = "`r`n" $tabChar = "`t" $separator = "---------------" Write-Output "Local Processes $newLine $separator" # let's loop through the list, and pick out the stuff we need foreach($item in $processList) { $itemObject = $item | Select-Object $itemName = $itemObject.Name $itemId = $itemObject.Id $itemUser = $itemObject.UserName $itemPath = (Get-Process -Id $itemId).path Write-Output "$itemId $tabChar $itemUser $tabchar $itemPath" }

BarryStevens

wow, that is cool.
Thank you

DonBakke

@Opto_Will I don't have a ready answer to the primary question. I haven't done much work or testing with those configuration options but I think others on our team have so I'll let them chime in.

OENGINE.dll is the binary that consumes a license. However, it is not an executable binary, so it must always be loaded by another process. Depending on how the engine is loaded, you might see OpenEngine as a sub-process in the Task Manager. Thus, it is possible for OpenEngine to be running (and thus a license is being consumed) but you won't see any reference to it in the Task Manager. You should, however, see the executable binary process that loaded the engine (continue reading).

Desktop OpenInsight (OINSIGHT.exe) is the most common executable binary that loads OENGINE.dll. If the engine is configured to be visible, the Task Manager shows OpenEngine as a sub-process to OpenInsight. You will not see OENGINE.exe as a process for this engine because it wasn't used to load it.

The simplest way to use OENGINE.exe is to launch it directly (i.e., double-click the file or use a shortcut). When it runs, it will attempt to load OpenEngine as a sub-process. Normally you'll see OENGINE.exe as a process with OpenEngine as a sub-process in the Task Manager.

OECGI4.exe (or OECGIx.exe to represent any version) will call OENGINE.exe to load OpenEngine as needed. If the OEngineServer was run as a Windows Service, then OENGINE.exe appears in the Background processes section of the Task Manager but no OpenEngine sub-processes are visible. This is because these engines are run invisibly. If the OEngineServer was run from a command prompt, OENGINE.exe appears in the Apps section of the Task Manager and an OpenEngine sub-process should be visible (although it is still possible to configure the engine as invisible even if loaded this way).

If multiple engines are spun up these should appear as additional sub-processes.

Because OpenEngine is a loaded sub-process, it is possible for it to be closed but the executable binary to still remain running. However, since it is possible to load OpenEngine and it not be visible, the Task Manager is not a fool proof way to determine how many engines (and thus licenses) are being consumed at a given time.

Opto_Will

What is that I hear?




Why, thats the sound of @DonBakke bursting my bubble




Just when I thought I was getting a grasp of things.

DonBakke

Pretty funny...but did I confuse you or did I illuminate you? More importantly, did I provide you any clues to troubleshoot the problem with? BTW, I also recommend making the Command line visible for processes so you can get even more information on what called OENGINE.exe.

Opto_Will

@DonBakke

A bit of both to be honest ;-)
You confirmed a couple of things for me. You also made me start to consider holes in my theories!

I always thought it was the OEngine.dll (OpenEngine) access that 'used' the licence but was struggling to have the proof to convince others. I get what you are saying about it not being foolproof, especially if dependant on configuration to be visible. We run the OEngines as a Service on all client sites but they are visible in the 'Details' tab of the Task Manager.

I sort of built myself up methods to justify what I could see in the SYSLOGINS table as well as what might be the gap to hitting the 'Too Many Users' count. My musings may not be 100% accurate but definately seem to be well within the 80/20 rule. As it is a contstantly changing target (Users on/off via Desktop OI, Engines up and down) it is generally enough to show clients they are actually using licences they didn't think they were so it is time to buy more if they dont want to hit limitations.

Only occassionally we might get a site that has some old bugs that @FrankTomeo mentioned in my March post where there are some orphans kicking around but fortunately not too many.

I have been trying to rollout 9.4.6 whereever I can (there are still some 9.3.x sites out there) to minimize known issues and ensure there is a matching 9.4.6 OEngine licence file (we have sites with mismatches here where they are running OI versions newer than their licence file version and that has caused some unexpected grief. Try fixing code onsite when the debugger wont work!). I have also started deploying the Universal Driver 4.7.2, especially where sites are complaining about licences and issues running multiple OI systems.

As far as this site is concerned, I still dont understand how so many Engines spun up nor why they dont get killed, even if it was a hung process. Its almost like they were orphaned somewhere after starting so the Engine 'manager' doesnt know they exist to kill them off yet they still get counted as a used licence. More investigation is required. Maybe tracing the starting point to what they were spun up for as you suggested may provide a clue....

DonBakke

We run the OEngines as a Service on all client sites but they are visible in the 'Details' tab of the Task Manager.

Please confirm that you are seeing OpenEngine and not OENGINE.exe in the 'Details' tab. Or do you see both?

Opto_Will

@DonBakke, sorry for the confusing statement.

I see OEngine.exe in the Details tab, not OpenEngine.

BarryStevens

OK, now I see they are all closed - so forget below - but am sure they were up for longer than 1 minute (as per the config file)

Hi all,
My observation is that if OI is not running , engines (OENGINE.EXE) open and close in the time limits.
BUT if OI is running OR has been running when the engines opened (OECGI4) when doing web app calling http frameworks, they are NOT getting closed. - they build up.
There appears to be an issue somewhere - albeit a setting.

Nothing running.

Opto_Will

@BarryStevens,

In my case the OEngine is always spawned from an OECGI request but OI is pretty much always running (they have like 50 users and are a almost 24 hour operation).

I have obersved them on a weekend when they have as few as 7 users active (which makes monitoring much easier) and engines spun up and down exactly as I would have expected.

There may be a specific request from a workday user that gums up the works but I would need to see via console what the current request is, as @DonBakke alluded to.

BarryStevens

@opto_will

In my case the OEngine is always spawned from an OECGI request

Same situation here

BarryStevens

A bit of testing has shown that oengine is 'shared' until another OECGI request is made using a different OI account which then is spawning another oengine.exe.