Welcome to the SRP Forum! Please refer to the SRP Forum FAQ post if you have any questions regarding how the forum works.

rlist question

edited July 2019 in OpenInsight
The second argument to the rlist function is a number. What is the difference between arguments 5 and 6?
I have always used 5, but I tried 6, and my program became significantly faster. I read the docs, but I don't know what "resolving the list" means:

For reference, this is the query:
rlist("SELECT ORYX_WEB_MENU_DEF BY PARENT_LEVEL",6)
I simply want to iterate over each record in the table.

TARGET_ACTIVELIST$ 5
Resolve the list but do not save it to a named or default list. When RList returns, the list remains active. This is only useful when RList is called procedurally from another SSP.

TARGET_LATENTLIST$ 6
Do not resolve the list unless absolutely necessary, and do not save it to a named or default list. When RList returns, the list remains active. This is only useful when RList is called procedurally from another SSP.

Using argument 5, this took 5 seconds, using argument 6, it happened instantaneously.

Comments

  • Resolving the list means that the query has already been resolved fully before control is returned back to the calling routine. All applicable Key IDs have been identified and placed in a cursor so you can work off of. Resolved (or "active") lists do take a longer on the front-end because of the need to execute the query.

    Unresolved (or "latent") lists start sending you Key IDs immediately once they have been identified as matching your query conditions. This gives the impression that the select is faster if only because you can begin processing right away. However, typically the overall length of processing should not be any faster. When you use the Select statement you are automatically creating a latent list.

    There are two negatives to using an unresolved list:
    1. On tables that are experiencing high activity, the results can be unreliable. This is because the latent select traverses the table at the same time the table is being updated. It is possible that changes to the table can interfere with the reliability of the results and can even cause your cursor to reach the end of file prematurely.
    2. You don't know the final row count before you begin processing. Resolved lists already have all Key IDs known, so you also know the total row count and can use this to display progress results.
  • Hi, thanks for the explanation, but it's much faster. Have a look at the code:

    open "ORYX_CODES_REF" to oryx_codes_ref else debug Set_Property(tree_ctrl$, "OLE.Redraw", 0) clearselect eof = false$ rlist("SELECT ORYX_WEB_MENU_DEF BY PARENT_LEVEL",6) i = 0 Loop readnext code else eof= true$ until eof i+= 1 parent = field(code,"*",1) child = field(code,"*",2) name = "" if not(get_Property(tree_ctrl$, "OLE.Item[":parent:"]")) then read rec from oryx_codes_ref, 701:"*":parent then name = rec<1> end item = "" item<1,1> = 2 item<1,2> = parent item<1,3> = name : " " : parent Send_Message(tree_ctrl$, "OLE.AddItems", "",item) end read rec from oryx_codes_ref, 701:"*":child then name = rec<1> end item = "" item<1,1> = 2 item<1,2> = child item<1,3> = name : " " : child Send_Message(tree_ctrl$, "OLE.AddItems", parent, item) repeat Set_Property(tree_ctrl$, "OLE.Redraw", 1) rv = Send_Message(tree_ctrl$, "OLE.Sort", "", "Ascending", 1)
  • edited July 2019
    btw, I am trying to initialize the tree control with our hierarchical data.

    We have 2200 nodes at the moment. Each node corresponds to a record in the table ORYX_WEB_MENU_DEF. The table has no fields, just a key, the structure of which is PARENT*CHILD.

    Do you think the way I am creating the tree is optimal? Can you see a better way to do this? It takes 3-4 seconds to load the whole tree, which is fine. But if the nodes increase, then performance could become prohibitive.

    It's quite annoying how selecting records from a table takes so long in OI. But in our Sybase database I can select tens of thousands of records instantaneously...

    Not sure if this is a "feature" of oi, or if our oi is broken.
  • use Itemlist to initialise the tree.
    Use the addItems method if you want to 'add' stuff after the fact, generally due to some user interaction.

    The OI select is one of the biggest drawbacks of OI. Yours isn't broken. It's consistent.
    With all the years spent on OI10 you'd hope there'd be significant improvement in that area. I can't speak from personal experience whether or not that's the case.
  • @josh - As Mark pointed out, use ItemList instead of AddItems. You probably haven't seen it, but we wrote a blog article that explains when it is better to use properties or methods when it comes to populating our ActiveX controls.

    Your code by itself doesn't explain anything or show me how it is faster. However, I trust you if you say it performs faster using the latent select versus the active select. If I had to guess, it's because your select does not apply a filter, just a sort. Therefore, it is possible that RList is pulling the keys quickly and immediately because it doesn't need to resolve the select. I'm curious how much faster your entire routine will be once you refactor the code to use ItemList.

    I also concur with you and Mark that OI queries and retrievals have never been as fast as other environments, but that's because OI is optimized for different use cases. When I run into situations where I need to have quick access to a large sorted list of Key IDs then I consider using indexes or writing an MFS to maintain this list for me so I can quickly get it in one pass.
  • Hi, I tried to use the itemlist property, but I couldn't figure out how to make it work with the way our data is stored. It's not the fault of the itemslist property, it's due to how we store the tree hierarchy in the table.
  • Are you suggesting that because your data doesn't allow you to build your array in a predictable order that this is what is preventing you from using ItemList? I'm pretty sure there is a way for you to still code this, but I want to understand your perception of the challenge so I can help guide you.
  • edited July 2019
    I spent a really long time trying to make itemlist work, but I gave up. Let me show you a sample of our data:

    Each record in the OI table consists of two numbers separated by a star. The number to the left of the star is the parent of the number to the right of the star. When I see a record, I need to decide where to put it in the itemlist array. That is my problem. I can't figure our how to do this.

    Assumptions:
    * The children of the root all have a number or letter as their first symbol followed by 4 0s. Eg: 80000,F0000,70000
    * All nodes at level 1 have a 0 for their second symbol, all nodes at level 2 have a 2 for their second symbol, all nodes at level 3 have a 3 for their second symbol, and so on.
    * Given a parent, all of its children will have the same first symbol (e.g., all of the children of 70000, have a 7 as their first symbol)



    80000*82084
    L0000*L2022
    50000*52073
    F0000*F2099
    F0000*F2109
    60000*62138
    10000*12363
    10000*12378
    10000*12477
    G0000*G2361
    72016*73255
    72016*73420
    70000*72016

    This is what it should look like:


  • I think this is doable. I just have two questions before I work out a proof of concept for you:
    1. Is the Tree always 2-tier?
    2. Is it possible for the child keys to duplicate across different parents? For instance, I see you have a Key ID of 80000*82084. Parent is 80000 and child is 82084. Is it possible for another parent to have a child with a Key ID of 82084?
  • oh, i just found another table in our data base that stores the data differently in a way that might make this possible. I will have a try and get back to you.
  • edited July 2019
    In regards to your first question, what do you mean by 2 tier? However, there are at most 5 levels in the tree. If that's what you are asking.

    In regards to your second question, no it's not possible for a child key to be duplicated.
  • Your sample list of Key IDs and your sample screen shot of the Tree control only shows a 2-tier relationship (which makes sense if all your Key IDs are 2-part). I was just curious if that is true for all your Key IDs and how the Tree control will look if it is fully populated. You seem to indicate this is not the case. I take it then that you have Key IDs that have more than 2 parts.

    The reason for the second question is because I see that you are using the child key by itself as the Tree item unique key. If the child key could be duplicated then I would recommend that you keep the full record Key ID (e.g., 80000*82084) as the unique key for the Tree item. I still suggest you consider that since this will make it future proof.
  • edited July 2019
    "Your sample list of Key IDs and your sample screen shot of the Tree control only shows a 2-tier relationship (which makes sense if all your Key IDs are 2-part). I was just curious if that is true for all your Key IDs and how the Tree control will look if it is fully populated. You seem to indicate this is not the case. I take it then that you have Key IDs that have more than 2 parts."

    Hi,

    no we only have IDs that consists of 2 parts.

    So take a look at this sample data:

    72016*73255
    72016*73420
    70000*72016

    This is saying that the first node is 70000, his child is 72016, and his children are 73255 and 73420 .

    "The reason for the second question is because I see that you are using the child key by itself as the Tree item unique key. If the child key could be duplicated then I would recommend that you keep the full record Key ID (e.g., 80000*82084) as the unique key for the Tree item. I still suggest you consider that since this will make it future proof."
    The child key can't be duplicated though. Each child and parent are unique.
  • Aha...gotcha. I overlooked that you actually portray this relationship in the previous Tree control screen shot. Thank you for helping me understand this better. I can now appreciate what you were explaining earlier regarding the challenges. Let me see if I can still figure out a way to make this work.
  • edited July 2019
    Hi, don't try too hard, as I have found another table in our system that stores the data like this:

    Each record in this table is a leaf, and it stores along with the leaf the full chain of parents.

    So If i simply do this:

    select ORYX_WEB_MENU_WEBSITE by LEVEL1_CODE by LEVEL2_CODE by LEVEL3_CODe by LEVEL4_CODe by LEVEL5_CODe

    I get this

    70000*72010***
    70000*72011***
    70000*72015***
    70000*72016*73044**
    70000*72016*73050**
    70000*72016*73092**
    70000*72016*73135**
    70000*72016*73173**
    70000*72016*73217**

    Where the first number is the ultimate parent of the leaf, the second number the child of this parent, and so on, all the way down to the leaf.

    Which is in exactly the correct order required for your itemlist array to work. so this seems perfect.
  • I'll make you a deal. I won't pursue this any further since you seem to have a tailor made data set. If, however, you run into another roadblock then let me know and I'll happily revisit this.
  • edited July 2019
    @AusMarkB

    Hi, I just thought of a way to make selecting all records fast...and it works.

    If the table you want to select all records for has a btree index, just call btree.extract on the index but make it look for all values that are not equal to a, where a is an invalid value...

    I tried it on a table with 12,000 records, and i got all of the keys in 280 ms...

    I am actually going to use this technique to create the tree...unless it's a bad idea that will break OI? Seems like a hack.

    250,000 records takes 3 seconds to get using this method.
  • @josh relying on OI indexes is always your best bet. I didn't suggest this earlier because somehow I was under the impression you didn't have indexes on this table. I'm not sure why I thought that. Btree.Extract is an excellent tool to use when you have Btree indexes. There are other techniques and routines in OI that can implement index based queries. Each one is well suited for specific use cases.
  • edited July 2019
    Well, I went to all the trouble of getting it to work -- including case insensitive sorting . And the whole thing takes 3 seconds. But the problem with loading the whole tree in one go is that it will only get slower as the users add more nodes. So we have now opted for lazy loading of the tree...

    So if a node, has a child, we will give it a fake child, and defer loading its actual children till the user expands the node.
  • So if a node, has a child, we will give it a fake child, and defer loading its actual children till the user expands the node.


    I think you will find this is how other systems work when the number of items is sufficiently large. You can also spin up a TIMER event that continues to update the tree.
  • Yeah i did think of that, but how do you know if the user is idle? I don't want the program to hang intermittently, but if you just load tiny parts of the tree in each timer event, it probably won't be noticeable. I will give this a try.
  • The TIMER event isn't intrusive. It waits for the user to be idle. You also have control over how frequently the TIMER event fires. So, you can experiment to find the right balance. The trick is to make sure that each time the TIMER event fires you only take a minimal amount of time to update the Tree.
Sign In or Register to comment.