SDK: most recent version of v14.x.x
Platform: v24.x.x
Python: v3.11
Isolation mode: venv
Current Behavior
I have tried and compared three approaches to count the number of entities per type created by users in a tree app (across all projects/workspaces):
Approach 1: API calls tracking EVENTS
Approach 2: Audit EVENTS directly from the website (I obtain a .csv file for each year and merge them)
Approach 3: API calls retrieving every entities
When comparing Approach 1 with Approach 2, I did not obtain the same number of ENTITY.CREATE events between the REST API call and the Audit. The Audit had a lot more data (almost double); and I noticed this might be due to the API rate although I’m not certain.
When comparing Approach 3 with Approach 2, the number of entities presently existing (one API call per workspace, took over an hour) was significantly greater than the number of events associated with entities created. The difference was over 30%. This doesn’t make sens at all to me since the event of creation should be greater than the number of existing entities, especially when factoring hundreds of deleted ones.
Expected Behavior
I would expect that the three approaches would yield somewhat similar result or could be explained by another type of event (rough numbers).
Context (optional, but preferred)
We automate performance graphics for our applications to show how many entities are created, or user trends.
Note that I paid a lot of attention to starting date, ending date, API requests, making sure every result is associated with the entity unique id that is not duplicated, what sort of filtering should be used, as well as what other types of events that could be factored in. I came to the conclusion that either an important consideration is unknown to me, or there is a bug .
Hi, I think some of these discrepancies could be linked to the following:
Entities can be (recursively) copied which result in an ENTITY.COPY event instead of ENTITY.CREATE
When a workspace is initialized it could consist of initial entities, for example root level folders. These entities do not trigger an ENTITY.CREATE event.
I also wonder if you exclude development workspaces in this comparison?
For approach 1, what does your request including filtering look like?
That would explain why there’s a lot more ENTITY.COPY than I would think based off my observation of how users use our apps and perhaps part of the difference observed. The number of copy is disproportionate compared to the number of users who use the functionality (which is not small in itself).
I filter for each last-level entity types in our tree app (we have one app available to users, a tree app). Any root, higher, intermediate level is ignored. Although we do receive some of this data at first, it is filtered.
Yes, we exclude them.
For approach 1/2, we exclude developers/administrators. We only have one application available to users and that is the only one we consider (we explicitly include all display names that application ever had).
For Approach 3, we only exclude developers/administrators only. Since we have one application.
From the data collected, an excel was made linking each entity to the user who created it and the unique identifier associated to it.
I thought good to mention that the comparison was made first overall, but also for each entity types.
Since we ended up not retaining it, I didn’t find it however I have a pretty good recollection of it.
I followed the documentation on this page: Get Audit Events within a Time Window | VIKTOR Documentation. I added starting/ending date at intervals (I tried 1 year intervals and then 1 month intervals). The data of 3 years was merged into one.
That makes me think, that perhaps one cause could not explain the difference alone, but maybe if you factor ENTITY.COPY and a slight margin of error due to the different approaches, perhaps another event, this could be explained
Perhaps you could take a smaller interval (e.g. 1 month) and compare approaches 2 and 3 (use created_at_min and created_at_max). You could also check if any EVENT.COPY or EVENT.COPY_RECURSIVE is present within this range to see if it affects the comparison.
I still wonder why approaches 1 and 2 differ. The REST API docs you are using is exactly how our frontend uses the API as well. I would expect the results to be equal, only the client differs.