Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In order to avoid long-running transactions (leading e.g. to issues like

JIRA
serverEvolveum Jira
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId701b45f2-090c-3276-8ac9-f45eedf731bc
keyMID-4414
) we have changed the way how searchObjectsIterative method is implemented.

 There are three four iteration methods (see IterationMethodType):

Iteration methodDescription
SINGLE_TRANSACTIONFetches objects in single DB transaction. Not supported for all DBMSs. For those that support it (e.g. PostgreSQL) it can lead to long-running transactions spanning hours or even days. Moreover, current transaction isolation settings cause the objects returned to have values that were stored in the database when the search was initiated.
SIMPLE_PAGING

Uses the "simple paging" method: takes objects (e.g.) numbered 0 to 49, then 50 to 99, then 100 to 149, and so on. The disadvantage is that if the order of objects is changed during operation (e.g. by inserting/deleting some of them) then some objects can be processed multiple times, where others can be skipped.

STRICTLY_SEQUENTIAL_PAGINGUses the "strictly sequential paging" method: sorting returned objects by OID. This is (almost) reliable in such a way that no object would be skipped. However, custom paging cannot be used in this mode, except for setting maxSize parameter.
FETCH_ALL

This is a workaround for situations when STRICTLY_SEQUENTIAL_PAGING cannot be used because a client-supplied paging is present. All the objects are fetched as in regular searchObjects() call and then send to the client one-by-one. (This defeats the basic purpose of searchObjectsIterative but can be safely used for small numbers of objects.)

So this method is a safe fallback that is used when STRICTLY_SEQUENTIAL_PAGING is implicitly chosen but a custom paging exists, provided that the paging contains maxSize clause with a number not greater than a specified limit (maxObjectsForImplicitFetchAllIterationMethod). New in 3.9.

Before midPoint 3.9

Before 3.9, majority of iterations were done in this mode - unless configured otherwise (globally in config.xml or per individual searches):

...

Elimination of long-running transactions could lead to a better performance and freshness of data (

JIRA
serverEvolveum Jira
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId701b45f2-090c-3276-8ac9-f45eedf731bc
keyMID-4414
). However, STRICTLY_SEQUENTIAL_PAGING means that custom paging cannot be used, except for maxSize parameter. In stock midPoint, we typically don't use paging combined with searchObjectsIterative method. But if you have a custom code, e.g. in scripts, make sure you either replace searchObjectsIterative with searchObjects, eliminate the paging (except for maxSize), or change the default iteration method (see below).

If an iterative search encounters a combination of implicitly-set STRICTLY_SEQUENTIAL_PAGING and non-compatible custom paging, it:

  • If the custom paging has maxSize at most 500 (or the current value of maxObjectsForImplicitFetchAllIterationMethod system parameter), then switches iteration method to FETCH_ALL.
  • Otherwise it issues an warning message to the log and switches iteration method to SIMPLE_PAGING.

In particular, beware of bulk actions (midPoint scripting). Searching objects there uses searchObjectsIterative method.

...