There are some situations when administrator needs to carry out actions on a set of objects. Some examples:
- recompute all users without fullName,
- delete all users that have no accounts,
- disable all users with passwords older than 100 days,
- assign an account on a resource to a people meeting given criteria,
- assign a role to a people meeting given criteria.
MidPoint provides a specialized language to define such bulk actions. It is based on pipes-and-filters architectural pattern, where execution components (responsible for gathering data and acting on them) are connected into chains, pushing an output of an upstream action to be an input of the action that lies downstream. Just like good old Unix pipelines.
The scripts are currently represented using standard midPoint prism structures, externalized as XML, JSON or YAML documents. A specialized text-based language was once conceived (and experimentally implemented), but it is not currently available. Hoping its time will come it is described here.
It is possible to specify the target resource by OID (example 4a). In the future it might be possible to specify it also by name (example 4b), or using arbitrary filter.
The language and its execution model
The basic building block of the language is a scripting expression. The expression is a piece of code that may have an input, does some processing, and (optionally) produces an output. Currently there are the following kinds of expressions:
Retrieves a set of objects via searchObjects model call.
|action||An action that can be carried out on a given piece of data that comes at its input. Typical actions are add, modify, delete, enable, disable, assign, resolve, log, ...|
|select||Selects a given item (container, reference, property) from the input data and copies its value(s) into output. For example, accepts a list of users and selects only their accounts.|
|filterContent||Removes selected items from the input data. For example, give a list of users, removes all the data except for names and password values. (Since 3.6.)|
|pipeline||Chains a set of expression together. They are executed one after another; input sent to the pipeline as a whole is sent to the first expression. Output of the last expression is considered to be the output of the whole pipeline.|
|sequence||Sequence of command expressions. They are executed one after another; input sent to the sequence as a whole is then sent individually to each expression. Output of the last expression is considered to be the output of the whole sequence.|
Other planned expression types: are constant expressions, initialization and use of variables, or filtering input values.
An action modifies the input data (or acts on it in any other way).
In addition to the input data, an action may have one or more parameters. For example,
assign action must know the role or resource to be assigned;
modify action must have the delta that has to be applied.
Currently, there are the following actions:
|add||Adds an object coming as input to the repository, which must be a PrismObject. (***)||-||-|
|modify||Modifies an object coming as input, which must be a PrismObject. (*) (***)||delta||Delta to be applied to the object.|
|delete||Deletes an object coming as input, which must be a PrismObject. (*) (***)||-||-|
|enable, disable||Enables or disables an object coming as input (must be a FocusType or ShadowType). (*) (***)||-||-|
|assign||Assigns a role or a resource account to a FocusType. (*) (***)||resource||Resource(s) on which account(s) have to be assigned. **|
|role||Role(s) to be assigned. **|
|recompute||Recomputes a user (must be PrismObject<UserType>). (*) (***)||-||-|
|execute-script||Executes a script against the input data. (Since midPoint 3.4.1)||script||A value of type ScriptExpressionEvaluatorType.|
If the script provides any output that is to be processed further, the item definition has to be given here. It is in the form of URI, pointing to item name (e.g. user) or item type (e.g. UserType). "Unqualified" URIs like the two examples here are allowed.
But note that outputting data from scripts is currently only experimental.
|forWholeInput||The script would get the whole pipeline as input (since 3.7, experimental).|
|resume||Resumes a suspended task.||-||The task must be in a suspended state. Since 3.7.2.|
|resolve||Resolves a reference, e.g. data coming from a c:linkRef, into a PrismObject.||noFetch||Whether noFetch option has to be applied (default: false).|
|purge-schema||Removes all schema information from a given resource(s) coming as input (PrismObject<ResourceType>).||-||-|
|discover-connectors||Discovers all connectors on a given connector host(s), given as PrismObject<ConnectorHostType>.||rebindResources||Searches for all resources using now-outdated versions of newly discovered connectors and re-links them to current connectors.|
|test-resource||Tests a given resource(s) coming as input (PrismObject<ResourceType>).||-||-|
|validate||Validates a resource - i.e. provides a set of issues just like in Resource Wizard (since 3.5)||-||-|
|generate-value||Generates value(s) for object(s) coming as input.||items||Description of what and how to generate.|
|notify||Sends a notification event for each of objects at input (since 3.5) - i.e. it generates a custom Event with the content driven by action parameters.||subtype||Subtype of the event created.|
|handler||Ad-hoc event handler that should be used to process the event. Normally this parameter should not be needed, because event handling should be driven by the system configuration. However, for ad-hoc events we can specify handler directly within the event.|
|forWholeInput||Whole input (i.e. all items in the pipeline) should be sent as event object. The default behavior is to generate one event for each input object.|
|status||Status to be put into event (success, failure, inProgress, alsoSuccess, onlyFailure). Default is "success".|
|operation||Operation to be put into event (add, modify, delete). Default is "add".|
|log||Logs debugDump form of the data.||level||info (the default), debug or trace|
|message||Custom message that is prepended to the data.|
(*) In the future these actions will support also PrismReferences instances as their input.
(**) These are to be specified as PrismObjects, PrismReferences, or PrismProperties encapsulating either ObjectReferenceTypes or Strings (understood as OIDs - in the future, string containing resource/role names could be accepted as well). Since 3.7 it is possible to specify queries or search filters here, so it is possible to assign role/resource by its name (see this sample).
(***) Since 3.5, these actions support
dryRun parameter that (if set to "true") causes executing "preview changes" instead of real modifications. They also (except for recompute) support
raw parameter for applying the operation in raw mode. And since 3.7 these actions (again except for recompute) support
skipApprovals parameter, and
options parameter, as a generalization of these two (
skipApprovals) that can be used to set arbitrary model execution options (see this sample).
Some simple examples of scripts in XML form can be found in resources/scripting directory in model-intest module and in tasks/bulk-actions directory in the samples module.
executeScript action and
Embedding in tasks
Scripts can be run within tasks. That is extremely useful for long-running scripts. More information is on this page.
Data being passed
The common data format to be passed between expressions, accepted as script input, or provided as script output is the list of prism values (corresponding to objects, containers, references, or properties). For example an output of a
search command is the list of PrismObjectValues. Or, the output from
search UserType | select linkRef command is the list of PrismReferenceValues. Each of these values can be accompanied by an
OperationResult depicting the state of processing that value. So, for example, after selecting 100 users and attempting to disable them, one can easily determine what users were processed correctly and what were not.
Serialization of the data is described here. (TODO)
As in other scripting languages, midPoint scripting also provides an easily-understandable text output of individual commands. An example:
Of course, detailed trace of commands executed along with their results is available in the form of OperationResult objects mentioned above. However, the "console output" feature is meant to be a quick and easy way to convey the administrator the result of the script execution. In current implementation, each action puts there information on actions taken (users enabled, disabled, deleted, modified, ...), along with warnings and errors. For any other information, the operation result should be analyzed and displayed.
Currently, the policy is "stop on any exception". For example, when a "modify" or "delete" operation throws an ObjectNotFoundException, the script execution simply stops at that moment. This is for safety reasons.
Note that actions themselves are also a bit picky. When they get an object they cannot act upon (e.g. a PrismPropertyValue in situations where they expect PrismObjectValue, or a ResourceType when they expect UserType), they treat this like a fatal error and stop the execution of the whole script. Also this behavior could be made configurable in the future.
TODO (GUI, Eclipse plugin, command-line client)