This information is applicable for midPoint 3.6. It describes migration from midPoint 3.5.x CSVFile connector to midPoint 3.6 CSV connector.
Up until version 3.6 midPoint included a legacy CSVFile connector. The legacy CSVFile Connector was replaced by new CSV Connector in midPoint 3.6. The new CSV connector is a rewrite from scratch. The old CSVFile connector was written even before midPoint project started and it was not designed for real deployment use. We have maintained and improved the connector during the years. But it was not maintainable any more. Also the ConnId framework evolved over the time and we needed a connector that will use these features. Therefore we have decided to rewrite the connector completely. MidPoint 3.6 no longer bundles the old connector. New CSV connector is bundled instead. Old CSV connector can still be used and it is still supported for deployments that purchased midPoint subscription before midPoint 3.6 was released. As the old connector is not bundled with midPoint any more you have to download the connector JAR and deploy it explicitly.
Migration: New CSV Connector
The new CSV connector is very similar in its operation and configuration as the old CSVFile connector. However it is not entirely backwards compatible with the old connector. There are several major differences:
- connector name and bundle
- some connector configuration properties are different and some are missing
- attribute names: native column names are used instead of icfs:name and icfs:uid
- the new connector does not support script execution yet
Connector Name and Bundle
The new CSV connector is using bundle and connector name from com.evolveum.polygon namespace. The configuration XML namespace also reflects that:
Bundle name (connectorBundle)
Connector name (connectorType)
CSV Connector has configuration schema similar to old connector, but there are differences. The new configuration properties are summarized below:
|multivalueDelimiter||Multivalue delimiter||Multivalue delimiter character user for splitting multivalue attributes.|
|commentMarker||Comment marker||Comment marker character.|
|filePath||File path||Path to CSV file with records.|
|quoteMode||Quote Mode||What field should be quoted. Allowed values are ALL, MINIMAL, NON_NUMERIC, NONE.|
|ignoreEmptyLines||Ignore empty lines||Whether connector should ignore empty lines in CSV file.|
|trailingDelimiter||Trailing delimiter||Whether connector should use trailing delimiter.|
|encoding||Encoding||CSV file encoding.|
|fieldDelimiter||Field delimiter||Delimiter character between fields in one CSV record (escaped as regexp, if needed).|
|ignoreSurroundingSpaces||Ignore surrounding spaces||Whether connector should ignore surrounding spaces.|
|trim||Trim||Whether fields should be trimmed.|
|headerExists||Header exists||Whether header exists in csv file.|
|passwordAttribute||User password attribute name||User password attribute name is not required. Its used only in authenticate operation.|
|tmpFolder||Tmp folder||Folder where csv connector can write temporary files. By default the same folder as where csv file resides.|
|readOnly||Read only||Whether file is for read only acces only. Default is false.|
|preserveOldSyncFiles||Preserve old sync files||How many old sync filed do we want to preserve.|
|objectClassDefinition||Object class definition||File which contains definitions for other object classes.|
|recordSeparator||Record separator||Record separator string.|
|uniqueAttribute||Unique attribute||Unique attribute name.|
Configuration samples for CSV resources were updated to the new CSV connector. The Configuration Samples are located at the usual places.
The shadows in the old connector looked like this:
Shadow for new connector looks like this:
The new CSV connector is using native names of attributes and object classes and therefore it has obviously cleaner and better data structure. But this data structure is different and currently there is no way how to automatically transform the shadows.
There is a known issues in midPoint that prohibits the use of native attribute with the name of 'id'. ( MID-3872 - Attribute 'id' Open ). If the attribute name in the CSV file cannot be changed then the workaround is to force the use of legacy schema. In that case midPoint will use the legacy ConnId attribute names (icfs:name and icfs:uid) even with the new CSV connector.
We recommend the following migration procedure:
Add resource that will use the new CSV connector. Configure it as a completely new resource using the configuration equivalent to the old one.
Change assignment enforcement level to a permissive value (none or positive).
Set up a correlation expression to correlate users with the CSV accounts. Reconcile the new CSV resource. The result should be that the CSV accounts on the new resource are linked.
Modify definitions of role and/or direct assignments to point to the new CSV resource instead of the old one. Resource reference (resourceRef) needs to be changed, but also any mappings for identifier attributes (icfs:name and icfs:uid in the old connector).
Delete old CSVFile resource and all shadows that belong to that resource (there is now an option to do this efficiently in the “Repository Objects” GUI page).
Recompute the users. This should remove the orphaned linkRefs in user objects.
Double-ckeck that every thing is switched to the new resource (roles, assignments, shadows exist and are linked to users).
Change assignment enforcement level to the original.
Migration: Legacy CSVFile Connector
The legacy CSVFile connector is still available and it still can be used. This avoids the need for data (shadow) migration.
To use the legacy CSVFile connector in midPoint 3.6 or later please follow these steps:
Download JAR of the legacy CSVFile connector and deploy it into midPoint
In all the resource definitions change connectorRef to point to the newly discovered legacy CSVFile connector.
No change in configuration, shadows or roles is needed.
The legacy CSVFile connector will be maintained for a reasonable migration period which mostly depends on the requirements of midPoint subscribers. After that period the legacy connector will no longer be supported. Therefore please plan the migration to the new connector accordingly.