Skip to end of metadata
Go to start of metadata

This information is applicable for midPoint 3.6. It describes migration from midPoint 3.5.x CSVFile connector to midPoint 3.6 CSV connector.

Synopsis

Up until version 3.6 midPoint included a legacy CSVFile connector. The legacy CSVFile Connector was replaced by new CSV Connector in midPoint 3.6. The new CSV connector is a rewrite from scratch. The old CSVFile connector was written even before midPoint project started and it was not designed for real deployment use. We have maintained and improved the connector during the years. But it was not maintainable any more. Also the ConnId framework evolved over the time and we needed a connector that will use these features. Therefore we have decided to rewrite the connector completely. MidPoint 3.6 no longer bundles the old connector. New CSV connector is bundled instead. Old CSV connector can still be used and it is still supported for deployments that purchased midPoint subscription before midPoint 3.6 was released. As the old connector is not bundled with midPoint any more you have to download the connector JAR and deploy it explicitly.

Migration: New CSV Connector

The new CSV connector is very similar in its operation and configuration as the old CSVFile connector. However it is not entirely backwards compatible with the old connector. There are several major differences:

  • connector name and bundle
  • some connector configuration properties are different and some are missing
  • attribute names: native column names are used instead of icfs:name and icfs:uid
  • the new connector does not support script execution yet

Connector Name and Bundle

The new CSV connector is using bundle and connector name from com.evolveum.polygon namespace. The configuration XML namespace also reflects that:

Bundle name (connectorBundle)

com.evolveum.polygon.connector-csv

Connector name (connectorType)

com.evolveum.polygon.connector.csv.CsvConnector

Configuration namespacehttp://midpoint.evolveum.com/xml/ns/public/connector/icf-1/bundle/com.evolveum.polygon.connector-csv/com.evolveum.polygon.connector.csv.CsvConnector

Connector Configuration

CSV Connector has configuration schema similar to old connector, but there are differences. The new configuration properties are summarized below:

Property nameLabelDescription
multivalueDelimiterMultivalue delimiterMultivalue delimiter character user for splitting multivalue attributes.
commentMarkerComment markerComment marker character.
filePathFile pathPath to CSV file with records.
quoteModeQuote ModeWhat field should be quoted. Allowed values are ALL, MINIMAL, NON_NUMERIC, NONE.
ignoreEmptyLinesIgnore empty linesWhether connector should ignore empty lines in CSV file.
trailingDelimiterTrailing delimiterWhether connector should use trailing delimiter.
encodingEncodingCSV file encoding.
fieldDelimiterField delimiterDelimiter character between fields in one CSV record (escaped as regexp, if needed).
ignoreSurroundingSpacesIgnore surrounding spacesWhether connector should ignore surrounding spaces.
quoteQuoteQuote character.
trimTrimWhether fields should be trimmed.
headerExistsHeader existsWhether header exists in csv file.
passwordAttributeUser password attribute nameUser password attribute name is not required. Its used only in authenticate operation.
tmpFolderTmp folderFolder where csv connector can write temporary files. By default the same folder as where csv file resides.
readOnlyRead onlyWhether file is for read only acces only. Default is false.
preserveOldSyncFilesPreserve old sync filesHow many old sync filed do we want to preserve.
escapeEscapeEscape character.
nameAttribute  
objectClassDefinitionObject class definitionFile which contains definitions for other object classes.
recordSeparatorRecord separatorRecord separator string.
uniqueAttributeUnique attributeUnique attribute name.

Configuration samples for CSV resources were updated to the new CSV connector. The Configuration Samples are located at the usual places.

Shadows

The shadows in the old connector looked like this:

Shadow: Legacy CSVFile connector

Shadow for new connector looks like this:

Shadow: New CSV connector

The new CSV connector is using native names of attributes and object classes and therefore it has obviously cleaner and better data structure. But this data structure is different and currently there is no way how to automatically transform the shadows.

There is a known issues in midPoint that prohibits the use of native attribute with the name of 'id'. ( MID-3872 - Attribute 'id' Open ). If the attribute name in the CSV file cannot be changed then the workaround is to force the use of legacy schema. In that case midPoint will use the legacy ConnId attribute names (icfs:name and icfs:uid) even with the new CSV connector.

Migration steps

We recommend the following migration procedure:

  1. Add resource that will use the new CSV connector. Configure it as a completely new resource using the configuration equivalent to the old one.

  2. Change assignment enforcement level to a permissive value (none or positive).

  3. Set up a correlation expression to correlate users with the CSV accounts. Reconcile the new CSV resource. The result should be that the CSV accounts on the new resource are linked.

  4. Modify definitions of role and/or direct assignments to point to the new CSV resource instead of the old one. Resource reference (resourceRef) needs to be changed, but also any mappings for identifier attributes (icfs:name and icfs:uid in the old connector).

  5. Delete old CSVFile resource and all shadows that belong to that resource (there is now an option to do this efficiently in the “Repository Objects” GUI page).

  6. Recompute the users. This should remove the orphaned linkRefs in user objects.

  7. Double-ckeck that every thing is switched to the new resource (roles, assignments, shadows exist and are linked to users).

  8. Change assignment enforcement level to the original.

Migration: Legacy CSVFile Connector

The legacy CSVFile connector is still available and it still can be used. This avoids the need for data (shadow) migration.

To use the legacy CSVFile connector in midPoint 3.6 or later please follow these steps:

  1. Download JAR of the legacy CSVFile connector and deploy it into midPoint

  2. In all the resource definitions change connectorRef to point to the newly discovered legacy CSVFile connector.

  3. No change in configuration, shadows or roles is needed.

The legacy CSVFile connector will be maintained for a reasonable migration period which mostly depends on the requirements of midPoint subscribers. After that period the legacy connector will no longer be supported. Therefore please plan the migration to the new connector accordingly.

See Also

  • No labels