Gutenberg

Release 4.0 is a twenty eighth midPoint release code-named Gutenberg. The 4.0 release is a major milestone in midPoint development. It brings long-awaited features such as archetypes and support for multiple equivalent accounts. Many internal mechanisms were significantly improved. There is also a number of exciting experimental features. However, perhaps the most important aspect of midPoint 4.0 is the long-term support (LTS). Gutenberg will be supported for many years to come.

Release date: 8th September 2019

Johannes Gutenberg (c.1400 - 1468) was German blacksmith, goldsmith, printer and publisher who introduced printing to Europe with the printing press. Information sharing that was enabled by printing caused a cultural and scientific revolution. Modern period of human history was born. The effect of Gutenberg's inventions can hardly be overstated. However, it was not just the printing press itself that made a difference. Gutenberg created entire printing system: the press, adjustable molds, oil-based ink, mechanical movable type and the alloy for casting the type. Those simple elements combined together to create an efficient and economically feasible system for producing books.

Similarly to Gutenberg's printing system, midPoint 4.0 is a revolutionary release. It bring a couple of long-awaited features. However what really matters is a huge amount of improvements and smaller features. Those are designed to work together with existing midPoint features to create a comprehensive and consistent system for identity management and governance. There are also numerous internal improvements and cleanups that enable a long-term maintenance of midPoint 4.0.

Credits

Majority of the work on the Gutenberg release was done by the Evolveum team. However, this release would not be possible without the help of our partners, customers, contributors, friends and families. We would like to express our thanks to all the people that contributed to the midPoint project both by providing financial support, their own time or those that maintain a pleasant and creative environment for midPoint team. However, midPoint project would not exist without proper funding. Therefore we would like to express our deepest gratitude to all midPoint subscribers that made midPoint project possible.

Features

There are too many features in midPoint 4.0 to list them in details. The Features page lists the features of most recent midPoint release.

Changes with respect to version 3.9

New Features and Improvements

Deprecation, Feature Removal And Incompatible Changes

Releases Of Other Components

Other Major Changes And Limitations

See upgrade instructions below for more details.

MidPoint 4.0 is a  major release. There are changes that are not strictly compatible with midPoint 3.x. Those incompatible changes are mostly removal of schema elements that are deprecated for a long time or elements that were never really used. Therefore major release should not significantly affect midPoint deployments that are maintained properly. However there are also some behavioral changes and changes in internal implementation. There are also changes in support routines, limitations and other non-technical aspects that can affect midPoint deployments.

It is strongly recommended to read those release notes very carefully.

Quality

Release 4.0 (Gutenberg) is intended for full production use in enterprise environments. All features are stable and well tested - except the features that are explicitly marked as experimental or partially implemented. Those features are supported only with special subscription contract.

Release 4.0 is also a long-term support (LTS) release that has a prolonged support lifetime. MidPoint 4.0 and subsequent maintenance updates are recommended as a base for deployments that prefer stability over new features.

Limitations

Platforms

MidPoint is known to work well in the following deployment environment. The following list is list of tested platforms, i.e. platforms that midPoint team or reliable partners personally tested with this release. The version numbers in parentheses are the actual version numbers used for the tests.

It is very likely that midPoint will also work in similar environments. But only the versions specified below are supported as part of midPoint subscription and support programs - unless a different version is explicitly agreed in the contract.

Support for some platforms is marked as "deprecated". Support for such deprecated versions can be removed in any midPoint release. Please migrate from deprecated platforms as soon as possible.

Java

OpenJDK 11 is a recommended Java platform to run midPoint.

Support for Java 8 is deprecated. Running midPoint on OpenJDK 8 is supported for midPoint 4.0 and the preliminary plan is to support for the usual lifetime of ordinary support of midPoint 4.0.x line (which means 3 years). But Java 8 support may be shortened, e.g. in case that Oracle or OpenJDK project will stop providing free updates to Java 8 platform. It is strongly recommended to upgrade to Java 11 as soon as possible.

Support for Oracle builds of JDK is provided only for the period in which Oracle provides public support (free updates) for their builds. End of free updates for Oracle JDK 11 were planned for March 2019, and the current status is not known. Which means that Oracle JDK 11 may not be supported at all for MidPoint 4.0. MidPoint is an open source project, and as such it relies on open source components. We cannot provide support for platform that do not have public updates as we would not have access to those updates and therefore we cannot reproduce and fix issues. Use of open source OpenJDK builds with public support is recommended instead of proprietary builds.

Web Containers

MidPoint is bundled with an embedded web container. This is the default and recommended deployment option. See Stand-Alone Deployment for more details.

Apache Tomcat is supported as the only web container for midPoint. Support for no other web container is planned. Following Apache Tomcat versions are supported:

Apache Tomcat 8.0.x is no longer supported as its support life is over (EOL).

Databases

MidPoint supports several databases. However, performance characteristics and even some implementation details can change from database to database. Since midPoint 4.0, PostgreSQL is the recommended database for midPoint deployments.

Our strategy is to officially support the latest stable version of each database (to the practically possible extent). It may be possible to support also older database versions. But as that means additional testing and support effort, we provide such service only with special support contracts. Contact Evolveum sales for the details.

Supported Browsers

Recent version of browser as mentioned above means any stable stock version of the browser released in the last two years. We formally support only stock, non-customized versions of the browsers without any extensions or other add-ons. According to the experience most extensions should work fine with midPoint. However, it is not possible to test midPoint with all of them and support all of them. Therefore, if you chose to use extensions or customize the browser in any non-standard way you are doing that on your own risk. We reserve the right not to support customized web browsers.

Microsoft Internet Explorer compatibility mode is not supported.

Important Bundled Components

ComponentVersionDescription
Tomcat8.5.31Web container
ConnId1.5.0.10ConnId Connector Framework
LDAP connector bundle2.3LDAP, Active Directory and eDirectory connector
CSV connector2.3Connector for CSV files
DatabaseTable connector1.4.3.0Connector for simple database tables

Download and Install


Release Form

Download

Install Instructions

Binary

http://evolveum.com/downloads/midpoint/4.0/midpoint-4.0-dist.zip

Installing midPoint v4.0

Source

From Git repository (tag "v4.0")
https://github.com/Evolveum/midpoint

Building MidPoint From Source Code

Java API

https://www.evolveum.com/downloads/midpoint/4.0/midpoint-api-4.0-javadoc/ [JAR]


Schema Doc

https://www.evolveum.com/downloads/midpoint/4.0/schema-4.0-schemadoc/ [ZIP]


Upgrade

MidPoint is software that is designed for easy upgradeability. We do our best to maintain strong backward compatibility of midPoint data model, configuration and system behavior. However, midPoint is also very flexible and comprehensive software system with a very rich data model. It is not humanly possible to test all the potential upgrade paths and scenarios. Also some changes in midPoint behavior are inevitable to maintain midPoint development pace. Therefore we can assure reliable midPoint upgrades only for midPoint subscribers. This section provides overall overview of the changes and upgrade procedures. Although we try to our best it is not possible to foresee all possible uses of midPoint. Therefore the information provided in this section are for information purposes only without any guarantees of completeness. In case of any doubts about upgrade or behavior changes please use services associated with midPoint subscription or purchase professional services.

Major Release 4.0

Even though midPoint minor releases are managed with almost complete compatibility in mind, midPoint 4.0 is different. MidPoint 4.0 is a  major release. This is a point in midPoint development lifecycle when we remove obsolete functionality and when we make major updates to midPoint schema, database data structures and functionality. Every experienced software engineer knows that it is rarely feasible to make such changes while keeping compatibility as the same time. Therefore midPoint 4.0 is not backwards-compatible with midPoint 3.x. But the situation is not as bad as it might seem. We have tried to avoid changes that were not necessary. Therefore vast majority of midPoint data schema is still compatible. It is just those little places where it is not. Those places are the cause that we cannot declare complete compatibility. And that is also the reason that there is no automatic upgrade path from midPoint 3.x that is 100% reliable.

The changes in midPoint schema and functionality is mostly limited to data items that were already deprecated for a long time, some of them going back even to midPoint 2.x. Those elements were removed or significantly changed. All such changes were marked as "planned removal in 4.0" in midPoint 3.9 schema. This plan was documented in midPoint 3.9 release notes therefore the users had sufficient time to prepare. You should be able to upgrade without any major issues if you haven't used any deprecated properties or if you have avoided the use of removed elements at the very least. But even in that case there may be some updates that need to be done manually. Please refer to the section that deals with midPoint schema for details. Please be especially careful about the iterationSpecification element described below.

Upgrade from midPoint 3.x

Upgrade path from MidPoint 3.x goes through midPoint 3.9. Upgrade to midPoint 3.9 first by using the documented upgrade techniques. Then upgrade from midPoint 3.9 to 4.0.

Upgrade from midPoint 3.9

MidPoint 3.9 data model is not completely backwards compatible with previous midPoint versions. However, vast majority of data items is compatible. Therefore the usual upgrade mechanism can be used. The usual SQL scripts for database schema upgrade are provided. There are some important changes to keep in mind:

Schema changes since 3.9

MidPoint schema was significantly changed since midPoint 3.9. There are many elements that are removed. Those were marked "for removal" in midPoint 3.9. Our Ninja tool can be used to detect the use of those elements even in midPoint 3.9. The "ninja" should be used to audit your use of deprecated data items before attempting to upgrade to midPoint 4.0.

However, there were also changes that were not foreseen at the time of midPoint 3.9 release or changes that cannot be easily detected by the means of our schema language. Those changes must be done manually either before upgrade or the configuration should be fixed after the upgrade:

Other removed items are automatically detected by midPoint parsing code and such elements should be automatically removed from the data. There will be a warning in the log file in case that such an element was removed during parsing. Please note that it takes an update of the object to store the data value without the removed elements. MidPoint does not do it proactively.

Even though this is midPoint 4.0, the numbers in the schema namespaces are still referring to version 3, e.g. http://midpoint.evolveum.com/xml/ns/public/common/common-3. This might seems strange and this decision was given a significant amount of consideration. Version number was introduced to the namespaces in early days of midPoint when such a practice was quite common in the XML world. However, the current consensus of midPoint architects is that the schema versioning mechanism in the XML namespace is far from being ideal. A better versioning mechanism will be needed in the future. The preliminary design is to remove version number from the namespace entirely and use explicit schema versioning that could reflect semantic versioning principles. The preliminary plan is to address this in midPoint 5.0. Which would mean that the namespaces will need to change now and there will be another change in few years when midPoint 5.0 is released. We have decided that the current change from "common-3" to "common-4" would not bring any significant advantage. However, it would significantly complicate the upgrade from midPoint 3.x to midPoint 4.0. Therefore the decision was to keep the "common-3" namespaces. Even though it might look strange, we are doing a very pragmatic decision here that makes midPoint migration much easier for everybody.

Upgrade process

Flowing steps are an outline of an upgrade process:

  1. Complete (i.e. approve or reject) any open approval work items and wait for consequent execution of approval tasks is complete. This is because the approval processes will be lost after the upgrade.
  2. Back up your configuration and data.
  3. Make sure that you are not using elements that are planned for removal. If they are used then reconfigure them to their new equivalents. It is recommended to do the same with all deprecated configuration elements.
  4. Stop all tasks, disable end user access to user interface.
  5. Shutdown midPoint.
  6. Update midPoint distribution binaries.
  7. Run database migration scripts.
  8. Start midPoint. Make sure that tasks are stopped and end user access is disabled.
  9. Update customized initial objects as needed.
  10. Re-add iterationSpecification element to object templates.
  11. Update connector references in the resources.
  12. Run shadow refresh task to populate primaryIdentifierValue in shadow objects.
  13. Mind the warnings in log files. There may be warnings about removal of deprecated items.
  14. Test new configuration.
  15. Start the tasks. Do it with caution. Start the tasks one-by-one, watch for any errors.
  16. Re-enable user access to user interface.

Those steps are just a rough outline. Actual steps needed to upgrade to midPoint 4.0 may be different as the upgrade procedure depends on midPoint customizations, environment and other deployment details.

Changes in initial objects since 3.9

MidPoint has a built-in set of "initial objects" that it will automatically create in the database if they are not present. This includes vital objects for the system to be configured (e.g. role superuser and user administrator). These objects may change in some midPoint releases. But to be conservative and to avoid configuration overwrite midPoint does not overwrite existing objects when they are already in the database. This may result in upgrade problems if the existing object contains configuration that is no longer supported in a new version. Therefore the following list contains a summary of changes to the initial objects in this midPoint release. The complete new set of initial objects is in the config/initial-objects directory in both the source and binary distributions. Although any problems caused by the change in initial objects is unlikely to occur, the implementors are advised to review the following list and assess the impact on case-by-case basis: 

Bundled connector changes since 3.9

Behavior changes since 3.9

Public interface changes since 3.9

Important internal changes since 3.9

These changes should not influence people that use midPoint "as is". These changes should also not influence the XML/JSON/YAML-based customizations or scripting expressions that rely just on the provided library classes. These changes will influence midPoint forks and deployments that are heavily customized using the Java components.

Known Issues and Limitations

As all real-world software midPoint 4.0 has some known issues. Full list of the issues is maintained in jira. As far as we know at the time of the release there was no known critical or security issue.

MidPoint 4.0 is a major release that brings many changes in midPoint code. Some of those changes may be quite disruptive. As midPoint is a very flexible product it is almost impossible to test all the possible use-case scenarios. Therefore the first maintenence release (4.0.1) is planned to follow quickly. Focus of midPoint 4.0.1 will be fixes of issues that are reported by midPoint users. There will be fixes for issues that were not discovered during midPoint 4.0 testing or that could not be fixed during 4.0 development cycle. However, only issues that are covered by midPoint support contract will be fixed.

There is currently no plan to fix the known issues of midPoint 4.0 en masse. These issues will be fixed in future maintenance versions of midPoint only if the fix is covered by a support agreement or subscription. No other issues will be fixed - except for severe security issues that may be found in the future.

The known issues of midPoint 4.0 may or may not be fixed in following maintenance releases or in midPoint 4.1. This depends on the available time, issue severity and many variables that are currently difficult to predict. The only reliable way how to make sure that an issue is fixed is to purchase midPoint support. Or you can fix the bug yourself. MidPoint is always open to contributions.

This may seem a little bit harsh at a first sight. But there are very good reasons for this policy. And in fact it is no worse than what you get with most commercial software. We are just saying that with plain language instead of scrambling it into a legal mumbo-jumbo.

Some of the known issues are listed below:

Disclaimer

Planned release dates are just that: they are planned. We do not promise or guarantee release dates. Software development is a creative activity that includes a lot of inherent risk. We are trying really hard to provide the best estimates. We are not able to provide precise dates for releases or deliveries. Do not rely on midPoint release dates. Plan your project properly to address the risk of delayed midPoint releases.

Planned scope of midPoint releases is also an estimate. MidPoint development process always includes the balancing of the iron triangle. Therefore planned release scope may change at any time. There is a method to make sure that midPoint releases will work well for your project and that method is platform subscription.

We do not make any claims that midPoint is perfect. Quite the contrary. MidPoint is a practical software, developed by living and breathing developers and deployed in a real world. There are both known and unknown issues in midPoint. Also, midPoint is not feature-complete. New features are introduced in midPoint all the time. But not all of them are completed. There are always some limitations. As the license states, midPoint is provided "AS IS". Please do not rely on midPoint functionality that you have not tested to make sure that it works. MidPoint support and subscription programs are a way how to handle those issues. But even with support service, do not rely on functionality that is not documented. If you plan to use undocumented or non-existing functionality, platform subscription is the right service for you.

See Also