The Provenance Repository implementation. The default value is
org.apache.nifi.provenance.PersistentProvenanceRepository. Two additional repositories are available as well. To store provenance events in memory instead of on disk (in which case all events will be lost on restart, and events will be evicted in a first-in-first-out order), set this property to
org.apache.nifi.provenance.VolatileProvenanceRepository. This leaves a configurable number of Provenance Events in the Java heap, so the number of events that can be retained is very limited.
As of Apache NiFi 1.2.0, a third and fourth option are available:
org.apache.nifi.provenance.EncryptedWriteAheadProvenanceRepository. This implementation was created to replace the
PersistentProvenanceRepository was originally written with the simple goal of persisting Provenance Events as they are generated and providing the ability to iterate over those events sequentially. Later, it was desired to be able to compress the data so that more data could be stored. After that, the ability to index and query the data was added. As requirements evolved over time, the repository kept changing without any major redesigns. When used in a NiFi instance that is responsible for processing large volumes of small FlowFiles, the
PersistentProvenanceRepository can quickly become a bottleneck. The
WriteAheadProvenanceRepository was then written to provide the same capabilities as the
PersistentProvenanceRepository while providing far better performance. Changing to the
WriteAheadProvenanceRepository is easy to accomplish, as the two repositories support most of the same properties.
Note Well, however, the following caveat: The
WriteAheadProvenanceRepository will make use of the Provenance data stored by the
PersistentProvenanceRepository. However, the
PersistentProvenanceRepository may not be able to read the data written by the
WriteAheadProvenanceRepository. Therefore, once the Provenance Repository is changed to use the
WriteAheadProvenanceRepository, it cannot be changed back to the
PersistentProvenanceRepository without deleting the data in the Provenance Repository. It is therefore recommended that before changing the implementation, users ensure that their version of NiFi is stable, in case any issue arises that causes the user to need to roll back to a previous version of NiFi that did not support the
WriteAheadProvenanceRepository. It is for this reason that the default is still set to the
PersistentProvenanceRepository at this time.