Oracle Berkeley DB Java Edition 11G R2 Change Log

Library Version 11.2.4.1, Release 4.1.27

Changes in 4.1.27

  1. Made DbPreUpgrade_4_1 and DbRepPreUpgrade_4_1 more easily scriptable by exiting with an appropriate status code. These now exit with status 1 if the utility fails, and exit with status 0 if the utility succeeds or the environment has already been upgraded to JE 5. [#21405] (4.1.22)

  2. Fixed a bug that caused data corruption (for example, resulting in an EnvironmentFailureException with a LOG_FILE_NOT_FOUND message) under certain circumstances, when many Databases are created and not all Databases are kept open. In fact two bugs were fixed that contributed to the problem. The first bug was a memory size calculation error and caused heavy/costly cache eviction, even when the cache would normally be large enough to avoid such eviction. This calculation error occurred during log cleaning, and only when closed databases had been previously evicted from the cache. The second bug caused Btree corruption when cache eviction was heavy and many closed databases were evicted. [#21686] (4.1.23)

  3. Fixed a bug where the following exception could be thrown during heavy concurrent write and read operations. This did not cause data loss and did not invalidate the Environment instance. 
     
com.sleepycat.je.EnvironmentFailureException: 
(JE 5.0.34) com.sleepycat.je.utilint.RelatchRequiredException 
UNEXPECTED_EXCEPTION: Unexpected internal Exception, may have side effects. 
at com.sleepycat.je.EnvironmentFailureException.unexpectedException(EnvironmentFailureException.java:286) 
at com.sleepycat.je.tree.BIN.fetchTarget(BIN.java:1268) 
at com.sleepycat.je.Cursor.checkForInsertion(Cursor.java:3006) 
at com.sleepycat.je.Cursor.retrieveNextAllowPhantoms(Cursor.java:2926) 
at com.sleepycat.je.Cursor.retrieveNextNoDups(Cursor.java:2789) 
at com.sleepycat.je.Cursor.retrieveNext(Cursor.java:2763) 
at com.sleepycat.je.Cursor.getNext(Cursor.java:1116) 
...
    Thanks to OTN user user591209 for reporting this and testing the fix. [#21121] (4.1.24)

  4. Removed an overzealous assertion that could fire during periods of heavy eviction with a stack trace such as the following: 
    java.lang.AssertionError
 at com.sleepycat.je.tree.INTargetRep$Sparse.updateCacheStats(INTargetRep.java:296)
 at com.sleepycat.je.tree.INArrayRep.updateCacheStats(INArrayRep.java:133)
 at com.sleepycat.je.tree.INArrayRep.noteRepChange(INArrayRep.java:158)
 at com.sleepycat.je.tree.INArrayRep.noteRepChange(INArrayRep.java:163)
 at com.sleepycat.je.tree.INTargetRep$None.set(INTargetRep.java:331)
 at com.sleepycat.je.tree.INTargetRep$None.set(INTargetRep.java:303)
 at com.sleepycat.je.tree.IN.updateNode(IN.java:1498)
 at com.sleepycat.je.tree.IN.fetchTarget(IN.java:1332)
    This (like any assertion) would invalidate the Environment, but otherwise did not cause any persistent damage. [#21990] (4.1.24)

  5. Made several improvements to log cleaning and eviction. [#22100] (4.1.25)

  6. Fixed a bug that caused two EnvironmentStats -- getNCachedBINs and getNCachedUpperINs -- to be incorrect when calling Environment.removeDatabase or truncateDatabase. When a database was removed/truncated, the number of BINs/INs belonging to that database were not subtracted from these two stats, even though they were evicted from the cache correctly. So in an application where databases are removed/truncated, these two stats were larger than the actual number of BINs/INs in cache and would increase over time.

    Also added a new cache statistic to help diagnose potential memory leaks: EnvironmentStats.getDataAdminBytes.

    [#22100] (4.1.26)


  7. Fixed a bug that caused a NullPointerException during log cleaning when a database is truncated or removed within a small processing window. An example of the stack trace follows. 
    Feb 11, 2013 3:32:13 PM com.sleepycat.je.utilint.DaemonThread run
SEVERE:  caught exception, java.lang.NullPointerException Continuing
java.lang.NullPointerException
    at com.sleepycat.je.cleaner.FileProcessor.processLN(FileProcessor.java:808)
    at com.sleepycat.je.cleaner.FileProcessor.processFile(FileProcessor.java:674)
    at com.sleepycat.je.cleaner.FileProcessor.doClean(FileProcessor.java:289)
    at com.sleepycat.je.cleaner.FileProcessor.onWakeup(FileProcessor.java:148)
    at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:163)
    at java.lang.Thread.run(Thread.java:679)
    [#22202] (4.1.27)

  8. DbVerify has been improved to add Btree level information for internal databases, and internal database names may now be specified with the -s option. Note that DbVerify is used primarily for analysis and debugging. [#22209] (4.1.27)

  9. Fixed a bug in EnvironmentConfig.setConfigParam where a boolean "true" property specified with leading or trailing whitespace was treated as if it were "false", and no exception was thrown. The same bug impacted boolean properties specified in the je.properties file with trailing whitespace.

    WARNING: This fix could change the behavior of JE if a boolean EnvironmentConfig property is specified as "true" but with leading or trailing whitespace. Previously such a property would be set to false, and now it will be set to true.

    [#22212] (4.1.27)


Changes in 4.1.21

  1. Fixed a bug that could cause the contents of a database to be deleted after a crash, when the database was renamed just prior to the crash. A database can be renamed explicitly with Environment.renameDatabase, or implicitly when a DPL Renamer mutation is used to rename an entity class or a secondary key field. This bug was introduced in JE 4.1.20. [#21537]

Changes in 4.1.20

  1. Fixed a bug that caused an EnvironmentFailureException with LOG_FILE_NOT_FOUND, for example: 
    com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception:
(JE 5.0.34) fetchTarget of 0xc3d4/0x25dab parent IN=2372138 IN class=com.sleepycat.je.tree.IN
lastFullVersion=0xc4ca/0xd2aad lastLoggedVersion=0xc4ca/0xd2aad parent.getDirty()=false state=0
LOG_FILE_NOT_FOUND: Log file missing, log is likely invalid.
Environment is invalid and must be closed.
...
    The bug occurred after deleting a range of keys in a deferred-write database (DatabaseConfig.setDeferredWrite(true) or StoreConfig.setDeferredWrite(true)), closing the Environment, and then opening the Environment. It is most likely to occur when deleting large numbers of consecutive keys, and when opening and closing the Environment often, with the log cleaner enabled.

    Because a log file is incorrectly deleted as a result of this bug, restoring from a backup is necessary, i.e., there is no way to repair the environment. Many thanks to Vishal Vishnoi and Vladimir Egorov for reproducing the problem and helping us to diagnose it.

    [#21348]


  2. Fixed a bug in the DbPreUpgrade_4_1 and DbRepPreUpgrade_4_1 utilities that prevented them from working properly due to concurrent log cleaning activity while the utility is running. The symptom of the bug is that, after running the utility without errors, when then opening the Environment with JE 5, the following exception would occur: 
      com.sleepycat.je.EnvironmentFailureException: (JE 5.0.XX) Before upgrading to
  JE 5.0, the following utility must be run using JE 4.1: DbPreUpgrade_4_1
  using the JE old version. See the release notes.
    WARNING: Due to this bug, JE 4.1.20 or later should be used to run the DbPreUpgrade_4_1 and DbRepPreUpgrade_4_1 utilities. See the JE 5.0 change log for information on running these utilities.

    Thanks to Vinoth for reporting the problem and helping us with the diagnosis and testing. [#21304]


  3. Fixed a bug that could cause Environment.removeDatabase or truncateDatabase to loop in a "livelock" pattern along with active log cleaner threads, where the removeDatabase or truncateDatabase thread is not making forward progress. Also fixed a bug that could cause this state to eventually cause an EnvironmentFailureException during a subsequent recovery, and prevent opening the environment. [#20816]

  4. Fixed a bug where the wrong stack trace was sometimes output for an owner or waiter thread, when using EnvironmentConfig.TXN_DEADLOCK_STACK_TRACE for debugging.

  5. Added additional fixes to prevent looping and long operations times for Environment.removeDatabase and truncateDatabase (see [#20816] above). The additional fixes address long operation times due to log cleaning with large file sizes in combination with eviction. Also fixed a bug that slows down log cleaning (contributing to the long operation times as well) when large numbers of databases are present, and not all databases are kept open by the application. [#21015]

  6. Fixed a bug that sometimes caused a ConcurrentModificationException when calling Environment.setMutableConfig, when exception listeners were previously configured. [#21177]

Changes in 4.1.17

General Changes

  1. Fixed a bug introduced in JE 4.1.10 which could occasionally result in this exception: 
    Caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: 
(JE 4.1.10) ...  unexpected get for VLSN: 37,054,546 already waiting for VLSN: 37,054,545
 UNEXPECTED_STATE_FATAL: Unexpected internal state, unable to continue. Environment is invalid and must be closed.
        at com.sleepycat.je.EnvironmentFailureException.unexpectedState(EnvironmentFailureException.java:391)
        at com.sleepycat.je.rep.vlsn.VLSNIndex.setupWait(VLSNIndex.java:550)
        at com.sleepycat.je.rep.vlsn.VLSNIndex.waitForVLSN(VLSNIndex.java:502)
        at com.sleepycat.je.rep.vlsn.VLSNIndex.awaitConsistency(VLSNIndex.java:1747)
        at com.sleepycat.je.rep.impl.RepImpl.awaitVLSNConsistency(RepImpl.java:1378)
        at com.sleepycat.je.recovery.Checkpointer.doCheckpoint(Checkpointer.java:712)
        at com.sleepycat.je.recovery.Checkpointer.onWakeup(Checkpointer.java:507)
    This problem had been introduced in JE 4.1.10 and can only be seen in that release. The problem is transient and there is no change to the persistent data. After closing and reopening the environment, the environment will be fully consistent and reusable.

    Thanks to Cesar for reporting this on OTN. [#20165]


  2. The master node in a replication group now rebroadcasts election results on a periodic basis, to help restore normal functioning of a replication group after a network partition has been resolved. The default period is a minute and can be modified by the new ReplicationConfig parameter ReplicationConfig.ELECTIONS_REBROADCAST_PERIOD. Please review its javadoc for further details. [#20220]

  3. Fixed a bug introduced in JE 4.1.7 which could cause an incomplete (not committed) transaction to be applied, as if it were committed, under certain circumstances. The circumstances that cause the bug are:
    1. Using an explicit Transaction, the first record in the Transaction is written in a particular timing window at the beginning of a checkpoint.
    2. The checkpoint finishes and then the application crashes before Transaction.commit or abort can be called by the application, or if NoSync durability is used, before the commit or abort can be flushed to the log.
    3. When the application opens the Environment after the crash, the state of the database is as if the transaction were committed.
    [#20270]

  4. Fixed a problem arising from a network partition event, that would result in an unnecessary rollback of committed transactions. This problem typically manifests itself in the application receiving a RollbackProhibitedException if the number of transactions exceeds je.rep.txnRollbackLimit. The following describes a scenario leading to the problem:

    1. Consider a three node replication group: A, B, C with A as the master and nodes: B, C serving as replicas.
    2. A network partition isolates A from B and C, resulting in the partitions: (A) and (B,C).
    3. B and C hold and election and B becomes the master.
    4. There are now two masters: the pre-existing A which is isolated and cannot perform durable writes, and a newly elected B.
    5. The majority side (B,C) is accessible and continues to make progress performing durable writes.
    6. The master on the majority side B goes down. There is now no master on the (B,C) side since there is no quorum.
    7. Some time later, the partition is healed.
    8. C now sees A as an established master and syncs with it potentially rolling back committed transactions as a result.

    The fix changes the final step above so that an election is held before the rollback is allowed to proceed. The election results in C being elected the new master. Node A encounters a MasterReplicaTransitionException which it must handle by closing and re-opening its environment handle so it can resume operations as a replica. [#20258] [#20572]

  5. Fixed a bug in the processing of internal BDBJE metadata which would result in the following stack trace when a replicated environment was re-opened: 
    java.lang.IndexOutOfBoundsException: Index: 110, Size: 110
    at java.util.ArrayList.RangeCheck(ArrayList.java:547)
    at java.util.ArrayList.get(ArrayList.java:322)
    at com.sleepycat.je.rep.vlsn.VLSNBucket.findPopulatedIndex(VLSNBucket.java:441)
    at com.sleepycat.je.rep.vlsn.VLSNBucket.removeFromTail(VLSNBucket.java:730)
    at com.sleepycat.je.rep.vlsn.VLSNIndex.pruneDatabaseTail(VLSNIndex.java:1087)
    at com.sleepycat.je.rep.vlsn.VLSNIndex.merge(VLSNIndex.java:1274)
    at com.sleepycat.je.rep.vlsn.VLSNIndex.init(VLSNIndex.java:1161)
    at com.sleepycat.je.rep.vlsn.VLSNIndex.(VLSNIndex.java:382)
    at com.sleepycat.je.rep.impl.RepImpl.preRecoveryCheckpointInit(RepImpl.java:374)
    at com.sleepycat.je.recovery.RecoveryManager.recover(RecoveryManager.java:238)
    at com.sleepycat.je.dbi.EnvironmentImpl.finishInit(EnvironmentImpl.java:549)
    at com.sleepycat.je.dbi.DbEnvPool.getEnvironment(DbEnvPool.java:237)
    at com.sleepycat.je.Environment.makeEnvironmentImpl(Environment.java:229)
    at com.sleepycat.je.Environment.(Environment.java:211)
    at com.sleepycat.je.Environment.(Environment.java:176)
    at com.sleepycat.je.rep.ReplicatedEnvironment.(ReplicatedEnvironment.java:443)
    at com.sleepycat.je.rep.ReplicatedEnvironment.(ReplicatedEnvironment.java:318)
    at com.sleepycat.je.rep.ReplicatedEnvironment.(ReplicatedEnvironment.java:379)
    There was no corruption to the persistent data, but without this fix, the environment would repeatedly fail to open. [#20796]

  6. Fix a bug that could cause a EnvironmentFailureException of type LOG_FILE_NOT_FOUND during recovery (while opening an environment). The bug has not been reported in the field and was discovered by a long running, internal stress test. The signature of the bug is a log file -- one of the last files in the environment -- with the lastEntryInPrevFileOffset FileHeader field set to zero, as shown by DbPrintLog: 
    <entry lsn="0x1791f/0x0" type="FileHeader/-1" prev="0x0" size="24"
cksum="659948021"><FileHeader num="0x1791f"
lastEntryInPrevFileOffset="0x0"
logVersion="0x8" time="2011-09-26 10:36:07.055"/></entry>
    [#20732]

Utility Changes:

  1. Added the com.sleepycat.je.rep.util.DbResetRepGroup utility, which resets the members of a replication group, replacing the group with a new group consisting of a single new member.

    This utility is useful when the user wishes to create a new replication group, starting with a copy of an existing replicated environment. The utility can also be used to change the group name associated with the environment.

Changes in 4.1.10

  1. Fixed a bug that caused a hard deadlock between a thread performing an Environment.sync (or a checkpoint when deferred-write databases are used) and another thread (which could be an application thread) that is performing eviction. The problem occurs when evicting cached information for a database that is no longer in use, and is most likely to occur after closing a database and/or when the JE cache is small enough to cause heavy eviction activity. An example thread dump is below. 
    Thread performing eviction (in this case the JEEvictor thread):
    com.sleepycat.je.recovery.DirtyINMap.coordinateEvictionWithCheckpoint(DirtyINMap.java:92)
    waiting to lock <0xaeae7118> (a com.sleepycat.je.recovery.DirtyINMap)
    at com.sleepycat.je.recovery.Checkpointer.coordinateEvictionWithCheckpoint(Checkpointer.java:203)
    at com.sleepycat.je.evictor.Evictor.coordinateWithCheckpoint(Evictor.java:1073)
    at com.sleepycat.je.evictor.Evictor.access$000(Evictor.java:124)
    at com.sleepycat.je.evictor.Evictor$1RootEvictor.doWork(Evictor.java:707)
    at com.sleepycat.je.tree.Tree.withRootLatchedExclusive(Tree.java:334)
    at com.sleepycat.je.evictor.Evictor.evictRoot(Evictor.java:741)
    at com.sleepycat.je.evictor.Evictor.evictBatch(Evictor.java:630)
    at com.sleepycat.je.evictor.Evictor.doEvict(Evictor.java:536)
    at com.sleepycat.je.evictor.Evictor$BackgroundEvictTask.run(Evictor.java:1215)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)

Thread calling Environment.sync:
    sun.misc.Unsafe.park(Native Method) parking to wait for
    <0xaeafb698> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:941)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1261)
    at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:594)
    at com.sleepycat.je.latch.SharedLatch.acquireShared(SharedLatch.java:149)
    at com.sleepycat.je.tree.Tree.withRootLatchedShared(Tree.java:344)
    at com.sleepycat.je.dbi.DbTree.getHighestLevel(DbTree.java:1615)
    at com.sleepycat.je.recovery.DirtyINMap.selectDirtyINsForCheckpoint(DirtyINMap.java:236)
    locked <0xaeae7118> (a com.sleepycat.je.recovery.DirtyINMap)
    at com.sleepycat.je.recovery.Checkpointer.doCheckpoint(Checkpointer.java:495)
    locked <0xaeae7058> (a com.sleepycat.je.recovery.Checkpointer)
    at com.sleepycat.je.dbi.EnvironmentImpl.invokeCheckpoint(EnvironmentImpl.java:1816)
    at com.sleepycat.je.Environment.sync(Environment.java:1472) ...
    [#19567]

  2. Fixed a bug that caused an infinite loop when calling the Transaction.commit or Transaction.abort method, when the transaction is used to both truncate and remove a single database, using Environment.truncateDatabase and removeDatabase. Now an exception will be thrown by Environment.removeDatabase in this situation, and the application must call Transaction.abort operation to end the txn. Although this is not a final solution (ideally it should be possible to both truncate and remove a database in a single transaction), this fix prevents the infinite loop in the commit and abort methods. Thanks to Christoph for reporting this on OTN.
    [#19636]

  3. Fixed a race condition that sometimes caused Environments opened with EnvironmentConfig.setSharedCache(true) to not use the shared cache and instead use a private cache. This happened only when opening multiple Environments concurrently in more than one thread. Thanks to Tim Underwood for reporting this on OTN and identifying the specific problem in the JE source code. [#19680]

  4. Fixed a bug that causes an EnvironmentFailureException when a JE HA environment is restarted. The stacktrace looks like this: 
     ... recoveryTracker should overlap or follow on disk last VLSN of 52,678 
recoveryFirst= 52,680 UNEXPECTED_STATE_FATAL: Unexpected internal state,
 unable to continue. Environment is invalid and must be closed.
at com.sleepycat.je.EnvironmentFailureException.unexpectedState
 (EnvironmentFailureException.java:391)
at com.sleepycat.je.rep.vlsn.VLSNIndex.merge(VLSNIndex.java:1230)
at com.sleepycat.je.rep.vlsn.VLSNIndex.init(VLSNIndex.java:1141)
at com.sleepycat.je.rep.vlsn.VLSNIndex.(VLSNIndex.java:367)
at com.sleepycat.je.rep.impl.RepImpl.preRecoveryCheckpointInit(RepImpl.java:373)
at com.sleepycat.je.recovery.RecoveryManager.recover(RecoveryManager.java:238)
at com.sleepycat.je.dbi.EnvironmentImpl.finishInit(EnvironmentImpl.java:534)
 ...
    Thanks to Jan for reporting this problem on OTN and working with us to fix it. [#19754]

Changes in 4.1.7

General Changes

  1. Fix a bug that could cause an EnvironmentFailureException with LOG_FILE_NOT_FOUND during recovery, meaning that the JE environment cannot be opened. We strongly recommend that all applications using JE 4.1.6 upgrade immediately. The bug was introduced in JE 4.1.6 and is not present in earlier releases. [#19346]

  2. Fixed a bug that prevents using a DPL class converter mutation for a proxy class. Previously, an exception such as the following was thrown: 
    Exception in thread "main" java.lang.ClassCastException:
com.sleepycat.persist.raw.RawObject cannot be cast to
com.sleepycat.persist.model.PersistentProxy
at com.sleepycat.persist.impl.ProxiedFormat.newInstance(ProxiedFormat.java:91)
at com.sleepycat.persist.impl.RecordInput.readObject(RecordInput.java:174)
at com.sleepycat.persist.impl.ReflectionAccessor$ObjectAccess.read(ReflectionAccessor.java:406)
at com.sleepycat.persist.impl.ReflectionAccessor.readNonKeyFields(ReflectionAccessor.java:285)
...
    Thanks to James Li for reporting this on OTN and helping us to identify the problem in the source code. [#19312]

  3. Fix a bug that caused a hard deadlock when attempting to abort a transaction in one thread, while performing operations using the transaction in another thread. Now, rather than a hard deadlock, an IllegalStateException will be thrown in this circumstance. Thanks to Jerven on OTN for reporting this. [#19321]

Changes in 4.1.6

Performance Enhancements

When a BDB JE application has a data set that does not fit entirely in cache, AND there is no particular working set of data that fits in cache, the application will see the best performance when as much of the internal btree nodes, or index, are kept in cache. This release includes improvements to make JE's cache management more efficient, and provides statistics to help determine cache efficiency.
  1. Added concurrent eviction, which can be a benefit to any BDB JE application whose data set does not fit in cache. In past releases, cache eviction has been carried out by JE daemon threads, application threads which call JE operations, and an optional single evictor thread. The actual eviction operation was serialized, and could create a bottleneck where many threads could be seen waiting upon the method com.sleepycat.je.evictor.Evictor.doEvict().

    In this release, cache eviction is no longer serialized, and can be executed concurrently. In addition, JE now has a dedicated, configurable thread pool which will do cache eviction when memory limits are reached. Eviction is done by this dedicated pool, by other JE daemon threads, and by application threads. JE attempts to skew the eviction workload toward the pool and daemon threads, in order to offload application threads.

    The eviction pool is enabled by default, but can be disabled by the je.env.runEvictor property. The properties je.evictor.coreThreads, je.evictor.maxThreads and je.evictor.keepAlive are used to configure the core, max and keepalive attribute for the pool. EnvironmentStats has new statistics that try to give an indication of whether eviction is executed by the eviction pool, by operations with specific com.sleepycat.je.CacheMode settings, by explicit calls to Environment.evictMemory, by operations that will cause the cache to go over budget, or by JE daemon threads. [#18626]


  2. This release introduces several new kinds of in-memory btree node compression, targeted toward reducing the memory needed to hold internal btree nodes in cache. This is particularly helpful for applications which have data sets which do not fit in cache.

    One of the optimizations reduces the in-memory footprint of an internal btree node when only a small portion of it has been referenced, as might be the case when data records are accessed in a random order, or when a subset of data is accessed. It does not help if the application is doing a database-wide cursor traversal. [#18623]

    Another optimization takes effect when the key less than or equal to 16 bytes in size, which can be true either when the key is naturally small, or when key prefixing is enabled through Database.setKeyPrefix(). [#18624]


  3. Optimized Database.sync and Database.close for deferred-write databases, so that no file write or fsync is performed if nothing in the database is dirty. Calling these methods will now have very low cost when no changes to the database have been made. [#18402]

  4. The javadoc for com.sleepycat.je.Durability states that COMMIT_NO_SYNC is the default durablity for replicated transactions.The documentation is correct, but the actual default used was incorrect, and was COMMIT_SYNC. This has been fixed. [#18900]

General Changes

  1. The following exception could be seen in rare circumstances from a replicated environment. It is not an actual error, and was the result of an over zealous assertion. The assertion has been corrected. 
    
<DaemonThread name="INCompressor"/> caught exception: 
 com.sleepycat.je.EnvironmentFailureException:
 Node 1(1):rrep0 Read invisible log entry at 0x98c/0x2c711 
<hdr type="LN_TX/7" <lsn v="6,505,875"> 
 isInvisible="1" prev="0x2c6dd" size="28" cksum="501157099"/>
 LOG_INTEGRITY:
 Log information is incorrect, problem is likely
 persistent. fetchTarget of 0x98c/0x2c711 parent IN=55088 
IN class=com.sleepycat.je.tree.BIN lastFullVersion=0x9c2/0x24a54 parent.getDirty()=true state=1

at com.sleepycat.je.log.LogManager.getLogEntryFromLogSource(LogManager.java:927)
at com.sleepycat.je.log.LogManager.getLogEntry(LogManager.java:781)
at com.sleepycat.je.log.LogManager.getLogEntryAllowInvisibleAtRecovery(LogManager.java:742)
at com.sleepycat.je.tree.IN.fetchTarget(IN.java:1309)
at com.sleepycat.je.tree.BIN.fetchTarget(BIN.java:1321)
at com.sleepycat.je.tree.BIN.compress(BIN.java:804)
at com.sleepycat.je.incomp.INCompressor.compressBin(INCompressor.java:501)
at com.sleepycat.je.incomp.INCompressor.doCompress(INCompressor.java:411)
at com.sleepycat.je.incomp.INCompressor.onWakeup(INCompressor.java:341)
at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:162)

  2. Fixed a bug that causes a EnvironmentFailureException LOG_FILE_NOT_FOUND when using a particular sequence of operations with a Database configured for duplicates (DatabaseConfig.setSortedDuplicates(true)) or a DPL secondary key with a MANY_TO_ONE or MANY_TO_MANY relationship. The sequence of operations that causes the bug is:
    1. A single record exists (no duplicates) for key A
    2. New transaction T1 starts
    3. Key A is deleted in T1
    4. Two or more duplicates are inserted for key A in T1
    5. The T1 transaction aborts
    Later, the log file containing the original version of the record is cleaned and deleted, causing an exception when that record is subsequently accessed. An example stack trace is below. 
    (JE 4.0.92) var/db/bdb_userNode fetchTarget of 0x3151/0x44b2638 parent
IN=729921304 IN class=com.sleepycat.je.tree.DBIN
lastFullVersion=0x3189/0x13616b0 parent.getDirty()=false state=0
LOG_FILE_NOT_FOUND: Log file missing, log is likely invalid. Environment is
invalid and must be closed.
at com.sleepycat.je.tree.IN.fetchTarget(IN.java:1241)
at com.sleepycat.je.tree.BIN.fetchTarget(BIN.java:1300)
...
    Thanks to "helg" on OTN for reporting this and working with us to diagnose the problem. [#17252] (fixed in 4.0.114)

  3. Added documentation for the je.rep.node.priority configuration property. [#17684] (fixed in 4.0.114)

  4. Fixed a bug which could cause PreloadStatus.EXCEEDED_TIME to be incorrectly returned from the Database.preload() method. [#18577] (fixed in 4.0.114)

  5. Improved the exception thrown in the following case: JE does not support opening an environment with first a non replicated and then a replicated environment handle. For example, if the user executes these statements in the following order: 
         Environment env = new Environment( envHome, envConfig)
     ReplicatedEnvironment repEnv = new ReplicatedEnvironment(envHome, repConfig, envConfig)
    she would see a java.lang.ClassCastException, instead of the more informative java.lang.UnsupportedOperationException. This has been fixed. Note that it is permissible to open an environment as a ReplicatedEnvironment, and then follow by opening it as a read only Environment. This was initially reported on the OTN forum. [#18649]

  6. Added the property EnvironmentConfig.CLEANER_LAZY_MIGRATION to provide finer control over log cleaner, checkpointing and eviction behavior. See the javadoc for details. [#18650] (fixed in 4.0.114)

  7. Fixed a bug that causes a fatal exception during recovery (opening of a JE Environment) under certain conditions. An example of the exception stack trace is below. 
    Exception in thread "main" com.sleepycat.je.EnvironmentFailureException:
(JE 4.0.92) last LSN=0x424/0xe4fc1 LOG_INTEGRITY: Log information is incorrect,
problem is likely persistent. Environment is invalid and must be closed.
    at com.sleepycat.je.recovery.RecoveryManager.traceAndThrowException(RecoveryManager.java:3052)
    at com.sleepycat.je.recovery.RecoveryManager.readINs(RecoveryManager.java:867)
    at com.sleepycat.je.recovery.RecoveryManager.buildINs(RecoveryManager.java:621)
    at com.sleepycat.je.recovery.RecoveryManager.buildTree(RecoveryManager.java:513)
    at com.sleepycat.je.recovery.RecoveryManager.recover(RecoveryManager.java:175)
    at com.sleepycat.je.dbi.EnvironmentImpl.finishInit(EnvironmentImpl.java:529)
    at com.sleepycat.je.dbi.DbEnvPool.getEnvironment(DbEnvPool.java:204)
    at com.sleepycat.je.Environment.makeEnvironmentImpl(Environment.java:230)
    at com.sleepycat.je.Environment.(Environment.java:212)
    at com.sleepycat.je.Environment.(Environment.java:166)
    ...
Caused by: com.sleepycat.je.EnvironmentFailureException: (JE 4.0.92)
fetchTarget of 0x89/0x4c2e29 parent IN=4883580 IN
class=com.sleepycat.je.tree.DIN lastFullVersion=0x424/0xdcec4
parent.getDirty()=false state=0 LOG_FILE_NOT_FOUND: Log file missing, log is
likely invalid. Environment is invalid and must be closed.
    at com.sleepycat.je.tree.IN.fetchTarget(IN.java:1241)
    at com.sleepycat.je.tree.DIN.fetchTarget(DIN.java:520)
    at com.sleepycat.je.tree.IN.findParent(IN.java:2704)
    at com.sleepycat.je.tree.Tree.getParentINForChildIN(Tree.java:879)
    at com.sleepycat.je.recovery.RecoveryManager.replayINDelete(RecoveryManager.java:1770)
    at com.sleepycat.je.recovery.RecoveryManager.replayOneIN(RecoveryManager.java:945)
    at com.sleepycat.je.recovery.RecoveryManager.readINs(RecoveryManager.java:846)
    ... 11 more
Caused by: java.io.FileNotFoundException: 00000089.jdb (The system cannot find
the file specified)
    at java.io.RandomAccessFile.open(Native Method)
    at java.io.RandomAccessFile.(Unknown Source)
    at java.io.RandomAccessFile.(Unknown Source)
    at com.sleepycat.je.log.FileManager$1.(FileManager.java:993)
    at com.sleepycat.je.log.FileManager.openFileHandle(FileManager.java:992)
    at com.sleepycat.je.log.FileManager.getFileHandle(FileManager.java:888)
    at com.sleepycat.je.log.LogManager.getLogSource(LogManager.java:1073)
    at com.sleepycat.je.log.LogManager.getLogEntry(LogManager.java:779)
    at com.sleepycat.je.log.LogManager.getLogEntryAllowInvisibleAtRecovery(LogManager.java:743)
    at com.sleepycat.je.tree.IN.fetchTarget(IN.java:1225)
    ... 17 more

    The bug only occurred under the following conditions:

    No data loss occurs as a result of this bug. By using a version of JE with the bug fix, such environments can be opened and used normally. Thanks to OTN user justindthomas for reporting the problem and working with us to diagnose and fix it. [#18663] (fixed in 4.0.114)


  8. Fixed a bug that causes an IllegalStateException when a DPL Converter mutation is used for class evolution, and Replicas are upgraded first (as prescribed) in a replication group. [#18690] (fixed in 4.0.114)

  9. Fixed a bug which can make log utilization inaccurate and prevent log cleaning, when Environment.removeDatabase or Environment.truncateDatabase is called, and the program crashes or exits before any other information is written to disk. For example, if the Environment is closed normally after the removal/truncation, or a scheduled checkpoint occurs, then the bug will not occur. [#18696] (fixed in 4.0.114)

  10. All public JE exception and statistics classes should be serializable. Some were not, and have been fixed. The following classes are now certified to be serializable. [#18738]

    (fixed in 4.0.114)

  11. Fixed a bug which caused a NullPointerException under certain circumstances when key prefixing is enabled. This was originally reported on the OTN Forum. [#18773] (fixed in 4.0.114)

  12. Fixed a bug that caused log cleaner threads to hang with the following stack trace. This could occur infrequently in applications which use EnvironmentMutableConfig.setExceptionListener() and have a high level of concurrent log cleaning. [#18791] (fixed in 4.0.114) 
       java.lang.Thread.State: RUNNABLE
        at java.util.HashMap.put(HashMap.java:374)
        at java.util.HashSet.add(HashSet.java:200)
        at com.sleepycat.je.dbi.EnvironmentImpl.registerExceptionListenerUser(EnvironmentImpl.java:730)
        at com.sleepycat.je.utilint.DaemonThread.(DaemonThread.java:60)
        at com.sleepycat.je.cleaner.FileProcessor.(FileProcessor.java:110)
        at com.sleepycat.je.cleaner.Cleaner.doClean(Cleaner.java:461)
        at com.sleepycat.je.dbi.EnvironmentImpl.invokeCleaner(EnvironmentImpl.java:1879)
        at com.sleepycat.je.Environment.cleanLog(Environment.java:1559)

  13. Fixed a problem where Database.count() could return the wrong value for replicated databases with duplicates, after the environment had experienced a replication rollback. [#18816] (fixed in 4.0.114)

  14. Fixed a problem which was intermittent and seen only on the master node of a replication group which using synchronous transaction commits. The application would see the following stack trace. The problem was transient, no data was lost, and the system could re-open successfully. [#18882] (fixed in 4.0.114) 
    java.io.FileNotFoundException: ... (The system cannot find the file specified)
    at java.io.RandomAccessFile.open(Native Method)
    at java.io.RandomAccessFile.(Unknown Source)
    at java.io.RandomAccessFile.(Unknown Source)
    at com.sleepycat.je.log.FileManager$1.(FileManager.java:998)
    at com.sleepycat.je.log.FileManager.openFileHandle(FileManager.java:998)
    at com.sleepycat.je.log.FileManager.getFileHandle(FileManager.java:893)
    at com.sleepycat.je.rep.stream.FeederReader$SwitchWindow.fillNext(FeederReader.java:523)
    at com.sleepycat.je.log.FileReader.readData(FileReader.java:758)
    at com.sleepycat.je.log.FileReader.readNextEntryAllowExceptions(FileReader.java:259)
    at com.sleepycat.je.log.FileReader.readNextEntry(FileReader.java:230)
    at com.sleepycat.je.rep.stream.FeederReader.scanForwards(FeederReader.java:284)
    at com.sleepycat.je.rep.stream.MasterFeederSource.getWireRecord(MasterFeederSource.java:62)
    at com.sleepycat.je.rep.impl.node.Feeder$OutputThread.run(Feeder.java:659)

  15. When a disk corruption occurs that overwrites a log entry with non-zero data, a checksum error is now properly reported via an EnvironmentFailureException, and tools such as DbScavenger and DbVerifyLog work properly. Earlier, if a corruption overwrote specific parts of the log entry, another exception was mistakenly throw, such as: 
    java.lang.ArrayIndexOutOfBoundsException at
java.util.zip.Adler32.update(Adler32.java:47) at
com.sleepycat.je.log.ChecksumValidator.update(ChecksumValidator.java:59) at
com.sleepycat.je.log.ChecksumValidator.update(ChecksumValidator.java:55) at
com.sleepycat.je.log.LogManager.getLogEntryFromLogSource(LogManager.java:928)
...
    Thanks to Jean-Christophe on OTN for reporting this bug. (fixed in 4.0.114)

  16. Opening a com.sleepycat.bind.serial.StoredClassCatalog on a replica node in a JE HA replication group incorrectly resulted in a ReplicaWriteException. This has been fixed. Thanks to user13067358 on OTN for reporting this problem. [#18938] (fixed in 4.0.114)

  17. A DPL class evolution bug caused an exception or incorrect behavior when renaming a field, when the name change caused the alphabetical order of fields in the class to change. An example of such an exception follows. However, other types of exceptions could also occur. 
    java.lang.IllegalArgumentException: Can not set java.lang.Integer field 
com.sleepycat.persist.test.EvolveClasses$RenameSecField.new_secKey2 to java.lang.String
at java.lang.reflect.Field.set(Field.java:657)
at com.sleepycat.persist.impl.ReflectionAccessor$ObjectAccess.read(ReflectionAccessor.java:422)
at com.sleepycat.persist.impl.ReflectionAccessor.readSecKeyFields(ReflectionAccessor.java:253)
at com.sleepycat.persist.impl.ComplexFormat$EvolveReader.readObject(ComplexFormat.java:2127)
at com.sleepycat.persist.impl.PersistEntityBinding.readEntity(PersistEntityBinding.java:115)
at com.sleepycat.persist.impl.PersistEntityBinding.entryToObjectInternal(PersistEntityBinding.java:83)
at com.sleepycat.persist.impl.PersistEntityBinding.entryToObject(PersistEntityBinding.java:64)
at com.sleepycat.persist.PrimaryIndex.get(PrimaryIndex.java:597)
at com.sleepycat.persist.PrimaryIndex.get(PrimaryIndex.java:584)
...
    This has been fixed. [#18961] (fixed in 4.0.114)

  18. Fixed a bug that causes Cursor.getNextDup or EntityCursor.nextDup to advance to a following key under certain rare conditions. This also has a side effect of causing Database.delete to delete duplicate records for the following key under the same conditions. The conditions that lead to the bug are:
    1. A Database is configured for duplicates or a DPL secondary key with a MANY_TO_XXX relationship is used.
    2. The environment or transaction is explicitly configured for Serializable Isolation (this is not the default isolation mode).
    3. One or more records for key A exist.
    4. The key following key A, key B, has two or more duplicates, and the first duplicate for key B is deleted.
    5. Timing and conditions are such that the deleted duplicate for key B has not been internally compressed out of the Btree.
    6. Cursor.getNextDup or EntityCursor.nextDup is called when positioned on the last record for key A, or Database.delete is called for key A. In this case, the record(s) with key B will mistakenly be returned or deleted.
    [#19026] (fixed in 4.0.114)

  19. Fixed a DPL bug that prevented use of a non-public persistent class. Previously, an exception such as the following was thrown when the constructor was public but the class was non-public. 
    ...
Caused by: java.lang.IllegalAccessException: Class
com.sleepycat.persist.impl.ReflectionAccessor can not access a member of class
XXX with modifiers "public"
at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:65)
at java.lang.reflect.Constructor.newInstance(Constructor.java:505)
at com.sleepycat.persist.impl.ReflectionAccessor.newInstance(ReflectionAccessor.java:151)
...
    Thanks to Sheila on OTN for reporting this problem and working with us. [#19100]

  20. After changing stats collection settings on the JE JConsole plugin, the user may see a delay before the jconsole tab is repainted and changes take effect. This has been improved to provide a more immediate response. [#19101]

  21. Fixed a rare problem that could possibly cause a LogFileNotFoundException if the following actions occurred in a very particular ordering: [#19160]

  22. The javadoc for com.sleepycat.je.rep.ReplicatedEnvironment referred to a je.rep.group.joinGroupTimeout property. That property doesn't exist, and the documentation has been corrected to refer to je.rep.envSetupTimeout.

  23. In a JE replication system, it was possible in some cases for a master node to transfer to replica state and fail to properly reinitialize internal state. The problem is that some data record locks were not properly released, and future access to those data records could cause deadlocks.

    To resolve this, a new exception, com.sleepycat.je.rep.MasterReplicaTransitionException is thrown in these cases. MasterReplicaTransitionException is a RestartRequiredException, and the application must close and reopen its environment handle, thereby properly reinitializing the node. [#19177]


  24. Fixed a bug that replicated parameters in je.properties can't be recognized if opening a read only standalone Environment on a replicated Environment home. [#19080]

  25. Fixed a bug that if hard recovery tries to truncate a transaction that spread multiple log files will cause an infinite recovery loop. [#19463]

Utility Changes:

  1. DbTruncateLog formerly will only truncate log entries in the log file where the truncated log entry lives, now it will also delete the log files following that log file. [#19463]