Skip to content

[COLLECTIONS-885] PatriciaTrie. Prevent silent mutations, prevents CMEs when iterating.#672

Open
griffinjm wants to merge 1 commit intoapache:masterfrom
griffinjm:collections-885-patriciatrie-silent-mutation-bugfix
Open

[COLLECTIONS-885] PatriciaTrie. Prevent silent mutations, prevents CMEs when iterating.#672
griffinjm wants to merge 1 commit intoapache:masterfrom
griffinjm:collections-885-patriciatrie-silent-mutation-bugfix

Conversation

@griffinjm
Copy link

@griffinjm griffinjm commented Mar 19, 2026

Some methods in AbstractPatriciaTrie use a pattern where they silently mutate the trie when searching for a key which is not an exact match for an existing key in the Trie. Said methods are invoked by subMap/tailMap/headMap/prefixMap under the hood. This results in CMEs for even simple shared read access.

ceilingEntry() floorEntry() higherEntry() lowerEntry() operations:
1. insert a phantom node into the trie with the search key
2. increment the modCount
3. find the neighbor of that phantom node via nextEntry()/previousEntry()
4. remove the phantom node
5. increment the modCount again
6. then decrement modCount by 2 to hide the 2 modifications (add + remove)

The referenced modCount is used as a fail-fast mechanism in the iterators.

This fix prevents the issue by instead:

1. Finding the nearest entry via getNearestEntryForKey().
2. Determining whether the search key is less than or greater than the found entry using isBitSet() at the first differing bit.
3. Walking forward (nextEntry()) or backward (previousEntry()) to locate the correct ceiling/floor neighbor.

See https://issues.apache.org/jira/browse/COLLECTIONS-885 for all details.

…iews with subMap, headMap, tailMap, and prefixMap. Fixes shared usage where even simple read access among multiple iterating threads results in ConcurrentModificationExceptions.
@griffinjm griffinjm marked this pull request as draft March 19, 2026 20:31
@griffinjm griffinjm changed the title [COLLECTIONS-885] PatriciaTrie. Prevent silent mutatations, prevents CMEs when iterating. [COLLECTIONS-885] PatriciaTrie. Prevent silent mutations, prevents CMEs when iterating. Mar 19, 2026
@griffinjm griffinjm marked this pull request as ready for review March 19, 2026 20:32
* A null-safe utility method for calling {@link KeyAnalyzer#compare(Object, Object)}
*/
final boolean compareKeys(final K key, final K other) {
final boolean keysAreEqual(final K key, final K other) {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed as this method name is a little misleading, it's actually checking that the keys are equal, in a null-safe manner, returning a boolean, not an int as you would expect.

assertTrue(trie.prefixMap(prefixString).containsKey(longerString));
}

@Test
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added these simple tests to verify continued correctness of the modified methods.

I could also add a threaded test which verifies there is no CME when one thread is iterating and another is creating a new sub/tail/head/prefix map view, but it's not guaranteed to result in a CME depending on timing. From my own local testing using a constantly iterating thread and a second thread which constantly creates new map views it fixes the issue, no CMEs on the new branch.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @griffinjm

This class is not advertised as thread-safe so we don't need to further complicate the code with multi-threaded testing.

Copy link
Member

@garydgregory garydgregory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello Hello @griffinjm
If I only apply the test side of the patch, all the tests still pass, the new tests should fail. This means a regression could undo whatever the main changes claim to make.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants