[server] Normalize legacy remote log manifest ranges

[gyang94]

Purpose

Fixes #4272.

Loading a legacy manifest containing [10,30) followed by [10,20) can make offset 20 unreadable: the shorter segment replaces the longer one in the offset index. Nested ranges with different starts can similarly hide existing records, and subsequent merges can shrink readable coverage.

This change establishes a consistent logical view before indexing or merging legacy manifests, independently of entry order.

Brief change log

• Add deterministic logical-range normalization: discard contained ranges and clip overlapping suffixes when a segment extends coverage. Preserve existing logical bounds, gaps, and the highest copied end offset.
• Reuse already ordered, non-overlapping logical views after an O(n) scan, avoiding segment-list allocation and sorting. This checks logical bounds rather than relying on manifest version or optional JSON fields.
• Use the normalized manifest for the tablet's indexes, statistics, and current snapshot. Normalize before deletion in trimAndMerge() so removing a visible segment cannot restore a hidden range.
• Keep raw JSON deserialization unchanged so orphan cleanup still sees all persisted physical-file references. Loading does not rewrite snapshots or delete remote files.

As with the existing merge logic, normalization assumes overlapping committed records agree; it does not reconcile divergent WAL histories.

Tests

Passed targeted verification on Java 11:

mvn -o -pl fluss-common,fluss-server -am verify \
  '-Dtest=RemoteLog*Test,DefaultRemoteLogStorageTest,TieredLocalSegmentTTLTest,CommitRemoteLogManifestITCase' \
  -Dsurefire.failIfNoSpecifiedTests=false

• The broader suite above passed on 74c9c6f65: 162 relevant cases (20 in common and 142 in server).
• The final changes passed targeted verify with -Dtest=RemoteLogManifestOverlapTest,RemoteLogManifestJsonSerdeTest,RemoteLogTabletOverlapTest,RemoteLogFetcherTest: 58 cases (22 in common and 36 in server), including empty, single-segment, contiguous, gapped, clipped, and unordered views.
• Reproduced the legacy loading and coverage-shrinking merge failures before applying the fix.
• Added coverage for 120 entry permutations, identical/nested ranges, idempotence, existing clipped ranges, gaps, TTL deletion, copy progress, and persistence/reload.
• Verified KV recovery against real overlapping remote files. FetchLog v0 coverage now sends an actual RPC request through TabletService at the shorter segment's end offset, after local copies have been removed. It checks the returned segment ID and byte position, then reads records using the response's path and position and verifies offsets and checksums in both manifest orders.
• Temporarily bypassing load-time normalization makes the longer-first, same-start case fail at the RPC response error assertion, confirming that the new test detects the original bug.
• Spotless, Checkstyle, and RAT passed.

API and Format

Uses the existing version-1 manifest fields and logical-range model. No RPC or storage-format changes.

Documentation

No new configuration or user-facing feature; no documentation changes.

Generative AI used: Codex (GPT-6).

[platinumhamburg]

@gyang94 Thanks for the fix! The normalization addresses the legacy overlap issue, and the explicit checks for version 1 and the absence of logical offset fields make the legacy-format coverage clear and deterministic.
One small suggestion: would you consider exercising the actual FetchLog RPC path instead of the custom read loop in fetchChecksumsUsingFetchV0()? Checking the returned segment and position at the shorter segment’s end offset could provide more direct regression coverage.

[platinumhamburg]

LGTM +1.