Conversation
`_write_archive` pins each file to the stat it was selected with and rejects the upload if anything moved. On Windows that check fires on unchanged files: a path `stat` can be served from cached directory metadata while `fstat` reads the handle directly, and NTFS settles that metadata asynchronously, so `st_ctime_ns` drifts by roughly 1-30 ms with nothing having touched the file. Repeated runs of the packaging tests failed a different, shifting set of cases every time. `st_ctime_ns` earns its place on POSIX, where it is the inode-change time and moves for a chmod or rename that leaves size and mtime alone. It buys nothing on Windows, where it does not hold still. Drop it from the comparison there and keep device, inode, size, and mtime, which still catch replacement and truncation, alongside the existing post-read check for a file that grew during the read. Both comparisons now go through one `_stat_identity` helper instead of two copies of a six-clause condition. Fixes usestrix#1258 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
|
This was referenced Sep 12, 2026
…g ctime On Windows a path stat reports the creation time as st_ctime_ns while fstat on a handle reports the last metadata change, so pinning files with lstat and checking them with fstat compared two different timestamps and rejected every file modified after it was created. The previous commit excluded st_ctime_ns on Windows, which also stopped a chmod or rename from being detected there. Selection now takes fstat from the file's own handle, after checking it is the same file lstat saw, and reads the archive header from that handle. Both sides of the comparison mean the same thing on every platform, so st_ctime_ns is part of the identity everywhere again.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1258
Root cause
select_sourcepins every file withpath.lstat(), and_write_archivechecks it again withos.fstat()on the open handle. On POSIX both calls fillst_ctime_nswith the inode-change time. On Windows they don't: CPython fills it from different Win32 calls. A path stat reports the file's creation time, whilefstatreports its last metadata change.So the guard compares two different timestamps. They only agree for a file that hasn't changed since it was created. That explains both symptoms in this issue:
lstatgave the NTFS creation time, andfstatgave a value equal tost_mtime_ns, the last change. Any file edited after it was created fails every time.Measured on Windows 11 with Python 3.12, on 3,000 freshly written files (a third or half of them appended to after creation):
lstatctime vsfstatctime, asmaincompares themfstatvsfstaton the same handle, before and after readingfstatvsfstatfrom two separate handles, opened 0.5 s apartOn Windows,
fstat's ctime does move on a chmod and on a rename away and back, exactly like POSIX ctime, whilelstat's doesn't.The fix: compare like with like
Instead of leaving
st_ctime_nsout on Windows, selection now pins each file with the same call the archiver checks it with:select_sourcestill useslstatto reject symlinks and non-regular files, then opens the file once and takesfstatfrom that handle. It checks that the handle's(st_dev, st_ino)match thelstat, so a file swapped in between is refused. The archive-magic check reads its header from the same handle instead of opening the file a second time._stat_identityis a single definition for every platform: dev, inode, size, mtime and ctime. There's no longer a platform flag._write_archiveis unchanged. Its before-read and after-readfstatnow compare against a pin of the same kind.What that keeps compared with dropping ctime on Windows: a metadata-only change is still caught there, just as on POSIX. That covers a chmod or attribute change, or a rename away and back, that leaves size and mtime untouched, whether it happens between selection and archiving or during the read. With ctime excluded on Windows, such a file would be uploaded as if nothing had changed.
Behaviour note: because selection now opens each regular file, a file that can be stat'ed but not opened fails with the same
could not safely read ...error that archiving already raised, just earlier.Tests
Four tests replace the two earlier ones:
maintest_archive_accepts_a_path_stat_with_a_different_ctime:lstatreports a different ctime thanfstat, reproducing the Windows case anywheretest_archive_accepts_a_windows_file_modified_after_creation: a real file appended to after creation, with thelstat/fstatdisagreement asserted firsttest_archive_rejects_a_metadata_only_change_after_selection: a real chmod between selection and archivingtest_archive_rejects_a_metadata_change_during_the_read:fstatctime moves between the before and after checksThe first two fail against
main'ssource_upload.py. The last two fail against the previous version of this PR, which left ctime out on Windows. All four pass here.Flakiness, running
tests/test_cloud_source_upload.pyfive times on Windows:On the full suite on Windows, all 28 tests that fail on this branch also fail on
main. They're Windows-specific test assumptions in other modules (19 of them intest_threat_model_tool.py), which #1297 addresses.mainadditionally failed 5 tests intest_cloud_source_upload.pyon that run, and those pass here. The pinnedruff0.15.20 check and format are clean.mypyreports nothing new insource_upload.pyfor--platform linux,darwinorwin32.