• Re: Bug reporting

    From Andy Smith@3:633/10 to All on Tuesday, January 06, 2026 22:30:01
    Hi,

    On Tue, Jan 06, 2026 at 06:01:12PM +0000, Lars Olsson wrote:
    It also happens if I instead try to copy the files made on Windows to
    my Debian computer via scp (haven't tested with ftp).

    This seems to be the simplest of all your error cases as if I've
    understood correctly it is a single file transferred from Windows to
    your Debian on a local filesystem without using any archival/compression/encryption tool like 7z.

    I suggest doing this:

    1. Take a large file that's on your Windows machine.

    2. Do a checksum of it with some tool like md5sum, sha256sum or whatever
    Windows has.

    3. Copy the file to your Debian machine using scp, sftp or rsync. It
    should be sent to a local filesystem of some typical type like ext4
    or btrfs,

    4. Run the same checksum on the file that's now on your Debian machine.

    If the checksums match then you haven't managed to reproduce your
    issue. Try to work out what is different between the above and what you
    do when you do see the problem. Try again.

    If the checksums do not match then it seems you have experienced data corruption at some point. Off the top of my head these points could be:

    a) Some aspect of reading data from your Windows machine. Could perhaps
    be debunked by obtaining files from some other source where you know
    the checksum and seeing whether the corruption does or does not
    happen. If it still happens then it's not the Windows machine.

    b) In your network. Seems unlikely since a TCP stream like scp should
    have its own checksumming.

    c) In your Debian machine's hardware like storage controller, drives or
    memory. Seems unlikely since your operating system would just totally
    malfunction relatively quickly.

    d) A bug in your Debian machine's kernel.

    There are no doubt other possibilities but if you can replicate to this
    stage then I think it's worth reporting a bug against the Debian kernel package. To do this ruyn:

    $ reportbug kernel

    while booted into the kernel you replicate this with.

    The error should be easy but rather time-consuming to reproduce. Just
    do something like below on a huge folder with thousands of files. The
    archive files I make are put on a usb-drive and I tested with two
    different usb-drives (actually one SSD in a cheap cabinet and one M.2
    NVMe in an ASUS TUF cabinet, highly recommended).

    Try to exclude USB from your tests. Transfer things over the network to
    local filesystems on storage that's attached by some more traditional
    means like SATA, M.2, SAS, etc. I have seen a LOT of USB storage issues
    like this.

    Thanks,
    Andy

    --
    https://bitfolk.com/ -- No-nonsense VPS hosting

    --- PyGate Linux v1.5.2
    * Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)
  • From Andy Smith@3:633/10 to All on Tuesday, January 06, 2026 22:50:01
    Hi,

    On Tue, Jan 06, 2026 at 04:36:07PM -0500, Dan Ritter wrote:
    Andy Smith wrote:
    c) In your Debian machine's hardware like storage controller, drives or
    memory. Seems unlikely since your operating system would just totally
    malfunction relatively quickly.

    I once saw this happen with a PCI extender (a right-angle
    adapter for a rackmount case) and a 3Ware RAID card.

    Right, yeah. It isn't clear to me if OP is storing all these files on
    USB attached storage. If that is the case then bugs with the driver or
    failing of that hardware would become my #1 guess, and the fact that
    it's only for data not the OS would explain why OP's Debian computer
    doesn't just completely fail.

    Hopefully easy for OP to exclude by testing all this using only the same storage that the OS is on.

    Thanks,
    Andy

    --
    https://bitfolk.com/ -- No-nonsense VPS hosting

    --- PyGate Linux v1.5.2
    * Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)
  • From Dan Ritter@3:633/10 to All on Tuesday, January 06, 2026 23:00:01
    Andy Smith wrote:
    c) In your Debian machine's hardware like storage controller, drives or
    memory. Seems unlikely since your operating system would just totally
    malfunction relatively quickly.


    I once saw this happen with a PCI extender (a right-angle
    adapter for a rackmount case) and a 3Ware RAID card.

    Since the RAID volumes did not include the root filesystem,
    data corruption happened after successful booting and during the
    normal course of operations.

    -dsr-

    --- PyGate Linux v1.5.2
    * Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)
  • From Max Nikulin@3:633/10 to All on Wednesday, January 07, 2026 04:10:01
    On 07/01/2026 4:25 am, Andy Smith wrote:
    d) A bug in your Debian machine's kernel.

    That case I expect to see errors in "journalctl -b ..." with proper boot
    ID or, alternatively, with suitable time range.

    Try to exclude USB from your tests.

    I consider USB as suspicious here as well.

    Since it happens if files are copied from another machine, I would not
    exclude RAM failure, so memtest86+ may be tried.

    I believe it is unlikely, but if it is really 7z bug then it might be
    related to switching to another 7z implementation happened in bookworm.

    --- PyGate Linux v1.5.2
    * Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)
  • From Anssi Saari@3:633/10 to All on Wednesday, January 07, 2026 14:40:01
    Lars Olsson <larso64@outlook.com> writes:

    I make two big 7z archives of each folder, but when I test the archives with ¯7z t¯ 7z reports that the archive is full
    of errors.

    What does this mean, full of errors? Some files bad or all files bad?

    4 If I take the time to make the archives on Windows they are error-free (as expected), and when I then move them to my
    Debian computer with or via an external USB-drive, the errors occur on the files when they are copied to Linux
    regardless if the external usb-drive is formatted with ext4, ntfs or exfat. (you can imagine the amount of time I've
    put on testing ?).

    What about testing the files directly on the USB in Linux without
    copying them anywhere? If they work from there but not after copying to
    Linux, transfer or storage is bad.

    5 It also happens if I instead try to copy the files made on Windows to my Debian computer via scp (haven't tested with
    ftp).

    Another easy thing to check is copying the files back from Linux. If
    they're still good by Windows, at least your transfer method and storage
    are solid. If not, they got corrupted somehow, either during transport
    or storing to Linux.

    The error should be easy but rather time-consuming to reproduce. Just do something like below on a huge folder with
    thousands of files. The archive files I make are put on a usb-drive and I tested with two different usb-drives (actually
    one SSD in a cheap cabinet and one M.2 NVMe in an ASUS TUF cabinet, highly recommended).

    7z a -mx0 -p****** -r -sccUTF-8 -scsUTF-8 -t7z -v2g /media/myname/233gb/users/archive-251231.7z Folderwithmanyfiles

    I think you'll have to do a little better than that for a bug report.
    At least try to find out some figure on how many files are needed to
    cause this. Or is it enough to have just one file in the archive?

    --- PyGate Linux v1.5.2
    * Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)