I have used an IDE drive for a long time in my Amiga 600, starting with a 250MB spinning disk in the 90s and later on using a compact-flash to IDE converter. Over all these years I had no noticeable problems with that setup. But recently, while testing data transfer between my PC and the Amiga, I noticed some weird file corruption happening sometimes even though the different transfer software I used have some sort of checksumming.
1. What was happening?
I finally was able to reproduce it by simple copying a file of half a megabyte or so from the IDE device to ram. I tried different tools and while my main tool of choice, Directory Opus, worked fine, the command line tool "copy" created a corrupted file. I tested some other tools like cp from a GNU installation with mixed results. Some worked fine, others created corrupted files too.
Eventually I wrote a simplified file copy program with configurable buffer size and I was finally able to find out that using buffers larger than 128k make the problem appear. The AmigaOS tool "copy" also has a option "BUF" to choose the buffer size which is a multiple of 512 byte blocks. Up to a value of 256, the copy was correct, larger values created corrupted files.
I suspected either a hardware problem with my machine or the compact flash card, or some transfer size limitation of the IDE controller. After some web research, I found that the IDE controller indeed apparently only supports transfers of 255 512 bytes blocks, or there is some compatibility problems with newer ATA versions:
Either way, the maximum transfer size must be limited to avoid data corruption.
The whole time my disk was configured with a default value of the maximum transfer size of 16 MByte. I then used the HDToolBox program to modify the value to 0x1fe00 (255*512) and finally all copy operations worked fine again.
You can make the adjustment on the live partition, you don't need to reformat anything. To apply the changes, open the HDToolBox program, select your drive and choose "Partition Drive". For each partition on that drive, show the "Advanced Options" and click on "Change File System for Partition". In that dialog, you can enter a new value for "MaxTransfer". Choose "Ok" and repeat for any other partition. When done, click on "Ok" partition window and in the main window click on "Save Changes to Drive". I'm not sure if the parameters take immediate effect, so probably just reboot to be safe.
2. Conclusion
I can't believe the IDE compact flash was misconfigured all the time and I never noticed any data corruption, probably due to the fact that most the file operations I was doing was done by Directory Opus, which apparently uses a smaller buffer size when copying files.Considering that there is no hint about the correct value while preparing an IDE drive (spinning or flash based), I think it might be helpful to share this information (even though probably not many need that information anyway nowadays).