Blog overview

Latest entries:

2022-03-21: Create OPML feed file from YouTube subscriptions

There used to be a way to directly download an OPML file to import all youtube subscriptions into a RSS reader such as thunderbird but this no longer exists.

But there is another way to access youtube channel updates as a RSS feed. Basically you can query the subscription list as a CSV file and use the contained channel IDs to create an OPML file and import that into thunderbird. Here is how to do it:

  • Go to your youtube account
  • Select menu entry "Your data in youtube"
  • Select "Download youtube data"
  • In section "Create a new export" click on "All youtube data included" and select "subscriptions"
  • Click on "Next step"
  • Select "Export once"
  • Download the archive once it's done and extract the csv file

The CSV file contains the channel ID you need for the RSS link and the channel name. You can use this simple python script to convert the CSV file into an OPML file:

create_opml.py (1 kB)

Execute the script with the CSV file as an argument:

./create_opml.py file.csv
and you get file of the same name with the .opml ending.

Import that file into thunderbird to get all channels as RSS subscriptions.

2022-01-14: Serial data transfer on the Amiga

While implementing serial support for my tool APCComm I stumbled across some weird issues with the serial port at the Amiga and I could not find much information about those specific issues. Basically I was following the good documentation at https://wiki.amigaos.net/wiki/Serial_Device

However, some issues arose during implementation and testing:

  1. For high speed communication with no data loss you normally want to use hardware flow control (CTS/RTS). In the serial.device at the Amiga this is called 7-wire support.
    To make this work correctly, you not only have to set the SERF_7WIRE in the IORequest->io_SerFlags field when opening devices (as described in the manual), but also when adjusting parameters afterwards with SDCMD_SETPARAM. If 7WIRE is not set in this call, it will fall back to 3-wire communication. At least this happens on my Amiga OS 2.05 system.
  2. Sending data is pretty much error free, but receiving data comes with some issues.
    There are basically two error types (returned in io_Status of the IORequest) which can happen during regular transmission.
    • First there is the SerErr_BufOverflow, which basically says that the internal device buffer for incoming data is full. There are two ways to prevent this from happening:
      1. Use a larger buffer (io_RBufLen in SETPARAM) to get more time processing the data.
      2. Query data more often.
      However, I think with proper 7-wire communication, this error should not happen as the Amiga will indicate that the buffer is full stopping the other side from sending data.
    • The second error is more severe. It is SerErr_LineErr (Hardware data overrun).
      I did not find any detailed documentation about when this error happens so the following is my interpretation of what happens.

      The UART responsible for serial transfer has a single shift register and a single byte buffer. Whenever a full byte is received, it is transferred to the buffer and a flag indicates the availability. If another bytes is received without the previous byte being read, the bit 15 (OVRUN) of the register SERDATR ($018) is set indicating data loss.

      I think when this bit is set, the error SerErr_LineErr is returned. I have seen this error happening sporadically when there is high CPU load or a lot of interrupts going on. My guess is that the interrupt for getting the serial byte is not executed fast enough when there are other interrupts routines running.

      With multitasking enabled, there is not much to do about this error. Here are some possible workarounds:

      1. Use a specific communication protocol so that the other communication side is only sending data when you are actually within the receive loop of your code. You could send a single character to let the other side start sending data. Otherwise the other side might already send data while your code is doing something else which could trigger interrupts (like disk access or so).
      2. Reduce serial speed. This is obvious, a slower speed means more time between bytes thus reducing the chance of a buffer overflow.
      3. Disable multitasking/interrupts and poll hardware registers directly. Controlling the serial port is relatively straight forward and when you have full control over the hardware, you can actually make sure you have CPU cycles available for reading the byte buffer from the UART. Downside is of course, that the OS is basically disabled during that time with other consequences.
  3. Support for OS < 2.0
    The current OS documentation usually covers the current library functions but access to the serial.device is possible with Kickstart 1.3 functions as well. But it is hard to find documentation about how to do stuff in older OS version.

    All you have to do is to use the function

    CreatePort("somename",0)
    instead of
    CreateMsgPort()
    and
    CreateExtIO
    instead of
    CreateIORequest

    For cleaning up, you do

    DeleteExtIO
    instead of
    DeleteIORequest
    and
    DeletePort
    instead of
    DeleteMsgPort

    The calls to OpenDevice and DoIO can remain the same.

Conclusion

While it is relatively easy to program the serial.device on the Amiga, having a reliable high speed connection seems to be hard to do with full multitasking enabled. Especially the hardware overrun error is hard to avoid since you can't control the priority of interrupts.
tags: amiga10

2021-08-15: Data corruption on Amiga 600/1200 IDE controller

I have used an IDE drive for a long time in my Amiga 600, starting with a 250MB spinning disk in the 90s and later on using a compact-flash to IDE converter. Over all these years I had no noticeable problems with that setup. But recently, while testing data transfer between my PC and the Amiga, I noticed some weird file corruption happening sometimes even though the different transfer software I used have some sort of checksumming.

1. What was happening?

I finally was able to reproduce it by simple copying a file of half a megabyte or so from the IDE device to ram. I tried different tools and while my main tool of choice, Directory Opus, worked fine, the command line tool "copy" created a corrupted file. I tested some other tools like cp from a GNU installation with mixed results. Some worked fine, others created corrupted files too.

Eventually I wrote a simplified file copy program with configurable buffer size and I was finally able to find out that using buffers larger than 128k make the problem appear. The AmigaOS tool "copy" also has a option "BUF" to choose the buffer size which is a multiple of 512 byte blocks. Up to a value of 256, the copy was correct, larger values created corrupted files.

I suspected either a hardware problem with my machine or the compact flash card, or some transfer size limitation of the IDE controller. After some web research, I found that the IDE controller indeed apparently only supports transfers of 255 512 bytes blocks, or there is some compatibility problems with newer ATA versions:

Either way, the maximum transfer size must be limited to avoid data corruption.

The whole time my disk was configured with a default value of the maximum transfer size of 16 MByte. I then used the HDToolBox program to modify the value to 0x1fe00 (255*512) and finally all copy operations worked fine again.

You can make the adjustment on the live partition, you don't need to reformat anything. To apply the changes, open the HDToolBox program, select your drive and choose "Partition Drive". For each partition on that drive, show the "Advanced Options" and click on "Change File System for Partition". In that dialog, you can enter a new value for "MaxTransfer". Choose "Ok" and repeat for any other partition. When done, click on "Ok" partition window and in the main window click on "Save Changes to Drive". I'm not sure if the parameters take immediate effect, so probably just reboot to be safe.

2. Conclusion

I can't believe the IDE compact flash was misconfigured all the time and I never noticed any data corruption, probably due to the fact that most the file operations I was doing was done by Directory Opus, which apparently uses a smaller buffer size when copying files.

Considering that there is no hint about the correct value while preparing an IDE drive (spinning or flash based), I think it might be helpful to share this information (even though probably not many need that information anyway nowadays).

tags: amiga10

All blog entries: all