[markad] überarbeiteter Decoder

  • vpv

    If you like, you can update the Finnish translation file from plugin/po/fi_FI.po and post it here. This file realy needs a native speaker review.

    VDR
  • Die Version 3.0.8 ist auf vdr-plugin-markad verfügbar.

    Diesmal ein spezieller Fix für Finnische Sender.

    Und wie immer: viele kleine Fehlerbereinigungen und Optimierungen.


    Version 3.0.8 is available. This time a special fix for Finnish channels (thx vpv for reporting)

    - ignore change of real frame rate after first TS file

    got invalid real frame rate from libav in second TS file in some Finnish recordings

    - some minor bug fixes and optimizations, see git

    VDR
  • I tested automatic logo detection. So far I have saved the logos under /var/lib/markad and let markad use those. I found out that when I run markad manually, it can't find the logo in my recordings from the Finnish channel 6 (or "Kutonen"). However, when markad runs automatically by VDR, it will find the logo.


    Has anyone seen a case like this? I'm calling markad like this: markad --log2rec --loglevel 3 --autologo 2 --logocachedir /tmp - 2021-07-22.19.58.6-0.rec/


    kfb77 I can send you logs if you want to debug this.

  • The "logocachedir /tmp" tells markad to search only in /tmp for logos. With "autologo 2" markad will generate logos and store them there as well.

    You need to specify "logocachedir=/var/lib/markad" for your demand.

    --
    vdr User #2022 - hdvdr2: Lenovo SFF M83, Intel(R) Core(TM) i5-4670S, 14 GB Ram, zram-swap/tmp, ubuntu-focal, softhddevice-vdpau
    ddbridge-6.x mit 2xDVB-S2 und (Flex) 2xDVB-C/T Tunern, nvidia-GF1050Ti SFF (nvidia-dkms-470), System SSD btrfs,

    snapper, 8TB HDD XFS/cow /srv/vdr, yavdr-ansible-2.5.6-seahawk, vdr-epg-daemon mit Frodo-plugins, Kernel 5.13.4-xfsscrub

    vdradmin-am, live+webstreaming, vdrmanager (Smartphones als FB), ffmpeg-4.3.1-libfdk_aac, vdr-plugin-hbbtv.

  • The "logocachedir /tmp" tells markad to search only in /tmp for logos. With "autologo 2" markad will generate logos and store them there as well.

    You need to specify "logocachedir=/var/lib/markad" for your demand.

    I'll try to explain what I mean. It would be nice if markad could detect the logo automatically from each recording without me saving the logos manually to /var/lib/markad (or anywhere else). The logos tend to change over time.


    I used /tmp to get markad to use an almost empty directory where it has access to. In that case the automatic logo detection does not work with a manual markad run. If I set --logocachedir /var/lib/markad when running markad manually, then it will detect the logo (named Kutonen_HD-A16_9-P0.pgm) from the recording and mark ads correctly.


    I also set -l /tmpin /etc/vdr/conf.d/50-markad.confand when markad is running during the recording, started by VDR, it will detect the logo from the same channel and same show. It seems to me I've found a case where manual markad runs and automatic markad runs produce different results.

  • wmautner answer "--logocachedir=/var/lib/markad" will solve your problem. Some addional informations:


    "--logocachedir=/tmp" (not "--logocachedir /tmp") is only used if your distribution provides logos to /var/lib/markad/ and you never want to use them. In this case you have to set the logo cache directory to an existing directory where no logo will be found (/tmp exists on all systems).


    Thats is how i recomment to operate. Logos may change, some channel even uses different logos for different shows and worse on different corner. My server takes in average 13s to find the logo, I don't care about this CPU time.


    There are many special quirks for german channel used to find the logo but it is no problem to add these for Finnish channels too. Let me download an example and I will add it.

    VDR
  • Now we had overlapping posts.

    It would be nice if markad could detect the logo automatically from each recording without me saving the logos manually to /var/lib/markad

    markad never save any logo to /var/lib/markad, it only read from this directory if you or your distribution had saved logos there and you tell markad to read from this directory "--logocachedir=/var/lib/markad"


    With --autologo=2 (this is default) it work as you requested. If there is no logo in "--logocachedir=..." it will be searched in the recording. Maybe not with your Finnish recordings, but this can be fixed.


    I used /tmp to get markad to use an almost empty directory where it has access to. In that case the automatic logo detection does not work with a manual markad

    t seems to me I've found a case where manual markad runs and automatic markad runs produce different results.

    That should not be, please post markad.log. Maybe a bug in sending the parameters from the plugin to markad.

    VDR

    The post was edited 2 times, last by kfb77 ().

  • Because I have not saved all the logs, the situation is not quite like I first thought, sorry. I have looked at two recordings. The second recording was actually fine during recording and fine when using manual markad. Contrary to what I said, there is no case where markad by VDR works and manual markad does not.


    The first recording is more interesting. I have uploaded the log as markad-successful-7-tries.txt. In this recording, markad says "no logo found in recording, retry in 6nd recording part" and then "cMarkAdStandalone::CheckLogo(): logo search failed". This is what confused me, I always stopped markad after these messages. But when markad goes to ProcessFiles, it finds the logo!


    When I add --cut to this recording's markad run, markad segfaults. It tries to load 0004.ts, which does not exist. I have uploaded that log as markad-segfault.txt.

  • The first recording is more interesting. I have uploaded the log as markad-successful-7-tries.txt. In this recording, markad says "no logo found in recording, retry in 6nd recording part" and then "cMarkAdStandalone::CheckLogo(): logo search failed". This is what confused me, I always stopped markad after these messages. But when markad goes to ProcessFiles, it finds the logo!

    Thats how it works: start logo search after pre-timer, retry 6 times each 5 min to get a clear and good logo result. I you have a bright background, a transparent logo, a special logo from this show or a very short recording this can fail.

    Then start processing the recording. If we have still no logo do one retry from start of the recording (usually the broadcast beforehand) and take what ever you got. Better a bad/false logo than nothing.

    So what you found in the log is totaly expected.


    Two things are not expected: Of cause the crash in closing the last file with --cut and the 3rd try of logo search has a good result, but it does not pass the validation test.

    I have to take a deeper look, please tar the recording directory and provide me a download link via PM.

    VDR

    The post was edited 2 times, last by kfb77 ().

  • One more comment: I recomment using parameter "nice" if you call markad in command line. This gives markad a low CPU and I/O priority which will help all other applications on your system to keep best performance:


    sudo -u vdr markad --log2rec --loglevel=3 --logocachedir=/tmp nice <path to recording>


    --autologo 2 is not necessary , it is default.

    VDR

    The post was edited 1 time, last by kfb77 ().

  • Thank you once again, kfb77 ! These discussions have been useful. I'll send you a PM, but getting things ready will take a while.


    Edit: I noticed this specific recording has a "Tokyo 2020" Olympic Games sign below the logo. I'm guessing that affects the logo detection.

  • I noticed this specific recording has a "Tokyo 2020" Olympic Games sign below the logo. I'm guessing that affects the logo detection.

    Yes, that is quite possible. But this will happen more often in the near future and not only in Finnish recordings. So I can try to adapt markad in this regard.

    VDR
  • In fact, the reason was the Olympic symbol. Fix is in the git, branch V03. Now I get the logo on first try.


    I can not reproduce your crash but I saw some not good looking free when I debug memory management. This issue is still in progress.

    VDR

    The post was edited 2 times, last by kfb77 ().

  • Found the bug: If last stop mark is before last ts file, I do not open the last file but close it. Not a good idee, this results in your crash. Fix is in branch V03.

    VDR
  • I took a deeper look in this recording: This is a horor movie for logo detection. A logo with very thin lines and a lot of white backgrounds in the logo area. I will try to optimize the detection.

    VDR

    The post was edited 2 times, last by kfb77 ().

  • I added a commit to optimize the logo detection of logos with a very low pixel count.

    The changes essentially only have an internal effect, because markad can filter out most of false logo detection.

    VDR
  • Die Version 3.0.9 ist auf vdr-plugin-markad verfügbar.

    - update Finnish translation (author: vpv )

    - MarkadCut: fix crash if last ts file is not used (thx vpv )

    - fix compile error with libavcodec 59.3.102 (current code base of ffmpeg git master)

    - some minor bug fixes and optimizations, see git

    VDR