You are not logged in.

Dear visitor, welcome to VDR Portal. If this is your first visit here, please read the Help. It explains in detail how this page works. To use all features of this page, you should consider registering. Please use the registration form, to register here or read more information about the registration process. If you are already registered, please login here.

1

Monday, April 16th 2012, 4:34pm

xmltv2vdr plugin and channel names

Hi,

In a thread about xmltv2vdr, somebody asked for an explanation about the usage of channelnames used by xmltv2vdr. So, let me try to give some help:

In fact, xmltv2vdr does not use the channelnames available in the channels.conf of the vdr; it also does not use the channelnames provided for example by the grabbers of the xmltv package; in fact, the aim is to define an own set of channelnames that could be used by all the grabbers, so that in the long run, the matching between channels in an xmltv file and the channels of the vdr could be automated.

There already is a short list with channelnames for xmltv2vdr; you can find it here:
http://vdr-wiki.de/wiki/index.php/Xmltv2vdr-plugin

If you are writing a grabber importing epg data for one of the channels already present in the list that I linked above, please use the name of that list for the id of the channel in the xmltv file or stream created by the grabber.

We are far from having a system, that automatically matches the channelnames from the xmltv file or stream to the channels of the channels.conf; so this step has to be done manually by the user in the OSD of xmltv2vdr.

But once the user has performed the matching of the id in the xmltv file or stream and the channels in his VDR, he does not have to do it anymore if he replaces his grabber by another one, that also uses the ids of the list provided in the link above.

Finally, you might ask, why did the xmltv2vdr developers decide to create an own new channellist. Let's assume that you have a setup with dvb-s and dvb-c or that there is a channel that you receive in SD or HD. From the point of view of the epg, there is no difference for the schedule if the channels is received by the dvb-s or dvb-c card; the schedule also does not change for a channel is received in HD or SD. But for the VDR, these are all different channels. As xmltv2vdr allows to map one id of the xmltv file to multiple channels, all these variants of one channel can get their epg from only one id of the xmltv file.

We are far from having an automated matching between the ids of the xmltv file or stream and the channels of the VDR; but considering the other project of the channelpedia that has been started not very long ago, such an automated match might be possible in the long run.

Finally, what is essential for the developers of grabbers for xmltv2vdr? They should use the channelnames defined in the following page for the ids of the channels in their xmltv file or xmltv stream; this way, if every grabber uses the same ids, the matching between the ids and the channels of the VDR has only to be done one time.
http://vdr-wiki.de/wiki/index.php/Xmltv2vdr-plugin


I hope to have been helpful, but there is one problem left: what to do for the channels that are not available in list? I hope that the xmltv2vdr developers will step in with a procedure to expand the list.

Cheers,

Ludi

This post has been edited 1 times, last edit by "ludi" (Apr 16th 2012, 5:00pm)


hepi

Sage

Posts: 5,243

Location: Heidelberg

  • Send private message

2

Monday, April 16th 2012, 4:44pm

Hi,

I'm trying to find a sophisticated approach for the generation of unique IDs that I have tried to describe here:
[Announce] Entwurf: Channelpedia generiert automatisch unique Kanal-IDs

I'm happy to translate that into English if needed.

Cheers,
hepi
Must-read: yaVDR-FAQ | Aktuelle Kanallisten findet Ihr in der Channelpedia | Meine momentan aktive Hardware: VDR1: yaVDR 0.5alpha1 auf Zotac ZBOX ID40 + 2x Technisat Skystar USB HD + Gyration GYR3101 FB & KB + LCD-Display Linux4Media 4243:ee08 | VDR2: yaVDR 0.5.alpha1 auf Acer Aspire Revo 3600 + 2x TechnoTrend S2-3600 DVB-S2 USB | VDR 3: Seagate Goflex Net mit Debian Squeeze + Kernel 3.1.10 + VDR 1.7.22 + HDHomeRun 2x DVB-C

3

Monday, April 16th 2012, 4:47pm

I hope to have been helpful, but there is one problem left: what to do for the channels that are not available in list?


If you write an grabber and your epgsource provides channels that not in that *) list, then you need add the channels to that list (its wiki, everyone can do this). Its forbidden **) to develop an grabber that uses channel ids that are not in this list.

Sure, the German wiki is not ideal, but until someone find a better solution... hepi started a topic about that [Announce] Entwurf: Channelpedia generiert automatisch unique Kanal-IDs

BTW @ludi: Thanks for explain this fine in English. Nice to have an link to point to.

cu

*) http://vdr-wiki.de/wiki/index.php/Xmltv2vdr-plugin
**) not really ;) but the idea are that all grabber use only ids that are in this list.

Mein VDR

Mein VDR
Digitainer2xBouget DVB-SDebian Squeeze (Kernel 2.6.35.3 von kernel.org)Softdevice Ausgabepluginvdr 1.6.0-3 (Extensions Patch 72) und viele Plugins von SourceMedion X10 FernbedienungSDC-Megtron Display (240x128x1) mit GraphLCD-PluginFreevo 1.9.0
Vodcatcher Helper in ein freundliches DEB verpackt, Tester Willkommen: http://dl.dropbox.com/s/705bh6ydgisfrqu/index.htmlFingerprint: 8A104A00D5031773A9F72A19BAEE135EA7860149

hepi

Sage

Posts: 5,243

Location: Heidelberg

  • Send private message

4

Monday, April 16th 2012, 5:50pm

The question came up how unique channel ids (used by xmltv2vdr or other plugins/addons) should be crafted:

I'm of the opinion that an automated (coded) approach to generate those ids has a few substantial advantages compared to hand-crafting those ids (without using tight rules). An automated approach tries to harvest as many useful meta information as possible from channel list files and tries to create a little database out of that.

Until now, the list of channel id's for the xmltv2vdr plugin was manually extended whenever somebody had written a new grabber. But why should we not instead generate id's for all those channels in the channels.conf straight away and in advance (only excluding unimportant junk channels). We can also use these id's for channel logo assignments. For logo assignments, we need ID's in the same way we need them for xmltv2vdr.

If we are able to parse a channels.conf file line by line and get nice unique ids based on the channel names in the channels.conf file, we also are able to assign the sid-nid-tid of this channel to the unique id. As with different DVB providers those sid-nid-tid's may vary, we are still able to collect all the sid-nid-tid's that match the channel. But this can only work if we take many different channels.confs from many different providers into consideration. Therefore only a channel list directory like Channelpedia has all the data at hand to do it.

So we are not only generating strings based on channel names from the channels.conf (relatively useless on their own), but we store them together with the related sid-nid-tid's so that we collect data to safely find the right unique id for any channel string from a channels.conf. This is useful knowledge later on.

During the process of the id generation we should use one function that will modify the original channel name string according to our needs:
  • The function enforces lowercase to avoid clashes with mixed case (KIKA, KiKa, Kika)
  • The function should for example delete some special characters that we don't want to be part of an id - including blanks (zdf.info, zdfinfo, zdf-info). Stripping special chars can solve many matching problems that would arise if we don't strip them.
  • The function should add meta information whether the channel is a TV channel or a radio channel (because namesakes may exist).
  • And the function should assign a language based or region based identifier that claryfies the main audience that the channel is broadcasted to (the main audio language of the channel is helpful to determine this, also the manually created grouping rules in Channelpedia help here to a great extend).
  • The function should also have access to a list of channel name variants to know that the German channel "Pro7" can also be called "Prosieben" (those alias lists indeed need to be maintained manually. But they only need to contain stuff that isn't solved by taking out the special characters and forcing lowercase).
  • Inter-channel-relations:
    • Channels that exist in a SDTV and HDTV version should get different IDs on one hand, but those id's need to have some kind of relation to each other so that the relation can be seen by software using this data.
    • The IDs of regional channel variants (WDR Hintertupfingen, WDR Unterammergau) should also have some kind of relation to each other.
    • Timeshift-channels (+1, +24) should have an relation to their "parent" channels.

  • The ID strings should in the end still be human readable although grammar rules and word separation are neglected. But even if they are human readable, it might not be easy or sensible to learn them by heart. It is not necessary to learn them by heart or to remember them correctly because humans won't need to deal with them.

Most of what I describe above I have tried to do with the unique-id generation demo in Channelpedia. (http://channelpedia.yavdr.com/gen/de_uniqueIDs.html)

When we try to find an existing channel id that matches a channel we can now follow different strategies depending if we only know a channel name or if we also know all the channel parameters in the channels.conf (sid-nid-tid, audio languages, type radio / type TV). In the first case, we would need to do a name based matching. This means we need a function that converts the name given in the same way we have created the channel id from a channels.conf and then look for string matches. In the second case, we can look up the sid-nid-tid is in our database and will find out the correct unique id this way.

For each of these theoretical thoughts there already exists practical source code in Channelpedia that I can link to if you are interested.
The current source code of the unique id demo in channelpedia can be found here:
https://github.com/yavdr/channelpedia/bl…s/uniqueIDs.php

This stuff is not only relevant for xmltv2vdr, but also for example for automated channel logo assignment.

Cheers,
hepi
Must-read: yaVDR-FAQ | Aktuelle Kanallisten findet Ihr in der Channelpedia | Meine momentan aktive Hardware: VDR1: yaVDR 0.5alpha1 auf Zotac ZBOX ID40 + 2x Technisat Skystar USB HD + Gyration GYR3101 FB & KB + LCD-Display Linux4Media 4243:ee08 | VDR2: yaVDR 0.5.alpha1 auf Acer Aspire Revo 3600 + 2x TechnoTrend S2-3600 DVB-S2 USB | VDR 3: Seagate Goflex Net mit Debian Squeeze + Kernel 3.1.10 + VDR 1.7.22 + HDHomeRun 2x DVB-C

This post has been edited 24 times, last edit by "hepi" (Apr 16th 2012, 7:40pm)


5

Monday, April 16th 2012, 6:45pm

So, if I get it right, an id for a channel in the channelpedia consists mainly of the channel name, a region or country code and maybe some other metadata about whether it is a radio or tv channel.

Instead of continuing now with two separate lists (the unique ids in the xmltv2vdr wiki and those of the channelpedia), I would suggest the following:

hepi: you add an interface to the channelpedia, where the grabber developer can get the ids to use in the xmltv file or xmltv stream of the grabber he is writing. To get the id, he could for example enter data like the channel name, the region where it is broadcasted etc

Joe_D: you remove the channelnames from the xmltv2vdr wiki and tell people to use the names provided by channelpedia, ideally by providing a link to the interface to search for the unique channel names.

Remains the problem about what to do with the binding channelnames of xmltv2vdr that already exist. One approach would be to change xmltv2vdr to recognize the new and old names. Another approach might be to have channelpedia provide these already existing names. Maybe there is also a third, cuter approach.

Honestly, the sooner an interface to get unique channelnames appears in channelpedia, the better it will be for xmltv2vdr, the developers working on the logos, and probably other people. Anyway, there surely are problems that will only appear once the channelpedia gets really used and I expect that it will have to go through various versions until it offers the various needed features.

Cheers,

Ludi

PS: How do I get my channels.conf into the channelpedia, so that it also contains data of the channels that I am using?

PS2: Another point that might be interesting for the channelpedia is to enhance xmltv2vdr with an option where the user can choose to upload to the channelpedia the matching between ids of the xmltv file and the channels.conf; this would provide data to the channelpedia about what channel of the channels.conf corresponds to what channel of the xmltv file. @Joe_D: you might perhaps talk to hepi about it.

This post has been edited 6 times, last edit by "ludi" (Apr 16th 2012, 7:45pm)


hepi

Sage

Posts: 5,243

Location: Heidelberg

  • Send private message

6

Monday, April 16th 2012, 11:39pm

So, if I get it right, an id for a channel in the channelpedia consists mainly of the channel name, a region or country code and maybe some other metadata about whether it is a radio or tv channel.

As much information as necessary to make sure it is a unique string and at the same time human readable and human understandable, meaning: If a human happens to look at the id's what is not mandatory he will still be able to grasp what channel is meant.

Instead of continuing now with two separate lists (the unique ids in the xmltv2vdr wiki and those of the channelpedia), I would suggest the following:

hepi: you add an interface to the channelpedia, where the grabber developer can get the ids to use in the xmltv file or xmltv stream of the grabber he is writing. To get the id, he could for example enter data like the channel name, the region where it is broadcasted etc


The thing is that using a strict naming convention is one thing and using Channelpedia data is another thing. If we could set up a naming convention based on the bullet list above people can even create unique channel ids for channels that are not in Channelpedia.


Joe_D: you remove the channelnames from the xmltv2vdr wiki and tell people to use the names provided by channelpedia, ideally by providing a link to the interface to search for the unique channel names.

Remains the problem about what to do with the binding channelnames of xmltv2vdr that already exist. One approach would be to change xmltv2vdr to recognize the new and old names. Another approach might be to have channelpedia provide these already existing names. Maybe there is also a third, cuter approach.

Honestly, the sooner an interface to get unique channelnames appears in channelpedia, the better it will be for xmltv2vdr, the developers working on the logos, and probably other people. Anyway, there surely are problems that will only appear once the channelpedia gets really used and I expect that it will have to go through various versions until it offers the various needed features.


I don't think Joe_D should remove anything from the wiki at this time. On one hand I like my approach to solve the id topic, but on the other hand I don't see why Joe_D's approach has to be stopped urgently: I guess it just works and does what it's supposed to do for now.

I don't want to put pressure on anybody and also not on myself. Like I said, my unique id stuff has "draft" status an I was asking for comments in the other thread. I'm convinced that Channelpedia can do a good job in providing id's but there's still work to be done and the best thing would be if at some stage some people will start using it because they feel it helps them. If it's only three to four people discussing this topic, it just is not important enough. :D It's just that you have started a thread regarding channel names and I dumped all my current ideas that went through my head while working on it.

Regarding xmltv2vdr current ids: I guess there could be a way to keep it compatible with both id systems, couldn't there?

Problems will arise. For the time being I'm only creating IDs for channels from de, at, ch. Until this is all sorted out I will not extend the id generation.

Channelpedia gets "really" used all the time while we are talking here. But its main use up to now is to provide sorted channel lists of which some are updated regularly and automatically.

PS: How do I get my channels.conf into the channelpedia, so that it also contains data of the channels that I am using?


You tell me what you have to offer. Which DVB providers do you receive? We can only add those that are not yet part of Channelpedia. Have you followed the Channelpedia announce thread where everything is explained in detail?

PS2: Another point that might be interesting for the channelpedia is to enhance xmltv2vdr with an option where the user can choose to upload to the channelpedia the matching between ids of the xmltv file and the channels.conf; this would provide data to the channelpedia about what channel of the channels.conf corresponds to what channel of the xmltv file. @Joe_D: you might perhaps talk to hepi about it.

Like I wrote on the other thread: I don't get that last idea.

Cheers,
hepi
Must-read: yaVDR-FAQ | Aktuelle Kanallisten findet Ihr in der Channelpedia | Meine momentan aktive Hardware: VDR1: yaVDR 0.5alpha1 auf Zotac ZBOX ID40 + 2x Technisat Skystar USB HD + Gyration GYR3101 FB & KB + LCD-Display Linux4Media 4243:ee08 | VDR2: yaVDR 0.5.alpha1 auf Acer Aspire Revo 3600 + 2x TechnoTrend S2-3600 DVB-S2 USB | VDR 3: Seagate Goflex Net mit Debian Squeeze + Kernel 3.1.10 + VDR 1.7.22 + HDHomeRun 2x DVB-C

Similar threads