svdrpsend CLRE doesn't clear EPG completely if there's active timers

-Dis · Aug 2nd 2021

While I was making xmltv2vdr.pl working I came across possible bug with command svdrpsend CLRE. It doesn't clear EPG completely from channels which has active timers. Is this bug or intended behavior?

This is problematic while using xmltv because if program's start and end time changes program with wrong start and end time is also displayed in EPG.

EPG before CLRE: before.txt

After CLRE: after.txt

Without active timer output is:

Code

telkka@telkka:~$ svdrpsend CLRE 11
220 telkka SVDRP VideoDiskRecorder 2.4.7; Mon Aug 2 23:03:32 2021; UTF-8
250 EPG data of channel "11" cleared
221 telkka closing connection
telkka@telkka:~$ svdrpsend LSTE 11
220 telkka SVDRP VideoDiskRecorder 2.4.7; Mon Aug 2 23:03:34 2021; UTF-8
215-C C-0-32-13208 TLC HD
215-c
215 End of EPG data
221 telkka closing connection
telkka@telkka:~$

Display More

VDR version

Code

telkka@telkka:~$ vdr -V
vdr (2.4.7/2.4.7) - The Video Disk Recorder
conflictcheckonly (0.0.1) - Direct access to epgsearch's conflict check menu
dbus2vdr (31) - control vdr via D-Bus
desktop (0.0.3) - desktop apps menu
devstatus (0.4.1) - Status of dvb devices
epgfixer (0.3.1) - Fix bugs in EPG
epgsearch (2.4.1) - search the EPG for repeats and more
epgsearchonly (0.0.1) - Direct access to epgsearch's search menu
live (3.0.10) - Live Interactive VDR Environment
markad (3.0.9) - Mark advertisements
menuorg (0.5.2) - Reorganizes the main menu
noepg (0.0.5) - block/allow EPG for selected channels
osd2web (0.2.55-GIT) - osd2web plugin
osdteletext (2.2.0) - Displays teletext on the OSD
pulsecontrol (0.2.1) - control settings of pulseaudio
quickepgsearch (0.0.1) - Quick search for broadcasts
skindesigner (1.2.17) - Skin Designer
softhddevice (1.2.2) - A software and GPU emulated UHD device
streamdev-server (0.6.1-git) - VDR Streaming Server
tvguideng (0.3.3) - TV Guide for Skindesigner Skins
tvscraper (0.2.0) - Scraping movie and series info
vnsiserver (1.8.0) - VDR-Network-Streaming-Interface (VNSI) Server

Display More

kls · Aug 2nd 2021

The code in cSVDRPServer::CmdCLRE() is supposed to set the events of all timers to NULL, so the intended behavior is that ALL events are cleared by CLRE.

You might want to check cSchedule::Cleanup(time_t Time) to see if it really deletes all events, and if not, why that happens.

-Dis · Aug 3rd 2021

kls wrote:

The code in cSVDRPServer::CmdCLRE() is supposed to set the events of all timers to NULL, so the intended behavior is that ALL events are cleared by CLRE.

You might want to check cSchedule::Cleanup(time_t Time) to see if it really deletes all events, and if not, why that happens.

I'm sorry but how I can check that?

I set timer to TLC HD. There's also timers to other channels and after clearing EPG all timers are still in timers.conf. Same result when running svdrpsend CLRE without channel number.

kls · Aug 3rd 2021

CLRE doesn't delete timers. Use DELT to do that.

-Dis · Aug 3rd 2021

Oh okay. I don't want to delete timers

~~But CLRE should delete EPG even if there's timers?~~ Edit. you already said that.

-Dis · Aug 3rd 2021

I think I misunderstood you earlier. So SVDRPServer::CmdCLRE() should set all events to NULL on cSchedule::Cleanup(time_t Time) should clean them? How I can debug it? It looks like CLRE deletes EPG events to the moment when first recording timer is set.

kls · Aug 3rd 2021

Let's assume you're using CLRE w/o any additional parameters, which executes this code in cSVDRPServer::CmdCLRE():

Code

LOCK_TIMERS_WRITE;
LOCK_SCHEDULES_WRITE;
for (cTimer *Timer = Timers->First(); Timer; Timer = Timers->Next(Timer))
Timer->SetEvent(NULL); // processing all timers here (local *and* remote)
for (cSchedule *Schedule = Schedules->First(); Schedule; Schedule = Schedules->Next
Schedule->Cleanup(INT_MAX);
cEitFilter::SetDisableUntil(time(NULL) + EITDISABLETIME);
Reply(250, "EPG data cleared");

Here it sets the event of all timers to NULL and calls Cleanup() for all schedules with a time (INT_MAX) that is way in the future.

So now all schedules should be empty.

In cSchedule::Cleanup(time_t Time) we have

Code

cEvent *Event;
while ((Event = events.First()) != NULL) {
if (Event->HasTimer()) dsyslog("event has timer");//XXX
if (!Event->HasTimer() && Event->EndTime() + EPG_LINGER_TIME < Time)
DelEvent(Event);
else
break;
}

See the line marked with XXX. Add that line to your file and recompile.

If you issue the CLRE command and get that log message, the event's 'numTimers' member apparently got out of sync and we'll need to further investigate there.

-Dis · Aug 4th 2021

Here is syslog after starting modified vdr and I then run CLRE.

kfb77 · Aug 4th 2021

kls

I can easy reproduce the problem of -Dis :

1. take an unsed channel x

svdpsend clre x

all events of this channels are deleted

2. take an unused channel y

add a timer (e.g. at the end of the current day)

svdrpsend clre y

only events earlier like the timer of this channel are deleted

Maybe this can help.

kls · Aug 4th 2021

kfb77 I tried that and all events were cleared just as expected.

Are you using plain vanilla VDR 2.5.6 without any patches and with only the minimum number of plugins necessary?

What I did see while testing this is that if "CLRE <number>" is issued while the Schedule menu of that channel is open, the list isn't immediately cleared. Closing and reopening it shows that all events are gone (including those at and after the timer). I'll fix this.

kfb77 · Aug 4th 2021

kls wrote:

Are you using plain vanilla VDR 2.5.6

No, its 2.4.7 from yavdr repository with satip, epg2vdr, epgsearch, live and markad plugin. But I can test it later with 2.5.6 plain vanilla.

kls wrote:

What I did see while testing this is that if "CLRE <number>" is issued while the Schedule menu of that channel is open, the list isn't immediately cleared. Closing and reopening it shows that all events are gone (including those at and after the timer). I'll fix this.

I use live and "svdrpsend lste" as test frontend, this is a headless vdr server.

kfb77 · Aug 4th 2021

Same result with current plain vanilla vdr version.

epg2vdr is the only plugin who can have an impact to events, but test channels are not in epg2vdr channel map, so they will be untouched.

Code

root@VDR-2004-Dev:/usr/src/git# vdr --version
vdr (2.5.6/2.5.6) - The Video Disk Recorder
conflictcheckonly (0.0.1) - Direct access to epgsearch's conflict check menu
epg2vdr (1.1.118-GIT) - epg2vdr plugin
epgsearch (2.4.1) - search the EPG for repeats and more
epgsearchonly (0.0.1) - Direct access to epgsearch's search menu
live (3.0.10) - Live Interactive VDR Environment
markad (3.0.9) - Mark advertisements
quickepgsearch (0.0.1) - Quick search for broadcasts
satip (2.4.1) - SAT>IP Devices

channel without timer: svdrpsend clre 54 && svdrpsend lste 54

Code

220 VDR-2004-Dev SVDRP VideoDiskRecorder 2.5.6; Wed Aug 4 12:12:08 2021; UTF-8^M
215-C S19.2E-133-5-662 1-2-3.tv
215-c
215 End of EPG data^M
221 VDR-2004-Dev closing connection^M

channel with timer at the end of today (UHREN SALE, 23:00 - 00:00 Uhr): svdrpsend clre 55 && svdrpsend lste 55

Code

220 VDR-2004-Dev SVDRP VideoDiskRecorder 2.5.6; Wed Aug 4 12:13:15 2021; UTF-8^M
215-C S19.2E-1-1055-5502 1-2-3.tv HD
215-E 6293 1628110800 3600 50 14
215-T UHREN SALE
215-S 1-2-3.tv Programmvorschau 04.08.2021, 23-0 Uhr
215-D In dieser Stunde erwarten Sie folgende Highlights:|Preissturzauktion Auguste Reymond Cotton Club|Preissturzauktion PortaS Uhr 'Elstra'|Preissturzauktion PortaS Uhr 'Lichtenberg'|Preissturzauktion Raoul U. Braun 'RUB05-0477'|Preissturzauktion Raoul U. Braun RUB05-T40D12|Preissturzauktion ADELSBERGER 'Polar 6'|Preissturzauktion Bellevue Unisexuhr|Preissturzauktion DELMA Tarragona Partneruhren|Preissturzauktion MANAGER Chronograph BOLD|Preissturzauktion ADELSBERGER Fliegerzeit II
215-G A6
215-e
215-E 6294 1628121600 10800 50 14
215-T Night Deals
215-D Night Deals
215-G A6
215-e
215-E 6295 1628132400 3600 50 14
215-T Morning Deals
215-D Morning Deals
215-G A6
215-e
215-c
215 End of EPG data^M
221 VDR-2004-Dev closing connection^M

Display More

Both channel had the same events before (1-2-3.tv HD and non HD) and only the 2 events (Night Deals and Morning Deals) after today. A good channel for this tests.

kls · Aug 4th 2021

I tried to reproduce this, but everything worked as expected:

Code

220 vdr3 SVDRP VideoDiskRecorder 2.5.6; Wed Aug 4 23:12:16 2021; ISO-8859-1
lstt
250 7 9:70:2021-08-04:2255:0020:50:50:UHREN SALE:
lste 70
215-C S19.2E-1-1055-5502 1-2-3.tv HD
215-E 6290 1628100000 3600 50 14
215-T SCHUH SALE
215-S 1-2-3.tv Programmvorschau 04.08.2021, 20-21 Uhr
215-D In dieser Stunde erwarten Sie folgende Highlights:|Preissturzauktion Dr. FeetLeder Sandaletten|Preissturzauktion aloeloe Damensandalen|Preissturzauktion aloeloe Damensandalen|Preissturzauktion aloeloe Damenpantoletten|Preissturzauktion Brütting Outdoorschuhe|Preissturzauktion FITTERS FOOTWEAR Pantoletten|Preissturzauktion TOPWAY FLEX FOAM Schnürer|Preissturzauktion Millie&Co Damensneaker|Preissturzauktion FITTERS FOOTWEAR Sandalen|Preissturzauktion Fashion Damenpantoletten
215-G A6
215-e
215-E 6291 1628103600 3600 50 14
215-T UHREN SALE
215-S 1-2-3.tv Programmvorschau 04.08.2021, 21-22 Uhr
215-D In dieser Stunde erwarten Sie folgende Highlights:|Preissturzauktion Auguste Reymond Elegance A40|Preissturzauktion PortaS Uhr 'Erlau'|Preissturzauktion Uhrenbox für 12x schwarz|Preissturzauktion Pierre Chaubert Funkuhr|Preissturzauktion deLorean Uhr 'Fast Lane'|Preissturzauktion CONSTANTIN WEISZ Happy Monk|Preissturzauktion CONSTANTIN WEISZ ETA 2369|Preissturzauktion INVICTA 'DC Comics Batman'|Preissturzauktion DETOMASO Automatik VIAGGIO|Preissturzauktion ADELSBERGER Meisterzeit
215-G A6
215-e
215-E 6292 1628107200 3600 50 14
215-T UHREN SALE
215-S 1-2-3.tv Programmvorschau 04.08.2021, 22-23 Uhr
215-D In dieser Stunde erwarten Sie folgende Highlights:|Preissturzauktion deLorean Uhr 'Square'|Preissturzauktion CONSTANTIN WEISZ Dresswatch|Preissturzauktion SWISS MILITARY by Chrono Uhr|Preissturzauktion CONSTANTIN WEISZ 'Skywalker'|Preissturzauktion KIENZLE Herren-Taucheruhr|Preissturzauktion Auguste Reymond Cotton Club|Preissturzauktion PortaS Uhr 'Erlau'|Preissturzauktion Raoul U. Braun 'RUB05-T49'|Preissturzauktion ADELSBERGER Meisterzeit
215-G A6
215-e
215-E 6293 1628110800 3600 4E 10
215-T UHREN SALE
215-S 1-2-3.tv Programmvorschau 04.08.2021, 23-0 Uhr
215-D In dieser Stunde erwarten Sie folgende Highlights:|Preissturzauktion Auguste Reymond Cotton Club|Preissturzauktion PortaS Uhr 'Elstra'|Preissturzauktion PortaS Uhr 'Lichtenberg'|Preissturzauktion Raoul U. Braun 'RUB05-0477'|Preissturzauktion Raoul U. Braun RUB05-T40D12|Preissturzauktion ADELSBERGER 'Polar 6'|Preissturzauktion Bellevue Unisexuhr|Preissturzauktion DELMA Tarragona Partneruhren|Preissturzauktion MANAGER Chronograph BOLD|Preissturzauktion ADELSBERGER Fliegerzeit II
215-G A6
215-e
215-E 6294 1628121600 10800 4E 10
215-T Night Deals
215-D Night Deals
215-G A6
215-e
215-E 6295 1628132400 3600 50 14
215-T Morning Deals
215-D Morning Deals
215-G A6
215-e
215-c
215 End of EPG data
clre 70
250 EPG data of channel "70" cleared
lste 70
215-C S19.2E-1-1055-5502 1-2-3.tv HD
215-c
215 End of EPG data
lstt
250 7 9:70:2021-08-04:2255:0020:50:50:UHREN SALE:

Display More

kfb77 · Aug 5th 2021

-Dis sorry for hijacking your thread.

kls

I have done some debugging with this changes in your code:

Diff

diff --git a/epg.c b/epg.c
index d917e669..3302ea53 100644
--- a/epg.c
+++ b/epg.c
@@ -1135,6 +1135,7 @@ void cSchedule::Cleanup(time_t Time)
{
cEvent *Event;
while ((Event = events.First()) != NULL) {
+ if (Event->HasTimer()) dsyslog("CLRE event has timer: %s", Event->Title());
if (!Event->HasTimer() && Event->EndTime() + EPG_LINGER_TIME < Time)
DelEvent(Event);
else
diff --git a/svdrp.c b/svdrp.c
index b993577f..a0040697 100644
--- a/svdrp.c
+++ b/svdrp.c
@@ -1323,6 +1323,7 @@ void cSVDRPServer::CmdCLRE(const char *Option)
LOCK_SCHEDULES_WRITE;
cSchedule *Schedule = NULL;
ChannelID.ClrRid();
+ dsyslog("CLRE: ChannelID: %s", *ChannelID.ToString());
for (cSchedule *p = Schedules->First(); p; p = Schedules->Next(p)) {
if (p->ChannelID() == ChannelID) {
Schedule = p;
@@ -1331,8 +1332,10 @@ void cSVDRPServer::CmdCLRE(const char *Option)
}
if (Schedule) {
for (cTimer *Timer = Timers->First(); Timer; Timer = Timers->Next(Timer)) {
- if (ChannelID == Timer->Channel()->GetChannelID().ClrRid())
+ if (ChannelID == Timer->Channel()->GetChannelID().ClrRid()) {
+ dsyslog("CLRE: timer file name: %s", Timer->File());
Timer->SetEvent(NULL);
+ }
}
Schedule->Cleanup(INT_MAX);
cEitFilter::SetDisableUntil(time(NULL) + EITDISABLETIME);
diff --git a/timers.c b/timers.c
index 57aa349b..b206b4a6 100644
--- a/timers.c
+++ b/timers.c
@@ -905,6 +905,7 @@ bool cTimer::SetEventFromSchedule(const cSchedules *Schedules)
bool cTimer::SetEvent(const cEvent *Event)
{
+ dsyslog("CLRE SetEvent called");
if (event != Event) {
if (event)
event->DecNumTimers();

Display More

After second clre 107 (at this time we have only events after the timer) I got:

Code

Aug 5 10:37:44 VDR-2004-Dev vdr: [89289] CLRE: ChannelID: S19.2E-1-1051-28724
Aug 5 10:37:44 VDR-2004-Dev vdr: [89289] CLRE: timer file name: The Beatles: Eight Days a Week
Aug 5 10:37:44 VDR-2004-Dev vdr: [89289] CLRE SetEvent called
Aug 5 10:37:44 VDR-2004-Dev vdr: [89289] timer 8 (107 2153-2355 'The Beatles: Eight Days a Week') set to no event
Aug 5 10:37:44 VDR-2004-Dev vdr: [89289] CLRE event has timer: The Beatles: Eight Days a Week
Aug 5 10:37:44 VDR-2004-Dev vdr: [89289] SVDRP VDR-2004-Dev < 127.0.0.1:36460 connection closed
Aug 5 10:37:44 VDR-2004-Dev vdr: [89289] SVDRP VDR-2004-Dev < 127.0.0.1:36460 server destroyed
Aug 5 10:37:45 VDR-2004-Dev vdr: [89267] CLRE SetEvent called
Aug 5 10:37:45 VDR-2004-Dev vdr: [89267] timer 8 (107 2153-2355 'The Beatles: Eight Days a Week') set to event Fr. 06.08.2021 21:55-23:45 (VPS: 06.08. 21:55) 'The Beatles: Eight Days a Week'

There is something not working on our systems, but on yours it works. As you see, after calling SetEvent(NULL) this events still reports a timer, it can not be deleted and exits the loop with break. If I quick and dirty remove the check "!Event->HasTimer()" in epg.c all events of this channel are deleted.

I assume the problem in "bool cTimer::SetEvent(const cEvent *Event)" but I do not understand, what you are doing here in case of called with NULL.

-Dis · Aug 5th 2021

kfb77 No problem. Good to see some else can reproduce this problem and has time to debug it

I tried to uninstall epgsearch and vnsiserver but that didn't fix anything.

kfb77 · Aug 5th 2021

I have also check our common plugins and tested without markad, no changes too.

Can you test without live, this is the last common plugin. I will have no frontend if I remove live because it is a headless server.

kls · Aug 5th 2021

Can you please add these lines and post the results?

Diff

--- epg.c 2021/04/29 09:19:58 5.6
+++ epg.c 2021/08/05 09:30:51
@@ -259,6 +259,7 @@
numTimers++;
if (schedule)
schedule->IncNumTimers();
+ dsyslog("IncNumTimers %d %s", numTimers, Title());
numTimersMutex.Unlock();
}
@@ -268,6 +269,7 @@
numTimers--;
if (schedule)
schedule->DecNumTimers();
+ dsyslog("DecNumTimers %d %s", numTimers, Title());
numTimersMutex.Unlock();
}

Display More

-Dis · Aug 5th 2021

kfb77 wrote:

I have also check our common plugins and tested without markad, no changes too.

Can you test without live, this is the last common plugin. I will have no frontend if I remove live because it is a headless server.

Tried without live plugin with previously set timers and after uninstalling live CLRE command did clear all epg events.

kfb77 · Aug 5th 2021

Now we know who is guilty.

kfb77 · Aug 5th 2021

kls wrote:

Can you please add these lines and post the results?

with live plugin:

Code

Aug 5 13:07:31 VDR-2004-Dev vdr: message repeated 4 times: [ [109770] CLRE DecNumTimers 3 Die Rosenheim-Cops]
Aug 5 13:07:57 VDR-2004-Dev vdr: [109793] SVDRP VDR-2004-Dev < 127.0.0.1:49208 client connection accepted
Aug 5 13:07:57 VDR-2004-Dev vdr: [109793] SVDRP VDR-2004-Dev > 127.0.0.1:49208 server created
Aug 5 13:07:57 VDR-2004-Dev vdr: [109793] CLRE: ChannelID: S19.2E-1-1051-28724
Aug 5 13:07:57 VDR-2004-Dev vdr: [109793] CLRE: timer file name: The Beatles: Eight Days a Week
Aug 5 13:07:57 VDR-2004-Dev vdr: [109793] CLRE SetEvent called
Aug 5 13:07:57 VDR-2004-Dev vdr: [109793] CLRE DecNumTimers 1 The Beatles: Eight Days a Week
Aug 5 13:07:57 VDR-2004-Dev vdr: [109793] timer 8 (107 2153-2355 'The Beatles: Eight Days a Week') set to no event
Aug 5 13:07:57 VDR-2004-Dev vdr: [109793] CLRE event has timer: The Beatles: Eight Days a Week
Aug 5 13:07:57 VDR-2004-Dev vdr: [109793] SVDRP VDR-2004-Dev < 127.0.0.1:49208 connection closed
Aug 5 13:07:57 VDR-2004-Dev vdr: [109793] SVDRP VDR-2004-Dev < 127.0.0.1:49208 server destroyed
Aug 5 13:07:58 VDR-2004-Dev vdr: [109742] CLRE DecNumTimers 2 Die Rosenheim-Cops
Aug 5 13:07:58 VDR-2004-Dev vdr: [109742] CLRE DecNumTimers 2 Die Rosenheim-Cops
Aug 5 13:07:58 VDR-2004-Dev vdr: [109742] CLRE DecNumTimers 0 The Beatles: Eight Days a Week
Aug 5 13:07:58 VDR-2004-Dev vdr: [109742] CLRE DecNumTimers 2 Die Rosenheim-Cops
Aug 5 13:07:58 VDR-2004-Dev vdr: message repeated 4 times: [ [109742] CLRE DecNumTimers 2 Die Rosenheim-Cops]
Aug 5 13:07:58 VDR-2004-Dev vdr: [109742] CLRE IncNumTimers 3 Die Rosenheim-Cops
Aug 5 13:07:58 VDR-2004-Dev vdr: message repeated 6 times: [ [109742] CLRE IncNumTimers 3 Die Rosenheim-Cops]
Aug 5 13:07:58 VDR-2004-Dev vdr: [109742] CLRE SetEvent called
Aug 5 13:07:58 VDR-2004-Dev vdr: [109742] timer 8 (107 2153-2355 'The Beatles: Eight Days a Week') set to event Fr. 06.08.2021 21:55-23:45 (VPS: 06.08. 21:55) 'The Beatles: Eight Days a Week'
Aug 5 13:07:58 VDR-2004-Dev vdr: [109742] CLRE IncNumTimers 1 The Beatles: Eight Days a Week
Aug 5 13:07:59 VDR-2004-Dev vdr: [109742] CLRE DecNumTimers 2 Die Rosenheim-Cops
Aug 5 13:07:59 VDR-2004-Dev vdr: message repeated 6 times: [ [109742] CLRE DecNumTimers 2 Die Rosenheim-Cops]
Aug 5 13:07:59 VDR-2004-Dev vdr: [109742] CLRE IncNumTimers 3 Die Rosenheim-Cops
Aug 5 13:07:59 VDR-2004-Dev vdr: message repeated 6 times: [ [109742] CLRE IncNumTimers 3 Die Rosenheim-Cops]
Aug 5 13:07:59 VDR-2004-Dev vdr: [109742] CLRE IncNumTimers 2 The Beatles: Eight Days a Week

Display More

without live (now I am without frontend

Code

Aug 5 13:17:29 VDR-2004-Dev vdr: [109939] SVDRP VDR-2004-Dev < 127.0.0.1:51824 client connection accepted
Aug 5 13:17:29 VDR-2004-Dev vdr: [109939] SVDRP VDR-2004-Dev > 127.0.0.1:51824 server created
Aug 5 13:17:29 VDR-2004-Dev vdr: [109939] CLRE: ChannelID: S19.2E-1-1051-28724
Aug 5 13:17:29 VDR-2004-Dev vdr: [109939] CLRE: timer file name: The Beatles: Eight Days a Week
Aug 5 13:17:29 VDR-2004-Dev vdr: [109939] CLRE SetEvent called
Aug 5 13:17:29 VDR-2004-Dev vdr: [109939] CLRE DecNumTimers 0 The Beatles: Eight Days a Week
Aug 5 13:17:29 VDR-2004-Dev vdr: [109939] timer 8 (107 2153-2355 'The Beatles: Eight Days a Week') set to no event
Aug 5 13:17:29 VDR-2004-Dev vdr: [109939] SVDRP VDR-2004-Dev < 127.0.0.1:51824 connection closed
Aug 5 13:17:29 VDR-2004-Dev vdr: [109939] SVDRP VDR-2004-Dev < 127.0.0.1:51824 server destroyed

numTimers values of all timers are 1 bigger with live than without ???

Share