The "Added:" prefix for new programmes
#1
I've been using the following script, for some time :-

date  >~/iPlayer/playerfiles/iplayprogs.txt | get_iplayer | grep -iv -f ~/iPlayer/playerfiles/match.txt >>~/iPlayer/playerfiles/iplayprogs.txt

This effectively creates a file of the output of the command, with unwanted data filtered out, with a date/time stamp at the start. match.txt is the file which contains data not wanted. It works well, and still does, but the nicest part is that it appends "Added" to all entries new, since I ran the script the previous time. However, for some reason I cannot understand, the "added" function has failed to be included, over the past couple of weeks. I've checked the filter list, (match.txt), to see if added has somehow got on the list, but there's nothing there which could be interpreted as this. In any case no changes had been made prior to the effect becoming evident. if anyone could explain, I'd be most grateful. The issue, BTW is v3.01 running on Mageia 5

Cheers, Rod Goslin
#2
v3.01 is broken by BBC changes and no longer supported. Update to 3.06 and try again. My guess is that your scheme still won't work, albeit for a different reason. I don't see how you could be appending anything by using only grep, so I assume you are referring to the "Added:" prefix that get_iplayer prepended to its dump of newly-indexed programmes.  get_iplayer no longer produces such a dump. You can filter the "Added:" records (if there are any) from the output from get_iplayer --verbose, but it's cleaner to use --since=N to list programmes added in the last N hours. If N is less than your cache update interval, the list will contain only programmes added in the update just performed. You also appear to be relying on the fact that when invoked without arguments, get_iplayer dumps a list of its entire cache contents. That behaviour is deprecated and will be removed in the next release. You will need to explicitly request a full dump by using ".*" as a search string.
#3
Thanks, Dinky, for the prompt reply. I only noticed, today, that v 3.06 was out. Unfortunately, Mageia repositories still only hold v2.94, so no help there! I'll look round for an installable download. The grep operation is a negative grep. It excludes all titles that have a string listed in the text file, so cutting down the list, Cbeebies and such. I used to run the script in chrontab, at a specific time, so the --since=N option would work ok, but lately I've changed to running the command whenever I feel I have the time to download programmes of interest. If the "Added" prepend is no longer produced, I'll have to live with it. A pity, though. It saved me scrolling down a long list of programmes that I'd seen before, since it indicated a short list of unique entries. The alternative was to trawl through the BBC iPlayer site and select the files at source.

Cheers, Rod Goslin
#4
Ummm. Any idea where I might get a copy of 3.06? the latest one I've been able to find has been the rpm package for 3.05. Most sites stop at v3.01, or even earlier!

Cheers, Rod
#5
(29-10-2017, 01:19 AM)rodgos Wrote: If the "Added" prepend is no longer produced, I'll have to live with it.
As I said, they are still there if you really want them. For your case it's as simple as:  
Code:
get_iplayer --verbose | grep 'Added:'
(29-10-2017, 01:42 AM)rodgos Wrote: Any idea where I might get a copy of 3.06?
See Linux manual installation doc in wiki
#6
Thanks. v3.06 duly loaded. Nothing, however, seems to bring back the 'Added' programmes. I've written a command which saves the latest listing as a text file, and a program which renames the previous file to 'old', renames the latest as 'new'. Since the Index is always changing, a cut command removes that. A diff command gives the difference. Stripping out the references to the programmes removed from the listing, leaves me with only those which were not in the earlier listing. It's a bit untidy, with diffs editing notes but does give me the information. Anything that interests I can download from the pid
#7
Ummm, How does one indicate that the thread is solved, on this forum?
#8
(01-11-2017, 02:54 AM)rodgos Wrote: Nothing, however, seems to bring back the 'Added' programmes.
They are there, but of course will only appear when the cache is actually updated (default 4 hrs). You can add --refresh to --verbose to force an update.
(01-11-2017, 02:57 AM)rodgos Wrote: Ummm, How does one indicate that the thread is solved, on this forum?
Full edit your original post and select the [SOLVED] prefix in the subject.
#9
I assumed that that was the way to add 'solved', as in other forums. But, in this case there is no 'edit' function on the first post. Only 'delete' 'quote' and 'report' (and 'find'). 'edit' does appear on the latest post for the usual period.
The command get_iplayer --verbose | grep 'Added:'' did not work either, even after a 12 hour+ interval. And that was copied and pasted, so verbatim However, I've refined my shell script, somewhat, and it gives me the results I want. So 'Solved' is enough for me.

Cheers, Rod Goslin
#10
(01-11-2017, 04:47 PM)rodgos Wrote: But, in this case there is no 'edit' function on the first post.
My bad. Forgot about the time limit for editing. I haven't typically used thread prefixes very much, only a few times in an attempt to curb duplicate posting.
(01-11-2017, 04:47 PM)rodgos Wrote: The command get_iplayer --verbose | grep 'Added:'' did not work either,
We'll have to disagree on that. Maybe someone else knows how you might be suppressing those messages in verbose output. AFAIK, it isn't possible. But as I said, you also have option of using --since=1 (instead of --verbose) to show everything added in the past hour, which would encompass the last update.
#11
Sorry to be a bother, but strange things are happening, which I'm unable to comprehend. The program I wrote works ok, but the result shows no differences between an old search and a new search, and therefore no output. Opening up the two files, there indeed seems to be no change in the programmes available. Thinking to start from scratch, I renamed the .get_iplayer file to take it out of consideration, on the basis that restarting the program would re-establish the setup directory. This it did, but now, there is nothing in the return of running get_iplayer, as below:-

get_iplayer Output:

[rod@down ~]$ get_iplayer
get_iplayer v3.01, Copyright (C) 2008-2010 Phil Lewis
 This program comes with ABSOLUTELY NO WARRANTY; for details use --warranty.
 This is free software, and you are welcome to redistribute it under certain
 conditions; use --conditions for details.

 NOTE: A UK TV licence is required to legally access BBC iPlayer TV content


INFO: Getting tv Index Feeds (this may take a few minutes)

INFO: Using concurrent indexing
..................................

INFO: 0 Matching Programmes
[rod@down ~]$
#12
You said you installed v3.06 - use it, or you'll never get any TV programmes indexed.
#13
That's odd. I thought that v3.06 had been installed. Certainly the v3.06 files are the only ones held on the machine. I'd already deleted all the 3.01 file structure. But using the whole path name it worked. I'm even more puzzled than before. But, thanks it did the trick. I'd not noticed the version info in the header. In fact, the short program I wrote to run the thing is as below, and the full pathname of the executive file is there.
Code:
#!/bin/bash
# iplayprog
cd ~/iplayprogs
\mv newcutprogs.txt oldcutprogs.txt
date > newprogs.txt
~/get_iplayer/get_iplayer-3.06/get_iplayer | grep -iv -f ~/iPlayer/playerfiles/match.txt>> newprogs.txt
cut -f2 newprogs.txt >newcutprogs.txt
diff -f2 oldcutprogs.txt newcutprogs.txt> latestprogs.txt
cat newprogs.txt >> latestprogs.txt
konsole --noclose --workdir ~/iplayprogs -e more latestprogs.txt

From that, the absence of 'Added:' is probaby due to there not being an update in the old and the new files, so they were both identical. Presumably get_iplayer compares both of the caches, old and new and does something similar to my program!!!

Cheers, Rod Goslin
#14
The absence of the 'Added:' lines in get_iplayer output will be permanent because you decided not to use --verbose as I suggested. Your diffs will reflect programmes that have been expired from the cache, which seems like unnecessary clutter. You're really making extra work for yourself here, but it's your show.

For anyone else who wants to filter new cache entries, just use get_iplayer --refresh --verbose [other options] | grep "Added:". Replace grep with findstr on Windows.
#15
Well, I did say that it was a bit untidy. If, now, the old way works, I shall be only too pleased to go back to it. Or something like it. I was giving some thought to eliminating the expired files, using the left and right arrows. It's been years since I did any program writing, and I'm not anxious to re-learn it all!

Cheers Rod Goslin