Have started having problems with programmes that are over 7 days old again today.

It seems that some programmes that have gone past 7 days are refusing to download the default version even though it shows as available in and shows modes as available for the default version.

An example is http://www.bbc.co.uk/iplayer/episode/b00...-episode-1 which is fully playable on iPlayer but cannot seem to be downloaded.

Output with




Shows that default does exist and has modes available.

Output from download attempt with




Also http://www.bbc.co.uk/iplayer/episode/b03...d-galloway is refusing to download saying that only the versions are available. The shows that there is a default version there but there are no modes for it.

The Slammer downloads fine here. There are no default version streams for Escape to the Country. It happens sometimes, particular with old programmes - blame the Beeb. If you see a reference to a shortened version it suggests you are using a recent version of get_iplayer from GitHub, but the output from your Slammer download suggests that you are using 2.90 as released. Save us both some time and go get the very latest version of get_iplayer from GitHub. See Git HEAD section in wiki installation page. If those programmes still don't download, run with and attach the log as a .txt file. I can't tell anything from the above.

The version I was using was the git HEAD version from 28/11/2014. Have now updated to the most recent version and still have the same problem.

When I update from the git HEAD I save the file and then replace it manually in the get_iplayer folder rather than running the copy and rename script in the wiki. Could that be why the update doesn't show in the output?

The output for the Slammer is the same as above.

is the line in there that I'm unsure of though. Seems to be after it's checked the JSON playlist.

Interestingly, if I change the script for a slightly older git HEAD version which I downloaded on the 12th, the Slammer downloads fine.

That's not the output you should get from Git HEAD, so something is going wrong when you update. HEAD doesn't pull JSON playlists unless you don't have the XML::Simple Perl module installed. If that is the case, then you somehow have an incomplete or damaged installation. A version from 12 Nov won't have those changes, so perhaps you do have an incomplete installation. I can't duplicate the problem anyway - any version works for me. Grab Git HEAD from today (2 Dec) or later, then run for that programme. If the output has a "desclong" field, then you're running the correct script. And in your get_iplayer console window, run . If you don't see an error message, then the module is installed.

"desclong" is showing in the --info and shows no errors.

Have also installed 2.90 on another machine and updated it with the most recent script with the same error.

Try Git HEAD a few more times with and these additional options:




Post the output as .txt files. Don't worry about the file names. I can tell from the contents which is which.

Attempts 3 & 4 worked and the programme downloaded. All outputs attached.

attempts12.txt
attempt3.txt
attempt4.txt

Thanks. Those results at least confirm what is happening, but not why it is happening. It's still impossible for me to duplicate. Instead of this:






- which means you are getting the old, semi-busted metadata extraction - you should see something like:






Check a few BBC One or Two programmes to see if you get the same results, or whether the new metadata extraction is used. The implication of your test is that a check of the PID value is failing for some reason in your version of get_iplayer. Just to check if sanity still reigns, run the following in your get_iplayer console window:






It should print "1".

I'm going to remove the options and re-enable the screen scraping by default when using the old metadata extraction method. It turns out that many CBBC programmes now appear to be missing any playlist data, so screen scraping is the only way to go in that case. Scraping is not needed with the new method.

Also: you don't need with .

Sanity check comes back with "1"

I still can't see any reference to the .xml playlist though when it tries to download a programme. Is seems to go straight for the JSON.

It does use mediaselector5 though to get the actual download data.

Have tried with last weeks Watchdog and Not Going Out. Output attached.

watchdog-not-going-out-output.txt

The mediaselector reference was irrelevant - copy/paste error. "iPlayer metadata URL" refers to the XML playlist. The URL doesn't have a .xml extension.

Still unable to reproduce this behaviour, so whatever it is appears to be peculiar to your system. If you know some Perl, I can point you towards the place in the code to investigate. Otherwise I'm afraid I must throw in the towel. Can't fix a problem I can't see.

I've updated GitHub and removed the options. TV episode pages will be scraped by default (turn off with ), so hopefully no more problems with CBBC programmes.

Thanks for your help.

I don't know any Perl.

The odd thing is that it is affecting 2 machines in the same way. One is Win 7 and the other is XP.

Maybe when version 2.91 is released, whatever it is that's causing the problem might be solved by a proper install?

I will update to the latest HEAD version and see how it goes.

Thanks

What do you mean by "proper install"? If you didn't use the get_iplayer installer and then update over it with Git HEAD, or if you're not using the version of Perl delivered with the installer, then all bets are off. And you shouldn't count on the next release to fix anything. The particular bit of code in question isn't going to change before then. It already works fine here on XP/7/8. Even so, the old metadata code will keep working for you - it will be just be a bit slower and less accurate.

Quote:Quote: What do you mean by “proper install”?

I mean get_iplayer_latest.exe. It could be something I've done when updating to the HEAD version that's causing me these problems. The next installer will do it properly rather than me replacing the script.

I upgraded to 2.90 and then updated to the HEAD version over the top of that. All I did was save the git HEAD file and replace the original one with that.

Is there a command I can use to force the new metadata method to be used?

No. It's the default. It only requires the programme have recognisable a PID. So in your case, either something is wrong with all your PIDs, or as you suggest, you've mad an error in installation. You can try replacing this line:





with this:





to avoid the PID check, but I've no idea if that will work.

I think I might of sorted it.

It seems that the new metadata method relies on AtomicParsley being installed. Without AtomicParsley, it drops back to the old method.

Now that AtomicParsley is installed I am getting





Had a problem yesterday though with an episode of Escape to the Country from last weekend. Unfortunately it is now no longer available but when I tried to download, there was no default version but there was a "lengthened" version available but this only had flashhigh and flashstd available. I had to revert to an older script to get the flashvhigh stream to download. (I think it was the original script that came with 2.90 that I reverted back to)

Quote:Quote:
It seems that the new metadata method relies on AtomicParsley being installed. Without AtomicParsley, it drops back to the old method.

Well-spotted. Fixed in Git.

Quote:Had a problem yesterday though with an episode of Escape to the Country from last weekend. Unfortunately it is now no longer available but when I tried to download, there was no default version but there was a “lengthened” version available but this only had flashhigh and flashstd available. I had to revert to an older script to get the flashvhigh stream to download. (I think it was the original script that came with 2.90 that I reverted back to)

If you don't give me an exact episode, I can't check it.

Quote:Quote:
If you don’t give me an exact episode, I can’t check it.

I thought that would probably be the case but I thought I'd report it anyway in case it came up again.

It was http://www.bbc.co.uk/programmes/b00psvz4 but was only available for 7 days.

From the metadata that remains, it looks like one version of the programme was marked as both lengthened and audio described, so the streams for the audio described version were used instead (flashhigh best available). get_iplayer should now be able to pick out the audio described version separately, at least in some cases.

I think you are right. Default had no modes, lengthened only showed flashhigh and flashstd and I think the audiodescribed version showed the same.

I did try but it told me that flashvhigh was not available and I should try flashhigh or flashstd.

The data reflected this.

Possibly if I had run on it it may well have found the flashvhigh version because that would also have scraped for versions wouldn't it?

Reverting to the original 2.90 .pl file probably did this for me so the flashvhigh download worked. It downloaded it as the default version using the original .pl

would probably work in this case. It doesn't work in every case, so I'm trying to work out the correct juju to avoid any extra scraping. Latest code will discard any versions that don't have streams, and tries to to use any alternate version (e.g., "shortened") as default if listed default version doesn't have any streams, but there may be some as-yet-unseen permutations that will confuse it.