I've installed get_iplayer on Linux (Sailfish OS v.2.2.0.29). When I perform a refresh, the following errors are generated. The errors happen several times, so I've trimmed the output to remove repeats.


The errors happen on only two out of the three phones I'm using to test. All the phones are running the same OS, but may not all have the same packages installed, so it's probably some underlying dependency issue. The errors don't seem to be preventing it refreshing the cache (I'm not sure, but it may not be picking up all the programmes), and downloads are working fine.

Does anyone have any idea what the problem might be, or which dependencies I might be missing?

I'm using a local::lib, so all the perl dependencies were installed like this:

The following is the versioning info for the dependencies:

• get_iplayer - 3.14
  
• Perl - 5.16.1
  
• JSON::PP - 2.97001
  
• LWP - 6.34
  
• LWP::Protocol::https - 6.07
  
• Mojolicious - 7.84
  
• XML::LibXML - 2.0132
  
• IO::Socket::SSL - 2.056
  

The error suggests Mojolicious is accessing an HTTP stream after it has been closed. No idea why that happens on your phone. Some things to try:

• Refresh with --verbose. That will surface any underlying HTTP or connection errors. If your cache was empty to start, the total above would be close to what I'd expect, but you need to run with --verbose to see if indexing of any schedule pages produced no entries due to errors.
  
• The HTTP stream implementation in Mojolicious has changed recently. You could roll back to 7.82 or earlier to see if that makes a difference.
  
• You can refresh with --no-index-concurrent to take Mojolicious out of the equation altogether, but that is much slower.
  
• Test with a clean local::lib, created without --self-contained. That may have had no effect, but you could have acquired an incompatible module along the way. At any rate, you should never need --self-contained for get_iplayer with a proper Perl installation. Start with the minimal set of modules required by get_iplayer dependencies and work out from there. Your system's version of Perl is woefully obsolete, but it should be OK for get_iplayer, provided nobody has chopped and changed it.
  

Thanks for all of the useful suggestions. I've not had a chance to roll back Mojolicious yet, but I'll give it a go to see whether it affects anything. In the meantime, here's the output of the refresh again, but this time with --verbose added. The failures are still appearing, but I'm afraid my log-fu is weak and it's still not clear to me whether it shows feeds being dropped.

https://pastebin.com/7HtpqKkZ

I had to remove some of the 'Added' lines to satisfy pastebin, but hopefully I didn't remove anything important.

The logging in that area will be slightly more intelligible in the next release, but I can still see that you are in fact losing cache entries from the pages that generate errors. Some of those entries are recovered through indexing of later repeats or regional variants, in the case of BBC Two.

The log doesn't shed any further light on the underlying problem. get_iplayer currently ignores any connection errors on the first two attempts to download a schedule page (though that will be changing), so nothing there. I note that the Mojo error messages only appear for pages with a failed first attempt. But without a reproducible test case, I can't even guess at cause and effect. If the errors always occur on the same pages, that could be illustrative, but that seems unlikely to be the case.

You can get additional debugging traces from Mojolicious by setting environment vars (see Mojo docs re: MOJO_CLIENT_DEBUG and MOJO_IOLOOP_DEBUG). I don't know if that will get you any closer, but it might expose underlying connection errors (beware huge spew of HTTP traffic). You might also try refreshing with --index-maxconn values less than 5 - maybe something is screwy with polling.

Thanks for all of your help with this @user-2. I tried the different logging options, and also tried all of the other Mojolocious parameters that can be controlled using environment variables, but none of them fixed the issue or provided more insight. After a bundle of testing, it seems that on the devices that exhibit this problem, they can handle two connections without any problem, but anything higher than this will start to trigger failure messages.

If I ever get to the bottom of it I'll post the results here for completeness, Sadly I've not been able to find the underlying reason yet, but it's pretty clear the issue is with Mojolicous on particular devices/perl installations, and not with get_iplayer. And I also want to thanks @revad for his support testing and debug on his devices.