These forums are archived

See this post for further info

get_iplayer forums

Forum archived. Posting disabled.

Repeated indexing due to cache not being updated

user-910

I've just noticed perhaps a small glitch under the command line in Windows 10 (in a command prompt) using 3.14.0.

If I do a simple check for a program (for example get_iplayer submarine) it goes away, does its normal concurrent indexing (if it's the first run of the day) and comes up with a list of matching shows.

However if I then do a second one (for example get tomb) it then goes off and does the indexing again before showing up the list of matches. And then for every subsequent search again it does the indexing before showing results.

The available programs are all there as they should be, and downloading works fine. It's just that doing the indexing takes a few minutes to complete each time. The first one is of course usually necessary and fine, but the repeated ones (adding nothing to the list normally) are redundant.

Can anyone reproduce this, or is it something in my set-up? It didn't do this in 3.13.0 or earlier versions that I can recall.

user-30

Unable to reproduce this issue.

Have you somehow set a preference to refresh by default?

user-910

Not knowingly, although --show-prefs does include one about refreshfuture:

excludechannel = "BBC Alba"
fields = name,episode
modes = default
refreshfuture = 1
tvmode = better
fileprefix = <name><-senum><-episodeshort>
whitespace = 1

If I do "--prefs-del refreshfuture" it just says updated and moves it to the bottom of the list (or more accurately reorders the whole list)?

tvmode = better
modes = default
fields = name,episode
fileprefix = <name><-senum><-episodeshort>
excludechannel = "BBC Alba"
whitespace = 1
refreshfuture = 1

user-2

Follow our instructions to provide a proper report so we can see what is happening on your system. You'll need to attach the logs from two consecutive searches. Indicate which is which. Before each search, run get_iplayer --show-cache-age and post the output. Again, indicate which is which.
Quote:If I do "--prefs-del refreshfuture" it just says updated and moves it to the bottom of the list
Don't delete before testing. If you should want to delete it later, see "Setting preferences" in documentation for correct syntax

user-910

No problem. It's get_iplayer 3.14.0 running under Windows 10 build 1803. Run just after a download, hence the low first cache age. Everything's on the local HD - it's all working fine except doing the indexing every time.

This is the sequence (logs uploaded to a pastebin to keep things small), repeating the same sequence as in my first post:

get_iplayer --show-cache-age

191 secs

get_iplayer>get_iplayer submarine --verbose > c:/users/darren/desktop/sub.txt 2>&1

https://paste.ubuntu.com/p/5cbbgGgT42/

get_iplayer --show-cache-age

18 secs

get_iplayer>get_iplayer tomb --verbose > c:/users/darren/desktop/tomb.txt 2>&1

https://paste.ubuntu.com/p/TqmCBq3gSb/

If you need anything else please let me know.

user-2

Thanks. However, I must ask you to do it again using --debug instead of --verbose. There is a useful bit of information that only appears with --debug. --debug will produce a large spew, unfortunately. Feel free to upload the complete files. If you want to dig out the relevant nuggets, the line to look for in each case looks like this:

Code:
DEBUG: Cache expiry time for tv is 14400 secs - refresh in 5337 secs

user-910

OK here's the new sequence:

get_iplayer --show-cache-age

INFO: tv cache age: 8681 secs

get_iplayer submarine --debug > c:/users/darren/desktop/sub2.txt 2>&1

DEBUG: Cache expiry time for tv is 14400 secs - refresh in 5704 secs

get_iplayer --show-cache-age

INFO: tv cache age: 29 secs

get_iplayer tomb --debug > c:/users/darren/desktop/tomb2.txt 2>&1

DEBUG: Cache expiry time for tv is 14400 secs - refresh in 14352 secs

The full logs are in the zip file below - you can see which is which from their filenames.

Logs.zip

user-2

Thanks. It appears that new programmes are not being added to your cache, which is triggering the repeated updates. These two lines are the same in all four logs:

Code:
INFO: Got 7523 file cache entries for tv
INFO: Cache refresh limit (tv): 2018-05-26T00:30:08+00:00

It's unlikely, though not impossible, that the number of cache entries would be exactly the same for updates a couple of hours apart. The cache refresh limit is based on the availability date of the newest programme in the cache, so it should advance with every update that adds new programmes.

File corruption could be to blame, but there is no point in wasting time looking for it. Back up your cache files and delete them. Use --cache-rebuild to rebuild them separate from any other activity.

user-910

@user-2 - thanks for that. Spot on the money as usual, the rebuild has fixed it.

This one can be closed off as resolved.

These forums are archived

See this post for further info