Page 1 of 1

Download age issue

PostPosted: Sun Aug 05, 2018 6:17 pm
by brindell
I opened newsbin the other day like every other time I use it, and now for some magical reason. I find that I can only see files using the internet search from 2017-2018. My news server is usenet which has a retention of at least 3000 plus days. I don't go all super wizard ninja with newsbin just download basic stuff not any particular groups using the internet search. I reinstalled newsbin completely and deleted the previous settings just in case I somehow changed something by accident. Did something change my download age? or my storage age? both are at 3000.. I changed everything I could,nothing changes.. I could search for the word tree or motorbike and still get 2017-2018? any help out there? :)

Re: Download age issue

PostPosted: Sun Aug 05, 2018 9:15 pm
by Quade
1 - What kinds of messages do you see in the logs when you download headers?

2 - Down where it says "Cache" do you see a number in parents? "(XXX)"

3 - If you wipe and re-install, you'll need to re-download headers. For some groups it's a huge amount of data. Have you re-downloaded headers?

Re: Download age issue

PostPosted: Sun Aug 05, 2018 11:18 pm
by brindell
What kinds of messages do you see in the logs when you download headers?

I never used groups, since the internet search came into play. I just dump what I am looking for in the search bar and go from there.


Down where it says "Cache" do you see a number in parents? "(XXX)"

Yep it says 400/400 (0)


If you wipe and re-install, you'll need to re-download headers. For some groups it's a huge amount of data. Have you re-downloaded headers?

I haven't even used anything different or changed any setting in like two years except the newsbin updates. So like no I haven't the only groups i have are the default ones.


I just searched using the word jazz, any and all returned searches are only 2018...? my download age is 5000 and storage is 5000, display 30. I am pretty much at a loss here ? :D

Re: Download age issue

PostPosted: Mon Aug 06, 2018 10:14 am
by Quade
I opened newsbin the other day like every other time I use it, and now for some magical reason. I find that I can only see files using the internet search from 2017-2018.


Ok, so you're using search. I assumed headers. How many results are you getting from your search? Each search is capped at 2000 results. If you're using a broad search term, you might simply run out of resutls.

You can return more results using the left facing curly arrow on the search tab.

Re: Download age issue

PostPosted: Mon Aug 06, 2018 11:31 am
by brindell
ok here it is as clear as I can make it.

I have used newsbin the same way every time for the past at least 6 years. The search function only right click download to new folder then done..

Now like scary magic it does not work.

PDF - date range (7/26/2018-8/6/2018) date count and size 31.61 GB (2000)

star trek - date range (10/23/2017-8/5/2018) date count and size 1.87 TB (1946)

Book - date range - (5/8/2018-8/6/2018) date and count size 214.90 GB (1991)

It seem the average return is 2000, using basic search words like book, you should get like tribillions of useless posts at the least. Did the server Usenet purge everything? how can I fix this? :D

Re: Download age issue

PostPosted: Mon Aug 06, 2018 11:45 am
by brindell
Ok

I see what I need to do, but somehow I never had to load older posts before. Jamming the load older post button of course loads older posts increasing returned searches but like book still limits returned post to the same date range. Never had issues before? never had to keep using load older post to get earlier returns?
How can I return all post without downloading group headers using just the search function?

Re: Download age issue

PostPosted: Mon Aug 06, 2018 11:57 am
by dexter
The 2000 result per search cap has been in place since the inception of the Search feature. It could be in the past you had some kind of filter in place, like a size filter, that narrowed your search and returned older results? Or in the past you used less broad terms that returned more focused results? We have a cap in place to ensure consistent performance for all users. If we allowed you to download everything, a few users could monopolize resources and slow everyone else down. We figured giving you the capability but making it a more manual process was a good tradeoff.

Re: Download age issue

PostPosted: Fri Nov 30, 2018 8:35 am
by peacs
dexter wrote:The 2000 result per search cap has been in place since the inception of the Search feature. It could be in the past you had some kind of filter in place, like a size filter, that narrowed your search and returned older results? Or in the past you used less broad terms that returned more focused results? We have a cap in place to ensure consistent performance for all users. If we allowed you to download everything, a few users could monopolize resources and slow everyone else down. We figured giving you the capability but making it a more manual process was a good tradeoff.

I just started using the search facility and found this 2000 limit. It gets me about half the results I want.

Is there any way to tell the filter/search to get results OLDER than a set days.

Thus I could do the search and get my most recent 2000 files and then do another search and get the next 2000 files that are older than say 712 days? And repeat for next 2000 if there were any?

Second Question. When i do an internet search on xyz[.]18 the search returns files with xyz.17.02.18 and that is not regex behaviour. NOTE: I am using the suggested search in the user guide Quote "Or typing [.]jpg would filter for files with the .jpg file extension." (V660-Searching and Filtering)
So the question is "how can I do searches for xyz.18 without getting files with xyz and 18 in the subject."

Re: Download age issue

PostPosted: Fri Nov 30, 2018 10:15 am
by Quade
You can click the left pointing curved arrow to get the next 2000 and so on. You can't set an age to get results "older than". You can set an age to get results "newer than" though.

The search engine doesn't use regex. There's basically two different dialects. Local search of downloaded headers uses regular expressions but internet search uses something closer to a google search. You can create a filter profile with all the regexes you want to match then apply it to the search results after you search.

Re: Download age issue

PostPosted: Fri Nov 30, 2018 10:37 am
by peacs
Quade wrote:You can click the left pointing curved arrow to get the next 2000 and so on. You can't set an age to get results "older than". You can set an age to get results "newer than" though.


Oh good. Thats exactly what I need. Thank you very much

Shame that the search doesn't use regex. But now I know, thank you.

Re: Download age issue

PostPosted: Fri Nov 30, 2018 12:40 pm
by Quade
I tend to use search with a relatively broad search term then use regex in the find box or a filter profile to get what I really want.

For example, I have some automated searches that run every hour with a * search (wildcard) but then the results are fed through a filter profile leaving just what I want to accumulate in the automated search. I then load it like a group and see what it has caught.