Intelligent, scripted, filter of headers

Technical support and discussion of Newsbin Version 6 series.

Intelligent, scripted, filter of headers

Postby spike6479 » Sat Sep 24, 2011 4:46 pm

Hi there,

Is there any way to write a script to filter what is displayed for a group. If there is could you please point me to the docs?

It would be really nice to optionally pass the list to a python script before display. For example I'd like to find all the sample video files.

Thanks. By the way I have found V6.01B5 Build 1211 to be very stable.
spike6479
Occasional Contributor
Occasional Contributor
 
Posts: 40
Joined: Fri Aug 06, 2010 8:10 am

Registered Newsbin User since: 08/01/10

Re: Intelligent, scripted, filter of headers

Postby Quade » Sat Sep 24, 2011 8:14 pm

I'm probably going to be adding some more filtering to headers. The intent of the remote control interface was to let someone programatically browse the groups and make filtering decisions. The API for remote control will never be feature complete. I can always add more stuff to it.

What do you think about using Python to remotely control Newsbin and run your own filters on what's in the groups? You can make download decisions completely in your own program and never use the GUI if that's what you want.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44882
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Intelligent, scripted, filter of headers

Postby spike6479 » Sat Sep 24, 2011 10:43 pm

Sounds great!
spike6479
Occasional Contributor
Occasional Contributor
 
Posts: 40
Joined: Fri Aug 06, 2010 8:10 am

Registered Newsbin User since: 08/01/10

Re: Intelligent, scripted, filter of headers

Postby Quade » Sat Sep 24, 2011 11:15 pm

You can pretty much do it how using the existing remote control interface in 6.1.

viewtopic.php?f=45&t=28590

Here's some of the spec. Dex has moved this project to source-forge. I don't know the actual URL. Jonny has built an IOS remote controller to remote control Newsbin with an iPhone or ipad. There's some C# sample code for various functions to look at too.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44882
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Intelligent, scripted, filter of headers

Postby apsen » Mon Sep 26, 2011 1:38 pm

spike6479 wrote:Is there any way to write a script to filter what is displayed for a group. If there is could you please point me to the docs?


If you just need to filter headers you could run something like

Code: Select all
path_to_newsbin/sqlite3.exe path_to_DB\SPOOL_V6\group_name\Storage.db3 'select distinct NN_Subject from Compact_V4'


and filter output with python or whatever else you like - I use perl.

I actually do it all the time as internal search used to be flaky and there were also problems with its history (plus others).
Maybe they have been fixed but at this point I have my own custom scripts that do the job better so I just do not care to test.
apsen
Active Participant
Active Participant
 
Posts: 73
Joined: Wed Feb 18, 2009 4:45 pm

Registered Newsbin User since: 03/13/07

Re: Intelligent, scripted, filter of headers

Postby Quade » Mon Sep 26, 2011 2:02 pm

Yeah, that's nice. Nice thing about using remote control is that you can both filter and add to the download list in one go.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44882
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Intelligent, scripted, filter of headers

Postby spike6479 » Sat Oct 01, 2011 12:53 pm

The telnet interface is working pretty well for me. I've been able to obtain lists of entries and select and request downloads so far. Is there a method to get newsbinpro to update the headers for a group? Is there a way to mark all entries as old in one command.

Terminology becomes a bit convoluted when you can both download stuff from newsbinpro (vis telnet) and download to newsbinpro (via nntp).

Thanks,
Spike
spike6479
Occasional Contributor
Occasional Contributor
 
Posts: 40
Joined: Fri Aug 06, 2010 8:10 am

Registered Newsbin User since: 08/01/10

Re: Intelligent, scripted, filter of headers

Postby spike6479 » Sat Oct 01, 2011 1:30 pm

Another question. What is the format of the download path in the Loaded Download command (I tried both single and double backslashes). I send the following, but the download still goes to the newsbinpro option download folder.

Telnet(127.0.0.1,118): send 'Loaded Download 556 z:\\newsbin\\samples\r\n'
Telnet(127.0.0.1,118): recv '200 Loaded Download - Succeeded\r\n'


Thanks,
Spike
spike6479
Occasional Contributor
Occasional Contributor
 
Posts: 40
Joined: Fri Aug 06, 2010 8:10 am

Registered Newsbin User since: 08/01/10

Re: Intelligent, scripted, filter of headers

Postby Quade » Sat Oct 01, 2011 1:42 pm

http://sourceforge.net/projects/nbremote/?_test=beta

http://help.newsbin.com/index.php/Versi ... rface_Spec

Is the current interface spec. It needs to be updated though.

There's currently no "mark all old" though I can probably add it.

Currently you can't download outside of the download path as a security measure. I can consider making that optional in the future. The format would be a normal windows path. It is assumed that all filenames and folders are UTF-8.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44882
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Intelligent, scripted, filter of headers

Postby spike6479 » Mon Oct 03, 2011 6:02 pm

I would appreciate a MarkAllOld command. I didn't actually time it, but it just took me minutes to Mark 280 entries old.

It would also be nice to have a way to force update of headers.

Thanks,
Spike
spike6479
Occasional Contributor
Occasional Contributor
 
Posts: 40
Joined: Fri Aug 06, 2010 8:10 am

Registered Newsbin User since: 08/01/10

Re: Intelligent, scripted, filter of headers

Postby Quade » Mon Oct 03, 2011 6:27 pm

"Update" will update all the headers for all active groups.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44882
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Intelligent, scripted, filter of headers

Postby spike6479 » Tue Oct 04, 2011 7:50 am

Thanks much, I'll try that tomorrow night.

Spike
spike6479
Occasional Contributor
Occasional Contributor
 
Posts: 40
Joined: Fri Aug 06, 2010 8:10 am

Registered Newsbin User since: 08/01/10


Return to V6 Technical Support

Who is online

Users browsing this forum: Google [Bot] and 2 guests