Page 1 of 1

Advice to speed up local header searches

PostPosted: Sat Jan 19, 2019 1:34 pm
by rob2k77
Was hoping for some hardware tweak suggestions to speed up locally store header searches. (I'm jealous of your online index search)

I have maybe 12 newgroups downloaded and it looks like the oldest records are 2008.

I'm trying to speed up local searches through at large number of headers. (uses about 6gb of drive space)

Using an 3-4 year old HP server with windows server 2016
I added 3 SSDs (OS, newsbin program/headers, and a data volume)
i think 2 xeon six core processors.
a bunch of memory.

Are there any switch to use more or all memory?

I've try creating a 20gb RAM-Drive and moving the newbin program and DBs to the RAM-Drive drive, but didnt see any improvement.

Any suggestion on hardware upgrades to improve local searches?

Thanks a bunch
Rob

Re: Advice to speed up local header searches

PostPosted: Sat Jan 19, 2019 2:57 pm
by Quade
Compacting the databases might improve it. Reducing the number of records could improve it.

The issue is that there's no way to use a database index when doing free text searching so, every record has to be touched. The greater the number of records, the slower the search. If you didn't really need to go back to '08 you could reduce the storage age. SSD's should make it faster than spinning disks but the performance is mostly impacted by "table scanning". The part where you can't use an index.

It might be possible to write an external utility that bolt real text searching onto the database. I'd probably google search "free text searching" and Sqlite on google.