Very High HDD activity

Technical support and discussion of Newsbin Version 6 series.

Very High HDD activity

Postby GrouchoBarx » Tue Sep 04, 2018 4:45 pm

After loading Newsbin Pro I am seeing very high disk activity on the Newsbin Data Folder. I have the Data Folder on a dedicated 3TB (2.72 TB actual) HDD. I have the download folders on a separate HDD.

I am not downloading any files or headers, so I was wondering what might cause this disk activity.

I recently added UsenetServer as a second server (thanks for the heads up on the great deal!). I wen back ad disabled Giganews for header down loads and did a <Download All Headers> for a number of groups. The header downloads have completed, but I still see the high disc activity.

I am using 6.8B2 build 5065

Thanks!
User avatar
GrouchoBarx
Active Participant
Active Participant
 
Posts: 50
Joined: Fri Jun 26, 2009 4:03 pm

Registered Newsbin User since: 06/03/07

Re: Very High HDD activity

Postby itimpi » Tue Sep 04, 2018 5:06 pm

On modern systems headers can be downloaded much faster than Newsbin can process them into the header database so it could well be this processing that is causing the high disk activity. If you look on the Status line does the entry for cache have a number in brackets? If so this is the number of header download temporary files that saved to disk and are waiting to be processed into the database.
The Newsbin Online documentation
The Usenettools for tutorials, useful information and links
User avatar
itimpi
Elite NewsBin User
Elite NewsBin User
 
Posts: 12604
Joined: Sat Mar 16, 2002 7:11 am
Location: UK

Registered Newsbin User since: 03/28/03

Re: Very High HDD activity

Postby GrouchoBarx » Tue Sep 04, 2018 8:27 pm

It looks like that's the answer. Thanks!
User avatar
GrouchoBarx
Active Participant
Active Participant
 
Posts: 50
Joined: Fri Jun 26, 2009 4:03 pm

Registered Newsbin User since: 06/03/07

Re: Very High HDD activity

Postby Quade » Tue Sep 04, 2018 10:22 pm

I recently added UsenetServer as a second server (thanks for the heads up on the great deal!). I wen back ad disabled Giganews for header down loads and did a <Download All Headers> for a number of groups. The header downloads have completed, but I still see the high disc activity.


For the future, You don't need to re-download headers just because you changed servers. The old server's headers can work on the new server.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Very High HDD activity

Postby V_M_F » Sat Sep 15, 2018 8:50 pm

itimpi wrote:On modern systems headers can be downloaded much faster than Newsbin can process them into the header database so it could well be this processing that is causing the high disk activity.


Is this due to the mechanical HD or is it a limit of Newsbin?

I'm running on Gigabit Internet, have an octa-core i5-8500, 32GB, and a pair of fast 480GB (boot) and 1TB (data storage) SSDs, run huge groups of headers in new groups (doing it right now as this is a new computer), and I never notice anything like this at all. Even on my laptop (with quad-core i7 + SSD) it's not an issue so it's very possible this is due to the mechanical HD (or the CPU) not being able to keep up.
V_M_F
Occasional Contributor
Occasional Contributor
 
Posts: 40
Joined: Fri Apr 03, 2009 10:06 am

Re: Very High HDD activity

Postby Quade » Sun Sep 16, 2018 1:00 am

5.59 didn't do anything with the headers. It just wrote them to disk into binary files. Then when you loaded a group, all the assembly and combining happened when the groups was loaded. Meaning it happened every time you loaded the group. 6.XX doesn't work that way. The headers are processed after download one time then fed to a database. That means the combining is a one time thing right after the header download. 6.XX loads the groups much faster as the cost of slower feed into the database.

Another problem with 5.59 is that search was slow and deleting posts meant the whole binary file had to be re-written. As groups grew this became less and less usable.

People don't seem to realize how much data we're talking about. Downloading all headers from a large group downloads TB's worth of headers.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97


Return to V6 Technical Support

Who is online

Users browsing this forum: Google [Bot] and 2 guests

cron