6.82 RC3 - downloaded posts not marked as downloaded

This is the place to help test and discuss Version 6 Beta releases.

6.82 RC3 - downloaded posts not marked as downloaded

Postby RayMark » Tue Dec 17, 2019 1:07 pm

Not sure if it is with RC3 only, probably not.
And it happens not with all the posts. Still perhaps NewsBin could be able to deal with those posts.

So here's what happens:
When I add a post to the download queue, the post is marked with a "v". When it is downloaded - the "v" is replaced by a solid green square - thus indicating that the post was downloaded.
After restarting Newsbin the downloaded posts remain being indicated as downloaded, with solid green squares.
But for some posts, the 'v" is not replaced by the solid green square after the post is downloaded. It remains being indicated with "v"
And after restarting NewsBin, such posts lose any indication that they were downloaded. No solid square, no more "v", not even the background color changed to white - nothing.

So recently all the posts of one especially prolific poster stopped being indicated as downloaded. Previously they always were.
A strange change. Is it a change in NewsBin, or the poster began posting differently?
Because I like bypassing duplicate detection and rely mostly on the indication that the post was already downloaded, I downloaded again unnecessarily hundreds of gigabytes.
Yes, a prolific poster:
s@nd.p (SP)
He posts mainly to a.b.hdtv.x264 group.
"s@nd" is enough for search by poster.

His older posts remain being indicated as downloaded, but from some time, approximately from 2019-11-05, his posts are no longer marked by NewsBin as downloaded.
So what could possibly be the reason?
And there are other such posts from other posters as well, but usually only occasionally, not all of them.
But here, as if I had applied a switch in settings, or something.
Is there any way to fix this thing? At your end, or perhaps at my end?
RayMark
Seasoned User
Seasoned User
 
Posts: 468
Joined: Sat Jul 21, 2007 10:40 pm

Registered Newsbin User since: 07/21/07

Re: 6.82 RC3 - downloaded posts not marked as downloaded

Postby Quade » Tue Dec 17, 2019 2:45 pm

"DownloadMarker.db3 is how "old" and "downloaded" is tracked. Maybe it's gotten corrupted? You might need to delete it and let it build back up again.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: 6.82 RC3 - downloaded posts not marked as downloaded

Postby RayMark » Tue Dec 17, 2019 11:56 pm

Quade, could you test if for you the newest posts of that poster s@nd are marked as downloaded?
Because today I deliberately tried some posts from other posters - and they are marked as downloaded correctly.
But not from that particular poster. Lots and lots of posts - and all not marked, but before approximately November 05 - all were marked.
Actually, I haven't tried his reality tv, or NFL posts, just scripted tv shows, they all are not marked.

Let me try a few of the very latest of his other posts.
Hm, mixed results, but mostly not marked as downloaded:
If the .nfo file and .srr file have exactly the same name as .mkv file - then the post is not marked as downloaded.
But if the .nfo and .srr files have a different name than the .mkv file - then the post is marked as downloaded.

So, for example - and I'm not sure if I can quote here the exact names of the shows, so I will just put the first letters of their titles, the season and episode numbers should help to identify the posts, they are currently the very latest posts of this poster:
- B.D.S07E11 both 720p and 1080p - NOT marked as downloaded, the three files have the same name in both cases.
- A.V.B.R.S00E45 both 720p and 1080p - marked as downloaded, the three files have different names
- M.T.M.S07E14 both 720p and 1080p - NOT marked as downloaded, the three files have the same name in both cases.
- K.U.W.T.K.S17E12 1080p - NOT marked as downloaded, the three files have the same name, but 720p - marked as downloaded, the three files have different names
- NFL - I tried a couple of those - NOT marked as downloaded, the three files have the same name in each case.
- M.R.S04E11 both 720p and 1080p - NOT marked as downloaded, the three files have the same name in both cases.

As to DownloadMarker.db3 - I found the following 3 files:
DownloadMarker.db3 (485,474,304 bytes)
DownloadMarker.DB3-shm (32,768 bytes)
DownloadMarker.DB3-wal (4,169,472 bytes)
I closed NewsBin, removed those three files, and restarted NewsBin (all the markings as downloaded were gone) and tried to download again:
No difference! Still not marked as downloaded!
So the reason for not marking as downloaded is not that the DB files got corrupted.

Quade, please try yourself.
Notice, that if the three files: .nfo, srr, and .mkv have the same name - the post is NOT marked as downloaded.
Furthermore, I tried a different poster, and the post was marked as downloaded.
So what about the three files in that post from the other poster?
Turns out, .nfo and .mkv have the same name, but .srr is missing altogether.
So, it seems, the problem is with .srr file. If it is present, it must have a different name than .mkv file, otherwise the post is NOT marked as downloaded.
Still, I cannot say with 100% certainty that in the case of other posters if all three files are present and have the same name, the post are never marked as downloaded. Not sure about that.
But let's concentrate on this poster, because it should be easy to reproduce the problem with his posts.


IMPORTANT UPDATE:

Tested with 6.82 RC3 - posts NOT marked as downloaded
Tested with 6.82 RC2 - posts NOT marked as downloaded
Tested with 6.82 Beta 3 build 5090 - posts ARE marked as downloaded, everything is OK!

I have no intermediate versions to test, possible later betas or RC1.
But Beta 3 is working fine.
So for now, I will be using beta 3.
Please fix the new bug introduced after Beta 3 in 6.82 before releasing 6.82 Final
Again, the problem seems to be related to the .srr file being present and having exactly the same name as .mkv (or perhaps other main) file.
If .srr is absent or if .srr has a different name - then the post is correctly marked as downloaded.
RayMark
Seasoned User
Seasoned User
 
Posts: 468
Joined: Sat Jul 21, 2007 10:40 pm

Registered Newsbin User since: 07/21/07

Re: 6.82 RC3 - downloaded posts not marked as downloaded

Postby Quade » Wed Dec 18, 2019 3:01 pm

I was looking at this. I see the problem. Not too clear on the fix yet. Shouldn't be hard to make it work but doing it and not breaking something else might be problematic.

If you expand one of these, you can see the problem. A bunch of small files are mixed in with the set. They throw things off.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: 6.82 RC3 - downloaded posts not marked as downloaded

Postby RayMark » Wed Dec 18, 2019 9:56 pm

Yes, I see 3 small files:
.nzb
.nfo
.srr

I don't think .nzb or .nfo are causing the problem, only .srr is.
Because I just recently tested with 6.82 RC3 on posts with .nfo and .nzb present, and they were marked as downloaded correctly.
Also, another of my favorite prolific posters always posts with .nfo and .nzb inside - and all his posts are also correctly marked as downloaded.
I did not test with his posts specifically, but I see from the dates of the latest downloaded posts that they were downloaded already with RC2 or RC3 - and still marked as downloaded.
So, when .nzb and .nfo are present but .srr is absent - everything is OK.
But, as I wrote in my previous long post here, among all those not marked as downloaded posts from the poster s@nd, there were several marked as downloaded.
You can see inside those posts, that .srr is present, but it has a different name.
In some cases - a shorter name. Examples:
A.V.B.R.S00E45, 720p
K.U.W.T.K.S17E12 720p
In other cases - a longer name. Example:
A.V.B.R.S00E45, 1080p
But here is something interesting that might be IMPORTANT:
In the examples of successful marking as downloaded above, all three small files: .nzb, .nfo, and .srr actually have exactly the same name as the main multi-part rar - contrary to what I just said.
But the .mkv file inside the rar has a different name. Not the same as the name of the rar itself, and therefore not the same as the name of the .srr file (and also not the same as the name of .nzb and .nfo files).
So when I said that the name is different, at that time I was looking at the resulting .mkv, not at the initial rar file.
Hower, also very interesting, and perhaps IMPORTANT:
The post gets marked as downloaded just after beginning the downloading, long before the file inside the rar is actually extracted. How could it be?
Seems that Newsbin is looking inside the rar file at a very early stage. Perhaps for early duplicate detection or for something. Even though I was downloading it with Dowload - Bypass Filters.

Again, for a post to be correctly marked as downloaded, the file inside the rar does not have to have a different name than the rar that contains it - it is enough that the .srr file is absent.
Only if the .srr file is present, only then the post is correctly marked as downloaded when the file inside the rar has a different name than the rar itself, and not marked as downloaded if the file inside rar has the same name as the rar.
And the marking as downloaded occurs just after beginning downloading the post, so the name of the file inside of the rar plays a very important role in this from the very beginning.

I hope this helps
RayMark
Seasoned User
Seasoned User
 
Posts: 468
Joined: Sat Jul 21, 2007 10:40 pm

Registered Newsbin User since: 07/21/07

Re: 6.82 RC3 - downloaded posts not marked as downloaded

Postby Quade » Thu Dec 19, 2019 1:19 am

In the specific case I looked at, it's the NZB. It has nothing to do with the file contents but the order. It could be an NFO or NZB or any other file that's not Part01.rar at the top. Newsbin is expecting a specific sort order and these posts aren't on that order.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: 6.82 RC3 - downloaded posts not marked as downloaded

Postby RayMark » Thu Dec 19, 2019 9:00 am

.nzb is always the first file, when expanded - both in the case when the post is correctly marked as downloaded, and in the case when it is not.
The order is always the same:
.nzb
.nfo
.part08.rar
.part09.rar
...
.par2
.part17.rar etc.
.srr
.vol000+01.par2
....

Again, compare
- K.U.W.T.K.S17E12 1080p - NOT marked as downloaded
and
- K.U.W.T.K.S17E12 720p - marked as downloaded
The same file order in both. But in the case of 720p the file inside the rar has a different name than the rar itself.

Absolutely no difference in the file order, but in one case the file inside the rar has the same name as the rar itself (not marked as downloaded), in the other case - it has a different name (marked as downloaded)
Unless you are speaking about some other file order, not as seen when expanded.

MAY BE IMPORTANT:

The subject of the aggregated compacted post is not the same as the subject of .nzb, .nfo, .srr and rar parts inside.
The same example:
- K.U.W.T.K.S17E12 720p (marked as downloaded): the outer subject has the episode name in it. Inside, subjects of all the files DO NOT have episode names in them. The files do not have as well. But the file inside the rar does.
- K.U.W.T.K.S17E12 1080p (NOT marked as downloaded): the outer subject does not have the episode name in it. The inner subjects also do not have the episode name in them, but the file names - do. And the file inside rar does too.
That may be the critically important difference.
Why the outer subject in the case of 720p gets the episode name in it, when the collected inside it items (both their subjects and files) do not have the episode name in them? But this works out OK - the post is correctly marked as downloaded.



BTW, it is strange that the first of rar parts is always part08, and part01 is further down, mixed between rar parts with higher numbers. It is always the same, as far as I can tell.
RayMark
Seasoned User
Seasoned User
 
Posts: 468
Joined: Sat Jul 21, 2007 10:40 pm

Registered Newsbin User since: 07/21/07

Re: 6.82 RC3 - downloaded posts not marked as downloaded

Postby Quade » Thu Dec 19, 2019 3:51 pm

What you see when you expand isn't necessarily what the program sees internally. I just suggested you expand to see the extraneous files. There's a lot going on you don't see from the display. For example the "Sand" files that worked even with the extraneous files only worked because of the leading "<" in the subject which triggers a completely different mode required because of how the search engine reports de-obscured posts.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: 6.82 RC3 - downloaded posts not marked as downloaded

Postby RayMark » Thu Dec 19, 2019 8:19 pm

You are right, I forgot to pay attention to the leading < in the subject, although I had some issues with < before.
In the above example,

- K.U.W.T.K.S17E12 1080p - NOT marked as downloaded - has no leading <, it begins with "[ "
- K.U.W.T.K.S17E12 720p - correctly marked as downloaded - it has the leading "<"

The leading "[ " contains one space after [ - as is in many posts, but in some other posts, from other posters, there is no space after [, so that might be yet another different case, or may not.

Still, look what you have changed after Beta 3 - everything is working fine in Beta 3, all the posts are marked as downloaded. Sure, there used to be some posts here and there not marked as downloaded even with Beta 3 and earlier (I am not sure if they were even downloaded successfully), but here we have almost all of them. At least from the poster s@nd.
Sorry I was not able to test with any versions between Beta 3 and RC2 and to narrow down even further - I missed them, not downloaded, not installed.
RayMark
Seasoned User
Seasoned User
 
Posts: 468
Joined: Sat Jul 21, 2007 10:40 pm

Registered Newsbin User since: 07/21/07

Re: 6.82 RC3 - downloaded posts not marked as downloaded

Postby Quade » Thu Dec 19, 2019 10:07 pm

It's already fixed and the release sent on to Dex.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44867
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97


Return to Newsbin Version 6 Beta Support

Who is online

Users browsing this forum: Google [Bot] and 3 guests