Corrupt FIles OSX SSD...

This forum is for anything not Reason related, if you just want to talk about other stuff. Please keep it friendly!
User avatar
plaamook
Posts: 2593
Joined: 22 Jan 2015
Location: Bajo del mar...

16 Mar 2021

So I've got these files, recordings from a few years ago, and they're corrupt. Reson can't read them. RX shows half the file to be blank. Playback on OSX and QT is half blank.
I go into TM backups and the files are fine from the first backup to the most recent. So TM isn't registering the file as having been changed at all.
OK, copy paste the backup file onto main SSD and we're back in play.
It's jsut 4 long-ish recordings all in the same folder but man this has given me the willies. How many more are there?!!!
So the next tiem I change HD SSD or computer I've got to keep all the original backups incanse of file corruption?

I've never had this in OSX. Not really since moving over from windows years back, I lost a few things.

So what it is? SSD? Just bad luck? Any ideas? Any suggestions for batch finding corrupts files? I've got hundreds of gigs of stuff I've recorded, stuff used in projects, etc. I can't go through it all manually.
Perpetual Reason 12 Beta Tester :reason:

You can check out my music here.
https://m.soundcloud.com/ericholmofficial
Or here.
https://www.youtube.com/channel/UC73uZZ ... 8jqUubzsQg

User avatar
jam-s
Posts: 3035
Joined: 17 Apr 2015
Location: Aachen, Germany
Contact:

16 Mar 2021

The SSD might be going bad. Did it get a little too hot perhaps?

User avatar
integerpoet
Posts: 832
Joined: 30 Dec 2020
Location: East Bay, California
Contact:

16 Mar 2021

plaamook wrote:
16 Mar 2021
So I've got these files, recordings from a few years ago, and they're corrupt. Reson can't read them. RX shows half the file to be blank. Playback on OSX and QT is half blank. I go into TM backups and the files are fine from the first backup to the most recent. So TM isn't registering the file as having been changed at all. OK, copy paste the backup file onto main SSD and we're back in play.
Which half of the files were gone? Was one channel of a stereo pair quiet? Or were there quiet spans of time in all channels?

User avatar
plaamook
Posts: 2593
Joined: 22 Jan 2015
Location: Bajo del mar...

17 Mar 2021

integerpoet wrote:
16 Mar 2021
plaamook wrote:
16 Mar 2021
So I've got these files, recordings from a few years ago, and they're corrupt. Reson can't read them. RX shows half the file to be blank. Playback on OSX and QT is half blank. I go into TM backups and the files are fine from the first backup to the most recent. So TM isn't registering the file as having been changed at all. OK, copy paste the backup file onto main SSD and we're back in play.
Which half of the files were gone? Was one channel of a stereo pair quiet? Or were there quiet spans of time in all channels?
Dead silence both channels for the last 30% of the file.
Perpetual Reason 12 Beta Tester :reason:

You can check out my music here.
https://m.soundcloud.com/ericholmofficial
Or here.
https://www.youtube.com/channel/UC73uZZ ... 8jqUubzsQg

User avatar
integerpoet
Posts: 832
Joined: 30 Dec 2020
Location: East Bay, California
Contact:

17 Mar 2021

plaamook wrote:
17 Mar 2021
integerpoet wrote:
16 Mar 2021
Which half of the files were gone? Was one channel of a stereo pair quiet? Or were there quiet spans of time in all channels?
Dead silence both channels for the last 30% of the file.
I suppose that might eliminate the possibility of some well-meaning but bone-headed app scanning your disk and trying to help you save space by getting rid of redundant signals. I mean, mono ought to be enough for anyone to get the point across, right? :-)

I can't think of a likely reason for actual zeroes to get written to the end of a file like that unless some app lost its marbles (or was poorly implemented and power was cut suddenly while it was in the middle of rewriting the file).

I would have guessed that if your disk were in the process of dying that on reading it would report errors to the driver and the driver would report those errors to the operating system and the OS would report them up to apps and then to you. But if some part of that chain of custody for the errors were broken, it might look as if part of the file were empty in that way.

It almost certainly can't hurt to try whatever disk diagnostic software you have to see if it can find any corruption on the disk that might confuse the OS into believing part of a file were empty like this. Most operating systems these days come with some. Third parties will tell you theirs is better.

User avatar
plaamook
Posts: 2593
Joined: 22 Jan 2015
Location: Bajo del mar...

17 Mar 2021

integerpoet wrote:
17 Mar 2021
plaamook wrote:
17 Mar 2021

Dead silence both channels for the last 30% of the file.
I suppose that might eliminate the possibility of some well-meaning but bone-headed app scanning your disk and trying to help you save space by getting rid of redundant signals. I mean, mono ought to be enough for anyone to get the point across, right? :-)

I can't think of a likely reason for actual zeroes to get written to the end of a file like that unless some app lost its marbles (or was poorly implemented and power was cut suddenly while it was in the middle of rewriting the file).

I would have guessed that if your disk were in the process of dying that on reading it would report errors to the driver and the driver would report those errors to the operating system and the OS would report them up to apps and then to you. But if some part of that chain of custody for the errors were broken, it might look as if part of the file were empty in that way.

It almost certainly can't hurt to try whatever disk diagnostic software you have to see if it can find any corruption on the disk that might confuse the OS into believing part of a file were empty like this. Most operating systems these days come with some. Third parties will tell you theirs is better.
Yeah, disk is verified.

Other interesting point is that the buggered files are all within a group of rec's in one folder. Can't see anything else anywhre. Obvs I'm just skipping around though.
Perpetual Reason 12 Beta Tester :reason:

You can check out my music here.
https://m.soundcloud.com/ericholmofficial
Or here.
https://www.youtube.com/channel/UC73uZZ ... 8jqUubzsQg

User avatar
jam-s
Posts: 3035
Joined: 17 Apr 2015
Location: Aachen, Germany
Contact:

17 Mar 2021

If all files have their End cut off at rougthly the same position it might be due to truncation. Did you copy those from a filesystem that might have some kind of filesize limit. Or did you transfer them via some kind of cloud upload or filesync utility?

User avatar
plaamook
Posts: 2593
Joined: 22 Jan 2015
Location: Bajo del mar...

17 Mar 2021

jam-s wrote:
17 Mar 2021
If all files have their End cut off at rougthly the same position it might be due to truncation. Did you copy those from a filesystem that might have some kind of filesize limit. Or did you transfer them via some kind of cloud upload or filesync utility?
No nothing like that. Just transferred over manually from Mac OS Journaled formatted LaCie rugged back up drives when I got a new computer.
Perpetual Reason 12 Beta Tester :reason:

You can check out my music here.
https://m.soundcloud.com/ericholmofficial
Or here.
https://www.youtube.com/channel/UC73uZZ ... 8jqUubzsQg

User avatar
plaamook
Posts: 2593
Joined: 22 Jan 2015
Location: Bajo del mar...

16 May 2021

So if I could ask the opinion of the local peanut gallery on this...I've clarified the problem abit more here.

I've found more read probelm files. They tend to be older files grouped in the same folder. Things that date back a few years. WAV and aif files that have the ends 'silenced' but it's not just silence. It's like a blank section that nothing can read, maybe 75% through the sample. It varies, not the same amount damaged in each file. I can't load it into any DAW or editor but I CAN read it via a finder preview (space bar function) and as it's Mojave I can see the wave form there, I can see the blank spot. (see attached pic)

Now, if I go into time machine back ups, even the most recent, the files are fine so I can just transfer the back ups to the current drive which is a 2TB Crucial SSD. TM isn't over writing the files with the corrupted version, which is good. I'm not losing data yet.

But, do I get a new drive (which I can't afford at the mo)? Do I do a clean install? Do I restore from TM backup?

Any input here would be very much appreciated.
Attachments
Screenshot 2021-05-15 at 21.38.28.png
Screenshot 2021-05-15 at 21.38.28.png (155.87 KiB) Viewed 700 times
Perpetual Reason 12 Beta Tester :reason:

You can check out my music here.
https://m.soundcloud.com/ericholmofficial
Or here.
https://www.youtube.com/channel/UC73uZZ ... 8jqUubzsQg

User avatar
integerpoet
Posts: 832
Joined: 30 Dec 2020
Location: East Bay, California
Contact:

16 May 2021

Can you say more about what you mean by "…it's not just silence. It's like a blank section that nothing can read…"? Because if at some point something overwrote zeroes onto the end of the file, that would probably sound a lot like silence, but it would be readable silence.

Also, when you say "…grouped in the same folder. Things that date back a few years…", that makes me wonder if this set of files was once on some other disk, damaged there, and then copied here. I know I do all kinds of copies and moves over a period of years which seem innocuous and I then forget. For example, I might have a subset of files on this or that disk and I can't be bothered to figure out which are already on some other disk, so I just merge to wherever I want the superset regardless of whether each and every file needs to be copied.

User avatar
plaamook
Posts: 2593
Joined: 22 Jan 2015
Location: Bajo del mar...

16 May 2021

integerpoet wrote:
16 May 2021
Can you say more about what you mean by "…it's not just silence. It's like a blank section that nothing can read…"? Because if at some point something overwrote zeroes onto the end of the file, that would probably sound a lot like silence, but it would be readable silence.

Also, when you say "…grouped in the same folder. Things that date back a few years…", that makes me wonder if this set of files was once on some other disk, damaged there, and then copied here. I know I do all kinds of copies and moves over a period of years which seem innocuous and I then forget. For example, I might have a subset of files on this or that disk and I can't be bothered to figure out which are already on some other disk, so I just merge to wherever I want the superset regardless of whether each and every file needs to be copied.
Well, the blank section at the end of the file in the screan shot sounds like silence. Reason can't read the file. Finder can. So can RX but it takes a while to load. That's all I can say unless you know some clever other way of telling what the data content in the file is.

As for the file location...I don't think it's been shuffled around much. Wrote to local disc in 2015 and backed up. Earliest backup is 2019 from the install of the SSD I think. I'm pretty organised with how I store files so there's not too much messing around generally.
Local SSD is 2TB. Back up drives are 4TB. I'ts pretty simple set up. But as I said, the backups are all fine, hence thefact that I can just grab older backed up copies and replace them.

Thanks for the help!
Perpetual Reason 12 Beta Tester :reason:

You can check out my music here.
https://m.soundcloud.com/ericholmofficial
Or here.
https://www.youtube.com/channel/UC73uZZ ... 8jqUubzsQg

User avatar
integerpoet
Posts: 832
Joined: 30 Dec 2020
Location: East Bay, California
Contact:

16 May 2021

plaamook wrote:
16 May 2021
Well, the blank section at the end of the file in the screan shot sounds like silence. Reason can't read the file. Finder can. So can RX but it takes a while to load. That's all I can say unless you know some clever other way of telling what the data content in the file is.
I do know of other ways of looking at the raw contents of a file to see if it's got a big pile of zeroes, but rather than clever they're merely obscure command-line ways, and I'm not sure how to describe what I'd try, but `man hexdump` might be a good place to start. There are probably GUI apps out there there let you look at the raw contents of a file. They'll probably present an ocean of data, but it should be pretty easy to see a pattern of zeroes toward the end.

Maybe it's more interesting, though, that some apps can read the files but others cannot. That sounds vaguely like it might mean that there's corrupt data inside the file which describes how long the data should be as opposed to how long the data actually is. Like maybe Reason trusts the file's internal data about how long the data should be and RX, being a repair tool, is less trusting. It's also interesting that RX takes an atypical amount of time to load one of these files, as if it has to think extra-hard about it. I don't remember opening a corrupt file with RX, so I don't know how it behaves, but that might be an interesting avenue of investigation. Maybe RX can be configured to announce problems within a file instead of or in addition to quietly making its best effort to overcome those problems. I don't see a way to do that, but I have only the peasant version of RX, so your mileage may vary.

User avatar
plaamook
Posts: 2593
Joined: 22 Jan 2015
Location: Bajo del mar...

17 May 2021

I've got RX8 Standard. But I don't see an option there.
Anyway, I'm just trying to decide how to address this. Like I said, new SDD, clean install, or TM rollback.
Perpetual Reason 12 Beta Tester :reason:

You can check out my music here.
https://m.soundcloud.com/ericholmofficial
Or here.
https://www.youtube.com/channel/UC73uZZ ... 8jqUubzsQg

User avatar
plaamook
Posts: 2593
Joined: 22 Jan 2015
Location: Bajo del mar...

12 Nov 2021

I’m digging this thread out in the hopes that someone can suggest some app that can scan your comp for corrupted files.

Thing is I have no idea at what point a certain file becomes corrupted right? So I can’t delete any back ups ever and it makes migrating data to a new drive tricky because I’ll be migrating corrupted if I move the current data.

Any help much appreciated.
Perpetual Reason 12 Beta Tester :reason:

You can check out my music here.
https://m.soundcloud.com/ericholmofficial
Or here.
https://www.youtube.com/channel/UC73uZZ ... 8jqUubzsQg

User avatar
plaamook
Posts: 2593
Joined: 22 Jan 2015
Location: Bajo del mar...

13 Nov 2021

Anyone have any experience with something like disk warrior?
Perpetual Reason 12 Beta Tester :reason:

You can check out my music here.
https://m.soundcloud.com/ericholmofficial
Or here.
https://www.youtube.com/channel/UC73uZZ ... 8jqUubzsQg

User avatar
plaamook
Posts: 2593
Joined: 22 Jan 2015
Location: Bajo del mar...

13 Nov 2021

No rush...
Perpetual Reason 12 Beta Tester :reason:

You can check out my music here.
https://m.soundcloud.com/ericholmofficial
Or here.
https://www.youtube.com/channel/UC73uZZ ... 8jqUubzsQg

User avatar
plaamook
Posts: 2593
Joined: 22 Jan 2015
Location: Bajo del mar...

14 Nov 2021

Is it possible that an app is actually corrupting the files? Like Reason? Or Rx? Or similar?
If the SSD is failing surely there would be more general system errors not just corrupt audio files. And thus far it really is just audio files that are corrupted, and only occasionally. Not films, not text files, not pdf's, not jpeg's. Whole rig is running fine in all respects.

Anyone?
Perpetual Reason 12 Beta Tester :reason:

You can check out my music here.
https://m.soundcloud.com/ericholmofficial
Or here.
https://www.youtube.com/channel/UC73uZZ ... 8jqUubzsQg

User avatar
Pepin
Posts: 450
Joined: 16 Jan 2015

14 Nov 2021

If it’s only audio files I could see how that might point to a software issue. But not all corruption is obvious.

Maybe try comparing backups of some other folders to the versions on disk. You can use the diff command in the terminal:
diff -rq path1 path2

User avatar
plaamook
Posts: 2593
Joined: 22 Jan 2015
Location: Bajo del mar...

14 Nov 2021

Pepin wrote:
14 Nov 2021
If it’s only audio files I could see how that might point to a software issue. But not all corruption is obvious.

Maybe try comparing backups of some other folders to the versions on disk. You can use the diff command in the terminal:
diff -rq path1 path2
I will. I’ve lost track of the most recent bad file sadly. After replacing it w an old back up I mean.
But the next one...I’m compiling a list of checks to try tracking this down.
Cheers
Perpetual Reason 12 Beta Tester :reason:

You can check out my music here.
https://m.soundcloud.com/ericholmofficial
Or here.
https://www.youtube.com/channel/UC73uZZ ... 8jqUubzsQg

User avatar
selig
RE Developer
Posts: 11685
Joined: 15 Jan 2015
Location: The NorthWoods, CT, USA

14 Nov 2021

The interesting fact is Finder preview showing some of the waveform. I have a very vague memory of problems with a file’s volume header that can mess up the ability to read the file properly even though the actual audio data is not corrupted. Sorry I can’t remember more, maybe someone else has experienced this and knows of a solution?
Selig Audio, LLC

User avatar
plaamook
Posts: 2593
Joined: 22 Jan 2015
Location: Bajo del mar...

14 Nov 2021

But why this would happen to single files seemingly spontaneously and to seemingly random files is peculiar.
Perpetual Reason 12 Beta Tester :reason:

You can check out my music here.
https://m.soundcloud.com/ericholmofficial
Or here.
https://www.youtube.com/channel/UC73uZZ ... 8jqUubzsQg

User avatar
plaamook
Posts: 2593
Joined: 22 Jan 2015
Location: Bajo del mar...

18 Nov 2021

Ok. Sorry to use this place as a think tank for this problem but I’m running out of ideas and things are getting worse.

I decided to try hunting down all the corrupt files, replace them w files from the archive, and start a new larger back up drive.
I did this by systematically loading nearly 190Gb of files in batches into an NNXT because reason will flag the bad files and this seemed the fastest way.
I only found about ten of them but afterwards it seems to have completely fucked my computer. Not only does Reason lag but all of OS X now lags and hangs and stalls.

So, is it possible that this little bug hunt has revealed a RAM problem? And that pushing all that data through has finished it off?
The corrupt files could have been due to failing RAM to begin with.

But, I’ve run Mac diagnostics and Rember and neither of them are throwing any faults up. So what next?

I’m stumped. And my computer is in a sorry state to boot.
Any ideas from you guys would again be greatly appreciated. I don’t have loads of money to just throw at this speculatively buying ram and so on. (Though I can do this as my older model MBP still allows easy Ram change outs)

Also I figured some of you might like a little tech puzzle as a change from moaning about R12, gripping though it is.

Thanks again in advance.
Perpetual Reason 12 Beta Tester :reason:

You can check out my music here.
https://m.soundcloud.com/ericholmofficial
Or here.
https://www.youtube.com/channel/UC73uZZ ... 8jqUubzsQg

User avatar
plaamook
Posts: 2593
Joined: 22 Jan 2015
Location: Bajo del mar...

19 Nov 2021

Anyway Check this out.
The 12.2.2 updte has sorted the graphics issue. I guess it over wrote some files or some damn thing but I've only got the file corruption to deal with now it seems. Still. This may change of course.
Perpetual Reason 12 Beta Tester :reason:

You can check out my music here.
https://m.soundcloud.com/ericholmofficial
Or here.
https://www.youtube.com/channel/UC73uZZ ... 8jqUubzsQg

User avatar
Pepin
Posts: 450
Joined: 16 Jan 2015

19 Nov 2021

To double check, when you verified your disk, did you run first aid from disk utility on each item up the hierarchy (APFS volume, APFS container, physical disk - described here)? I know it hides containers and drives by default.

User avatar
plaamook
Posts: 2593
Joined: 22 Jan 2015
Location: Bajo del mar...

19 Nov 2021

Pepin wrote:
19 Nov 2021
To double check, when you verified your disk, did you run first aid from disk utility on each item up the hierarchy (APFS volume, APFS container, physical disk - described here)? I know it hides containers and drives by default.
I did yeah. All fine there.
Perpetual Reason 12 Beta Tester :reason:

You can check out my music here.
https://m.soundcloud.com/ericholmofficial
Or here.
https://www.youtube.com/channel/UC73uZZ ... 8jqUubzsQg

Post Reply
  • Information