Help - Search - Members - Calendar
Full Version: Now we're talkin STORAGE
Luminous Landscape Forum > Equipment & Techniques > Digital Cameras & Shooting Techniques
Pages: 1, 2
CliffSamys
I thought some here may be interested in this article:

http://www.wired.com/news/technology/0,723...ml?tw=rss.index
alainbriot
Yes, things are moving :-) Seagate bought Maxtor. I just got several Maxtor 1.5 TB OneTouch III Turbo Edition Hard Drives and they are great. Small, silent, practical. Far better than the LaCie 1TB I used before. The new Maxtor are not as noisy, they are smaller and they stay cooler. They also come with Retrospect software, with One Touch Sync backup (couldn't be easier) are Mac formatted, and are ready to be configured as Raid 0 or 1.

http://www.maxtorsolutions.com/en/catalog/OTIII_Turbo/
Nill Toulme
Mais Alain, don't those drives consist of two 750GB drives chained together in one case, thereby doubling your chances of catastrophic drive failure for each one?

Nill
~~
www.toulme.net
Ray
Amazing and incredible! 50 terabits per square inch?? Wow! Couple that with a quantum computer and anything is possible biggrin.gif .
alainbriot
QUOTE (Nill Toulme @ Jan 3 2007, 11:51 PM)
Mais Alain, don't those drives consist of two 750GB drives chained together in one case, thereby doubling your chances of catastrophic drive failure for each one?

Nill
~~
www.toulme.net
*


I agree it is a risk, but I need large storage space. Right now I have about 4 TB online. Everything is backed up twice, so even if a drive crashes nothing is lost.
CliffSamys
QUOTE (Nill Toulme @ Jan 3 2007, 03:51 PM)
Mais Alain, don't those drives consist of two 750GB drives chained together in one case, thereby doubling your chances of catastrophic drive failure for each one?

Nill
~~
www.toulme.net
*



If configured as RAID 1, this would be a mirrored 750GB RAID. This would be safe, as everything is stored on both drives.
RAID 0 is striping, where data is written alternately to each drive to increase speed.

By the way, this looks like a nice product. I've been pushing Samy's to start carrying a CalDigit item:

http://www.caldigit.com/FireWireVR.asp

The Maxtor is better priced, though.
Nill Toulme
QUOTE (alainbriot @ Jan 4 2007, 12:17 AM)
I agree it is a risk, but I need large storage space.  Right now I have about 4 TB online.  Everything is backed up twice, so even if a drive crashes nothing is lost.
*

I don't understand the point of these doubled drives though. In what way are they better than two separate 750GB external drives? It seems to me all they accomplish is doubling your chances of losing twice as much data. (Does that equate to 4x the risk?)

If I had that much data (I only have about half a TB now) I'd have it all on a RAID5, backed up to another RAID5, with that backed up in turn to a bunch of 750GB external drives. As it is, I have the live data on a 1.1TB RAID5, which backs up automatically nightly to two external 400GB drives, which are in turn backed up manually weekly to a single 750GB external that lives offsite.

Nill
~~
www.toulme.net
Nill Toulme
QUOTE (CliffSamys @ Jan 4 2007, 12:39 AM)
...By the way, this looks like a nice product. I've been pushing Samy's to start carrying a CalDigit item:

http://www.caldigit.com/FireWireVR.asp
...
*

Something like that holding four drives instead of two, configurable as RAID5, and with a handle on top for portability, would seem just about ideal to me. SATA might be preferable to firewire though.

Nill
~~
www.toulme.net
Ray
Has it not been established that those who are the least bit paranoid about data loss, should back up their images on optical media like DVDs.

I know it's a bit tedious burning DVDs that hold only 4.4Gb, but we now have Blu-ray discs that hold 25Gb.
feppe
QUOTE (Ray @ Jan 3 2007, 09:20 PM)
Has it not been established that those who are the least bit paranoid about data loss, should back up their images on optical media like DVDs.

I know it's a bit tedious burning DVDs that hold only 4.4Gb, but we now have Blu-ray discs that hold 25Gb.
*


I don't see a reason to do DVD backups. Backing up 100GB, let alone 1TB on DVDs is not "a bit tedious," it's murder. Not to say unnecessary and ridiculously expensive compared to HDDs. Besides, you'll have to check the DVDs or re-burn them every year or so to ensure they're not rotting. Blu-ray and HD-DVDs seem to have higher (claimed) shelf-life but are not nearly as cost-effective as HDDs or even DVDs.

Hot-swappable RAID 0 or 5 setups are quite affordable these days, and external HDDs for off-site backing up cost even less. With HDD Mean-Time-To-Failure -rates at well over 1 million (!) hours and non-operating shock resistances in the hundreds of Gs you'll have to really try hard to get a failure leading to data loss with striped or double-backups.
alainbriot
QUOTE (Ray @ Jan 4 2007, 02:20 AM)
Has it not been established that those who are the least bit paranoid about data loss, should back up their images on optical media like DVDs.

I know it's a bit tedious burning DVDs that hold only 4.4Gb, but we now have Blu-ray discs that hold 25Gb.
*


I have everything backed up on DVD's as well.
feppe
QUOTE (alainbriot @ Jan 3 2007, 09:45 PM)
I have everything backed up on DVD's as well.
*


Umm..... 4 terabytes on DVDs? That's.... *calculates* .. *shocked* .. *doublechecks* ... almost one thousand (1,000) DVDs...

You, sir, either have the patience of a saint or a very patient assistant.

Well, I can see how it's not such a problem if you accumulate it over the years. But as I said, I would check that they work yearly in case they start rotting - even if I had double backups. Well, I guess if you check a DVD per day so that's a 3-year rotation which isn't that bad as long as you have two backups.

Still, a few external HDDs would be much more convenient.
Ray
QUOTE (feppe @ Jan 4 2007, 11:44 PM)
Besides, you'll have to check the DVDs or re-burn them every year or so to ensure they're not rotting.
*


Nonsense! Have you ever thought of engaging in a bit of introspection to check your level of paranoia biggrin.gif .
feppe
QUOTE (Ray @ Jan 3 2007, 10:21 PM)
Nonsense! Have you ever thought of engaging in a bit of introspection to check your level of paranoia  biggrin.gif .
*


Paranoia? It's a studied fact that DVDs rot. I have literally hundreds of DVD-Rs from less than 5 years back that are unreadable. Cheapest DVDs available with nothing I wouldn't miss on them, but the point remains. If you have two sets of backups you have to do some kind of rotated check to ensure they are still readable, and that you have time to copy them to new media when they start to do so without losing any data.

And as I pointed out, DVDs have currently nothing which makes them a better choice than HDDs as a backup media.

If a professional really wants to gamble with her income, be my guest. Investing a few hundred dollars on a decent backup system - whether it's DVD or HDD -based - is priceless in case of a equipment failure or fire.
CliffSamys
QUOTE (Nill Toulme @ Jan 3 2007, 06:13 PM)
I don't understand the point of these doubled drives though.  In what way are they better than two separate 750GB external drives?  It seems to me all they accomplish is doubling your chances of losing twice as much data.  (Does that equate to 4x the risk?)
*


For most people, doing a single backup is difficult enough. So to be able to write it to one device that automatically makes two copies makes sense. There are really no advantages to archiving to RAID 5 over RAID 1 in this type of scenario. In RAID 1 everything is written twice. It's only when people use RAID 0 (striping) that the failure rate increases.
CliffSamys
QUOTE (Nill Toulme @ Jan 3 2007, 06:18 PM)
Something like that holding four drives instead of two, configurable as RAID5, and with a handle on top for portability, would seem just about ideal to me.  SATA might be preferable to firewire though.
*


Hmmmm... Like this?

http://www.wiebetech.com/products/rt5.php

I like these too:

http://www.kanotechnologies.com/products/SV3X500R5S.cfm
Ray
QUOTE (feppe @ Jan 5 2007, 01:02 AM)
Paranoia? It's a studied fact that DVDs rot. I have literally hundreds of DVD-Rs from less than 5 years back that are unreadable.
*


Wow! I can't understand that at all. All my ultra budget CDs recorded more than 10 years ago are still perfectly readable. I have no instances of DVDs that are unreadable, recorded 5 years ago (or so, or more).

Have you been the victim of fraud? Perhaps some company selling a batch of reject DVDs that they managed to acquire for nothing?
feppe
QUOTE (Ray @ Jan 3 2007, 11:24 PM)
Wow! I can't understand that at all. All my ultra budget CDs recorded more than 10 years ago are still perfectly readable. I have no instances of DVDs that are unreadable, recorded 5 years ago (or so, or more).

Have you been the victim of fraud? Perhaps some company selling a batch of reject DVDs that they managed to acquire for nothing?
*


CDs and DVDs aren't really comparable as DVDs have much higher data density and as thus are more susceptible to rotting.

It's possible that I'm a victim of fraud. But there are plenty of studies which suggest that "normal" recordable DVDs have a _practically_ limited shelf life. Limited in the sense that I nor any professional should trust them to work indefinitely. If you just burn two sets once, put them in a climate-controlled archiving cabinet and leave them for ten years, I wouldn't be surprised if there are quite a few ones that aren't readable.

Thankfully (?) tech advances at such a pace that people are re-burning or re-archiving files to different media every few years.
alainbriot
QUOTE (feppe @ Jan 4 2007, 03:21 AM)
Umm..... 4 terabytes on DVDs? That's.... *calculates* .. *shocked* .. *doublechecks* ... almost one thousand (1,000) DVDs...

You, sir, either have the patience of a saint or a very patient assistant.
*


I have been burning them as I go, starting around 1995 with CD's and now with DVD's :-) Right now I am burning them at a rate of about 4 a week or so, with peaks and valleys, so to speak. I also have everything on external disk drives. 4 copies of everything. That keeps my paranoia in check.
Nill Toulme
QUOTE (CliffSamys @ Jan 4 2007, 04:19 AM)
For most people, doing a single backup is difficult enough. So to be able to write it to one device that automatically makes two copies makes sense. There are really no advantages to archiving to RAID 5 over RAID 1 in this type of scenario. In RAID 1 everything is written twice. It's only when people use RAID 0 (striping) that the failure rate increases.
*

Now that does make a bit of sense, although (a) I suspect that most of these drives are being used as 1.5TB RAID 0 (or JBOD or something worse?) drives, not as 750GB RAID 1, and (b) there are still advantages to simply having two redundant external drives over RAID 1, e.g., operator error, etc.

Clearly there's no benefit of RAID 5 over RAID 1 in a two-disk array, but my whole point is that I don't *see* the point of two-disk arrays. RAID 5's reliability benefits kick in at 3+ disks.

Nill
~~
www.toulme.net
Nill Toulme
QUOTE (CliffSamys @ Jan 4 2007, 04:24 AM)

Yes exactly! Those are a little pricy though. ohmy.gif Mac Gurus has some enclosures that I've been looking at...

Nill
~~
www.toulme.net
feppe
QUOTE (Nill Toulme @ Jan 4 2007, 09:00 AM)
Now that does make a bit of sense, although (a) I suspect that most of these drives are being used as 1.5TB RAID 0 (or JBOD or something worse?) drives, not as 750GB RAID 1, and (cool.gif there are still advantages to simply having two redundant external drives over RAID 1, e.g., operator error, etc.

Clearly there's no benefit of RAID 5 over RAID 1 in a two-disk array, but my whole point is that I don't *see* the point of two-disk arrays.  RAID 5's reliability benefits kick in at 3+ disks.

Nill
~~
www.toulme.net
*


I could be wrong, but my understanding is that with RAID 5 you lose all your data if your controller card malfunctions (short circuit, lightning damage, etc.). With RAID 1 you don't have that problem.
CliffSamys
QUOTE (Nill Toulme @ Jan 4 2007, 06:00 AM)
Now that does make a bit of sense, although (a) I suspect that most of these drives are being used as 1.5TB RAID 0 (or JBOD or something worse?) drives, not as 750GB RAID 1, and (cool.gif there are still advantages to simply having two redundant external drives over RAID 1, e.g., operator error, etc.

Clearly there's no benefit of RAID 5 over RAID 1 in a two-disk array, but my whole point is that I don't *see* the point of two-disk arrays.  RAID 5's reliability benefits kick in at 3+ disks.

Nill
~~
www.toulme.net
*


I don't see the advantages of 2 drives vs RAID 1. Do you mean being able to take one off site?
RAID 5 requires a minimum of 3 drives by definition. You can't have a 2 drive RAID 5. But I still don't see its advantages over RAID 1 for what we are discussing. And as poited out, if the controller goes on a RAID 5, you will be very unhappy.
Nill Toulme
QUOTE (CliffSamys @ Jan 4 2007, 05:14 PM)
I don't see the advantages of 2 drives vs RAID 1. Do you mean being able to take one off site?
*

There's that, plus if you accidentally permanently delete or otherwise hose a file or folder on a RAID 1 setup, by definition you do it on both disks, right? With two separate disks — i.e., two separate backups — you reduce that risk significantly it seems to me.

Nill
~~
www.toulme.net
djgarcia
QUOTE (feppe @ Jan 4 2007, 03:30 PM)
I could be wrong, but my understanding is that with RAID 5 you lose all your data if your controller card malfunctions (short circuit, lightning damage, etc.). With RAID 1 you don't have that problem.
*

If a controller malfunctions there's no telling what could happen, no matter what RAID level - you may get lucky or not. If it just stops functioning, no harm, just replace it. But if it goes medieval on your drives, well that's that. But the same would apply to a normal disk controller, so there's no saving grace here, RAID or not. I mean, lightning bolts don't discriminate against RAID disks particularly - any drive can be fried equally biggrin.gif. However, a fault-tolerant RAID array could minimize your probability of failure, because if only one drive is fried you can recover.

Some levels provide more than one drive tolerance. RAID 10 uses mirrored pairs, so in a multiple mirror array you could conceivably lose one drive in each pair and recover while still working in degraded performance mode. But lose both drives in the same mirror and you're hosed. I had that happen with a backup utility that couldn't handle a specific controller but had no problems with others.

Keep in mind there are many RAID configurations. Some address performance (RAID 0), some address fault tolerance (RAID 1), and some do both (10, 5, 6, 50, 60) in some way or another. But any way you cut it, RAID does not safeguard against my own stupidity - if I erase a folder and purge the trashcan, it's gone, RAID or not. Hence the existence of BACKUPS ... which RAID complements but does not substitute by any stretch of the imagination.

The main thing to remember about RAID is that it's transparent to the user and the OS (assuming hardware RAID, the only type worth having IMHO), until the problem occurs, then you normally get a chance to fix it relatively quickly and painlessly.

No, RAID isn't cheap, so it's up to you what it's worth to insure your images and to what point. But if a gas main explodes in your basement, well, may the Force be with you ...
narikin
no-one seems to be mentioning the platform difference here.

a windows based case can house many many drives internally, along with an advanced hardware raid card. it's one of the main reasons I switched to PC rather than Mac 4 years ago - Apple's ridiculous 2 drives internal for a top model was a joke, as is the idea of using endless firewire drives to store serious image library on.

In an upgrade a year ago, I settled on a PC case with 12 internal quick swap bays, (plus extra space for an OS drive etc) and put in an Areca raid card. total cost for 6Tb in Raid 6 (double redundancy of raid 5) was $2800. you would now get 9Tb for the same $. Throughput is simply amazing with 12 drives reading/writing all at once.

an x-raid was going to cost me over $10,000 more, for the same thing. hmmm.
narikin
oh, and check out the warranty on that Maxtor. I suspect its one year. which is ridiculous.

they are using the Seagate 750gb drives, which you can buy for $300 each with a 5 year warranty and put them in your case, attach them to your onboard MB raid and get the exact same thing - 1.5Tb storage/ backup (and get them replaced for free if they break down in 4 years time...)
tedchoi11
QUOTE (feppe @ Jan 4 2007, 04:37 AM)
CDs and DVDs aren't really comparable as DVDs have much higher data density and as thus are more susceptible to rotting.

It's possible that I'm a victim of fraud. But there are plenty of studies which suggest that "normal" recordable DVDs have a _practically_ limited shelf life. Limited in the sense that I nor any professional should trust them to work indefinitely. If you just burn two sets once, put them in a climate-controlled archiving cabinet and leave them for ten years, I wouldn't be surprised if there are quite a few ones that aren't readable.

Thankfully (?) tech advances at such a pace that people are re-burning or re-archiving files to different media every few years.
*

i've had quite a few dvd-r discs rot over time. sometimes as soon as a few months. rough guess is that ~10% will fail within 1-3 years. it's harder to get catastrophic failure with hundreds of independent image files on a dvd-r, but a patch of rot will stop a movie cold. i see that all the time.

ted
Ray
QUOTE (tedchoi11 @ Jan 6 2007, 02:48 AM)
i've had quite a few dvd-r discs rot over time. sometimes as soon as a few months. rough guess is that ~10% will fail within 1-3 years. it's harder to get catastrophic failure with hundreds of independent image files on a dvd-r, but a patch of rot will stop a movie cold. i see that all the time.

ted
*


Hmm! How odd! I was importing DVD movies from the US from the time the first DVD-ROM player became available in Australia; before there were hardly any titles available in Australia. I had my Creative Labs DVD player permanently set to Region 1, before I discovered a 'crack'.

I haven't come across a single disc, from those early years, that fails to play, whether from disc rot or any other cause. But I have bought the occasional pirated movie (from Malaysia and Thailand) that failed to play despite being new, and I have recorded images on a DVD disc which failed to play immediately after recording.

I have no reason to suppose that 'genuine' discs that have met standard QC requirements and that have been recorded properly should give any trouble. That's just my experience.
Jonathan Wienke
You're just lucky, Ray.
Ray
QUOTE (Jonathan Wienke @ Jan 6 2007, 05:50 AM)
You're just lucky, Ray.
*


Well, I'm certainly not going to argue that I'm not biggrin.gif .
kaelaria
For anyone needing them/interested - Hitachi and Seagate just announced their 1TB drives at CES, starting at $399!
tedchoi11
QUOTE (Jonathan Wienke @ Jan 5 2007, 08:50 AM)
You're just lucky, Ray.
*

i have to concur. at first i thought it was a disc quality issue so i only used name brand, store bought discs. Verbatim, Maxell, Apple. Not that they make their own, or do much beyond repackaging. I'm sure there's a list of who makes what somewhere on the internet. Following discussions of disc rot on the internet led me to a couple of supposedly high quality, japanese original manufacturer 'gold label' discs. much higher cost, but would be worth it if they didn't fail. or let me stop worrying about failure in my backups.

but it didn't help. not noticably, anyway. i think the real problem is in the burners. i've used pioneer burners since the 104 series, and the matsushita burner in my iMac. my guess is that the quality of the dot created by the burner is nowhere near as good as what can be done in high quality mass production, and are much more sensitive to media breakdown. i've tried burning at the slowest speeds rather than the fastest speeds, but that doesn't make a difference.

also, the reader/player is another variable. some are much better at reading messy discs than others, but i'm concerned it's through some sort of lossy error correction. this is essentially unnoticable on a movie, may possibly be noticable on a still image, and would be a big concern in an excel or filemaker file. i'd rather have the whole excel file fail to open than to have one or two cells with corrupted data that would lead to intermittent erroneous calculations!

bottom line: i only backup onto hard drives. i migrate my data often- probably every 6 or 12 months- onto the second-biggest drives available. so these days, all my data are on duplicate 500gb drives, as the 750s are prohibitiively expensive. when the 1tb drives come out, i'll start buying 750gb drives and sell the 500gb drives. so far, this is working well to keep the number of drives at a reasonable level.

ted
Ray
Ted,
I guess bad news is always more interesting than good news. The issue is complicated by the fact that, as a result of media reports of disc rot and physical deterioration becoming widely knowm, every time there's a problem reading a disc, the cause will likely be attributed to that physical deterioration.
feppe
QUOTE (Ray @ Jan 5 2007, 02:30 AM)
Hmm! How odd! I was importing DVD movies from the US from the time the first DVD-ROM player became available in Australia; before there were hardly any titles available in Australia. I had my Creative Labs DVD player permanently set to Region 1, before I discovered a 'crack'.

I haven't come across a single disc, from those early years, that fails to play, whether from disc rot or any other cause. But I have bought the occasional pirated movie (from Malaysia and Thailand) that failed to play despite being new, and I have recorded images on a DVD disc which failed to play immediately after recording.

I have no reason to suppose that 'genuine' discs that have met standard QC requirements and that have been recorded properly should give any trouble. That's just my experience.
*


You're also comparin apples to oranges: DVDs and recordable DVDs (DVD-R and DVD+R) are very different beasts as the recording surface is not the same. In short, recordable DVDs are much more susceptible to rotting. That's the reasons why your legit DVDs haven't started rotting, and some of your pirated have - many pirated movies are actually recordable DVDs and not pressed in a factory.

Also, I've had the same issue as tedchoi11 below. I had an early Lite-On burner which did both DVD-R and DVD+R (feature which is pretty much standard these days). Now I have a batch of DVDs which can only be read with that burner as others won't work with them. That's only with one brand.

So, once again, DVDs are way too finicky for me to trust for backups. This holds even now when the tech is mature, as there is a much better alternative in external HDDs and RAID arrays.
jani
QUOTE (feppe @ Jan 5 2007, 03:11 PM)
So, once again, DVDs are way too finicky for me to trust for backups. This holds even now when the tech is mature, as there is a much better alternative in external HDDs and RAID arrays.

So far, the "external HDDs and RAID arrays" crowd have been very good at dissing DVDs.

But you haven't been very good at showing that your alternative is better.

Harddisks are also prone to failure, they're susceptible to power surges, electromagnetic noise and physical shock, not to forget wear, tear and mechanical failures.

Perhaps you place too big a trust in some layman's reading of the term MTBF as the only meaningful measurement of a HDD's lifetime?

It is typical for hard drives to have a service life of three to five years (though most drives probably will live for ten or twelve, maybe more).

MTBF should only be considered in concert with the service life and/or warranty time of the drive.

So, what does this mean for you, in practice?

Well, you need to keep migrating your data to new media every few years, regardless of the choice of storage media.

And when you migrate your data to new media, the most important bit is to check that:

1) Your data isn't corrupt.
2) Your data isn't corrupt after migration.


An alternative to disk drive "backups" is to make tape backups, but then you need to avoid the bad kinds of backup tapes (DAT, for one) and try to find the ones that have a fairly long storage life (S-DLT, maybe, or LTO-3). But of course, even the cheap ones aren't really cheap in terms of absolute number of dollars.

And you're still not safe from the issue of at least checking your data integrity, or migrating your data.

Got a headache yet? Good, because I certainly do. smile.gif
Ray
QUOTE (feppe @ Jan 6 2007, 12:11 PM)
Now I have a batch of DVDs which can only be read with that burner as others won't work with them. That's only with one brand.
*


That's not a major problem, feppe. Migrate the data to another computer. Reburn it. The only insurmountable problem is when physical deterioration takes place so the disc is unreadable by any device.

I've seen pictures of such deterioration on this forum, which made me wonder what sort of environment could cause that. I've never experienced this myself and I've recorded literally thousands of CDs and DVDs over the years, often on the cheapest, best value media I could find in the local stores.

I have never, however, bought any blank discs over the internet, and I'm beginning to wonder if this impersonal type of internet trading is the cause of some dissatisfaction. If you are a retail store, you can't afford to sell bodgy products. If you are trading on the internet, you could probably sell a batch of reject blanks you'd picked up for next to nothing (perhaps from China) and then disappear.

I recently watched a documentary on the effects of atomic radiation. There's been a huge amount of research on the effects of the Chernobyl atomic power station accident. When watching this, I was reminded of this 'bit rot' situation with optical media. Bad reports get extrapolated and exaggerated. They fire our imagination and appeal to our basic fears about everything. Atomic radiation is a classic example. Over the years, public opinion has shifted from the viewpoint that the very word radium itself was symbolic of everthing good, hi-tech and wonderful, to the position that any amount of radiation, however small, is harmful.

As a result, the Chernobyl accident caused a great deal of consternation amongst people living in the vicinity of the accident. Women had abortions, as it now seems, for no good reason at all. As some great American president (or important historical figure) said (who was that?), the only thing we have to fear is fear itself.

Whilst it's true that some people died who were very close to the accident and who received massive doses of radiation (and I hope that I am not seen as trivialising their families' grief), there are lots of people who received moderate amounts of radiation, who've been quaking in their boots for the past 20 years, and who are still as fit as a fiddle.

An analysis of the wildlife in the close vicinity of the accident revealed very surprising results. The researchers expected to find all sorts of mutant rats and mice with two heads and three ears, or perhaps no wildlife at all. On the contrary, wildlife was thriving without a hiccup. Moderately strong radiation levels just seemed to have bounced off their back. In fact, it now seems that relatively small amounts of radiation, up to 100 millisieverts (per year) are actually beneficial to us. They stimulate our immune response. An analysis of areas in the United States that have strong, natural background radiation, have shown that people who live in such areas have less cancer, statistically, than people who live in areas with low background radiation.

I can't remember how many X-rays a 100 millisievert dosage represents, but it's into the thousands.

The message from me is, if you are worried about bit rot, don't worry. Get a life! (Unless of course you've been in the habit of buying ultra-cheap discs from the internet. You know, if something's too good to be true, it often is biggrin.gif ).
feppe
QUOTE (jani @ Jan 12 2007, 07:57 PM)
An alternative to disk drive "backups" is to make tape backups, but then you need to avoid the bad kinds of backup tapes (DAT, for one) and try to find the ones that have a fairly long storage life (S-DLT, maybe, or LTO-3). But of course, even the cheap ones aren't really cheap in terms of absolute number of dollars.

And you're still not safe from the issue of at least checking your data integrity, or migrating your data.

Got a headache yet? Good, because I certainly do. smile.gif
*


I stand by my choice of HDD backups. As you said, there is a need to re-migrate to new media every few years whether one goes for CD, DVD or HDD. But as I've pointed out earlier, migrating hundreds - or even tens - of DVDs is an order of a magnitude more pain in the ass than migrating a few HDDs.

While tape is a decent solution, they are way too expensive as you point out, and due to them lacking random access are limited in their use.
Ray
QUOTE (feppe @ Jan 13 2007, 11:47 PM)
I stand by my choice of HDD backups. As you said, there is a need to re-migrate to new media every few years whether one goes for CD, DVD or HDD. But as I've pointed out earlier, migrating hundreds - or even tens - of DVDs is an order of a magnitude more pain in the ass than migrating a few HDDs.

While tape is a decent solution, they are way too expensive as you point out, and due to them lacking random access are limited in their use.
*


The first few hard drives I ever owned (the first being an 850mb Western Digital) have all failed, after 10 or 12 years and sometimes well before that. Not a single optical disc has yet failed in that period. When it does, I'll let you know. (Are you going to be around, on this site, in another 10 or 20 years? Maybe I'll catch up with you on another site biggrin.gif ).
feppe
QUOTE (Ray @ Jan 12 2007, 10:29 PM)
The first few hard drives I ever owned (the first being an 850mb Western Digital) have all failed, after 10 or 12 years and sometimes well before that. Not a single optical disc has yet failed in that period. When it does, I'll let you know. (Are you going to be around, on this site, in another 10 or 20 years? Maybe I'll catch up with you on another site  biggrin.gif ).
*


I still haven't seen a single compelling argument to back up to optical media rather than hard disks. DVDs and CD-ROMs were the most convenient backup media until recently, but HDD tech has gone way past optical media in backups. When HD-DVD or BluRay recordables come down in price this _might_ change. But I have a feeling that HDD prices and capacities have galloped past by then, especially as it seems that multi-layer HD-DVD/BluRay recordables won't happen any time soon.

You're telling me you've used 10-year-old HDDs? That's, what, 300MB and slow as molasses? I don't know about you but the longest I've used a HDD was less than five years, and that's pushing it. Tech advances so fast that drives become cheaper and OS requirements steeper necessitating constant upgrading, unless one wants to stick to PS5 and 98SE.

Failure rate is largely a non-issue with redundant backups and integrity checks. And just as my failure rate is perhaps an anomaly, so is yours. Archival on newer optical media (DVDs and HD-DVD/BluRay) is arguably more prone to the elements and time than CD-ROMs due to higher data density (I know error correction has developed a lot).

Even if you do trust your current backups to last ten years redundancy is necessary, as there is the possibility of fire, flood, etc. which can only be avoided by redundant, off-site backups. But due to what I pointed above and in earlier posts, archival in the traditional sense is unnecessary these days as you can backup an entire hard disk from ten years ago - which cost hundreds of dollars - on a DVD which costs $1, or take up HDD space which costs cents.

In any case, I still can't believe anyone is backing up tens or hundreds of DVDs _and_ checking their integrity every once in a while - for that's necessary if you want to ensure your files stay intact, whether you use optical media or magnetic. It is obvious the convenience and cost of HDDs is unsurpassed. One can always gamble and rely on their backups working until they're needed, but that's asking for trouble.

Anyway, it appears to be a moot argument. It's your time and money you're throwing away.
kaelaria
If you want the most secure solution, build a dedicated raid 5 array, and rebuild it with new discs every 5 years. As they fail individually in between main builds, you lose nothing.

You can build a 1TB array with external cabinet for less than $750 right now, and that will only be cheaper in the future.

Big deal.

I run a 1TB raid 5 array in my system right now, and it cost me $600 in drives.
feppe
QUOTE (kaelaria @ Jan 13 2007, 01:09 AM)
If you want the most secure solution, build a dedicated raid 5 array, and rebuild it with new discs every 5 years.  As they fail individually in between main builds, you lose nothing.

You can build a 1TB array with external cabinet for less than $750 right now, and that will only be cheaper in the future. 

Big deal.

I run a 1TB raid 5 array in my system right now, and it cost me $600 in drives.
*


If your RAID card fries, so does your data (most likely and unless you want to spend thousands on data recovery service which might or might not recover anything). So you still need backups even if you're running a redundant RAID setup.

But damn, prices really have come down. I'm definitely getting a RAID 5 finally when I get a new computer.
Ray
QUOTE
You're telling me you've used 10-year-old HDDs? That's, what, 300MB and slow as molasses?


No. I wasn't clear. My practice has been to hand down my old computers to friends, relatives and ex-wife. Those initial hard drives (an 850MB and a 2.1GB) have long since given up the ghost. Another 2 or 3 hard drives around 5, 10 and 20GB have been the victim of a power surge. It's fortunate I wasn't relying upon such drives as a back-up for my images.

I had a selection of my slides transferred to Photo CD by Kodak a year or so before I bought my first computer. I used such discs to test computers in the store whilst shopping around for a suitable computer. This was the days when the average video card was 500kb. 1MB was a premium card and 2MB a rarity. That first computer was the most expensive I've ever bought and slow as molasses, as you say. It took a full 2 minutes for an 18mb image to be read and decompressed.

Those first CD-ROMs from Kodak, now about 14 or 15 years old, are still perfectly readable. My first hard drive (and 2nd, 3rd and 4th etc) are not.

QUOTE
Anyway, it appears to be a moot argument. It's your time and money you're throwing away.


I don't get you. It takes no more time to burn a CD or DVD than to brush your teeth. The cost is only significant when the technology is new. I wouldn't recommend burning to Blu-ray just yet.

However, in your situation I see the problem. To transfer 1TB of images to DVD would be a mammoth task. Pity you haven't been doing it regularly by degrees. You could then have peace of mind biggrin.gif .

On the cost issue, I should mention that a LaCie 300GB external hard drive, in Australia, costs about A$1 per gigabyte, maybe a bit less now. Blank DVDs are about 10-20 cents per gigabyte.
jani
QUOTE (feppe @ Jan 13 2007, 06:21 AM)
If your RAID card fries, so does your data (most likely and unless you want to spend thousands on data recovery service which might or might not recover anything).

That depends on how the controller "fries". If it destroys the information stored on the disks, too, then you're out of luck.

If it doesn't, however, you can usually replace it with an identical model controller or a newer controller from the same vendor.

At work, we have successfully upgraded a couple of our RAID 10 setups at least two times this way, and recovered from RAID controller failure once.

We have not experienced complete RAID failure from a fried RAID controller.

This is with the 3Ware 8xxx and 9xxx series controllers.

Unless other RAID controllers are worse, your worries are not very rational.

Worry about Windows corrupting your file system instead.

QUOTE
So you still need backups even if you're running a redundant RAID setup.

But damn, prices really have come down. I'm definitely getting a RAID 5 finally when I get a new computer.

As I've mentioned in another thread, RAID 5 isn't the "most secure" solution as kaelaria touts.

RAID 5 allows the failure of any single drive, but steals less capacity from the total number of drives; the equivalent of one drive per RAID is "lost" capacity.

RAID 6 beats RAID 5 by using another parity stripe set, and allows for failure of any two drives. The equivalent of two drives per RAID is "lost" capacity.

RAID 1 (mirror) and 10 (AKA RAID 1+0, stripe of mirrors) allows for failure of up to 50% of the drives in the RAID. The capacity of a RAID 1 or RAID 10 set is half of the total drive capacity.

3Ware sells controllers that are pretty efficient at RAID 6, and allow hot spares (unused drives that can be added automatically to the array if another drive fails). A home-building redundancy nut will probably either use mirrored RAID 5 or RAID 6 ("RAID 51" or "RAID 61"), or RAID 10, with such hot spares.

Redundancy nuts with money will purchase storage solutions from e.g. NetApp or EMC, with backup tape robots and a remote storage vault in a mountain. biggrin.gif


This redundancy nut isn't quite that paranoid currently, I "mirror" manually to two or three different drives, plus two different DVDs, after each shoot. I expect to go move to dual RAID 10 or something like that in a while.
jani
QUOTE (Ray @ Jan 13 2007, 02:22 AM)
As a result, the Chernobyl accident caused a great deal of consternation amongst people living in the vicinity of the accident. Women had abortions, as it now seems, for no good reason at all. As some great American president (or important historical figure) said (who was that?), the only thing we have to fear is fear itself.

Whilst it's true that some people died who were very close to the accident and who received massive doses of radiation (and I hope that I am not seen as trivialising their families' grief), there are lots of people who received moderate amounts of radiation, who've been quaking in their boots for the past 20 years, and who are still as fit as a fiddle.

I am one of those people who have received moderate amounts of radiation because of that accident.

I have, however, not been "quaking in my boots", but others certainly have.

I feel comfortable that the doses of radiation haven't been critically high, and that the chances of me developing cancer aren't significantly higher because of that accident.

QUOTE
An analysis of the wildlife in the close vicinity of the accident revealed very surprising results. The researchers expected to find all sorts of mutant rats and mice with two heads and three ears, or perhaps no wildlife at all. On the contrary, wildlife was thriving without a hiccup. Moderately strong radiation levels just seemed to have bounced off their back. In fact, it now seems that relatively small amounts of radiation, up to 100 millisieverts (per year) are actually beneficial to us. They stimulate our immune response. An analysis of areas in the United States that have strong, natural background radiation, have shown that people who live in such areas have less cancer, statistically, than people who live in areas with low background radiation.

The "strong, natural background radiation" is peanuts compared to the radiation in areas that were close the Chernobyl accident.

Areas closest to Chernobyl, such as the town of Zaborye in Brjansk, Russia, had concentrations of cesium-137 of above 100 curies per square km.

In an area of approximately 10 300 square km areas around Chernobyl, the concentration of cesium-137 is greater than 15 curies per square km, or more than half a million becquerel per square meter.

An additional 28 600 square km in Russia, Ukraine and Belarus measured 5 curies or more per square km.

Certain areas of Norway received concentrations of about 100 000 becquerel per square meter, enough to make lots of reindeer, sheep and mushrooms unfit for human consumption. (Peruse a map if you don't know how far away that is ...)

The half-life of cesium-137 is about 30 years.

Gomel in Belarus, 130 km northwest of Chernobyl, still has a concentration of 1-5 curies per square km.

A total of eight to ten million people still live under conditions that are more radioactive, but it's suspected that tens of millions risk serious health consequences because of radioactive materials transported by water. How great these risks are is debated, however.

(Feel free to read a bit more)
feppe
QUOTE (Ray @ Jan 13 2007, 02:49 AM)
However, in your situation I see the problem. To transfer 1TB of images to DVD would be a mammoth task. Pity you haven't been doing it regularly by degrees. You could then have peace of mind  biggrin.gif .

On the cost issue, I should mention that a LaCie 300GB external hard drive, in Australia, costs about A$1 per gigabyte, maybe a bit less now. Blank DVDs are about 10-20 cents per gigabyte.
*


Mr Briot backs up terabytes of data on DVDs as well, but he's doing just that: copying stuff as he goes. This is IMO the only sane way to do DVD backups. But that doesn't solve the issue of verifying their integrity every few years, something I don't consider paranoid at all. I'm in a position to choose my backup regimen as I don't have size-hungry (photographic) backups dating back years, and currently HDDs are the most convenient.

I just checked prices here in Europe. 3.5" hard drives cost ~€0.20 per gigabyte (with €20-50 for the external casing, bringing an external 500GB drive to 25 eurocents per gig) while DVDs cost €0.10 in 100 DVD spindles - their price has gone down drastically here in the past 6 months. So you were right, DVDs are cheaper. But if one puts any money on time spent backing up and verifying it's a no-brainer.

BTW, I doubt hard drives cost three times as much in Australia than in already expensive Europe as your numbers suggest. Perhaps you're looking at the 2.5" prices or prices of some ultra-fast server drives?
kaelaria
WOW I can't believe the level of absolute PARANOIA around here...lol!! Either that or some of you should have auditioned for Monk smile.gif

Some of you can find a theoretical fault with anything, so there IS NO solution for you, pure and simple.

Here in the corporate IT world we use the time tested methods, and are only bound by our departmental budgets.

My web, file and SQL servers are raid 5 arrays with monthly master and daily incrimental tape backups. OMG the raid card may go out! OMG the SCSI card could go out! OMG a meteor could crash through the roof and kill the whole system!!

At home I backup critical files one and a while from my raid array to an external drive that's only powered on for transfers. OMG that might break too!!!

You guys sure are entertaining when you try and give computer advice smile.gif
jani
QUOTE (feppe @ Jan 13 2007, 01:35 PM)
I just checked prices here in Europe. 3.5" hard drives cost ~€0.20 per gigabyte (with €20-50 for the external casing, bringing an external 500GB drive to 25 eurocents per gig) while DVDs cost €0.10 in 100 DVD spindles - their price has gone down drastically here in the past 6 months. So you were right, DVDs are cheaper. But if one puts any money on time spent backing up and verifying it's a no-brainer.

Since you're using harddrives, you'll be replacing them every three years or so anyway.

And for integrity's sake, let's assume that you're using RAID 1 or 10 (dual DVDs the competition).

Hard disk densities roughly quadruple every three years for the same price at a certain price point (around €100-200), let's assume that DVDs have a similar price curve, but last at least 20 years if stored properly.

If you shoot 10,000 images of 15 MB each year, and that you now currently have three year's worth of images, that's 450 GB of storage, requiring 2x500 GB drives at roughly €140 each. Then you need a RAID controller, that costs €37 for a four-port controller. (Prices from komplett.nl.)

In three years, you'll need another 2x1000 GB drives at €100 each (because prices don't drop that fast). In three more years, you'll need 2x1500 GB drives at €70 each, if prices for drives keep dropping. In three more years, 2x2000 GB at €50 each.

Hardware cost twelve years, presuming that you change no other components: €720.

DVD writer: €28
200 DVDs €174 + 200 DVDs €120 + 200 DVDs €90 + 200 DVDs €70.

DVD cost, presuming that you change no other components: €442.

But the drives use about 1.5 W in standby mode each. Over twelve years, that adds up to about 120 kWh, if they're turned on 25% of the day. Add 25% power supply overhead, and it's another €20. smile.gif

So, basically, hard drives are not that much more expensive that it would matter much financially as long as we're sticking to a simple mirrored setup. But if you want your working storage to be online and available, the cost doubles again.

The problem for hard drives is remote location storage of backups, where DVDs win for convenience of transport. If you have access to networked storage, hard drives win on convenience, too.

And arguably, it's easier to verify the data integrity with hard drives.

But given the very small additional cost -- both in time and money -- of creating DVD backups as you go, it is prudent to do both: redundant hard drives and redundant DVDs.
Ray
QUOTE (feppe @ Jan 14 2007, 10:35 AM)
BTW, I doubt hard drives cost three times as much in Australia than in already expensive Europe as your numbers suggest. Perhaps you're looking at the 2.5" prices or prices of some ultra-fast server drives?
*


I'm looking at prices at http://www.computeralliance.com.au/parts.aspx?qrySubCat=HBK which is a local computer store in Brisbane with some of the best prices in town. Internal hard drives vary from around 40 cents to 70 cents per GB, but external hard drives are more expensive, ranging from a bit under a dollar per GB to a bit more than a dollar. A$1 is 0.60 Euros. There are always a few run-out models that are on special and might bring those prices down a bit.

Currently in that store, the best value DVDs are by LG, 16x in spindles of 50 for A$19. That's just 8.4 cents per GB.
larryg
Strange things happen (what you least expect some times)

Our Novel system crashed. The Novel program itself. We have Raid (not sure which one) but the backup software only backed up the data (every night and on Friday with offsite storage) But since the system itself (Novel) crashed we couldn't get anything going.

Still working on restoring the Novel System so that we can get the whole system back up.

Redundancy seems to be essential in any arrangement. Backup Backup and have other supporting systems that will preserve the integrety of the data and the systems.

I Currently: Have a Raid system (raid 0) then backup on an external Maxtor
that stays on site another backup on Maxtor goes home (along with archival DVD Gold backups at home).

I can see moving to a 1.5 TB raid system with other drives as backup to that system.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2010 Invision Power Services, Inc.