Re: Wanted: volunteers with bandwidth/storage to help save climate data

[DaKnOb <daknob.mac () gmail com>]

We are currently working on a scheme to successfully authenticate and verify the integrity of the data. Datasets in 
https://climate.daknob.net/ are compressed to a .tar.bz2 and then hashed using SHA-256. The final file with all 
checksums is then signed using a set of PGP keys.

We are still working on a viable way to verify the authenticity of files before there are tons of copies lying around 
and there’s a working group in the Slack team I sent previously where your input is much needed!

Thanks,
Antonios 

> On 16 Dec 2016, at 18:30, Ken Chase <math () sizone org> wrote:
>
> Surfing through the links - any hints on how big these datasets are? Everyone's got
> a few TB to throw at things, but fewer of us have spare PB to throw around.
>
> There's some random #s on the goog doc sheet for sizes (100's of TB for the
> landsat archive seems credible), and there's one number that destroys
> credibility of the sheet (100000000000 GB (100 ZB)) for the EPA archive.
>
> The other page has many 'TBA' entries for size.
>
> Not sure what level of player one needs to be to be able to serve a useful 
> segment of these archives. I realize some of the datasets are tiny (<GB)
> but which ones are most important vs size (ie the win-per-byte ratio) isnt indicated.
> (I know its early times.)
>
> Also I hope they've SHA512'd the datasets for authenticity before all these
> myriad copies being flungabout are 'accused' of being manipulated 'to promote
> the climate change agenda' yadda.
>
> Canada: time to step up! (Cant imagine the Natl Research Council would do so
> on their mirror site, too much of a gloves-off slap in the face to Trump.)
>
> /kc
>
>
> On Fri, Dec 16, 2016 at 06:02:46PM +0200, DaKnOb said:
> > If you???re interested, there???s also a Slack team: climatemirror.slack.com
> >
> > You can find more info about that here:
> >
> > - https://climate.daknob.net/
> > - http://climatemirror.org/
> > - http://www.ppehlab.org/datarefuge
> >
> > Thank you for your help!
> >
> >
> > > On 16 Dec 2016, at 17:58, Rich Kulawiec <rsk () gsp org> wrote:
> > >
> > > This is a short-term (about one month) project being thrown together
> > > in a hurry...and it could use some help.  I know that some of
> > > you have lots of resources to throw at this, so if you have an
> > > interest in preserving a lot of scientific research data, I've set
> > > up a mailing list to coordinate IT efforts to help out.  Signup via
> > > climatedata-request () firemountain net or, if you prefer Mailman's web
> > > interface, http://www.firemountain.net/mailman/listinfo/climatedata
> > > should work.
> > >
> > > Thanks,
> > > ---rsk
>
> -- 
> Ken Chase - math () sizone org Guelph Canada