Posts in The Open Government Ninjas
- Later 20 posts
- Earlier 20 posts
Chch earthquakes generated 660 GB of GIS data that everyone needed now - how NZ te
From:
Nick Jones
Date:
Feb 01 14:06 NZDT
Tim - happy to chat about the setup. The service is based on a distributed file system called iRODS[1] and we were using the parallel streaming / pipelining features within GridFTP[2]. (for those interested in the tech, this is the technology behind the SaaS platform for transferring large scientific datasets, GlobusOnline[3]. We're currently in planning for a transition and expansion of this service, into NeSI[4], and with additional research institutions connected, heading towards a national distributed file system for research. There was an issue at the start as there were breakages in the KAREN network due to the quake and a hub in Christchurch feeding some of our research institutions with campuses elsewhere in NZ. As Gavin notes, TB drives were used initially. Once we had this resolved, we were routinely distributing data between multiple parties across the country, between Auckland, Napier, Palmerston North, and Christchurch. This routine sharing and processing went on for many weeks as different data sets were acquired and processed. re comments on licensing, this was a difficult one to navigate. As a service provider, we provided the security that was initially required, as defined by the license for the original satellite image data negotiated by the involved agencies. In the end licensing was resolved and the data has been available since around March 2011[5]. cheers, Nick [1] http://www.irods.org [2] http://www.globus.org/toolkit/data/gridftp/ [3] https://www.globusonline.org/ [4] http://www.nesi.org.nz [5] http://df.auckland.ac.nz/BeSTGRID/home/NZDF/
Interesting national resource, shame we don't have something similar...
From:
Glen Barnes
Date:
Jan 31 13:33 NZDT
The goal of the first data.govt.nz was to get things out there and working using an MVP/JFDI approach. This worked well to start with but I guess somewhere along the lines things stagnated. Maybe with a renewed focus on this there can be some resources put in place to start building a constantly evolving data catalogue? In my mind whatever we build needs to cover 2 use cases - People who are browsing and don't know much about data and those who need the hardcore. They shouldn't be mutually exclusive but it will take some pretty talented UX and dev peeps to get it done. I would be interested to find out what the mindset is in government to sharing resources and building something decent on one of the open source catalogue stacks. Glen On Sunday, 29 January 2012 at 2:48 PM, <email obscured> wrote: > Perhaps DOC, MfE, MAF, CRI's, Regional Councils, etc., could collaborate on a national resource such as this, rather than developing multiple limited focus (sometimes overlapping focus) catalogues, which seems to be the current trend. > > http://www.the-eis.com/aboutus.php > > While it is certainly easier to focus & fund the development of small bite-sized domains, these should form part of a cohesive national strategy & system. A lack of focus on the overall national picture will lead the results of small, domain specific efforts into limited value & obscurity. Much as described in the recent data.govt.nz review. > > NZ should at least agree on a common vocabulary for such catalogues & systems before going too much further... such as the GEMET list or the GCMD vocabularies, etc., to try & ensure such systems will at least have some level of interoperability. > > Someone with a national strategic role in geospatial data management could perhaps provide some valuable leadership here. Hint :-)
> > > Cheers, > > Brent Wood > > > > > ----------------------------------------- > Full text of this topic in The Open Government Ninjas: > http://groups.open.org.nz/r/topic/1KvxE4X9qsPZgmYYeRh2Ch > > To leave The Open Government Ninjas, email > <email obscured>?Subject=unsubscribe > > Start your own free groups and site with > OnlineGroups.Net http://onlinegroups.net > > Host your own online groups site with > GroupServer http://groupserver.org > >
Interesting national resource, shame we don't have something similar...
From:
jforne
Date:
Jan 31 12:30 NZDT
Thanks Brent Good point. I agree, we need a way(s) of ensuring we're talking about the same thing within and across domains. And controlled vocabularies seem to provide a way of achieving this semantic interoperability. However, in order for controlled vocabularies to be of more (maximum?) value - it is appropriate that they are agreed across a variety of domains, including the geospatial domain. Consequently, I propose that the (cross domain) Open Government Data and Information Reuse Working Group is a more appropriate forum for discussing/agreeing a controlled vocabulary for catalogues and systems (rather than limiting to the Geospatial Steering committee or the geospatial domain). I will raise this item with Keitha and Kevin to table with the Working Group accordingly. While controlled vocabularies are generally of interest to all data management communities, New Zealand Geospatial Office (NZGO) has a particular interest in the aspect of a controlled vocabulary that defines how we describe where things are, namely a gazatteer. To this extent I propose that NZGO's role is to bring all the parties with an interest in developing a gazatter service together. I have discussed this with Wendy Shaw (Secretary, New Zealand Geographic Board Ngā Pou Taunaha o Aotearoa) and Richard Fone (project manager, LINZ) and note that LINZ is developing a gazatter service that we hope to be accessible mid-2012. I am currently seeking a revised document describing the user requirements and have been advised that I can email this to you shortly. And in terms of the opportunity for interested parties, such as you, to get involved in the development of the gazatter, Wendy/Richard agreed that it would be good for you to participate during the review of the gazatter during the pilot phase. I trust this helps, John.
Interesting national resource, shame we don't have something similar...
From:
Brent Wood
Date:
Jan 29 14:48 NZDT
Perhaps DOC, MfE, MAF, CRI's, Regional Councils, etc., could collaborate on a national resource such as this, rather than developing multiple limited focus (sometimes overlapping focus) catalogues, which seems to be the current trend. http://www.the-eis.com/aboutus.php While it is certainly easier to focus & fund the development of small bite-sized domains, these should form part of a cohesive national strategy & system. A lack of focus on the overall national picture will lead the results of small, domain specific efforts into limited value & obscurity. Much as described in the recent data.govt.nz review. NZ should at least agree on a common vocabulary for such catalogues & systems before going too much further... such as the GEMET list or the GCMD vocabularies, etc., to try & ensure such systems will at least have some level of interoperability. Someone with a national strategic role in geospatial data management could perhaps provide some valuable leadership here. Hint :-) Cheers, Brent Wood
Looking for NZ government Open Data publishing standards/procedures
From:
Jeff Rothenberg
Date:
Jan 27 14:34 NZDT
Thanks! I'll follow up on this soon. Jeff On Jan 26, 2012, at 12:33 PM, Alison Stringer wrote: > Hi Jeff > > I work for the NZ Open Goverment Data and Information Secretariat. We are currently preparing guidance and advice to assist NZ Public Sector agencies release their high value, non-personal data for reuse. > > My colleague Keitha Booth and I are happy to discuss the New Zealand Open Goverment Data and Information work programme. Please contact Keitha or on [last name] [first initial] [at] linz.govt.nz > > Regards > Alison Stringer > > PS: Happy New Year Ninjas! I'm working with Keitha 0.5 FTE this year. Looking forward to working with you all.
> A > > > > > > ----------------------------------------- > Full text of this topic in The Open Government Ninjas: > http://groups.open.org.nz/r/topic/VpQ8d8bqa80Y5Ttz5v3N5 > > To leave The Open Government Ninjas, email > <email obscured>?Subject=unsubscribe > > Start your own free groups and site with > OnlineGroups.Net http://onlinegroups.net > > Host your own online groups site with > GroupServer http://groupserver.org
Looking for NZ government Open Data publishing standards/procedures
From:
Alison Stringer
Date:
Jan 27 09:30 NZDT
Hi Jeff I work for the NZ Open Goverment Data and Information Secretariat. We are currently preparing guidance and advice to assist NZ Public Sector agencies release their high value, non-personal data for reuse. My colleague Keitha Booth and I are happy to discuss the New Zealand Open Goverment Data and Information work programme. Please contact Keitha or on [last name] [first initial] [at] linz.govt.nz Regards Alison Stringer PS: Happy New Year Ninjas! I'm working with Keitha 0.5 FTE this year. Looking forward to working with you all. A
Looking for NZ government Open Data publishing standards/procedures
From:
Jeff Rothenberg
Date:
Jan 26 11:37 NZDT
Thanks! I'll digest the Declaration and will then contact you about following-up. Jeff On Jan 25, 2012, at 1:06 PM, Nadia Webster wrote: > Hi Jeff > > I am currently working on developing a process and guidance for the New Zealand Department of Internal Affairs on the prioritisation and release of high value public data for reuse, in accordance with the Declaration on Open and Transparent Government http://www.ict.govt.nz/programme/opening-government-data-and-information/open-and-transparent-government
> > You can contact me: [my first name].[my last name][at]dia.govt.nz > > Nadia > ----------------------------------------- > Full text of this topic in The Open Government Ninjas: > http://groups.open.org.nz/r/topic/3WBpxYCMtIeyLFLWSSCvBA > > To leave The Open Government Ninjas, email > <email obscured>?Subject=unsubscribe > > Start your own free groups and site with > OnlineGroups.Net http://onlinegroups.net > > Host your own online groups site with > GroupServer http://groupserver.org
Looking for NZ government Open Data publishing standards/procedures
From:
Jeff Rothenberg
Date:
Jan 26 11:36 NZDT
Thanks! I'll check out the DANS group and will try to set something up to talk with you about science data issues next week. Jeff On Jan 24, 2012, at 8:37 PM, David Medyckyj-Scott wrote: > Hi Jeff, > > Have you talked to the people at the Data Archiving and Networked Services (DANS) in the Netherlands? They would provide some useful input I am sure. > > Happy to talk to you offline about the issues, from a science data providers viewpoint, of working within the NZ content.
> > Regards > > David > > __________________________________________________________________ > > Dr David Medyckyj-Scott > Team Manager - Informatics > Landcare Research - Manaaki Whenua > New Zealand > > tel: 00 64 (0)6 353 4979 > email: <email obscured> > web: http://www.landcareresearch.co.nz/ > Team blog: http://wiki.zen.landcareresearch.co.nz/informatics/ > __________________________________________________________________ > > > > -----Original Message----- > From: <email obscured> <email obscured>] On Behalf Of Jeff Rothenberg > Sent: Wednesday, 25 January 2012 1:41 p.m. > To: <email obscured> > Cc: Jeff Rothenberg > Subject: [ninjas] Looking for NZ government Open Data publishing standards/procedures > > Dear Ninjas, > > I am working on a small project for a Dutch agency that is interested in improving the publication > and distribution of open data by the Dutch government. The stated goal of this project is to: > > Improve the supply and distribution process of government open data in The Netherlands > > The focus of the project is on the use of publication formats, data and metadata formats, procedures, > and policies that can make government data and publications easier to discover, access, acquire, > and utilize for the full range of purposes to which they can be put. The intended users of such data > include not just end-users (citizens, organizations, and other government agencies) but also various > intermediate entities, such as developers of applications or online databases that repackage, collate, > integrate, process, or republish such data for end-users. > > I have been asked to perform a small number of small-scale international case studies looking for ways > that other governments are approaching this issue. In particular, I am looking at four English-speaking > cases: the U.K. the U.S., Canada, and New Zealand. > > I am therefore looking for relevant agencies, programs, projects, offices, groups, or individuals who > are engaged in the production, publication, and management of open data in New Zealand, as well > as pointers to any specific standards and policies, approaches, or efforts that might provide useful > insights into the problem and/or models for the Dutch effort. Similar pointers to counterparts in the > U.K., Canada, or the U.S. would also be helpful, if you happen to know of any. > > I would greatly appreciate any individual or agency contacts, websites, issue papers, policy documents, > or standards specifications you can suggest that might help me understand what is being done in New > Zealand (and elsewhere) along these lines. > > > Appreciatively, > > Jeff Rothenberg > Information Science Consultant > www.JeffRothenberg.org > > > ----------------------------------------- > Full text of this topic in The Open Government Ninjas: > http://groups.open.org.nz/r/topic/2Sbcq3hvuXPXGTiO3bggj3 > > To leave The Open Government Ninjas, email > <email obscured>?Subject=unsubscribe > > Start your own free groups and site with > OnlineGroups.Net http://onlinegroups.net > > Host your own online groups site with > GroupServer http://groupserver.org > > Please consider the environment before printing this email > Warning: This electronic message together with any attachments is confidential. If you receive it in error: (i) you must not read, use, disclose, copy or retain it; (ii) please contact the sender immediately by reply email and then delete the emails. > The views expressed in this email may not be those of Landcare Research New Zealand Limited. http://www.landcareresearch.co.nz > > ----------------------------------------- > Full text of this topic in The Open Government Ninjas: > http://groups.open.org.nz/r/topic/3vWGPnbZO4QwEfRWyW6K9L > > To leave The Open Government Ninjas, email > <email obscured>?Subject=unsubscribe > > Start your own free groups and site with > OnlineGroups.Net http://onlinegroups.net > > Host your own online groups site with > GroupServer http://groupserver.org
Looking for NZ government Open Data publishing standards/procedures
From:
Nadia Webster
Date:
Jan 26 10:05 NZDT
Hi Jeff I am currently working on developing a process and guidance for the New Zealand Department of Internal Affairs on the prioritisation and release of high value public data for reuse, in accordance with the Declaration on Open and Transparent Government http://www.ict.govt.nz/programme/opening-government-data-and-information/open-and-transparent-government You can contact me: [my first name].[my last name][at]dia.govt.nz
Nadia
Looking for NZ government Open Data publishing standards/procedures
From:
medyckyj-scottd
Date:
Jan 25 17:37 NZDT
Hi Jeff, Have you talked to the people at the Data Archiving and Networked Services (DANS) in the Netherlands? They would provide some useful input I am sure. Happy to talk to you offline about the issues, from a science data providers viewpoint, of working within the NZ content.
Regards David __________________________________________________________________ Dr David Medyckyj-Scott Team Manager - Informatics Landcare Research - Manaaki Whenua New Zealand tel: 00 64 (0)6 353 4979 email: <email obscured> web: http://www.landcareresearch.co.nz/ Team blog: http://wiki.zen.landcareresearch.co.nz/informatics/ __________________________________________________________________ -----Original Message----- From: <email obscured> <email obscured>] On Behalf Of Jeff Rothenberg Sent: Wednesday, 25 January 2012 1:41 p.m. To: <email obscured> Cc: Jeff Rothenberg Subject: [ninjas] Looking for NZ government Open Data publishing standards/procedures Dear Ninjas, I am working on a small project for a Dutch agency that is interested in improving the publication and distribution of open data by the Dutch government. The stated goal of this project is to: Improve the supply and distribution process of government open data in The Netherlands The focus of the project is on the use of publication formats, data and metadata formats, procedures, and policies that can make government data and publications easier to discover, access, acquire, and utilize for the full range of purposes to which they can be put. The intended users of such data include not just end-users (citizens, organizations, and other government agencies) but also various intermediate entities, such as developers of applications or online databases that repackage, collate, integrate, process, or republish such data for end-users. I have been asked to perform a small number of small-scale international case studies looking for ways that other governments are approaching this issue. In particular, I am looking at four English-speaking cases: the U.K. the U.S., Canada, and New Zealand. I am therefore looking for relevant agencies, programs, projects, offices, groups, or individuals who are engaged in the production, publication, and management of open data in New Zealand, as well as pointers to any specific standards and policies, approaches, or efforts that might provide useful insights into the problem and/or models for the Dutch effort. Similar pointers to counterparts in the U.K., Canada, or the U.S. would also be helpful, if you happen to know of any. I would greatly appreciate any individual or agency contacts, websites, issue papers, policy documents, or standards specifications you can suggest that might help me understand what is being done in New Zealand (and elsewhere) along these lines. Appreciatively, Jeff Rothenberg Information Science Consultant www.JeffRothenberg.org ----------------------------------------- Full text of this topic in The Open Government Ninjas: http://groups.open.org.nz/r/topic/2Sbcq3hvuXPXGTiO3bggj3 To leave The Open Government Ninjas, email <email obscured>?Subject=unsubscribe Start your own free groups and site with OnlineGroups.Net http://onlinegroups.net Host your own online groups site with GroupServer http://groupserver.org Please consider the environment before printing this email Warning: This electronic message together with any attachments is confidential. If you receive it in error: (i) you must not read, use, disclose, copy or retain it; (ii) please contact the sender immediately by reply email and then delete the emails. The views expressed in this email may not be those of Landcare Research New Zealand Limited. http://www.landcareresearch.co.nz
Looking for NZ government Open Data publishing standards/procedures
From:
Jeff Rothenberg
Date:
Jan 25 14:05 NZDT
Thanks! On Jan 24, 2012, at 4:51 PM, Dan Randow wrote: > > Hi Jeff, > > Here are some links to get you started. > > There are some open data stories at <http://opendatastories.org/> and > <http://wiki.open.org.nz/Open_Data_Mini_Case_Studies>. > > NZ's Declaration and Principles on Open and Transparent Government is at > <http://www.ict.govt.nz/programme/opening-government-data-and-information/open-and-transparent-government>.
> And the New Zealand Government Open Access and Licensing (NZGOAL) > framework is at > <http://www.ict.govt.nz/guidance-and-resources/information-and-data/nzgoal>. > > cheers, > > Dan > > -- > @danrandow +64-27-431-4928 +64-3-377-5377 > Chief Wrangler http://onlinegroups.net > > ----------------------------------------- > Full text of this topic in The Open Government Ninjas: > http://groups.open.org.nz/r/topic/3kjrM9Kc2END9zZum1cAQa > > To leave The Open Government Ninjas, email > <email obscured>?Subject=unsubscribe > > Start your own free groups and site with > OnlineGroups.Net http://onlinegroups.net > > Host your own online groups site with > GroupServer http://groupserver.org
Looking for NZ government Open Data publishing standards/procedures
From:
Dan Randow
Date:
Jan 25 13:51 NZDT
Hi Jeff, Here are some links to get you started. There are some open data stories at <http://opendatastories.org/> and <http://wiki.open.org.nz/Open_Data_Mini_Case_Studies>. NZ's Declaration and Principles on Open and Transparent Government is at <http://www.ict.govt.nz/programme/opening-government-data-and-information/open-and-transparent-government>. And the New Zealand Government Open Access and Licensing (NZGOAL) framework is at <http://www.ict.govt.nz/guidance-and-resources/information-and-data/nzgoal>.
cheers, Dan -- @danrandow +64-27-431-4928 +64-3-377-5377 Chief Wrangler http://onlinegroups.net
Looking for NZ government Open Data publishing standards/procedures
From:
Jeff Rothenberg
Date:
Jan 25 13:41 NZDT
Dear Ninjas, I am working on a small project for a Dutch agency that is interested in improving the publication and distribution of open data by the Dutch government. The stated goal of this project is to: Improve the supply and distribution process of government open data in The Netherlands The focus of the project is on the use of publication formats, data and metadata formats, procedures, and policies that can make government data and publications easier to discover, access, acquire, and utilize for the full range of purposes to which they can be put. The intended users of such data include not just end-users (citizens, organizations, and other government agencies) but also various intermediate entities, such as developers of applications or online databases that repackage, collate, integrate, process, or republish such data for end-users. I have been asked to perform a small number of small-scale international case studies looking for ways that other governments are approaching this issue. In particular, I am looking at four English-speaking cases: the U.K. the U.S., Canada, and New Zealand. I am therefore looking for relevant agencies, programs, projects, offices, groups, or individuals who are engaged in the production, publication, and management of open data in New Zealand, as well as pointers to any specific standards and policies, approaches, or efforts that might provide useful insights into the problem and/or models for the Dutch effort. Similar pointers to counterparts in the U.K., Canada, or the U.S. would also be helpful, if you happen to know of any. I would greatly appreciate any individual or agency contacts, websites, issue papers, policy documents, or standards specifications you can suggest that might help me understand what is being done in New Zealand (and elsewhere) along these lines. Appreciatively, Jeff Rothenberg Information Science Consultant www.JeffRothenberg.org
I couldn't agree more. But I think we need more than just open formats - we need open and community derived standards for collecting data, describing it, storing it and making it available. And we need community support and will power to preserve this data long after we're gone.
There are a few links to arguments on both sides here http://poynder.blogspot.com/2012/01/springer-statement-on-us-research-works.html The article says that rwa is attempting to revert the prior law that allows nih to require oa publishing, but the way I ead it, the new proposal would 'forbid thos e mandate a altogether, which is not going back, but making it even worse. I will try to find the link to where the action against rwa is being aggregated.
Fabiana On Jan 21, 2012 9:36 AM, "Peter Harrison" <email obscured>> wrote:
Creative Commons put me onto this story - a bill that will put paywalls up, protecting the scientific publishing industry. http://www.thepetitionsite.com/207/support-the-open-access-movement-stop-the-research-works-act/?cid=FB_TAF
There is an argument that we should store the most important information in a form that is both durable and does not require special devices to read - except perhaps a microscope. Something like Microfiche. However, for structured digital data the need for open formats is critical.
On Thu, Jan 19, 2012 at 2:01 PM, Don Christie <email obscured>> wrote: > On Thu, 2012-01-19 at 13:45 +1300, Mick Crouch wrote: >> I am hoping that at least a few Ninjas will be keen to come along to >> the Future Pefect conference 26-27 March in Wellington. >> >> http://futureperfect.org.nz >> >> Future Perfect 2012: Digital Preservation by Design is all about >> learning how to store, preserve and access digital information. >> >> There is no open government data unless that data is carefully >> preserved first.... > > "...in an open format." > > I think you meant to add. > > Happy to talk about that at your conference :-) > > Cheers > Don > > > > ----------------------------------------- > Full text of this topic in The Open Government Ninjas: > http://groups.open.org.nz/r/topic/3OIksTA2KgUjMo4SzQ9jAi > > To leave The Open Government Ninjas, email > <email obscured>?Subject=unsubscribe > > Start your own free groups and site with > OnlineGroups.Net http://onlinegroups.net > > Host your own online groups site with > GroupServer http://groupserver.org
Of course I did, Don. Of course I did. Long live OOXML!
;-) M -----Original Message----- From: <email obscured> <email obscured>] On Behalf Of Don Christie Sent: Thursday, 19 January 2012 2:02 p.m. To: <email obscured> Subject: Re: [ninjas] Future Perfect 2012 On Thu, 2012-01-19 at 13:45 +1300, Mick Crouch wrote: > I am hoping that at least a few Ninjas will be keen to come along to > the Future Pefect conference 26-27 March in Wellington. > > http://futureperfect.org.nz > > Future Perfect 2012: Digital Preservation by Design is all about > learning how to store, preserve and access digital information. > > There is no open government data unless that data is carefully > preserved first.... "...in an open format." I think you meant to add. Happy to talk about that at your conference :-) Cheers Don ----------------------------------------- Full text of this topic in The Open Government Ninjas: http://groups.open.org.nz/r/topic/3OIksTA2KgUjMo4SzQ9jAi To leave The Open Government Ninjas, email <email obscured>?Subject=unsubscribe Start your own free groups and site with OnlineGroups.Net http://onlinegroups.net Host your own online groups site with GroupServer http://groupserver.org ==== CAUTION: This email message and any attachments contain information that may be confidential and may be LEGALLY PRIVILEGED. If you are not the intended recipient, any use, disclosure or copying of this message or attachments is strictly prohibited. If you have received this email message in error please notify us immediately and erase all copies of the message and attachments. Thank you. ====
On Thu, 2012-01-19 at 13:45 +1300, Mick Crouch wrote: > I am hoping that at least a few Ninjas will be keen to come along to > the Future Pefect conference 26-27 March in Wellington. > > http://futureperfect.org.nz > > Future Perfect 2012: Digital Preservation by Design is all about > learning how to store, preserve and access digital information. > > There is no open government data unless that data is carefully > preserved first.... "...in an open format." I think you meant to add. Happy to talk about that at your conference :-)
Cheers Don
I am hoping that at least a few Ninjas will be keen to come along to the Future Pefect conference 26-27 March in Wellington. http://futureperfect.org.nz Future Perfect 2012: Digital Preservation by Design is all about learning how to store, preserve and access digital information. There is no open government data unless that data is carefully preserved first.
Thanks, Mick
- Later 20 posts
- Earlier 20 posts
Privacy | Acceptable Use | Terms of Service | About OnlineGroups.Net | Contact OnlineGroups.Net
Start an OnlineGroups.Net site for easier email collaboration in your organization.
Powered by GroupServer, the open source web-based mailing list manager.