NIWA's data for download (was MORST event on Publicly-Funded Research Data)

Hi,

NIWA is in the process of making more of it's data holdings publicly available,
but users need to be aware of a few issues:

As a commercial business, NIWA undertakes a variety of research projects for
many organisations, in addition to tax/ratepayer funded ones. It manages data
from these projects on behalf of these clients. NIWA is not able to make such
data available to the public as it is not NIWA owned data. Even when a
local/regional/central govt dept is the data owner, on behalf of
tax/ratepayers, NIWA cannot make such data available without the client's
approval, generally not without the client contracting NIWA to make these data
available.

Some data owned by NIWA has been released to the public, under a no
guarantees/no liability "licence". NIWA has found that various businesses
build such data availability into their daily business practices, and hold NIWA
to task when such data becomes unavailable for any reason. NIWA is still
working around such issues, where expectations become, from some perspectives
at least, unreasonable for a free, no guarantee source.

Much of NIWA's data is derived directly from remote instruments. These can
fail, or generate invalid/incorrect data for a variety of reasons, so a range
of business rules and practices are put in place to ensure data quality is
maintained. This can include delays in the release of such data, which annoy
some potential users who desire more real time data. What level of corporate
responsibility is appropriate before such data is released is a contentious
issue, with differing perspectives often presented.

I'm presenting some details at a couple of venues next week on behalf of a NIWA
client about a NIWA web portal being developed to provide data to the public,
so there are ongoing moves to make more datasets & reports available to the
public.

Note these opinions are personal, & I'm not speaking on behalf of NIWA or
anyone else here :-)


Cheers,

  Brent Wood

--- On Sun, 7/11/10, Tim McNamara <email obscured>> wrote:

> From: Tim McNamara <email obscured>>
> Subject: [ninjas] NIWA's data for download (was Re: MORST event on
Publicly-Funded Research Data)
> To: <email obscured>
> Date: Sunday, July 11, 2010, 10:04 AM
> Morning Andrew,
>
> Thanks for that response.
>
> On 11 July 2010 09:39, Andrew Watkins <email obscured>>
> wrote:
>
> > CRIs like NIWA are expected by the government to pay
> for themselves as
> > much as possible.  It is sad to see how much time
> scientists have to
> > spend in counting hours, project management, finding
> funding etc.
> >
> > Even so practically all data collected by NIWA is
> available to the
> > public. The example given for Buller river flow is
> indeed a chart -
> > however data is also available for download from the
> same page -
> > unfortunately not real time enough to run the service
> that Jonathan
> > wants.  Almost all other physical measurements
> are available from the
> > Climate database in near real time.
> >
>
> I thought I would add a few comments from a my experience
> of that page. I
> wonder if you could add them to your long list of things
> that might be good
> to look at.
>
> *Please login*
> I receive the following error message, but there is no
> login/registration
> link on the page.
>
> "Download data
>
> If you were logged in you could download this data."
>
> *Licencing*
>
> At the bottom of that page, there is a notice that NIWA is
> the owner of the
> copyright to the page (and by inference the data). However,
> there's no
> indication of which licence those those copyrighted works
> are licenced
> under.
>
> Regards,
>
> Tim McNamara
>
> -----------------------------------------
> Full text of this topic in The Open Government Ninjas:
> http://groups.open.org.nz/r/topic/1wuUhbxToZytCGMFaZvPDF
>
> To leave The Open Government Ninjas, email
> <email obscured>?Subject=unsubscribe
>
> Start your own free groups and site with
> OnlineGroups.Net http://onlinegroups.net
>
> Host your own online groups site with
> GroupServer http://groupserver.org
>

On Sun, July 11, 2010 10:40 am, <email obscured> wrote:
> As a commercial business, NIWA undertakes a variety of research projects
> for many organisations, in addition to tax/ratepayer funded ones. It
> manages data from these projects on behalf of these clients. NIWA is not
> able to make such data available to the public as it is not NIWA owned
> data. Even when a local/regional/central govt dept is the data owner, on
> behalf of tax/ratepayers, NIWA cannot make such data available without the
> client's approval, generally not without the client contracting NIWA to
> make these data available.

I understand the issue you raise: my question is how can we move beyond that?

As with Digital NZ, some data is made available, but other data is
withheld because it requires additional permissions. It's not clear if
anyone is asking the questions up front, so how can we educate the
stakeholders to ensure public access and permissive licensing is baked in
from the beginning, and to actively revisit existing services with a view
to making the licensing less restrictive?

It seems to me that clear central government leadership is required to set
the agenda on this, similar to the statements from USA, UK and Aus. There
is only so much a 'grassroots' effort will achieve.

Regards
Jonathan
--
http://huntdesign.co.nz

On Sun, Jul 11, 2010 at 4:41 PM,  <email obscured>> wrote:
> I was pleasantly surprised recently...
> Lobbying govt to support such directions, be it via local MP's, or at
Ministerial level is working. Open Data intiatives in NZ are
> generally newer than similar Open Source ones, but have a closely related
focus, and could learn & benefit from some of the
> lobbying that the NZOSS & others have done over recent years (right
Nate/Peter/etc?)

Since I'm in this forum perhaps I should talk briefly about why I'm
here, because some might conclude I am here to push open source, and
using "Open Government" as a hammer for this goal. This isn't the
case.

My involvement in "Open Government" actually goes back to 1990 when I
joined the New Labour Party. I didn't join because I was a "leftist",
but because they opened the policy development process to members. The
idea that a average person could actually participate in developing
policy was exciting. It was also an unmitigated disaster, with many
loonies and extremists using the opportunity to push their agenda.
After seeing this process quickly close down I left the party. A few
years after I joined Tony Cooke's NZ Super Democratic Party, which
proposed phone referendums.

http://crash.ihug.co.nz/~tonycook/nzsdp/

However, I didn't find Tony understanding of new technology. He
rejected the idea of the Internet being used, as it was not
universally available. His following was also very limited, and I
realised that for participatory Government to be a success would
require mainstream acceptance by the major parties. The idea of
participatory democracy is not new, but before the Internet it would
have been academic. The Net has made it possible for practical
participatory democratic processes that simply could have not existed
before.

However, we must start to think beyond confrontational party politics.
I always hear that Democracy may not be perfect, but it's the best way
we know. Well we have another model; one that does not supplant or
replace democracy, but rather augments it. The model is the Scientific
Method; the process of using evidence and analysis with peer review to
distinguish good policy from bad.

And for this model to work everyone must have access to the raw
materials of good policy - data. Open data is not an end in itself, it
is a stepping stone to better policy and laws. While open source
software has a role, I don't believe we should over emphasize this
role to the detriment of the larger goal; engendering a participatory
Government. If laws and policy are developed thought these mechanisms
- ones that do not discriminate based on crude labels such as "left"
and "right" - we will see ideas from a far broader range of
perspectives.

Voting is a terribly blunt instrument. It gives you the power to
remove a whole party, but rarely the ability to reverse bad policy.
Participatory Government based on the Scientific Method would allow
input from the public, even from "interested parties", but would weigh
such input based on the strength of evidence and facts. Ultimately
Parliament will be the ones to make the decisions, but they will be
far better armed with the facts and evidence than ever before, and
will be deciding between policies that have had far wider review and
critique.

This is our future. Let's own it :)

Peter Harrison