All posts in the topic Unstructured open data (Short link)
Summary
- There are 4 posts — by 4 authors — in this topic.
- Latest post made by Glen Barnes at 2009 Jun 17 16:26 NZST
We should not forget government unstructured open data. Matthew Ross has been working on a standard for publishing alerts, job vacancies, etc. http://research.elabs.govt.nz/new-zealand-government-feed-standard-2009/ This is the type of information that you'd need to populate something like http://www.everyblock.com/, if it has a geocode field. What agencies need, are tools that support the concept. (Typically the people that produce the content don't control the system or tools). So perhaps an add-on to an open source CMS used by govt?
I'm working to establish a convention within govt for exposing datasets (along with other types of information). This would deliver a notification/discovery mechanism that any crawler/portal/aggregator could read. This would enable a open system that is not dependent on human effort and does not go stale over time. Feeds are a suitable technology for this. For example, this feed notifies the existence of a [supposed] dataset. /_assets/http/eg.elabs.govt.nz/feeds-2009/dataset-feed.xml And here's an example portal demonstrating aggregation of such notifications (click 'View', click on the item row): http://nz.elabs.govt.nz/ Additional/optional elements could be proposed and added over time. The Atom Category element would convey the values. CMS plugins could assist agencies publishing such feeds. Interested to hear feedback from the group. - Matt
There was also some work done through the Web Standards wiki on publishing spatial information that may have some relevance. <http://www.gis.org.nz/wiki/Government_Geospatial_Information_Web_Access_Guideline > Having a consistent namespace for publishing is a good start. Although I'd recommend against separating spatial from other forms of data, hence would like to see /geospatial deprecated in favour of /data. It could be that the document from the index of /data could provide a link to the feed listing data sources. One other thing that would need inclusion in a standard is a consistent use of terminology, particularly if Atom Category is used. Last thing we'd want is each agency creating their own slightly different terms from others. Cheers Gav
Looks good. When a department goes online with a data feed let me know and I'll write a scraper/updater for the ODC. If you could make sure that it incorporates as many of the fields tat we have here - http://opengovt.org.nz/cat/suggest-other-datasets/ that would be great. It would be nice to have all of this auto updating. Can you post any technical docs on the spec as I have had a few enquires from third party data suppliers who want to list on the ODC and I'm going to create another couple of links "SItes that Re-Publish this data" and "Sites that use this data". This will highlight the use of these datasets "in the wild". It may be better to use your spec instead of re-inventing the wheel.