I just wanted to draw attention to this great set of videos that give a great overview of one of the first large scale Geographical Information Systems, in this case the Canada Geographic Information System (CGIS). Dr Roger Tomlinson was the initiator, planner and director of this for the time very ambitious project, which meant that he has gone down in history as the “father” of GIS.  I feel quite proud to note that he wrote his PhD thesis here at University College London!

From the Wikipedia article:

The Canada Geographic Information System (CGIS) was developed in the 1960s and 1970s to assist in regulatory procedures of land-use management and resource monitoring. At that time, Canada was beginning to realize problems associated with its seemingly endless boundaries, in combination with natural resource availability. The government therefore decided to launch a national program to assist in management and inventory of its resources. The simple automated computer processes designed to store and process large amounts of data enabled Canada to begin a national land-use management program and become a foremost promoter of geographic information systems (GIS).

Just saw this post on the forum. After a long time where the idea of a OpenStreetMap ISI Driver for Manifold was thrown around on the forum, James K. finally stepped up to the plate!
James did a stellar job, not only generating a ISI Dll for the default OSM Mapnik Tile Layer, but also adding the Tiles@Home and Cloudmade Cycle Map. The Dll works fine in both x32 and x64, and finally allows map makers to quickly and easily add a high quality OSM background map to their products.

osm_isi_dr2

This is a significant step for map makers, as the OSM tiles offer a level of freedom in terms of usage license. Altough a acknowledgement of the OSM license is needed in any mapping products which contain any OSM data, this is a simple requirement when compared with the grey and murky licensing requirements for Google, Microsoft Virtual Earth and Yahoo tiles.

It remains to be seen with the imminent update to 9.0, if Manifold.net are going to support OSM XML as a Import/Export format. Altough there are existing scripts to enable the import of OSM XML data, these scripts have issues and don’t scale efficiently to bulk downloads. A native solution hopefully would enable bulk imports and the ability to save (or even upload directly to OSM) to OSM XML for update of OSM.

PS: AFAIK planet.osm, representing a database dump in XML form of the whole OSM dataset, is one of the largest publicily available vector datasets out there (over 364 million elements). Import and display of such a huge dataset could be a great benchmark test for upcoming versions of Manifold!

There have been numerous discussions in the past in the Manifold User Forums regarding a lack of (exploratory) spatial analysis/statistics tools, such as measures and visualisations, in the Manifold GIS. Altough it seems likely that a number of users have been sending in detailed suggestions for spatial analysis functionality, so far it seems that Manifolds development focus has been devoted to other more fundamental areas such as interfacing with spatial databases and the ability to efficiently use multithreading and CUDA.

Given that in the short to medium term, we most probably won’t see the integration of significant spatial analysis functionality into Manifold, a pragmatic approach is the integration of external software packages with Manifold. There are a number of software packages that offer spatial analysis and statistics capabilities such as for example Crimestat, Geoda and the R project. R, an open source project benefits from a wide support in academia as a platform for the implementation of statistical computing, and thus provides a very rich environment for the analysis of spatial data through a combination of free packages. R is a command-line environment, and although the syntax is relatively accessible, it does present a significant learning curve for any beginners.

Recently, my research project led me to investigate the spatial distribution of foreign investors into London. I needed to do a density analysis of historic investment patterns to identify likely agglomeration or dispersion processes between investors. Although Manifold doesn’t offer any relevant density estimation algorithms, R, and specifically the spatstat package, allows for the creation of Kernel Density Estimation (for the estimation of density) grids.

I took the opportunity to write a script that gives users a point and click front-end to both the kernel smoothed intensity function from a point pattern (KDE), and spatial smoothing (interpolation) of numeric values observed at a set of irregular locations (GKS) from inside Manifold.  The script takes care of building and maintaining the interface between Manifold and R, running the analysis in the background and creating a result surface component. I must acknowledge here the help and inspiration of numerous users on the forum which have been working with R.

Screenshot example of interface and output

Screenshot example of interface and output

Although R is completely hidden from the user of the script once installed, the successful installation relies on a basic understanding of the concepts of R and the installation of a few prerequisite software tools and R packages. Along with R , you need to have installed R(D)Com A all in one package for R and R(D)Com can be found at statconn. You also need a C:\temp\ directory (temporary files are stored there). You also need to have the following R packages installed:

  • spatstat, the main analysis package
  • maptools, helper package for the conversion of data to a spatstat compatible format
  • rgdal, package allowing the import and export of data from R to Manifold

I strongly advise anyone wanting to use this script to first read and understand the algorithms and outputs involved by consulting the relevant help pages from the spatstat package.  I also include two datasets that are suitable for experimentation with the script, one each for KDE and GKS.

Finally, there are some caveats to this script. I do not make any guarantees as to the output of the script, and I have to repeat that you need to have an understanding of what the algorithms do to fully comprehend the analysis.  Also, the script at the moment doesn’t take account of projections at all. I have personally only tested the script with projected point patterns (British National Grid). In the case of BNG, assigning the BNG projection to the created Image (Preserve Local Values ticked) should be sufficient. Your mileage may vary when using other projections.

You can find the download of the Manifold .map file with the script here!

As you may have guessed, this is only a first stab at the integration of Manifold with R, and it is still an unlikely couple which sometimes can have communication difficulties. Clearly is a lot of work left in integrating other most basic functionality, a few examples of functionalities being other interpolators such as IDW, Geographically Weighted Regression, LISA and Moran’s I … .

But this proof of concept shows the potential for added functionality to boost Manifold’s power from a pure GIS to an exploratory spatial statistics toolset.

Just saw this press release on the Manifold website:

Carson City, NV USA — 16 March 2009 — Manifold.net today announced a new world record for the number of processors used in a personal computer for Geographic Information Systems (GIS) processing. At the company’s 2009 European User Meeting in London, Manifold demonstrated an upcoming new software product that simultaneously utilized over 1440 processor cores to perform a remote sensing image computation at supercomputer speed with over 3.5 teraflops of performance. Manifold demonstrated the new software on a desktop 64-bit Windows PC equipped with three NVIDIA GTX 295 GPU cards costing less than $500 each. (Illustration at right shows the demonstration hardware.)

Wow, I didnt know that we witnessed a world record at the User Meeting back in February here at UCL! Well I am glad that Manifold came and did their demo of Release 9.0, even though they did so in their usual hyberbole style.

Also, good to see that they are showing commitment to a release date around June. This should imply a beta starting in the next weeks…!!!

heatmapapi

HeatMapApi.com is a new service which allows Google Maps mashups to integrate heat map representations easily. Heat maps,  or more generally point to raster interpolations allow the graphical representation of point patterns through the use of continuous colors identifying areas of higher or lower density of points. Areas where this has been employed are crime hotspots analysis or economic activity analysis.

A novel concept in Web 2.0 mashups, I was interested in finding out what the methodology was behind the generation of these rasters. Sadly, I couldn’t find a definitive answer on the algorithm that the website uses to generate its hotspot maps. They do expose in their API two variables that can influence the generation of the heatmaps, decay and boost, but without information on the algorithm behind it, the setting of these values remains a pure exercise in trial and error, and seeing what “looks” best. Also, because the parameters are set as “optional”, most developers will be tempted into a one size fits all approach, smoothing out interesting patterns in the data, or creating hotspots that are not statistically viable, creating masses of effectively meaningless maps.

Mashup developers thus will more than ever the spatial analysis literacy skills to understand the processes, models and algorithms that lie behind the pretty maps.

Note: This is not a new problem, but has been present all through the development of Exploratory Spatial Data Analysis over the past 20-30 years in academia and commercial settings, and a lot can be learned from this past experience.

About this blog

My name is Patrick Weber and I am a Engineering Doctorate Student at University College London. I also work for Think London as GIS Project Manager. On this site, you can find more information about me, my professional activities and my academic achievements, or have a look at my LinkedIn Profile.

-->
  • Rob: Summer 2007 also marks the start of the credit crunch and 'global downturn' - smaller budgets, fewer [...]
  • Muki Haklay: Hi Patrick, I like this analysis, and I would throw few other points into the thinking: 1) Another [...]
  • gustavo palminha: Hello. Its an interesting analisys, i suspected that tyou would reach results like this. we do not [...]
  • miken: scruss, common. "expensive joke", "shock-troops" and not using "layers". I have been trained in E [...]
  • Vincent Fréchette: I think this is a very good reading of the situation. [...]