Rendering local shapefiles with HTML5

Thanks to Tom Carden's javascript functions from his shapefile-js project and the new HTML5 local file access API, it is very easy to load and render any shapefile stored in the client's device.

You can test it here.

If you don't have a shapefile close at hand, get one of these: [China], [Europe], [South America], [United States].

Comments:

  • I have only tested it on Windows 7 + Google Chrome v. 15.x and Mozilla Firefox 8. I'm curious to see if it works on smartphones.
  • Choose a line or polygon SHP file (not a very big one).

Imagen de jsanz

Wow Juan Lucas, that's

Wow Juan Lucas, that's really interesting.

It works perfectly on Firefox on my Linux box with a 2MB shp.

Hi, can plz send the code

Hi, can plz send the code for it

And also quite fast with

And also quite fast with medium size shapes. What bother me is that Java -unlike C++/Qt with webkit renderer engine- has not good support for HTML5/Javascript (AFAIK there is not a mature widget working clossplatform). Wouldn't be nice to have a preview of your shapes in gvSIG before loading them? That would be utterly easy with good HTML5/Javascript support :)

Ubuntu 11.04 + Chromium

Ubuntu 11.04 + Chromium 15.0.874.106

Lo he probado con un shape de 88MB y 37000 features. Es muy rápido. Buen trabajo.

Imagen de mmontesinos

Miguel Montesinos Hi

Miguel Montesinos

Hi Juan,

the speed is really impressive! even with a 6 MB polygon shapefile (on Chrome).

Hi, can u plz share the

Hi,
can u plz share the code with me i am struggling to render the shape file with html5

Imagen de jsanz

Juan Lucas linked it on the

Juan Lucas linked it on the post

Nice job, Juan Lucas. It

Nice job, Juan Lucas. It works like a charm!
Cheers,
Antonio

Wow, it's very nice, great

Wow, it's very nice, great job!

Victor

I get your code. It's very

I get your code. It's very clear and I start to play with it.
But I think there is an issue in the rendering (I saw the same issue with other shape renderer).
I try with the map of Iceland and it seems really stretch.
So I thought that in shape file, coordinate are in latitude and longitude. The issue is that the more you go to the pole and the more the longitude are close from each other.
So I add this :

var DegToRad = 3.1416 / 180;

var fx = Math.cos(ring[k].y * DegToRad);
ring[k].x *= fx;

for each ring, and it's seems better.
Tell me if you are agree with my modification.
Thanks.

I don't know why, but

I don't know why, but finally if you want a perfect correction, you need to do :

var DegToRad = 3.1416 / 180;

var fx = Math.cos(ring[0].y * DegToRad);
ring[k].x *= fx;

with always the same point for y, I take the first one and it seems perfect. I compare with the shape of country in Googlemap.
If someone has an explanation, I'm interrested :)

Thanks.

Imagen de jsanz

Hi Olivier, I'm afraid Juan

Hi Olivier, I'm afraid Juan Lucas is on holidays, I'll remind you of your comments by email so he can reply you when he's available :-)

Jorge

Hello, Olivier. Thanks for

Hello, Olivier. Thanks for your message. I think the explanation is this:

- The coordinate system of those shapefiles (called WGS84 or EPSG:4326) is in lon/lat, so the countries which are close to the Poles appear stretched, as you say. That's the expected behavior, because that coordinate system is "ugly" in those countries and the script I provided does not perform any transformation or reprojection, it paints the shapefile as it is.

- Google Maps behaves in a special way: it shows you the coordinates in lon/lat when you use the mouse and lets you open KML files in lon/lat BUT the map is not painted in lon/lat. It is painted in Spherical Mercator. Google Maps is constantly reprojecting the lon/lat coordinates on the fly, so the user sees the coordinates in lon/lat but the map is painted in Spherical Mercator (because if you paint it in lon/lat, it is strecthed in many countries). The advantage is that Spherical Mercator shows proportionate (nice) maps everywhere.

- The change you have made (scaling the longitude using the cos of the latitude) has an effect which is very similar to the reprojection to Spherical Mercator, so what you have done is a "fast imitation" of the mechanism that Google Maps is using to show the KML files, for example.

- The exact change from lon/lat to Spherical Mercator can be found in many places, for example, the method "newGeo2Mercator(lon, lat)" in this Java class:

https://svn.prodevelop.es/public/labs/gvsigmobileonopenmoko/trunk/libGeo...

(the input lon/lat must be in degrees)

Be careful if you use it because the script I provided is also computing the bounding box of the shapefile. So you will have to apply that change to the bounding box too so that the map is centered in the screen.

- Another possibility is reprojecting the shapefile before using it. If the shapefile is Iceland, you can use Spherical Mercator and also EPSG:32627 for example. In this way, you would not need to modify the script.

Regards,
Juan Lucas

Will this work on any size

Will this work on any size shape file?
I tried it with US Census files for blocks for California 2010. It seems to render ok for their cartographic boundary shape file BUT not so for their much larger TIGER 2012 shape files. If the California TIGER 2012 block file renders then their are several large unconnected patches or my Firefox browser hangs!

Hello, Mark. Thanks for your

Hello, Mark. Thanks for your message.

- One way to increase performance is by reducing the line width (drawing thin lines is extremely fast)

- I also saw some badly drawn features in my tests but did not investigate further. It might be a bug in the renderer or maybe it depends on how the feature is closed in the SHP file (some SHP files use a CLOSE_SHAPE token at the end of an array of coordinates instead of a final LINE_TO coordinate) That's only a possible cause.

- The script is not doing any optimization whatsoever, simply draws every single vertex of every single feature. For large shapefiles like TIGER this does not make much sense if the canvas is small (the computer screen) because you get a useless map. If you were to draw a TIGER shapefile, there should be some previous conditioning and/or optimization. For example, draw the shapefile only if the scale of the map is above a certaing value, or maybe try to simplify features when you draw them, so that pixels are not painted again and again, etc.

Regards

Hello and thank you for the

Hello and thank you for the added info! I've taken out the rendering portion of your script since all I really want is a why to change TIGER block files to CSV files.

The problem still remains.

With this one 2012 California census block TIGER file I do get the error "Not a Polygon record (too small)" thrown once.

Also, I need still to check CLOSE_SHAPE and LINE_TO issues. Any other suggestions?

Hello, the error message you

Hello, the error message you mention perhaps happens in the last feature. Try to check whether the features have been already processed when that error happens.

The script is not using any external library because I copy-pasted all the functions written by Tom Carden, so if you do step-by-step execution (for example with Chrome Javascript Console) then you should see what the problem is.

If you only need to convert a SHP to CSV (with the geometries in well-known text, WKT) then you can use some other software. For example, the FWTools package provides some utility programs, such as ogrinfo which can print the content of the SHP (attributes and geometry in WKT). If your shapefile is called "california_blocks.shp", then you can type in a DOS-like console:

ogrinfo california_blocks.shp california_blocks >ca_blocks.txt

and ca_blocks.txt will be a text file with all the data. Then you would process it somehow to get a CSV file.

Chrome's console doesn't

Chrome's console doesn't show anything with that 2012 census file, it just pops up a page saying something is wrong and to try reloading. Firefox gave more info and it looks like the problem is happening with your getDoubleAt and getLongAt.

Maybe it's those TODO's?

There is a hex to floating point parser here

https://github.com/wavded/js-shapefile-to-geojson

stream.js seems to address those issues but I would have to look more.

Thank you for the lead to FWTools!!

Imagen de jsanz

Hi,Sorry to interfere but as

Hi,

Sorry to interfere but as web master of the site I've removed the moderation of comments until you finish this thread. Comments are moderated because of the spam we receive compared with the little conversation of the blog. But as this thread is active, I don't want to make you wait to have your comments posted so I will temporally disable the moderation. Sorry for the inconvenience these days, I hope this will boost the thread (if needed).

Cheers 

Jorge Gaspar Sanz Salinas

The script works fine on

The script works fine on files for all blocks in a county but not for the much bigger file for all blocks in the state.

Hello, can you provide a


Hello, can you provide a link to the SHP file that contains all the blocks in California? What's the output you are trying to get? By the way, the ogr2ogr utility program (available after installing FWTools) supports the CSV format for output, so you would get it directly with:

ogr2ogr -f CSV -lco "GEOMETRY=AS_WKT" blocks_folder cali_blocks.shp

That will create a folder called "blocks_folder" with the resulting CSV file.

Here is where you

Here is where you start.
http://www.census.gov/cgi-bin/geo/shapefiles2012/main
Select "blocks" (not "block groups"
Select California.
It is a big file!

I did try to get a geoJSON version of the county files instead of state. That worked BUT ogr2ogr left out the bounds of the features outer most ring. Using -lco WRITE_BBOX=YES does not help in providing that info! I used JavaScript on the geoJSON file and added bounding box, centroid and signed area data. The outputed the string infield JSON file content to a text area, Talk about jumping through hoops ... Ugh,

Hello. I think those huge

Hello. I think those huge SHP files must be considered a "dump file". In other words, they should not be used directly by applications which have a GUI. I think one should either export those shapefiles to a spatial database (PostGIS/Oracle Spatial, etc) or filter/crop them (ogr2ogr does that too) so that the final application deals with fewer features or takes profit of powerful indexes (thus faster performance) provided by databases.

Opciones de visualización de comentarios

Seleccione la forma que prefiera para mostrar los comentarios y haga clic en «Guardar las opciones» para activar los cambios.
Distribuir contenido