Commit Graph

240 Commits

Author SHA1 Message Date
Eric Fischer
cde1e60603 Use a string pool to avoid duplicating keys and values 2015-06-17 17:18:08 -07:00
Eric Fischer
55e93a5d37 Use variable-length zigzag for ints and long longs 2015-06-17 16:46:36 -07:00
Eric Fischer
8a1f0d83e1 Consistent indent style with clang-format 2015-06-03 11:22:13 -07:00
Eric Fischer
34b00eca73 Merge pull request #50 from mapbox/gl4096
Add a command line option to specify the minimum allowed tile extent
2015-06-03 11:11:16 -07:00
Eric Fischer
70291f0415 Merge pull request #51 from mapbox/fitsample
Flag to drop as many features as necessary to keep tiles under the size limit
2015-06-03 11:10:54 -07:00
Eric Fischer
bc2f243f0b Add a command line option to specify the minimum allowed tile extent 2015-06-01 15:01:46 -07:00
Tom MacWright
95997b50c4 Add tippecanoe -v 2015-05-29 10:50:11 -04:00
Tom MacWright
ce6a1aac88 Only error if minzoom is truly greater, not just equal 2015-05-29 10:33:44 -04:00
Tom MacWright
a0693446d5 Add check that min is less than max. Fixes #40 2015-05-28 13:21:38 -04:00
Eric Fischer
38dc80ec68 Fix a place that wasn't checking for disk write errors 2015-05-20 18:04:34 -07:00
Eric Fischer
167ec690a0 Warn if no Features have been seen after 50 JSON hashes are closed. 2015-04-17 10:48:03 -07:00
Eric Fischer
d64328ac35 Fix crash when encoding boolean properties. 2015-04-10 13:03:11 -07:00
Eric Fischer
4041811372 Add flags to disable simplification/coalescing/limits when unwanted 2015-04-10 11:36:30 -07:00
Eric Fischer
fd8de691eb Fix layer name crash when reading from the standard input 2015-03-25 14:07:34 -07:00
Eric Fischer
a076c5619d Merge remote-tracking branch 'origin/master' into multilayer 2015-03-25 13:54:29 -07:00
Eric Fischer
86925eea4c Write out geometries for all layers 2015-03-24 17:07:51 -07:00
Eric Fischer
73b63133e2 Multiple layers make it into JSON metadata now 2015-03-24 16:28:31 -07:00
Eric Fischer
2198bcc2a6 Handle multiple reading. Multiple writing still crashes. 2015-03-23 17:44:23 -07:00
Eric Fischer
f5135ebc63 Writing out the JSON metadata for multiple layers 2015-03-23 16:12:12 -07:00
Eric Fischer
eb24c6e21e Include the layer number in the serialized geometry 2015-03-23 15:37:49 -07:00
Eric Fischer
a880f44a91 Clean up the gamma semantics a little.
0 (default) means unchanged: maxzoom contains all the dots.

Positive number, even very small, means that it still checks
the gap size and thins out dots if they are close together.

Negative number means no gamma, and the old random instead of
uniform dropping at low zooms.
2015-03-23 14:44:21 -07:00
Eric Fischer
8002609f0c More scaffolding for multiple layers 2015-03-23 13:44:35 -07:00
Eric Fischer
b59a251924 Write metadata even when tiling fails so you can look at the partial map 2015-03-06 16:33:32 -08:00
Eric Fischer
f0a8e5b192 Progress indicator for reordering geometry 2015-03-06 15:32:52 -08:00
Eric Fischer
9343c5fcc1 Less wordy message about sorting 2015-03-06 14:35:39 -08:00
Eric Fischer
41b28b2a1b Add dot gamma processing 2015-03-06 13:12:32 -08:00
Eric Fischer
558a7a412c Restore old code for uniform instead of random dot-dropping
From ffe4c95376
2015-03-06 10:56:02 -08:00
Eric Fischer
572df8ad39 Reorder the geometries by index 2015-03-05 16:18:01 -08:00
Eric Fischer
a8b2db8d5a Fix double-close of top level geometry file 2015-03-05 15:22:58 -08:00
Eric Fischer
97d65e6b7d Bring back the index: now just a file position and a point 2015-03-05 15:15:56 -08:00
Eric Fischer
290e39f80c Improve function name 2014-12-19 11:56:40 -08:00
Eric Fischer
5a2003cb2c Fix failure to release temporary memory 2014-12-17 17:41:57 -08:00
Eric Fischer
48b5db6ae5 Keep the progress indicator progressing while working through big tiles 2014-12-17 16:01:33 -08:00
Eric Fischer
7f3551070e Remove unneeded debug output 2014-12-17 14:01:20 -08:00
Eric Fischer
92bbf27f72 Merge branch 'master' into topdown
Conflicts:
	geojson.c
2014-12-17 11:18:47 -08:00
Eric Fischer
cba1b8ae7f Remove unneeded old indexing code 2014-12-17 11:16:43 -08:00
Eric Fischer
0d0a546b1e No need to split features into child tiles when processing the final zoom. 2014-12-17 11:10:46 -08:00
Eric Fischer
ad17f1f282 Make -Z0 work again for file minimum zoom 2014-12-17 11:05:14 -08:00
Eric Fischer
a40192bcde Allow features to have null properties 2014-12-16 22:46:00 -08:00
Eric Fischer
c90ba8511f Fix compiler warning 2014-12-12 14:52:12 -08:00
Eric Fischer
34a6422c42 Merge branch 'master' into topdown 2014-12-12 11:36:20 -08:00
Eric Fischer
3f2818a814 More useful progress indicator 2014-12-11 17:59:22 -08:00
Eric Fischer
d69431e16b Partition each tile into children instead of repeatedly clipping the parent. 2014-12-11 15:46:54 -08:00
Eric Fischer
b068635acf Elevation in geometry is a warning, not an error.
Only complain about the first null geometry, not ever one.
2014-12-09 15:23:22 -08:00
Eric Fischer
40ecfc0668 Add missing newline to error message. 2014-12-09 15:17:20 -08:00
Eric Fischer
028fef470e Fix the major bug: forgetting to offset back to world coordinates 2014-12-04 14:08:36 -08:00
Eric Fischer
08ff40e42f Trying to follow Vlad's suggestion and work from the top down.
Doesn't quite work yet, but seems like the right thing to do.
From the bottom up, indexing is a mess because of the buffering,
and includes unneeded areas in the bounding boxes of big features.
2014-12-03 16:18:43 -08:00
Eric Fischer
eaeb55bf71 Distinguish between signed and unsigned in the temporary file 2014-12-02 18:03:07 -08:00
Eric Fischer
a1e7426956 Sprinkle consts and casts to make it valid C++ as well as C 2014-12-02 14:17:49 -08:00
Eric Fischer
cc58588724 More sensible error message when there are no valid geometries 2014-11-24 22:50:12 -08:00
Eric Fischer
ffe4c95376 Calculate point dropping during encoding instead of tiling
so it is consistent across tile boundaries, and so every point
that exists at zoom x always also exists at zoom x+1.
2014-11-14 12:42:50 -08:00
Eric Fischer
8210f83912 Make the progress indicator less chatty 2014-11-13 10:48:59 -08:00
Eric Fischer
5595a087c7 Quick reject by bounding box of linear features too small to draw 2014-11-12 15:57:45 -08:00
Eric Fischer
2b6ad9968f Add a flag to exclude all properties except ones explicitly named 2014-11-12 12:41:56 -08:00
Eric Fischer
b05c5a9583 Skip over properties with null values instead of considering it an error 2014-11-12 12:19:39 -08:00
Eric Fischer
ea292def47 Make duplicate trimming less bad: only trim features indexed >1 time 2014-11-06 11:27:10 -08:00
Eric Fischer
7f6a5dc005 The maxzoom=0 case couldn't have worked 2014-11-06 11:05:09 -08:00
Eric Fischer
241d96e57c Save a few more bytes per geometry in the temporary file
by using bytes instead of ints for the drawing operations
2014-11-05 14:37:54 -08:00
Eric Fischer
c24cfada61 Move feature type to index so point skipping can avoid looking up geometry
Which should avoid having to page the geometry in if there is
more data than will fit in memory
2014-11-05 12:18:03 -08:00
Eric Fischer
bc4527b25d Add an option to set the directory for the temporary files 2014-11-05 10:34:44 -08:00
Eric Fischer
a039aca860 Read-only memory map for merging, since there's no reason to write 2014-11-04 10:08:30 -08:00
Eric Fischer
3e3fdcbab4 Bring over the multistage sort from datamaps to avoid Mac crashes 2014-11-03 22:47:41 -08:00
Eric Fischer
984ac17d6c Use the same default ground precision whatever the base zoom is. 2014-10-27 14:36:13 -07:00
Eric Fischer
3cd6fcfbe0 Add option to set buffer size. Pass it through to clipping functions. 2014-10-27 13:20:17 -07:00
Eric Fischer
4cb56a6b11 The indexing side of buffering features at the edges of tiles 2014-10-27 12:56:51 -07:00
Eric Fischer
4b171c74b7 Constrain calculated center point to be within the bounding box
It could come out bigger because it is calculated from the center
of the densest tile, not actually the centroid.
2014-10-26 13:12:29 -07:00
Eric Fischer
46100c34df Move projection and tile arithmetic to its own file 2014-10-24 17:22:14 -07:00
Eric Fischer
754dbf152a Remove dead code 2014-10-24 15:23:35 -07:00
Eric Fischer
3bdcc3ca90 Heavy-handedly fix clipping errors at the edges of z0 and z1 tiles 2014-10-15 17:03:30 -07:00
Eric Fischer
43b27e2dfd Don't closepath twice in multipolygons 2014-10-15 16:30:33 -07:00
Eric Fischer
dc3c6138d1 Fix failure to close polygons. 2014-10-15 16:15:24 -07:00
Eric Fischer
e561931d53 Fix bug that would always index for z14 regardless of actual max zoom 2014-10-15 15:49:03 -07:00
Eric Fischer
ac34a292e9 Add an flag to remove all properties and use only geometry 2014-10-09 15:11:43 -07:00
Eric Fischer
65ec829de6 Report every 10000 features on input, since parsing GeoJSON is slow. 2014-10-09 12:23:43 -07:00
Eric Fischer
adbd9f6edb Add a flag to select the rate at which points are dropped at low zooms. 2014-10-07 13:54:13 -07:00
Eric Fischer
8f71604fe6 If reading from standard input, use the .mbtiles as default layer name 2014-10-07 12:02:12 -07:00
Eric Fischer
bda20fe8a6 Improve error reporting 2014-10-06 10:55:40 -07:00
Eric Fischer
41447a4678 Be clearer about error reporting 2014-10-04 14:11:59 -07:00
Eric Fischer
b292b66a7c Fix more (small) memory leaks 2014-10-01 10:33:22 -07:00
Eric Fischer
f0d2dbd2a8 Add -f option to overwrite existing mbtiles file 2014-09-29 15:09:21 -07:00
Eric Fischer
26e9feda02 Move mbtiles code to its own source file 2014-09-29 12:48:58 -07:00
Eric Fischer
5e64040859 Remove dead code 2014-09-29 12:18:58 -07:00
Eric Fischer
8118c13a71 Give the constant pool its own source file 2014-09-29 12:17:35 -07:00
Eric Fischer
fabeb4588d Stop duplicating pool initialization code 2014-09-29 12:12:54 -07:00
Eric Fischer
5ab7f89465 Add an option to exclude specified properties 2014-09-29 10:50:24 -07:00
Eric Fischer
c326762b6f Crunch out spaces and most punctuation from autogenerated layer names 2014-09-27 09:39:20 -07:00
Eric Fischer
6edaaa1104 Restrict to one input file, since that's all that will work 2014-09-26 14:33:03 -07:00
Eric Fischer
0a8420cc2e Add options to specify the base and low-zoom level of detail 2014-09-26 10:52:19 -07:00
Eric Fischer
528e0575e2 Actually write tiles directly to the .mbtiles file! 2014-09-25 16:34:17 -07:00
Eric Fischer
ce485148fe String formatting mess to encode the JSON metadata 2014-09-25 16:07:03 -07:00
Eric Fischer
86f4f2dd05 More metadata adding, and boilerplate from mbutil 2014-09-25 15:38:20 -07:00
Eric Fischer
eec0e14447 Start putting data into mbtiles instead of separate files 2014-09-25 15:20:17 -07:00
Eric Fischer
e722ee487e More useful progress indicator 2014-09-25 14:36:25 -07:00
Eric Fischer
201708f19c Add command line options for zoom, title, layer name, name, output dir 2014-09-25 12:09:31 -07:00
Eric Fischer
0b23e65803 Hardcode layer name a little less. 2014-09-24 23:22:14 -07:00
Eric Fischer
dc7c58169f Use a constant instead of assuming everything goes to z0 2014-09-24 22:58:17 -07:00
Eric Fischer
ce3cdfc5e9 Choose the center of the busiest tile for the map center 2014-09-24 12:14:35 -07:00
Eric Fischer
95c2fb67b2 Fix large features getting included multiple times at low zooms 2014-09-23 17:18:36 -07:00
Eric Fischer
8f77376b58 Calculate a bounding box and hopefully reasonable center point 2014-09-23 16:41:38 -07:00
Eric Fischer
5d7a2f50a9 Write the metadata file 2014-09-23 16:12:41 -07:00
Eric Fischer
91eefcbd16 Drop more points at each lower zoom level 2014-09-23 14:42:17 -07:00
Eric Fischer
7fc1c0cd24 Fix index mixup that kept points from working at all 2014-09-23 14:02:18 -07:00
Eric Fischer
454a5dd3c7 Index anything that entirely fits within the tile by its centroid 2014-09-23 13:33:40 -07:00
Eric Fischer
00a42796b4 Use lower resolution at lower zooms. Not sure it's worth it. 2014-09-23 10:57:01 -07:00
Eric Fischer
581105dc9a Much faster to search and add to the string pool as a binary tree 2014-09-22 22:31:55 -07:00
Eric Fischer
d7cdbec980 Fix bit shifting errors at z0 2014-09-22 18:08:37 -07:00
Eric Fischer
6851e914a9 Write out the tile as PBF 2014-09-22 17:46:48 -07:00
Eric Fischer
3df523a588 Write out geometry 2014-09-22 17:12:38 -07:00
Eric Fischer
c679fa176b Move whole tile-producing function onto the C++ side 2014-09-22 16:27:31 -07:00
Eric Fischer
74bdbfc76a Serialize geometry before tags
To make it easier to skip the tags if the geometry is optimized away
2014-09-22 16:17:55 -07:00
Eric Fischer
baf364dd53 Write string pool to tile 2014-09-22 16:06:44 -07:00
Eric Fischer
41981ec8e9 Make the string pools within each tile 2014-09-22 12:22:58 -07:00
Eric Fischer
94ef6afc17 Identify the start and end of each tile 2014-09-22 11:22:23 -07:00
Eric Fischer
647edc36d6 Index each feature in each tile rather than looking up features by tile later 2014-09-19 15:40:07 -07:00
Eric Fischer
eedb0850b2 Factor out code for finding ranges within the index 2014-09-18 17:25:33 -07:00
Eric Fischer
7c5254b1e3 Add progress indicator for tile iteration too 2014-09-18 17:02:09 -07:00
Eric Fischer
1554a59153 Add a progress indicator. Remember to run the optimizer. 2014-09-18 16:27:34 -07:00
Eric Fischer
9fd4da4f7d Unlink tmp files early in case the tiling doesn't run to completion 2014-09-18 14:04:52 -07:00
Eric Fischer
b5d28af005 Iterate through the bbox and look up geometries by tile 2014-09-18 14:01:22 -07:00
Eric Fischer
f4a40cb16c Use proper temporary files instead of fixed names 2014-09-18 12:12:59 -07:00
Eric Fischer
e8adeba40e Keep a file bounding box to know the area that needs to be tiled 2014-09-18 10:26:47 -07:00
Eric Fischer
14e7b648ed Error check when writing to temporaries 2014-09-17 16:21:25 -07:00
Eric Fischer
57fe46f423 Memory map the metadata pool too 2014-09-17 16:00:19 -07:00
Eric Fischer
a38ebd9804 Sort the index 2014-09-17 15:48:29 -07:00
Eric Fischer
6613246554 Write the index to a file and then memory map it 2014-09-17 15:44:07 -07:00
Eric Fischer
ee8c23b010 Start indexing; sanity check that the file positions are right 2014-09-16 17:54:23 -07:00
Eric Fischer
449da18c63 Sanity check serialization 2014-09-16 17:19:54 -07:00
Eric Fischer
88dc948b6b Serialize metadata and geometry to a file 2014-09-16 16:39:56 -07:00
Eric Fischer
a5b8a02ec0 More consistent naming for constants 2014-09-16 14:11:50 -07:00
Eric Fischer
d2f0aa0d37 Fix not saying moveto for top level points 2014-09-16 13:54:57 -07:00
Eric Fischer
e82edb90a3 Most of the way toward serializing the drawing operations 2014-09-16 13:51:22 -07:00
Eric Fischer
33747bdfd8 Project to web mercator and calculate bounding box 2014-09-16 11:16:51 -07:00
Eric Fischer
ff5b632cf5 MultiPoints are all movetos, no linetos 2014-09-15 17:57:16 -07:00
Eric Fischer
1dcfca9d51 Get closer to vector format 2014-09-15 17:33:54 -07:00
Eric Fischer
f54af30a40 Parse nested geometry 2014-09-15 16:32:06 -07:00
Eric Fischer
3d43c5afbd Flatten out error-handling so it isn't nested so deeply 2014-09-15 15:47:06 -07:00
Eric Fischer
efa3995d94 Clean up meta naming a little 2014-09-15 15:30:03 -07:00
Eric Fischer
0ee8d877ca Add stub main() for testing 2014-09-15 15:27:35 -07:00
Eric Fischer
567d57b864 Remove unneeded datamaps stuff 2014-09-15 15:18:08 -07:00
Eric Fischer
e61898b46e Start of GeoJSON from datamaps v2 2014-09-15 15:02:33 -07:00