1840 Commits

Author SHA1 Message Date
Erica Fischer
622084a507
Limit guessed maxzoom to avoid spending too many tiles on polygon fill (#23)
* Change sqlite3 schema to deduplicate identical tiles

* Limit guessed maxzoom to avoid spending too many tiles on polygon fill

* Fix test.

These dust polygons now have their area calculated because their
maxzoom is being guessed, so the attributes from the largest one
rather than the last one are preserved.

* Increase polygon limit to a million tiles

* Two million tiles ought to be enough for anyone, right?

* Add explanatory comments for mysterious numbers
2022-11-07 10:14:17 -08:00
Erica Fischer
1584543224
Fix the flatgeobuf build (#30) 2022-11-03 14:43:00 -07:00
zstadler
4317c826a9
Fix docker build compilation (#29)
Fixes #28
2022-11-03 14:24:31 -07:00
James Chevalier
c2214631f2
Update flatbuffers to v22.9.29 (#26)
* update flatbuffers files

How:
- Copied files in from the latest release (v22.9.29) of flatbuffers

Why:
- The files in place blocked compilation within Alpine Linux

Links:
- https://github.com/felt/tippecanoe/issues/24
- https://github.com/google/flatbuffers/releases/tag/v22.9.29

* update flatgeobuf generate files

How
- Created `feature_generated.h` & `header_generated.h` using the `src/fbs` files from the latest release (3.23.1) of flatgeobuf
    - Downloaded the `flatc` binary from the v22.9.29 release of flatbuffers
    - Copied `flatc` into the flatgeobuf repository
    - Created a new directory named `out` to collect the generated files
    - `./flatc --cpp -o out -I include src/fbs/feature.fbs`
    - `./flatc --cpp -o out -I include src/fbs/header.fbs`
    - Moved the generate files from `out` into this repository's `flatgeobuf` directory

Why
- This is a required step in the process of updating flatbuffers files

Links
- https://github.com/flatgeobuf/flatgeobuf/releases/tag/3.23.1
- https://github.com/google/flatbuffers/releases/tag/v22.9.29
- https://github.com/felt/tippecanoe/issues/24#issuecomment-1286476550
2022-11-03 14:24:02 -07:00
Erica Fischer
5b03b18f31
Generate label points after simplification, not before. (#27)
* Generate labels points after simplification, not before.

Previously there were some cases where dropping the smallest
polygons would never reduce the number of labels.

* Also wait until after polygon cleaning to make labels

* Remove the extra newlines after the TILES ONLY COMPLETE message
2022-10-21 13:59:36 -07:00
Erica Fischer
b763625862
Add an option to generate label points in place of polygons (#20)
* Add an option to generate label points in place of polygons

* Change all these places where I said "extent" but really meant "area"

* Revert "Change all these places where I said "extent" but really meant "area""

This reverts commit 403828d2f7c8fc21329f76408abf1ae61b6faf86.

* Add --order-smallest-first and --order-largest-first options

* Use Turf's center-of-mass algorithm for polygon label points

* If the label point isn't within the polygon, find one that is

* Don't choose a label point that is too close to a border

* Try a little harder to find an optimal label point

* Checkerboard which tiles labels are generated in, to reduce adjacency

* Use a label point for the general representative point for polygons

(Skipping the iteration to find one that is as far as possible from
the borders)

This makes the labels look better in many cases (like France at z1)
but unfortunately ripples into changing the sequence of polygons in
many tests, so the diff is big.

* Revert "Use a label point for the general representative point for polygons"

This reverts commit 2261adf05eba1d454f6792f08e1e107f4b8342b5.

* Checkpoint work on spiral labels

* Clip label spirals to the feature bounds

* Fix label test

* Be careful not to place spiral labels too close to borders either

* For spiral anchors, only check tile scale, not feature size

* Update test

* Only try to find a central label point for the largest ring

* In tiny polygon dust, keep the attributes of the largest feature
2022-10-13 14:21:36 -07:00
Erica Fischer
182093bdc7
Improve precision of polygon area calculations (#19)
* Improve precision of get_area by using long double

* Trying to get consistent polygon area results between ARM and x86

* Calculate polygon area closer to the origin for better precision

* Update changelog

* Also exercise tiny polygon dust in the ring area test

They previously behaved differently here between x86 and ARM

* On M1 Macs, long double is just double anyway, so don't use it

* Be more careful about overflow: scale the polygon ring down into range

* Fix the bug I just introduced in the scaled area calculation

* Use only the sign from the scaled-down area calculation

Co-authored-by: Roman Karavia <47303530+romankaravia@users.noreply.github.com>
2022-10-04 10:04:34 -07:00
Erica Fischer
a6abb0bc30
Add the option to use a different simplification level at maxzoom (#17)
* Add an option to specify a different simplification at maxzoom

* Add test
2022-10-03 12:18:02 -07:00
Erica Fischer
4ef430b913
Add an option for the Visvalingam simplification algorithm (#16)
* First attempt at porting Paul Mach's Visvalingam implementation

* Make indent

* Mostly working

* Approximate equivalence from Douglas-Peucker to Visvalingam levels

* Don't simplify away tile boundary crossing points

* Add a command-line option and test for Visvalingam simplification

* Update changelog

* Cleanup in response to review feedback

* Include <stdio.h> to fix compiler warnings
2022-09-30 12:00:07 -07:00
Erica Fischer
6de00c15ec
Fix tests that I missed updating in 2.6.2 (#15) 2022-09-27 10:37:02 -07:00
Erica Fischer
3086838a10
Fix wrong-way comparison that was causing buffer overflow crash in tile-join (#14)
* Fix wrong-way comparison that was causing buffer overflow in tile-join

* Update changelog
2022-09-27 09:15:54 -07:00
Erica Fischer
af1a7ed7ae
Once features can't possibly fit in a tile, stop trying (#9)
* Stop adding features to a tile if it can't possibly work

* Add --integer and --fraction options to tippecanoe-decode

* Carry the strategies field from tileset metadata through tile-join

* Update changelog

* Assign different codes to different kinds of error exits
2022-09-23 20:01:13 -04:00
Vincent Sarago
ed31b9ac84
Update README.md (#13)
protomaps -> felt
2022-09-23 19:54:57 -04:00
Erica Fischer
8e46650da7
Upgrade to protozero 1.7.1 (#5)
* Upgrade to protozero 1.7.1

* Bump version number
2022-09-13 11:24:01 -07:00
Erica Fischer
073700aa38
Reduce maxzooms being guessed a little: (#2)
* Reduce maxzooms being guessed a little:

* Use 1.5 standard deviations, not 2, as the minimum distinguishable
* Give overlapping polygons and linestrings more distinct indices

* Add another drop rate guessing options, from the same metrics -zg uses

* Guard against using -rp without -zg
2022-09-08 15:47:32 -07:00
Erica Fischer
a447dfc089 Extra coordinate precision; feature ordering; compression improvements
* Add an option to retain extra coordinate precision at maxzoom

* Make sure not to shift away the extra detail from coordinates

* Add an option to convert double-precision attributes to single

* Sort attribute values in tiles to make them compress a little better

* Slightly improve polygon simplification

By choosing a point that would be retained after simplification
to be the start/end point that always gets retained

* I regret making all of these tests involve polygons

* Add an option to specify the size of tiny polygons

* Fix accidental requiring of argument for --single-precision

* Guard against duplicate points when generating "sizes" for them

* Restore the intended behavior that tiny polygons don't get simplified

* Make the extra detail settable rather than always maximizing it

* Revert "Improve maxzoom guessing for tightly-clustered point data sources (#4)"

This reverts commit fec5e8354cff7ab6d64f00ea7db6da21c5c732c6.

* Add an option to prevent choosing a base zoom higher than the maxzoom

* Keep the drop rate high enough when the basezoom gets constrained

* Revert "Revert "Improve maxzoom guessing for tightly-clustered point data sources (#4)""

This reverts commit db6bc27d9e5bd6b5a218ba9a07117b4f1abb75fb.

* Add --order-by and --order-descending options

* Accept multiple --order-by and --order-descending-by sort keys
2022-09-06 13:08:11 -07:00
Erica Fischer
4ea8a37611
Accept tilestats limiting options in tile-join, not just tippecanoe (#8) 2022-08-12 13:57:04 -07:00
Erica Fischer
fec5e8354c
Improve maxzoom guessing for tightly-clustered point data sources (#4)
* Improve maxzoom guessing for tightly-clustered point data sources

* Go back to the old distance estimate, since it is less mysterious

* Update changelog and version
2022-08-11 10:13:54 -07:00
Erica Fischer
a9bf6cacd4
Add --smallest-maximum-zoom-guess to specify a minimum maxzoom to guess (#2)
* Add --maximum-zoom-at-least to specify a minimum maxzoom to guess

* Check whether the specified minimum maxzoom is out of range

* Rename the option to --smallest-maximum-zoom-guess

* Update tests, changelog, and version
2022-08-11 10:04:03 -07:00
Erica Fischer
86f3e98c02
Track the desired tile size for each zoom level in tileset metadata (#7) 2022-08-11 09:58:00 -07:00
Erica Fischer
c65d6938a4
Drop or coalesce points too when dropping or coalescing smallest (#6)
* Handle points too when dropping or coalescing the "smallest" features.

* Add statistics of tile size reduction strategies to tileset metadata

* Update changelog and version

* Update documentation
2022-08-09 15:29:03 -07:00
Erica Fischer
67cd9d8d85
Reduce tippecanoe memory usage (#5)
* Change JSON objects to a union type to use less memory

* Stop storing the string representation of JSON numbers

* Restore the ability to create features with large integer attributes

* Make sure large-integer feature IDs still behave as before

* Add missing #include

* Don't preallocate as much space for arrays and objects

* Treat inability to check free disk space as a warning, not an error

* Update changelog and version
2022-08-09 15:18:51 -07:00
Erica Fischer
6978a0e102
Reindent with clang-format (#3) 2022-07-28 11:19:30 -07:00
Brandon Liu
e6ab02ddd3 lambda default progress is public [#13] 2022-07-06 12:21:12 +08:00
Brandon Liu
fbc7b02123 improvements to lambda wrapper [#13] 2022-07-05 21:26:13 +08:00
Brandon Liu
f442e403c5 add logging struct with -u json options flag, log tiling progress [#13] 2022-07-05 15:32:11 +08:00
Brandon Liu
4adb22b74e suppress flatgeobuf message 2022-07-05 15:32:00 +08:00
Brandon Liu
9875ae97a8 add lambda info 2022-05-17 15:29:38 +08:00
Brandon Liu
d04bd09d11 fix struct name collision [#12] 2022-04-27 22:14:22 +08:00
Brandon Liu
7c726e4d2c add missing column types [#2] 2022-04-23 13:26:26 +08:00
Brandon Liu
170ddf73ff comments cleanup 2022-04-02 12:28:28 +08:00
Brandon Liu
7f678502af FlatGeobuf: read files with heterogeneous geometry types [#8] 2022-04-02 12:07:22 +08:00
Brandon Liu
19241f8373 FlatGeobuf: if fgb has index, assign feature IDs based on sequence number [#2] 2022-03-30 12:37:55 +08:00
Brandon Liu
fe6f21fe55 FlatGeobuf: support Point geometries [#2] 2022-03-28 00:01:08 +08:00
Brandon Liu
cc52f65939 update man page to reflect new README 2022-03-27 21:53:50 +08:00
Brandon Liu
9bf3326764 update README clone path 2022-03-27 21:53:20 +08:00
Brandon Liu
155f129616 readme updates 2022-03-27 19:27:41 +08:00
Brandon Liu
bb2aa65bcc version 2.1.0 2022-03-27 19:26:22 +08:00
Brandon Liu
35b576d0fd
Merge pull request #10 from protomaps/flatgeobuf
FlatGeobuf input support [#2]
2022-03-27 19:22:49 +08:00
Brandon Liu
e5856398dd
update LICENSE to reflect fork. 2022-03-27 19:21:05 +08:00
Brandon Liu
e1a5fd1221
README changes for FlatGeobuf [#2] 2022-03-27 19:18:09 +08:00
Brandon Liu
d2951d3437 FlatGeobuf style fixup, fix starting of queue with few features, buffer verification [#2,#6] 2022-03-27 19:14:35 +08:00
Brandon Liu
6e6cd29399 FlatGeobuf multithreading [#2] 2022-03-27 19:14:27 +08:00
Brandon Liu
245570feff FlatGeobuf multipoints [#2] 2022-03-27 19:13:43 +08:00
Brandon Liu
33a9aef758 Handle indeterminate length FlatGeobuf headers, linestring geometries [#2] 2022-03-27 19:13:22 +08:00
Brandon Liu
b350a8dda7 FlatGeobuf MultiPolygon parsing [#2] 2022-03-27 19:13:12 +08:00
Brandon Liu
88cedf54d4 FlatGeobuf column type parsing [#2] 2022-03-27 19:13:12 +08:00
Brandon Liu
8a5a2496c3 Implement FlatGeobuf Polygon type parsing [#2] 2022-03-27 19:13:03 +08:00
Brandon Liu
a3dec5116d add flatgeobuf dependencies [#2] 2022-02-14 11:33:34 +08:00
Brandon Liu
7f69b80381 version bump: original 1.36.0 same as fork version 2.0.0 2022-02-08 15:22:50 +08:00