Drop or coalesce points too when dropping or coalescing smallest (#6)

* Handle points too when dropping or coalescing the "smallest" features.

* Add statistics of tile size reduction strategies to tileset metadata

* Update changelog and version

* Update documentation
This commit is contained in:
Erica Fischer 2022-08-09 15:29:03 -07:00 committed by GitHub
parent 67cd9d8d85
commit c65d6938a4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
63 changed files with 574 additions and 18 deletions

View File

@ -1,3 +1,8 @@
## 2.3.0
* Drop and coalesce points too as part of smallest-as-needed dropping and coalescing
* Keep statistics in the tileset metadata of what tile size reduction strategies were used at each zoom level
## 2.2.0
* Reduce memory consumption when parsing large JSON objects

View File

@ -455,8 +455,8 @@ the same layer, enclose them in an `all` expression so they will all be evaluate
* `-as` or `--drop-densest-as-needed`: If a tile is too large, try to reduce it to under 500K by increasing the minimum spacing between features. The discovered spacing applies to the entire zoom level.
* `-ad` or `--drop-fraction-as-needed`: Dynamically drop some fraction of features from each zoom level to keep large tiles under the 500K size limit. (This is like `-pd` but applies to the entire zoom level, not to each tile.)
* `-an` or `--drop-smallest-as-needed`: Dynamically drop the smallest features (physically smallest: the shortest lines or the smallest polygons) from each zoom level to keep large tiles under the 500K size limit. This option will not work for point features.
* `-aN` or `--coalesce-smallest-as-needed`: Dynamically combine the smallest features (physically smallest: the shortest lines or the smallest polygons) from each zoom level into other nearby features to keep large tiles under the 500K size limit. This option will not work for point features, and will probably not help very much with LineStrings. It is mostly intended for polygons, to maintain the full original area covered by polygons while still reducing the feature count somehow. The attributes of the small polygons are *not* preserved into the combined features (except through `--accumulate-attribute`), only their geometry. Furthermore, the polygons to which nested polygons are coalesced may not necessarily be the immediately enclosing features.
* `-an` or `--drop-smallest-as-needed`: Dynamically drop the smallest features (physically smallest: the shortest lines or the smallest polygons) from each zoom level to keep large tiles under the 500K size limit.
* `-aN` or `--coalesce-smallest-as-needed`: Dynamically combine the smallest features (physically smallest: the shortest lines or the smallest polygons or the densest points) from each zoom level into other nearby features to keep large tiles under the 500K size limit. This option will probably not help very much with LineStrings. It is mostly intended for polygons, to maintain the full original area covered by polygons while still reducing the feature count somehow. The attributes of the small polygons are *not* preserved into the combined features (except through `--accumulate-attribute`), only their geometry. Furthermore, the polygons to which nested polygons are coalesced may not necessarily be the immediately enclosing features.
* `-aD` or `--coalesce-densest-as-needed`: Dynamically combine the densest features from each zoom level into other nearby features to keep large tiles under the 500K size limit. (Again, mostly useful for polygons.)
* `-aS` or `--coalesce-fraction-as-needed`: Dynamically combine a fraction of features from each zoom level into other nearby features to keep large tiles under the 500K size limit. (Again, mostly useful for polygons.)
* `-pd` or `--force-feature-limit`: Dynamically drop some fraction of features from large tiles to keep them under the 500K size limit. It will probably look ugly at the tile boundaries. (This is like `-ad` but applies to each tile individually, not to the entire zoom level.) You probably don't want to use this.

View File

@ -2341,7 +2341,8 @@ int read_input(std::vector<source> &sources, char *fname, int maxzoom, int minzo
std::atomic<unsigned> midx(0);
std::atomic<unsigned> midy(0);
int written = traverse_zooms(fd, size, meta, stringpool, &midx, &midy, maxzoom, minzoom, outdb, outdir, buffer, fname, tmpdir, gamma, full_detail, low_detail, min_detail, meta_off, pool_off, initial_x, initial_y, simplification, layermaps, prefilter, postfilter, attribute_accum, filter);
std::vector<strategy> strategies;
int written = traverse_zooms(fd, size, meta, stringpool, &midx, &midy, maxzoom, minzoom, outdb, outdir, buffer, fname, tmpdir, gamma, full_detail, low_detail, min_detail, meta_off, pool_off, initial_x, initial_y, simplification, layermaps, prefilter, postfilter, attribute_accum, filter, strategies);
if (maxzoom != written) {
if (written > minzoom) {
@ -2403,7 +2404,7 @@ int read_input(std::vector<source> &sources, char *fname, int maxzoom, int minzo
ai->second.maxzoom = maxzoom;
}
mbtiles_write_metadata(outdb, outdir, fname, minzoom, maxzoom, minlat, minlon, maxlat, maxlon, midlat, midlon, forcetable, attribution, merged_lm, true, description, !prevent[P_TILE_STATS], attribute_descriptions, "tippecanoe", commandline);
mbtiles_write_metadata(outdb, outdir, fname, minzoom, maxzoom, minlat, minlon, maxlat, maxlon, midlat, midlon, forcetable, attribution, merged_lm, true, description, !prevent[P_TILE_STATS], attribute_descriptions, "tippecanoe", commandline, strategies);
return ret;
}

View File

@ -571,9 +571,9 @@ compensate for the larger marker, or \fB\fC\-Bf\fR\fInumber\fP to allow at most
.IP \(bu 2
\fB\fC\-ad\fR or \fB\fC\-\-drop\-fraction\-as\-needed\fR: Dynamically drop some fraction of features from each zoom level to keep large tiles under the 500K size limit. (This is like \fB\fC\-pd\fR but applies to the entire zoom level, not to each tile.)
.IP \(bu 2
\fB\fC\-an\fR or \fB\fC\-\-drop\-smallest\-as\-needed\fR: Dynamically drop the smallest features (physically smallest: the shortest lines or the smallest polygons) from each zoom level to keep large tiles under the 500K size limit. This option will not work for point features.
\fB\fC\-an\fR or \fB\fC\-\-drop\-smallest\-as\-needed\fR: Dynamically drop the smallest features (physically smallest: the shortest lines or the smallest polygons) from each zoom level to keep large tiles under the 500K size limit.
.IP \(bu 2
\fB\fC\-aN\fR or \fB\fC\-\-coalesce\-smallest\-as\-needed\fR: Dynamically combine the smallest features (physically smallest: the shortest lines or the smallest polygons) from each zoom level into other nearby features to keep large tiles under the 500K size limit. This option will not work for point features, and will probably not help very much with LineStrings. It is mostly intended for polygons, to maintain the full original area covered by polygons while still reducing the feature count somehow. The attributes of the small polygons are \fInot\fP preserved into the combined features (except through \fB\fC\-\-accumulate\-attribute\fR), only their geometry. Furthermore, the polygons to which nested polygons are coalesced may not necessarily be the immediately enclosing features.
\fB\fC\-aN\fR or \fB\fC\-\-coalesce\-smallest\-as\-needed\fR: Dynamically combine the smallest features (physically smallest: the shortest lines or the smallest polygons or the densest points) from each zoom level into other nearby features to keep large tiles under the 500K size limit. This option will probably not help very much with LineStrings. It is mostly intended for polygons, to maintain the full original area covered by polygons while still reducing the feature count somehow. The attributes of the small polygons are \fInot\fP preserved into the combined features (except through \fB\fC\-\-accumulate\-attribute\fR), only their geometry. Furthermore, the polygons to which nested polygons are coalesced may not necessarily be the immediately enclosing features.
.IP \(bu 2
\fB\fC\-aD\fR or \fB\fC\-\-coalesce\-densest\-as\-needed\fR: Dynamically combine the densest features from each zoom level into other nearby features to keep large tiles under the 500K size limit. (Again, mostly useful for polygons.)
.IP \(bu 2

View File

@ -267,7 +267,57 @@ void tilestats(std::map<std::string, layermap_entry> const &layermap1, size_t el
state.json_end_hash();
}
void mbtiles_write_metadata(sqlite3 *outdb, const char *outdir, const char *fname, int minzoom, int maxzoom, double minlat, double minlon, double maxlat, double maxlon, double midlat, double midlon, int forcetable, const char *attribution, std::map<std::string, layermap_entry> const &layermap, bool vector, const char *description, bool do_tilestats, std::map<std::string, std::string> const &attribute_descriptions, std::string const &program, std::string const &commandline) {
std::string stringify_strategies(std::vector<strategy> const &strategies) {
std::string out;
json_writer state(&out);
bool any = false;
state.json_write_array();
for (size_t i = 0; i < strategies.size(); i++) {
state.json_write_hash();
if (strategies[i].dropped_by_rate > 0) {
state.json_write_string("dropped_by_rate");
state.json_write_number(strategies[i].dropped_by_rate);
any = true;
}
if (strategies[i].dropped_by_gamma > 0) {
state.json_write_string("dropped_by_gamma");
state.json_write_number(strategies[i].dropped_by_gamma);
any = true;
}
if (strategies[i].dropped_as_needed > 0) {
state.json_write_string("dropped_as_needed");
state.json_write_number(strategies[i].dropped_as_needed);
any = true;
}
if (strategies[i].coalesced_as_needed > 0) {
state.json_write_string("coalesced_as_needed");
state.json_write_number(strategies[i].coalesced_as_needed);
any = true;
}
if (strategies[i].detail_reduced > 0) {
state.json_write_string("detail_reduced");
state.json_write_number(strategies[i].detail_reduced);
any = true;
}
state.json_end_hash();
}
state.json_end_array();
if (any) {
return out;
} else {
return "";
}
}
void mbtiles_write_metadata(sqlite3 *outdb, const char *outdir, const char *fname, int minzoom, int maxzoom, double minlat, double minlon, double maxlat, double maxlon, double midlat, double midlon, int forcetable, const char *attribution, std::map<std::string, layermap_entry> const &layermap, bool vector, const char *description, bool do_tilestats, std::map<std::string, std::string> const &attribute_descriptions, std::string const &program, std::string const &commandline, std::vector<strategy> const &strategies) {
char *sql, *err;
sqlite3 *db = outdb;
@ -393,6 +443,18 @@ void mbtiles_write_metadata(sqlite3 *outdb, const char *outdir, const char *fnam
}
sqlite3_free(sql);
std::string strat = stringify_strategies(strategies);
if (strat.size() > 0) {
sql = sqlite3_mprintf("INSERT INTO metadata (name, value) VALUES ('strategies', %Q);", strat.c_str());
if (sqlite3_exec(db, sql, NULL, NULL, &err) != SQLITE_OK) {
fprintf(stderr, "set strategies: %s\n", err);
if (!forcetable) {
exit(EXIT_FAILURE);
}
}
sqlite3_free(sql);
}
if (vector) {
size_t elements = max_tilestats_values;
std::string buf;

View File

@ -41,11 +41,13 @@ struct layermap_entry {
}
};
#include "tile.hpp"
sqlite3 *mbtiles_open(char *dbname, char **argv, int forcetable);
void mbtiles_write_tile(sqlite3 *outdb, int z, int tx, int ty, const char *data, int size);
void mbtiles_write_metadata(sqlite3 *outdb, const char *outdir, const char *fname, int minzoom, int maxzoom, double minlat, double minlon, double maxlat, double maxlon, double midlat, double midlon, int forcetable, const char *attribution, std::map<std::string, layermap_entry> const &layermap, bool vector, const char *description, bool do_tilestats, std::map<std::string, std::string> const &attribute_descriptions, std::string const &program, std::string const &commandline);
void mbtiles_write_metadata(sqlite3 *outdb, const char *outdir, const char *fname, int minzoom, int maxzoom, double minlat, double minlon, double maxlat, double maxlon, double midlat, double midlon, int forcetable, const char *attribution, std::map<std::string, layermap_entry> const &layermap, bool vector, const char *description, bool do_tilestats, std::map<std::string, std::string> const &attribute_descriptions, std::string const &program, std::string const &commandline, std::vector<strategy> const &strategies);
void mbtiles_close(sqlite3 *outdb, const char *pgm);

View File

@ -531,14 +531,19 @@ int serialize_feature(struct serialization_state *sst, serial_feature &sf) {
}
}
} else if (sf.t == VT_LINE) {
double dist = 0;
for (size_t i = 1; i < sf.geometry.size(); i++) {
if (sf.geometry[i].op == VT_LINETO) {
double xd = sf.geometry[i].x - sf.geometry[i - 1].x;
double yd = sf.geometry[i].y - sf.geometry[i - 1].y;
extent += sqrt(xd * xd + yd * yd);
dist += sqrt(xd * xd + yd * yd);
}
}
// treat lines as having the area of a circle with the line as diameter
extent = M_PI * (dist / 2) * (dist / 2);
}
// VT_POINT extent will be calculated in write_tile from the distance between adjacent features.
}
if (extent <= LLONG_MAX) {
@ -559,7 +564,7 @@ int serialize_feature(struct serialization_state *sst, serial_feature &sf) {
long long midy = (sf.bbox[1] / 2 + sf.bbox[3] / 2) & ((1LL << 32) - 1);
bbox_index = encode_index(midx, midy);
if (additional[A_DROP_DENSEST_AS_NEEDED] || additional[A_COALESCE_DENSEST_AS_NEEDED] || additional[A_CLUSTER_DENSEST_AS_NEEDED] || additional[A_CALCULATE_FEATURE_DENSITY] || additional[A_INCREASE_GAMMA_AS_NEEDED] || sst->uses_gamma || cluster_distance != 0) {
if (additional[A_DROP_DENSEST_AS_NEEDED] || additional[A_COALESCE_DENSEST_AS_NEEDED] || additional[A_CLUSTER_DENSEST_AS_NEEDED] || additional[A_CALCULATE_FEATURE_DENSITY] || additional[A_DROP_SMALLEST_AS_NEEDED] || additional[A_COALESCE_SMALLEST_AS_NEEDED] || additional[A_INCREASE_GAMMA_AS_NEEDED] || sst->uses_gamma || cluster_distance != 0) {
sf.index = bbox_index;
} else {
sf.index = 0;

View File

@ -8,6 +8,7 @@
"maxzoom": "3",
"minzoom": "0",
"name": "tests/accumulate/out/-z3_-Ethesum@sum_-Etheproduct@product_-Ethemax@max_-Ethemin@min_-Ethemean@mean_-Etheconcat@concat_-Ethecomma@comma_-r1_-K100.json.check.mbtiles",
"strategies": "[ { \"coalesced_as_needed\": 78 }, { \"coalesced_as_needed\": 65 }, { \"coalesced_as_needed\": 43 }, { \"coalesced_as_needed\": 17 } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/accumulate/out/-z5_-Ethesum@sum_-Etheproduct@product_-Ethemax@max_-Ethemin@min_-Ethemean@mean_-Etheconcat@concat_-Ethecomma@comma.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 83 }, { \"dropped_by_rate\": 89 }, { \"dropped_by_rate\": 69 }, { \"dropped_by_rate\": 39 }, { \"dropped_by_rate\": 11 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/epsg-3857/out/-yNAME_-z5_-sEPSG@3857.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 240 }, { \"dropped_by_rate\": 262 }, { \"dropped_by_rate\": 239 }, { \"dropped_by_rate\": 208 }, { \"dropped_by_rate\": 121 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "20",
"minzoom": "0",
"name": "tests/highzoom/out/-z30.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 1 }, { \"dropped_by_rate\": 1 }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "0",
"minzoom": "0",
"name": "tests/loop/out/-z0_-O200_--cluster-densest-as-needed.json.check.mbtiles",
"strategies": "[ { \"coalesced_as_needed\": 999 } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "0",
"minzoom": "0",
"name": "tests/loop/out/-z0_-O200_--drop-densest-as-needed.json.check.mbtiles",
"strategies": "[ { \"dropped_as_needed\": 999 } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "0",
"minzoom": "0",
"name": "tests/loop/out/-z0_-O200_--drop-fraction-as-needed.json.check.mbtiles",
"strategies": "[ { \"dropped_as_needed\": 811 } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "3",
"minzoom": "0",
"name": "tests/multilayer/out/-ltogether_-z3.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 8 }, { \"dropped_by_rate\": 7 }, { \"dropped_by_rate\": 5 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "3",
"minzoom": "0",
"name": "separate",
"strategies": "[ { \"dropped_by_rate\": 8 }, { \"dropped_by_rate\": 7 }, { \"dropped_by_rate\": 5 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "11",
"minzoom": "11",
"name": "tests/muni/out/-Z11_-z11_-g2.json.check.mbtiles",
"strategies": "[ { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { \"dropped_by_gamma\": 1674 } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "13",
"minzoom": "11",
"name": "tests/muni/out/-Z11_-z13_-B15.json.check.mbtiles",
"strategies": "[ { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { \"dropped_by_rate\": 4727 }, { \"dropped_by_rate\": 4642 }, { \"dropped_by_rate\": 4209 } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "13",
"minzoom": "11",
"name": "tests/muni/out/-Z11_-z13_-Bf2000.json.check.mbtiles",
"strategies": "[ { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { \"dropped_by_rate\": 2910 }, { }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "13",
"minzoom": "11",
"name": "tests/muni/out/-Z11_-z13_-M10000.json.check.mbtiles",
"strategies": "[ { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { \"dropped_by_rate\": 18540, \"detail_reduced\": 4 }, { \"dropped_by_rate\": 6390, \"detail_reduced\": 3 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "13",
"minzoom": "11",
"name": "tests/muni/out/-Z11_-z13_-M10000_-aG.json.check.mbtiles",
"strategies": "[ { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { \"dropped_by_rate\": 18540, \"dropped_by_gamma\": 65, \"detail_reduced\": 4 }, { \"dropped_by_rate\": 5251, \"dropped_by_gamma\": 155, \"detail_reduced\": 2 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "13",
"minzoom": "11",
"name": "tests/muni/out/-Z11_-z13_-M10000_-ad.json.check.mbtiles",
"strategies": "[ { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { \"dropped_by_rate\": 4080, \"dropped_as_needed\": 107 }, { \"dropped_by_rate\": 2973, \"dropped_as_needed\": 232 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "13",
"minzoom": "11",
"name": "tests/muni/out/-Z11_-z13_-M10000_-pd.json.check.mbtiles",
"strategies": "[ { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { \"dropped_by_rate\": 7695, \"dropped_as_needed\": 93 }, { \"dropped_by_rate\": 4112, \"dropped_as_needed\": 87 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "13",
"minzoom": "11",
"name": "tests/muni/out/-Z11_-z13_-M5000_-as.json.check.mbtiles",
"strategies": "[ { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { \"dropped_by_rate\": 4080, \"dropped_as_needed\": 488 }, { \"dropped_by_rate\": 2973, \"dropped_as_needed\": 1306 }, { \"dropped_as_needed\": 3030 } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "13",
"minzoom": "11",
"name": "tests/muni/out/-Z11_-z13_-O100_--cluster-densest-as-needed.json.check.mbtiles",
"strategies": "[ { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { \"dropped_by_rate\": 4080, \"coalesced_as_needed\": 662 }, { \"dropped_by_rate\": 2973, \"coalesced_as_needed\": 1747 }, { \"coalesced_as_needed\": 4342 } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "13",
"minzoom": "11",
"name": "tests/muni/out/-Z11_-z13_-rf2000.json.check.mbtiles",
"strategies": "[ { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { \"dropped_by_rate\": 2542 }, { \"dropped_by_rate\": 1546 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "13",
"minzoom": "11",
"name": "tests/muni/out/-Z11_-z13_-rf2000_-Bg.json.check.mbtiles",
"strategies": "[ { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { \"dropped_by_rate\": 2546 }, { }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "13",
"minzoom": "11",
"name": "tests/muni/out/-Z11_-z13_-rf2000_-g2.json.check.mbtiles",
"strategies": "[ { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { }, { \"dropped_by_rate\": 1349, \"dropped_by_gamma\": 963 }, { \"dropped_by_rate\": 747, \"dropped_by_gamma\": 753 }, { \"dropped_by_gamma\": 513 } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "14",
"minzoom": "0",
"name": "tests/muni/out/-r1_-K20.json.check.mbtiles",
"strategies": "[ { \"coalesced_as_needed\": 4608 }, { \"coalesced_as_needed\": 4608 }, { \"coalesced_as_needed\": 4608 }, { \"coalesced_as_needed\": 4608 }, { \"coalesced_as_needed\": 4608 }, { \"coalesced_as_needed\": 4607 }, { \"coalesced_as_needed\": 4607 }, { \"coalesced_as_needed\": 4606 }, { \"coalesced_as_needed\": 6005 }, { \"coalesced_as_needed\": 5393 }, { \"coalesced_as_needed\": 5012 }, { \"coalesced_as_needed\": 4727 }, { \"coalesced_as_needed\": 4576 }, { \"coalesced_as_needed\": 4003 }, { \"coalesced_as_needed\": 3002 } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/ne_110m_admin_0_countries/out/-ae_-zg_-M5000_--drop-densest-as-needed.json.check.mbtiles",
"strategies": "[ { \"dropped_as_needed\": 170 }, { \"dropped_as_needed\": 186 }, { \"dropped_as_needed\": 164 }, { \"dropped_as_needed\": 118 }, { \"dropped_as_needed\": 58 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/ne_110m_admin_0_countries/out/-ae_-zg_-M5000_--drop-fraction-as-needed.json.check.mbtiles",
"strategies": "[ { \"dropped_as_needed\": 162 }, { \"dropped_as_needed\": 189 }, { \"dropped_as_needed\": 190 }, { \"dropped_as_needed\": 187 }, { \"dropped_as_needed\": 157 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/ne_110m_admin_0_countries/out/-ae_-zg_-M5000_--drop-smallest-as-needed.json.check.mbtiles",
"strategies": "[ { \"dropped_as_needed\": 173 }, { \"dropped_as_needed\": 197 }, { \"dropped_as_needed\": 183 }, { \"dropped_as_needed\": 156 }, { \"dropped_as_needed\": 62 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/ne_110m_admin_0_countries/out/-ae_-zg_-M5000_--force-feature-limit.json.check.mbtiles",
"strategies": "[ { \"dropped_as_needed\": 318 }, { \"dropped_as_needed\": 477 }, { \"dropped_as_needed\": 225 }, { \"dropped_as_needed\": 117 }, { \"dropped_as_needed\": 10 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "4",
"minzoom": "0",
"name": "tests/ne_110m_admin_0_countries/out/-z4_-yname_--drop-polygons.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 172 }, { \"dropped_by_rate\": 203 }, { \"dropped_by_rate\": 188 }, { \"dropped_by_rate\": 153 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/ne_110m_admin_0_countries/out/-z5_-M5000_--coalesce-densest-as-needed.json.check.mbtiles",
"strategies": "[ { \"coalesced_as_needed\": 704, \"detail_reduced\": 3 }, { \"coalesced_as_needed\": 211 }, { \"coalesced_as_needed\": 190 }, { \"coalesced_as_needed\": 148 }, { \"coalesced_as_needed\": 58 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/ne_110m_admin_0_countries/out/-z5_-M5000_--coalesce-fraction-as-needed.json.check.mbtiles",
"strategies": "[ { \"coalesced_as_needed\": 704, \"detail_reduced\": 3 }, { \"coalesced_as_needed\": 216 }, { \"coalesced_as_needed\": 219 }, { \"coalesced_as_needed\": 226 }, { \"coalesced_as_needed\": 180 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/ne_110m_admin_0_countries/out/-z5_-M5000_--coalesce-smallest-as-needed.json.check.mbtiles",
"strategies": "[ { \"coalesced_as_needed\": 1431 }, { \"coalesced_as_needed\": 1308 }, { \"coalesced_as_needed\": 801 }, { \"coalesced_as_needed\": 190 }, { \"coalesced_as_needed\": 72 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/ne_110m_admin_0_countries/out/-z5_-M5000_--drop-smallest-as-needed.json.check.mbtiles",
"strategies": "[ { \"dropped_as_needed\": 173 }, { \"dropped_as_needed\": 197 }, { \"dropped_as_needed\": 183 }, { \"dropped_as_needed\": 156 }, { \"dropped_as_needed\": 62 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "7",
"minzoom": "1",
"name": "tests/ne_110m_admin_1_states_provinces_lines/out/-lcountries_-P_-Z1_-z7_-b4_-xfeaturecla_-xscalerank_-acrol_-ps.json.check.mbtiles",
"strategies": "[ { }, { \"dropped_by_rate\": 109 }, { \"dropped_by_rate\": 125 }, { \"dropped_by_rate\": 140 }, { \"dropped_by_rate\": 125 }, { \"dropped_by_rate\": 123 }, { \"dropped_by_rate\": 98 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/ne_110m_admin_1_states_provinces_lines/out/-z5_-M500_--drop-smallest-as-needed.json.check.mbtiles",
"strategies": "[ { \"dropped_as_needed\": 92 }, { \"dropped_as_needed\": 97 }, { \"dropped_as_needed\": 115 }, { \"dropped_as_needed\": 113 }, { \"dropped_as_needed\": 106 }, { \"dropped_as_needed\": 102 } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "14",
"minzoom": "0",
"name": "tests/ne_110m_populated_places/out/-yNAME.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 242 }, { \"dropped_by_rate\": 268 }, { \"dropped_by_rate\": 255 }, { \"dropped_by_rate\": 248 }, { \"dropped_by_rate\": 207 }, { \"dropped_by_rate\": 139 }, { \"dropped_by_rate\": 78 }, { \"dropped_by_rate\": 33 }, { \"dropped_by_rate\": 13 }, { \"dropped_by_rate\": 5 }, { \"dropped_by_rate\": 4 }, { \"dropped_by_rate\": 2 }, { \"dropped_by_rate\": 1 }, { }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/ne_110m_populated_places/out/-yNAME_-Ccat_-z5.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 240 }, { \"dropped_by_rate\": 258 }, { \"dropped_by_rate\": 239 }, { \"dropped_by_rate\": 208 }, { \"dropped_by_rate\": 121 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "4",
"minzoom": "0",
"name": "tests/ne_110m_populated_places/out/-yNAME_-z4_--no-tile-stats.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 236 }, { \"dropped_by_rate\": 251 }, { \"dropped_by_rate\": 216 }, { \"dropped_by_rate\": 146 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "4",
"minzoom": "0",
"name": "tests/ne_110m_populated_places/out/-yNAME_-z4_-C.%tests%filter%remove.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 236 }, { \"dropped_by_rate\": 247 }, { \"dropped_by_rate\": 216 }, { \"dropped_by_rate\": 146 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "4",
"minzoom": "0",
"name": "tests/ne_110m_populated_places/out/-yNAME_-z4_-C.%tests%filter%rename.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 236 }, { \"dropped_by_rate\": 247 }, { \"dropped_by_rate\": 216 }, { \"dropped_by_rate\": 146 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "4",
"minzoom": "0",
"name": "tests/ne_110m_populated_places/out/-yNAME_-z4_-C.%tests%filter%rename_-c.%tests%filter%rename2.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 236 }, { \"dropped_by_rate\": 247 }, { \"dropped_by_rate\": 216 }, { \"dropped_by_rate\": 146 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/ne_110m_populated_places/out/-yNAME_-z5.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 240 }, { \"dropped_by_rate\": 262 }, { \"dropped_by_rate\": 239 }, { \"dropped_by_rate\": 208 }, { \"dropped_by_rate\": 121 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/ne_110m_populated_places/out/-yNAME_-z5_--drop-smallest-as-needed.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 240 }, { \"dropped_by_rate\": 262 }, { \"dropped_by_rate\": 239 }, { \"dropped_by_rate\": 208 }, { \"dropped_by_rate\": 121 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/ne_110m_populated_places/out/-yNAME_-z5_-B3.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 227 }, { \"dropped_by_rate\": 227 }, { \"dropped_by_rate\": 152 }, { }, { }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/ne_110m_populated_places/out/-yNAME_-z5_-c.%tests%filter%rename.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 240 }, { \"dropped_by_rate\": 262 }, { \"dropped_by_rate\": 239 }, { \"dropped_by_rate\": 208 }, { \"dropped_by_rate\": 121 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/ne_110m_populated_places/out/-yNAME_-z5_-ccat.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 240 }, { \"dropped_by_rate\": 262 }, { \"dropped_by_rate\": 239 }, { \"dropped_by_rate\": 208 }, { \"dropped_by_rate\": 121 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "5",
"minzoom": "0",
"name": "tests/ne_110m_populated_places/out/-yNAME_-z5_-r1.5.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 211 }, { \"dropped_by_rate\": 214 }, { \"dropped_by_rate\": 179 }, { \"dropped_by_rate\": 138 }, { \"dropped_by_rate\": 70 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@ -8,6 +8,7 @@
"maxzoom": "4",
"minzoom": "0",
"name": "tests/ne_110m_populated_places/out/-z4_-yNAME_-c.%tests%filter%remove.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 236 }, { \"dropped_by_rate\": 251 }, { \"dropped_by_rate\": 216 }, { \"dropped_by_rate\": 146 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "3",
"minzoom": "0",
"name": "tests/ne_110m_populated_places/out/-zg_-K20_-r1_-yNAME.json.check.mbtiles",
"strategies": "[ { \"coalesced_as_needed\": 207 }, { \"coalesced_as_needed\": 174 }, { \"coalesced_as_needed\": 108 }, { \"coalesced_as_needed\": 51 } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "4",
"minzoom": "0",
"name": "tests/nullisland/out/-b0_-z4.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 4 }, { \"dropped_by_rate\": 8 }, { \"dropped_by_rate\": 8 }, { \"dropped_by_rate\": 3 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -9,6 +9,7 @@
"maxzoom": "4",
"minzoom": "0",
"name": "tests/nullisland/out/-b0_-z4_-ANullIsland.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 4 }, { \"dropped_by_rate\": 8 }, { \"dropped_by_rate\": 8 }, { \"dropped_by_rate\": 3 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -8,6 +8,7 @@
"maxzoom": "4",
"minzoom": "0",
"name": "tests/nullisland/out/-b0_-z4_-NNullIsland.json.check.mbtiles",
"strategies": "[ { \"dropped_by_rate\": 4 }, { \"dropped_by_rate\": 8 }, { \"dropped_by_rate\": 8 }, { \"dropped_by_rate\": 3 }, { } ]",
"type": "overlay",
"version": "2"
}, "features": [

View File

@ -30,6 +30,7 @@
#include "evaluator.hpp"
#include "csv.hpp"
#include "text.hpp"
#include "tile.hpp"
#include <fstream>
#include <sstream>
#include <algorithm>
@ -1124,7 +1125,9 @@ int main(int argc, char **argv) {
}
}
mbtiles_write_metadata(outdb, out_dir, name.c_str(), st.minzoom, st.maxzoom, st.minlat, st.minlon, st.maxlat, st.maxlon, st.midlat, st.midlon, 0, attribution.size() != 0 ? attribution.c_str() : NULL, layermap, true, description.c_str(), !pg, attribute_descriptions, "tile-join", generator_options);
std::vector<strategy> strategies;
mbtiles_write_metadata(outdb, out_dir, name.c_str(), st.minzoom, st.maxzoom, st.minlat, st.minlon, st.maxlat, st.maxlon, st.midlat, st.midlon, 0, attribution.size() != 0 ? attribution.c_str() : NULL, layermap, true, description.c_str(), !pg, attribute_descriptions, "tile-join", generator_options, strategies);
if (outdb != NULL) {
mbtiles_close(outdb, argv[0]);

View File

@ -1221,6 +1221,8 @@ struct write_tile_args {
int wrote_zoom = 0;
size_t tiling_seg = 0;
struct json_object *filter = NULL;
std::atomic<size_t> *dropped_count = NULL;
atomic_strategy *strategy = NULL;
};
bool clip_to_tile(serial_feature &sf, int z, long long buffer) {
@ -1718,7 +1720,7 @@ static bool line_is_too_small(drawvec const &geometry, int z, int detail) {
return true;
}
long long write_tile(FILE *geoms, std::atomic<long long> *geompos_in, char *metabase, char *stringpool, int z, unsigned tx, unsigned ty, int detail, int min_detail, sqlite3 *outdb, const char *outdir, int buffer, const char *fname, FILE **geomfile, int minzoom, int maxzoom, double todo, std::atomic<long long> *along, long long alongminus, double gamma, int child_shards, long long *meta_off, long long *pool_off, unsigned *initial_x, unsigned *initial_y, std::atomic<int> *running, double simplification, std::vector<std::map<std::string, layermap_entry>> *layermaps, std::vector<std::vector<std::string>> *layer_unmaps, size_t tiling_seg, size_t pass, size_t passes, unsigned long long mingap, long long minextent, double fraction, const char *prefilter, const char *postfilter, struct json_object *filter, write_tile_args *arg) {
long long write_tile(FILE *geoms, std::atomic<long long> *geompos_in, char *metabase, char *stringpool, int z, unsigned tx, unsigned ty, int detail, int min_detail, sqlite3 *outdb, const char *outdir, int buffer, const char *fname, FILE **geomfile, int minzoom, int maxzoom, double todo, std::atomic<long long> *along, long long alongminus, double gamma, int child_shards, long long *meta_off, long long *pool_off, unsigned *initial_x, unsigned *initial_y, std::atomic<int> *running, double simplification, std::vector<std::map<std::string, layermap_entry>> *layermaps, std::vector<std::vector<std::string>> *layer_unmaps, size_t tiling_seg, size_t pass, size_t passes, unsigned long long mingap, long long minextent, double fraction, const char *prefilter, const char *postfilter, struct json_object *filter, write_tile_args *arg, atomic_strategy *strategy) {
int line_detail;
double merge_fraction = 1;
double mingap_fraction = 1;
@ -1759,6 +1761,7 @@ long long write_tile(FILE *geoms, std::atomic<long long> *geompos_in, char *meta
double fraction_accum = 0;
unsigned long long previndex = 0, density_previndex = 0, merge_previndex = 0;
unsigned long long extent_previndex = 0;
double scale = (double) (1LL << (64 - 2 * (z + 8)));
double gap = 0, density_gap = 0;
double spacing = 0;
@ -1870,9 +1873,17 @@ long long write_tile(FILE *geoms, std::atomic<long long> *geompos_in, char *meta
break;
}
if (sf.t == VT_POINT) {
double radius = sqrt(sf.index - extent_previndex) / 4.0;
sf.extent = M_PI * radius * radius;
extent_previndex = sf.index;
}
if (sf.dropped) {
if (find_partial(partials, sf, which_partial, layer_unmaps)) {
preserve_attributes(arg->attribute_accum, sf, stringpool, pool_off, partials[which_partial]);
strategy->dropped_by_rate++;
continue;
}
}
@ -1880,6 +1891,7 @@ long long write_tile(FILE *geoms, std::atomic<long long> *geompos_in, char *meta
if (gamma > 0) {
if (manage_gap(sf.index, &previndex, scale, gamma, &gap) && find_partial(partials, sf, which_partial, layer_unmaps)) {
preserve_attributes(arg->attribute_accum, sf, stringpool, pool_off, partials[which_partial]);
strategy->dropped_by_gamma++;
continue;
}
}
@ -1902,12 +1914,14 @@ long long write_tile(FILE *geoms, std::atomic<long long> *geompos_in, char *meta
}
preserve_attributes(arg->attribute_accum, sf, stringpool, pool_off, partials[which_partial]);
strategy->coalesced_as_needed++;
continue;
}
} else if (additional[A_DROP_DENSEST_AS_NEEDED]) {
indices.push_back(sf.index);
if (sf.index - merge_previndex < mingap && find_partial(partials, sf, which_partial, layer_unmaps)) {
preserve_attributes(arg->attribute_accum, sf, stringpool, pool_off, partials[which_partial]);
strategy->dropped_as_needed++;
continue;
}
} else if (additional[A_COALESCE_DENSEST_AS_NEEDED]) {
@ -1916,12 +1930,14 @@ long long write_tile(FILE *geoms, std::atomic<long long> *geompos_in, char *meta
partials[which_partial].geoms.push_back(sf.geometry);
coalesced_area += sf.extent;
preserve_attributes(arg->attribute_accum, sf, stringpool, pool_off, partials[which_partial]);
strategy->coalesced_as_needed++;
continue;
}
} else if (additional[A_DROP_SMALLEST_AS_NEEDED]) {
extents.push_back(sf.extent);
if (sf.extent + coalesced_area <= minextent && sf.t != VT_POINT && find_partial(partials, sf, which_partial, layer_unmaps)) {
if (sf.extent + coalesced_area <= minextent && find_partial(partials, sf, which_partial, layer_unmaps)) {
preserve_attributes(arg->attribute_accum, sf, stringpool, pool_off, partials[which_partial]);
strategy->dropped_as_needed++;
continue;
}
} else if (additional[A_COALESCE_SMALLEST_AS_NEEDED]) {
@ -1930,6 +1946,7 @@ long long write_tile(FILE *geoms, std::atomic<long long> *geompos_in, char *meta
partials[which_partial].geoms.push_back(sf.geometry);
coalesced_area += sf.extent;
preserve_attributes(arg->attribute_accum, sf, stringpool, pool_off, partials[which_partial]);
strategy->coalesced_as_needed++;
continue;
}
}
@ -1951,6 +1968,9 @@ long long write_tile(FILE *geoms, std::atomic<long long> *geompos_in, char *meta
if (additional[A_COALESCE_FRACTION_AS_NEEDED]) {
partials[which_partial].geoms.push_back(sf.geometry);
coalesced_area += sf.extent;
strategy->coalesced_as_needed++;
} else {
strategy->dropped_as_needed++;
}
preserve_attributes(arg->attribute_accum, sf, stringpool, pool_off, partials[which_partial]);
continue;
@ -2512,6 +2532,8 @@ long long write_tile(FILE *geoms, std::atomic<long long> *geompos_in, char *meta
arg->still_dropping = true;
}
line_detail++; // to keep it the same when the loop decrements it
} else {
strategy->detail_reduced++;
}
} else {
if (pass == 1) {
@ -2588,7 +2610,7 @@ void *run_thread(void *vargs) {
// fprintf(stderr, "%d/%u/%u\n", z, x, y);
long long len = write_tile(geom, &geompos, arg->metabase, arg->stringpool, z, x, y, z == arg->maxzoom ? arg->full_detail : arg->low_detail, arg->min_detail, arg->outdb, arg->outdir, arg->buffer, arg->fname, arg->geomfile, arg->minzoom, arg->maxzoom, arg->todo, arg->along, geompos, arg->gamma, arg->child_shards, arg->meta_off, arg->pool_off, arg->initial_x, arg->initial_y, arg->running, arg->simplification, arg->layermaps, arg->layer_unmaps, arg->tiling_seg, arg->pass, arg->passes, arg->mingap, arg->minextent, arg->fraction, arg->prefilter, arg->postfilter, arg->filter, arg);
long long len = write_tile(geom, &geompos, arg->metabase, arg->stringpool, z, x, y, z == arg->maxzoom ? arg->full_detail : arg->low_detail, arg->min_detail, arg->outdb, arg->outdir, arg->buffer, arg->fname, arg->geomfile, arg->minzoom, arg->maxzoom, arg->todo, arg->along, geompos, arg->gamma, arg->child_shards, arg->meta_off, arg->pool_off, arg->initial_x, arg->initial_y, arg->running, arg->simplification, arg->layermaps, arg->layer_unmaps, arg->tiling_seg, arg->pass, arg->passes, arg->mingap, arg->minextent, arg->fraction, arg->prefilter, arg->postfilter, arg->filter, arg, arg->strategy);
if (len < 0) {
int *err = &arg->err;
@ -2653,7 +2675,7 @@ void *run_thread(void *vargs) {
return NULL;
}
int traverse_zooms(int *geomfd, off_t *geom_size, char *metabase, char *stringpool, std::atomic<unsigned> *midx, std::atomic<unsigned> *midy, int &maxzoom, int minzoom, sqlite3 *outdb, const char *outdir, int buffer, const char *fname, const char *tmpdir, double gamma, int full_detail, int low_detail, int min_detail, long long *meta_off, long long *pool_off, unsigned *initial_x, unsigned *initial_y, double simplification, std::vector<std::map<std::string, layermap_entry>> &layermaps, const char *prefilter, const char *postfilter, std::map<std::string, attribute_op> const *attribute_accum, struct json_object *filter) {
int traverse_zooms(int *geomfd, off_t *geom_size, char *metabase, char *stringpool, std::atomic<unsigned> *midx, std::atomic<unsigned> *midy, int &maxzoom, int minzoom, sqlite3 *outdb, const char *outdir, int buffer, const char *fname, const char *tmpdir, double gamma, int full_detail, int low_detail, int min_detail, long long *meta_off, long long *pool_off, unsigned *initial_x, unsigned *initial_y, double simplification, std::vector<std::map<std::string, layermap_entry>> &layermaps, const char *prefilter, const char *postfilter, std::map<std::string, attribute_op> const *attribute_accum, struct json_object *filter, std::vector<strategy> &strategies) {
last_progress = 0;
// The existing layermaps are one table per input thread.
@ -2800,6 +2822,7 @@ int traverse_zooms(int *geomfd, off_t *geom_size, char *metabase, char *stringpo
args.resize(threads);
std::atomic<int> running(threads);
std::atomic<long long> along(0);
atomic_strategy strategy;
for (size_t thread = 0; thread < threads; thread++) {
args[thread].metabase = metabase;
@ -2850,6 +2873,7 @@ int traverse_zooms(int *geomfd, off_t *geom_size, char *metabase, char *stringpo
args[thread].passes = 2 - start;
args[thread].wrote_zoom = -1;
args[thread].still_dropping = false;
args[thread].strategy = &strategy;
if (pthread_create(&pthreads[thread], NULL, run_thread, &args[thread]) != 0) {
perror("pthread_create");
@ -2890,6 +2914,12 @@ int traverse_zooms(int *geomfd, off_t *geom_size, char *metabase, char *stringpo
maxzoom++;
}
}
if ((size_t) i >= strategies.size()) {
strategies.resize(i + 1);
}
struct strategy s(strategy);
strategies[i] = s;
}
for (size_t j = 0; j < TEMP_FILES; j++) {

View File

@ -19,9 +19,43 @@ enum attribute_op {
op_min,
};
long long write_tile(char **geom, char *metabase, char *stringpool, unsigned *file_bbox, int z, unsigned x, unsigned y, int detail, int min_detail, int basezoom, sqlite3 *outdb, const char *outdir, double droprate, int buffer, const char *fname, FILE **geomfile, int file_minzoom, int file_maxzoom, double todo, char *geomstart, long long along, double gamma, int nlayers);
struct atomic_strategy {
std::atomic<size_t> dropped_by_rate;
std::atomic<size_t> dropped_by_gamma;
std::atomic<size_t> dropped_as_needed;
std::atomic<size_t> coalesced_as_needed;
std::atomic<size_t> detail_reduced;
int traverse_zooms(int *geomfd, off_t *geom_size, char *metabase, char *stringpool, std::atomic<unsigned> *midx, std::atomic<unsigned> *midy, int &maxzoom, int minzoom, sqlite3 *outdb, const char *outdir, int buffer, const char *fname, const char *tmpdir, double gamma, int full_detail, int low_detail, int min_detail, long long *meta_off, long long *pool_off, unsigned *initial_x, unsigned *initial_y, double simplification, std::vector<std::map<std::string, layermap_entry> > &layermap, const char *prefilter, const char *postfilter, std::map<std::string, attribute_op> const *attribute_accum, struct json_object *filter);
atomic_strategy() :
dropped_by_rate(0),
dropped_by_gamma(0),
dropped_as_needed(0),
coalesced_as_needed(0),
detail_reduced(0) {
}
};
struct strategy {
size_t dropped_by_rate = 0;
size_t dropped_by_gamma = 0;
size_t dropped_as_needed = 0;
size_t coalesced_as_needed = 0;
size_t detail_reduced = 0;
strategy(const atomic_strategy &s) {
dropped_by_rate = s.dropped_by_rate;
dropped_by_gamma = s.dropped_by_gamma;
dropped_as_needed = s.dropped_as_needed;
coalesced_as_needed = s.coalesced_as_needed;
detail_reduced = s.detail_reduced;
}
strategy() = default;
};
long long write_tile(char **geom, char *metabase, char *stringpool, unsigned *file_bbox, int z, unsigned x, unsigned y, int detail, int min_detail, int basezoom, sqlite3 *outdb, const char *outdir, double droprate, int buffer, const char *fname, FILE **geomfile, int file_minzoom, int file_maxzoom, double todo, char *geomstart, long long along, double gamma, int nlayers, std::atomic<strategy> *strategy);
int traverse_zooms(int *geomfd, off_t *geom_size, char *metabase, char *stringpool, std::atomic<unsigned> *midx, std::atomic<unsigned> *midy, int &maxzoom, int minzoom, sqlite3 *outdb, const char *outdir, int buffer, const char *fname, const char *tmpdir, double gamma, int full_detail, int low_detail, int min_detail, long long *meta_off, long long *pool_off, unsigned *initial_x, unsigned *initial_y, double simplification, std::vector<std::map<std::string, layermap_entry> > &layermap, const char *prefilter, const char *postfilter, std::map<std::string, attribute_op> const *attribute_accum, struct json_object *filter, std::vector<strategy> &strategies);
int manage_gap(unsigned long long index, unsigned long long *previndex, double scale, double gamma, double *gap);

View File

@ -1,6 +1,6 @@
#ifndef VERSION_HPP
#define VERSION_HPP
#define VERSION "v2.2.0"
#define VERSION "v2.3.0"
#endif