Attribute Filtering Rules for Vector Tile Pipelines

Raw geospatial datasets routinely carry hundreds of columns — administrative codes, audit timestamps, internal identifiers, and legacy metadata — that are irrelevant to a browser renderer. When those attributes pass unfiltered into Tippecanoe, they inflate Protobuf payloads, degrade client-side parsing performance, and complicate style maintenance. Implementing deterministic, version-controlled attribute filtering ensures only rendering-relevant properties survive the transformation from source data to cached Mapbox Vector Tiles stored in an MBTiles SQLite container.

Prerequisites

Before implementing filtering logic, confirm this environment baseline:

For teams standardizing on columnar storage, GeoParquet input processing covers schema alignment and partition strategies that must be resolved before attribute selection begins.

Core Concept: The Tippecanoe Attribute Flag Set

Tippecanoe controls attribute inclusion and exclusion through four flags. Apply them to the same invocation; -y and -x can each be repeated.

When -y flags are active, Tippecanoe silently drops any attribute not on the allowlist. This prevents accidental metadata leakage into production tiles and is the preferred mode for production builds. The -x approach is better suited to exploratory pipelines where the full column list is unknown at invocation time.

How Attribute Bloat Propagates Through the MVT Format

The Mapbox Vector Tile specification stores attributes in a per-layer dictionary: keys are written once into a keys array, values are deduplicated into a values array, and each feature references them by integer index. An unused column still occupies entries in both arrays across every tile that contains it — even when the value is null.

The diagram below shows where filtering intercepts the pipeline and what remains unfiltered without it.

Step-by-Step Implementation

Step 1 — Audit Source Attributes Against Style Usage

Run a schema inventory before writing a single filter rule. Parse your MapLibre GL or Mapbox GL style JSON, extract every attribute referenced in paint, layout, and filter expressions, then diff that set against the source dataset schema.

import json, pathlib

def extract_style_attributes(style_path: str) -> set[str]:
    style = json.loads(pathlib.Path(style_path).read_text())
    attrs: set[str] = set()

    def walk(obj):
        if isinstance(obj, list):
            # Mapbox expression: ["get", "attr_name"] or ["==", ["get", "attr"], value]
            if len(obj) >= 2 and obj[0] == "get" and isinstance(obj[1], str):
                attrs.add(obj[1])
            for item in obj:
                walk(item)
        elif isinstance(obj, dict):
            for v in obj.values():
                walk(v)

    for layer in style.get("layers", []):
        walk(layer.get("paint", {}))
        walk(layer.get("layout", {}))
        walk(layer.get("filter", []))

    return attrs

used = extract_style_attributes("style.json")
print("Attributes used by style:", sorted(used))

Any column not in used — plus any column powering hover tooltips or click popups — is a candidate for removal. Columns that feed server-side analytics but not the frontend should be dropped at this stage. For a deep dive into systematic bloat elimination, see Dropping Unused Attributes to Reduce Tile Size.

Verify: Column count before vs after the audit. A typical OpenStreetMap buildings export has 30–60 columns; a styled buildings layer usually needs 4–8.

Step 2 — Define a Deterministic JSON Filtering Policy

Codify the audit output as a versioned JSON policy. Store it in version control alongside your tile configuration so every pipeline run is reproducible.

{
  "policy_version": "1.2",
  "layers": {
    "buildings": {
      "keep": ["height", "floors", "building_type", "name"],
      "drop": ["legacy_id", "internal_audit_flag", "created_at", "updated_by"],
      "coerce": {
        "height": "float",
        "floors": "int",
        "is_heritage": "bool"
      },
      "rename": {
        "bldg_type": "building_type"
      }
    },
    "roads": {
      "keep": ["maxspeed", "surface", "oneway", "name", "highway"],
      "drop": ["maintenance_schedule", "contractor_id", "audit_ts"],
      "coerce": { "maxspeed": "int" },
      "rename": {}
    }
  }
}

Verify: Policy file committed to git and peer-reviewed. Any change to keep or drop must trigger a tile rebuild.

Step 3 — Execute Pre-Filtering with pyarrow

Apply the policy before invoking the tiler. Pre-filtering reduces memory pressure during geometry simplification and ensures type coercion does not conflict with Tippecanoe’s internal type inference.

import pyarrow.parquet as pq
import pyarrow as pa
import pyarrow.compute as pc
import json, pathlib

def apply_filter_policy(
    input_path: str,
    output_path: str,
    policy_path: str,
    layer: str
) -> dict:
    policy = json.loads(pathlib.Path(policy_path).read_text())
    layer_policy = policy["layers"][layer]

    table = pq.read_table(input_path)
    schema_names = set(table.schema.names)

    # 1. Rename columns first so later selects use canonical names
    for old, new in layer_policy.get("rename", {}).items():
        if old in schema_names:
            idx = table.schema.get_field_index(old)
            table = table.rename_columns(
                [new if i == idx else table.schema.names[i]
                 for i in range(len(table.schema.names))]
            )

    # 2. Build keep list (geometry is mandatory)
    keep_cols = [c for c in layer_policy["keep"] if c in table.schema.names]
    geo_cols  = [c for c in ["geometry"] if c in table.schema.names]
    table = table.select(keep_cols + geo_cols)

    # 3. Coerce types to MVT-compatible values
    type_map = {"int": pa.int64(), "float": pa.float64(), "bool": pa.bool_()}
    for col_name, target in layer_policy.get("coerce", {}).items():
        if col_name in table.schema.names and target in type_map:
            idx = table.schema.get_field_index(col_name)
            table = table.set_column(
                idx, col_name,
                table[col_name].cast(type_map[target], safe=False)
            )

    pq.write_table(table, output_path, compression="snappy")
    return {"input_cols": len(schema_names), "output_cols": len(table.schema.names)}

stats = apply_filter_policy(
    "buildings_raw.geoparquet",
    "buildings_filtered.geoparquet",
    "filter_policy.json",
    "buildings"
)
print(f"Columns reduced: {stats['input_cols']} → {stats['output_cols']}")

Verify: pq.read_schema("buildings_filtered.geoparquet").names should match keep + ["geometry"] exactly.

Step 4 — Apply Tippecanoe -y Flags as a Final Gate

Use Tippecanoe’s -y / --include flag as a second enforcement layer. Even if pre-filtering succeeds, this prevents any accidental column reintroduction (e.g., from a schema migration or a new source file).

tippecanoe \
  --output=buildings.mbtiles \
  --layer=buildings \
  -y height -y floors -y building_type -y name \
  --drop-densest-as-needed \
  --maximum-zoom=16 \
  --minimum-zoom=10 \
  --coalesce-densest-as-needed \
  --no-tile-size-limit \
  buildings_filtered.geoparquet

For a full reference on production flag combinations including --drop-densest-as-needed, rate-control flags, and zoom-range tuning, consult Tippecanoe CLI Fundamentals.

Verify: tippecanoe-decode buildings.mbtiles 14/8239/5467 | python3 -c "import sys,json; d=json.load(sys.stdin); print(list(d['features'][0]['properties'].keys()))" — output should list only the four allowed keys.

Step 5 — Validate in CI/CD Before Promoting to Staging

Attribute filtering rules must be validated automatically before tiles reach staging or production.

#!/usr/bin/env bash
# ci/validate_tiles.sh
set -euo pipefail

TILE_PATH="$1"          # e.g., buildings.mbtiles
POLICY_PATH="$2"        # e.g., filter_policy.json
LAYER="$3"              # e.g., buildings
MAX_TILE_KB="${4:-500}"  # default 500 KB budget per tile

ALLOWED=$(python3 -c "
import json, sys
p = json.load(open('$POLICY_PATH'))
print(' '.join(p['layers']['$LAYER']['keep']))
")

# 1. Schema diff: sample tile at z14 center
SAMPLE=$(tippecanoe-decode "$TILE_PATH" 14/8239/5467 2>/dev/null \
  | python3 -c "import sys,json; f=json.load(sys.stdin)['features']; print(json.dumps(list(f[0]['properties'].keys()) if f else []))")

# 2. Check for forbidden keys
for key in $(sqlite3 "$TILE_PATH" "SELECT DISTINCT json_each.value FROM tiles, json_each(tilejson) LIMIT 1" 2>/dev/null || echo ""); do
  if ! echo "$ALLOWED" | grep -qw "$key"; then
    echo "FAIL: forbidden attribute '$key' found in tile" && exit 1
  fi
done

# 3. Tile size regression — 95th percentile
P95=$(sqlite3 "$TILE_PATH" \
  "SELECT length(tile_data) FROM tiles ORDER BY length(tile_data) DESC LIMIT 1" \
  | awk '{printf "%.0f", $1/1024}')

if [ "$P95" -gt "$MAX_TILE_KB" ]; then
  echo "FAIL: max tile size ${P95}KB exceeds budget ${MAX_TILE_KB}KB" && exit 1
fi

echo "PASS: schema and size checks passed (max tile: ${P95}KB)"

Integrate this script into your GitHub Actions or GitLab CI pipeline. Gate merges to main on a successful exit code. Store filter_policy.json in the same repository as your tile generation config and require peer review for any keep/drop modifications.

Optimization Knobs

Integration with Adjacent Pipeline Stages

Filtered GeoParquet output from Step 3 feeds directly into Tippecanoe (Step 4), which writes an MBTiles SQLite container. From there, the .mbtiles file is either served directly via a tile server like Martin, or converted to .pmtiles for cloud-native CDN delivery.

When binding filtered attributes to MapLibre GL style expressions for data-driven styling, the keep list in your policy becomes the contract between the pipeline and the frontend team. Any attribute a style expression references via ["get", "attr"] must appear in keep, or the expression silently returns null at runtime.

If your pipeline feeds a MapLibre GL JSON structure that targets multiple source layers, maintain one policy block per layer. Mismatched keys between layers cause style expression failures that only surface at render time, not at tile validation time.

Troubleshooting

Filtered tiles still contain unexpected attributes

Diagnosis: Pre-filter and Tippecanoe -y flags out of sync.

tippecanoe-decode buildings.mbtiles 14/8239/5467 \
  | python3 -c "import sys,json; d=json.load(sys.stdin); \
    props=d['features'][0]['properties']; print(sorted(props.keys()))"

Fix: Confirm filter_policy.json keep list matches -y flags exactly. Regenerate the filtered GeoParquet, then re-run Tippecanoe.

null values persist after coercion, causing type errors in style filters

Diagnosis: pa.bool_() cast on a column with null values raises ArrowInvalid in safe mode.

# Reproduce
table["is_heritage"].cast(pa.bool_(), safe=True)  # raises ArrowInvalid if nulls present

Fix: Fill nulls before casting: pc.if_else(pc.is_null(table["is_heritage"]), False, table["is_heritage"].cast(pa.bool_())).

Array-type columns cause Tippecanoe to emit WARNING: ... is not a string, number, or boolean

Diagnosis: MVT does not support arrays. Source column contains JSON arrays or list types.

ogrinfo -al -so buildings_filtered.geoparquet | grep "Array"

Fix: Flatten array fields during the coerce phase: extract the primary element with pc.list_slice(col, 0, 1) or join with pc.binary_join_element_wise.

Tile size budget exceeded after adding attributes to keep

Diagnosis: Adding attributes increases tile payload beyond the 500 KB ceiling at peak zoom levels.

sqlite3 buildings.mbtiles \
  "SELECT zoom_level, COUNT(*), MAX(length(tile_data))/1024 AS max_kb \
   FROM tiles GROUP BY zoom_level ORDER BY zoom_level"

Fix: Add --drop-densest-as-needed or reduce --maximum-zoom by one level. Alternatively, split the layer and apply a separate, shorter keep list to the high-density zoom range.

Renamed attributes not reaching the style layer

Diagnosis: rename in the policy was applied but the style still references the old key.

# Check properties in a decoded tile
tippecanoe-decode buildings.mbtiles 14/8239/5467 \
  | python3 -c "import sys,json; d=json.load(sys.stdin); \
    [print(f['properties']) for f in d['features'][:3]]"

Fix: Update style ["get", "old_name"] references to the new canonical key, or add old_name as an alias in keep temporarily during the migration window.

Child Pages

• Dropping Unused Attributes to Reduce Tile Size — quantifies MVT dictionary bloat from unused columns and shows how -y flags interact with Protobuf key/value encoding to shrink .pbf files by 15–60%.

Parent: Automated Generation Pipelines with Tippecanoe

Related:

• GeoParquet Input Processing — schema alignment and partition strategies that determine which columns are available for filtering.
• Geometry Simplification Algorithms — the downstream step where pre-filtered data enters vertex-reduction processing.
• Tippecanoe CLI Fundamentals — complete flag reference covering --drop-densest-as-needed, zoom-range strategy, and output format selection.
• MBTiles Architecture and Limits — how the SQLite tile container stores filtered attributes and what size constraints apply per tile.
• Dynamic Attribute Mapping in MapLibre — binding filtered tile properties to data-driven style expressions at render time.