Elasticsearch Index Corruption or Mapping Issues

Elasticsearch indices for DataHub are showing errors, search functionality is broken, or index mapping conflicts prevent new data from being indexed. This typically occurs after version upgrades, manual index modifications, or during high write load.

Common Error Messages:

• MapperParsingException: Field type mismatch
• ClusterBlockException: index read-only
• Circuit breaker exception
• Index not found

You Might Be Asking:

• How do I rebuild Elasticsearch indices?
• Why is search not returning any results?
• What do I do about mapping conflicts?

1. Check Elasticsearch health:

# Check cluster health
kubectl exec -it elasticsearch-0 -- curl localhost:9200/_cluster/health?pretty

# Check indices status
kubectl exec -it elasticsearch-0 -- curl localhost:9200/_cat/indices?v | grep datahub

# Check for errors
kubectl logs elasticsearch-0 | grep -i error

2. Rebuild all DataHub indices:

# Using DataHub CLI
datahub system reindex

# Or via API
curl -X POST http://localhost:8080/api/v2/system/reindex \
  -H "Authorization: Bearer YOUR_TOKEN"

# For specific entity type
curl -X POST http://localhost:8080/api/v2/system/reindex \
  -H "Content-Type: application/json" \
  -d '{"entityType": "dataset"}'

3. Fix read-only indices:

# Remove read-only block
kubectl exec -it elasticsearch-0 -- curl -X PUT "localhost:9200/_all/_settings" \
  -H 'Content-Type: application/json' \
  -d '{"index.blocks.read_only_allow_delete": null}'

# Check disk space (common cause)
kubectl exec -it elasticsearch-0 -- df -h

4. Delete and recreate specific index:

# List DataHub indices
kubectl exec -it elasticsearch-0 -- curl localhost:9200/_cat/indices?v | grep datahub

# Delete problematic index
kubectl exec -it elasticsearch-0 -- curl -X DELETE "localhost:9200/datasetindex_v2"

# Trigger reindex for that entity type
curl -X POST http://localhost:8080/api/v2/system/reindex \
  -d '{"entityType": "dataset"}'

5. Increase Elasticsearch resources for large datasets:

# In Helm values.yaml
elasticsearch:
  resources:
    requests:
      memory: "4Gi"
      cpu: "2"
    limits:
      memory: "8Gi"
      cpu: "4"

  # Increase heap
  env:
    - name: ES_JAVA_OPTS
      value: "-Xms4g -Xmx4g"

  # Increase circuit breaker limits
  esConfig:
    elasticsearch.yml: |
      indices.breaker.total.limit: 70%

Always backup Elasticsearch data before major operations. Reindexing can take significant time for large catalogs (10K+ entities). Consider running reindex during maintenance windows.

• Search & Discovery
• Elasticsearch Configuration