ElasticSearch 6 – Spatial Queries with RestHighLevelClient and Java – Part 1: geoDistanceQuery

This project has been designed/developed as a POC. Focus of the POC is to test some Spatial Queries against Elasticsearch (ES).

In the following posts, we’ll elaborate on 3 following types of Spatial Queries:

    • retrieve objects within certain distance
    • retrieve communes within rectangle/BoundingBox
    • retrieve communes within a polygon (implemented as a unit test)

This post elaborates entirely on the first use case.

Introduction – retrieving objects within certain distance

This particular post elaborates on the usage of the geoDistanceQuery. This query allows to
filter based on a specific distance from a specific geo location / point.

Our method will return a collection of locations, located within a certain distance and the returned collection will be sorted by distance AND …. the actual distance between the two locations will be included in our response.

For this POC, we use Elastic’s RestHighLevelClient , we rely entirely on REST and the code has been written in Java (Java 8 or Java 9). Following picture highlights the technical artifacts:

The Java REST client is the official client for Elasticsearch and comes in two flavors:

  1. Java low-level REST client: It allows communicating with an Elasticsearch cluster through HTTP and leaves requests marshaling and responses un-marshaling to users.
  2. Java high-level REST client: It is based on a low-level client and exposes API-specific methods, taking care of requests marshaling and responses un-marshaling.

The high-level client replaces the transport client, hence I invite all Java users to try it out and migrate to it if possible.

Dependencies

To get started you have to add the following dependency in your pom.xml file:

<!-- ES -->
<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-high-level-client</artifactId>
    <version>6.1.1</version>
</dependency>
<dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>6.1.1</version>
</dependency>

Curl 

As well known, you can run queries against ES by using Curl. Please find the complete example below:

curl -X GET -H "Content-Type:application/json" "localhost:9200/commune/doc/_search" -d'
{  
   "query":{  
        "bool" : {
            "must" : {
                "match_all" : {}
            },
            "filter" : {
                "geo_distance" : {
                    "distance" : "5km",
                    "myLocation" : {
                        "lat" : 51.152981,
                        "lon" : 4.455599
                    }
                }
            }
        }   
   }
}
' | json_pp 

Development

The code should be rather straightforward (and is available on GIT).

An important caveat is to retrieve the actual distance between the 2 locations and this can be done by adding the sort. You’ll have to repeat the location you are searching from and I’m still trying to clarify why the getSortValues returns an array of objects and how to deal with the different elements.

public Set<?> geoDistanceQuery(String objectType, double lat, double lon, int distance) throws IOException {
    Date startDate = new Date();

    Set<?> objectsWithinDistance = new LinkedHashSet<>();
    SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();

    QueryBuilder query = QueryBuilders.matchAllQuery();

    QueryBuilder geoDistanceQueryBuilder = QueryBuilders
            .geoDistanceQuery("myLocation")
            .point(lat, lon)
            .distance(distance, DistanceUnit.KILOMETERS);

    QueryBuilder finalQuery = QueryBuilders.boolQuery().must(query).filter(geoDistanceQueryBuilder);

    sourceBuilder.query(finalQuery).size(SIZE_ES_QUERY);

    SearchRequest searchRequest = new SearchRequest(objectType)
            .source(sourceBuilder.sort(SortBuilders.geoDistanceSort("myLocation", lat, lon)
                    .order(SortOrder.ASC)
                    .unit(DistanceUnit.KILOMETERS)));

    SearchResponse searchResponse = restClient.search(searchRequest);

    SearchHits hits = searchResponse.getHits();

    for (SearchHit hit : hits.getHits()) {
        objectsWithinDistance.add(GEO_Service.getObjectFromES_Hit(hit, objectType));
    }

    return timedReturn(LOGGER, new Object() {}.getClass().getEnclosingMethod().getName(), startDate.getTime(), objectsWithinDistance);
}

private static  T getObjectFromES_Hit(SearchHit hit, String objectType) {
    if (LOGGER.isDebugEnabled()) {
        LOGGER.debug("hit: id = {}", hit.getId());
        LOGGER.debug("hit: score = {}", hit.getScore());
        LOGGER.debug("hit: sort value = {}", hit.getSortValues()[0]);
    }

    if ("location".equals(objectType)) {
        LocationRequest.Location location = new LocationRequest.Location();

        Map<String, Object> source = hit.getSourceAsMap();

        location.setId((String) source.get("id"));
        location.setName((String) source.get("name"));
        location.setObjectId((String) source.get("objectId"));
        location.setScore(hit.getScore());
        return (T) location;
    }
    else {
        CommuneRequest.Commune commune = new CommuneRequest.Commune();

        Map<String,Object> source = hit.getSourceAsMap();

        commune.setCity((String) source.get("city"));
        //TODO : to clarify why sort values is an array and if we can simply take first element of array
        commune.setDistance((double) hit.getSortValues()[0]);
        return (T) commune;
    }
}

GitHub

Complete code with README, unit tests and Postman sample requests is available on github with this url.

I hope this article has enlightened you a little on how to run Spatial queries against Elastic Search. Don’t hesitate to ask questions or to recommend amendments.

All the best,

Wim Van den Brande.

Keywords: ElasticSearch, Java, rest-high-level-client

 

Advertisements

About IctDynamic.Be

IctDynamic designs and develops affordable software applications for the SME We focus on • Java Freelance missions • GIS and geographical solutions • Freelance missions as project manager (certified), analyst, architect, Java software engineer
This entry was posted in GIS, Java. Bookmark the permalink.