how to index data in elasticsearch using java

Index API is used to index documents into specific index and make them searchable. All the data stored in Elasticsearch is in jSON format. To index some data/object into elastic search, first we need to convert that object into json format.

Generate JSON Document
You can generate JSON document in number of ways.

1.   By manually doing it by yourself
2.   By using Elasticsearch helper classes.
3.   Using a third party library to serialize your beans such as Jackson, gson

By manually doing it by yourself
You can create json document by yourself like below.

String data = "{\"id\":514,\"firstName\":\"PTR\",\"lastName\":\"Nayan\"}";

By using Elasticsearch helper classes
Elasticsearch provides built-in helper classes to generate json document.

XContentBuilder builder = jsonBuilder().startObject().field("id", "514").field("firstName", "PTR").field("lastName", "Nayan").endObject();


Find complete application below.

import static org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder;
import java.io.IOException;
import org.elasticsearch.common.xcontent.XContentBuilder;

public class Main {
 public static void main(String args[]) throws IOException {

  XContentBuilder builder = jsonBuilder().startObject()
    .field("id", "514").field("firstName", "PTR")
    .field("lastName", "Nayan").endObject();

  System.out.println(builder.string());
 }
}


Output

{"id":"514","firstName":"PTR","lastName":"Nayan"}


Using a third party library to serialize your beans such as Jackson, gson
You can use third party libraries like GSON, Jakson to convert an object into json. Following application use gson to convert an object into json document.

public class Employee {
 private int id;
 private String firstName;
 private String lastName;

 public int getId() {
  return id;
 }

 public void setId(int id) {
  this.id = id;
 }

 public String getFirstName() {
  return firstName;
 }

 public void setFirstName(String firstName) {
  this.firstName = firstName;
 }

 public String getLastName() {
  return lastName;
 }

 public void setLastName(String lastName) {
  this.lastName = lastName;
 }

}


import java.io.IOException;

import com.google.gson.Gson;

public class Main {
 public static void main(String args[]) throws IOException {
  Employee emp = new Employee();
  emp.setId(514);
  emp.setFirstName("PTR");
  emp.setLastName("Nayan");

  Gson gson = new Gson();
  String json = gson.toJson(emp);
  System.out.println(json);
 }
}


Output
{"id":"514","firstName":"PTR","lastName":"Nayan"}

Go through following tutorial to learn about json.
http://self-learning-java-tutorial.blogspot.in/2014/11/json_14.html

Index JSON Document
For example, following Java application writes following document into index ‘organization’, type employee.

{"age":"27","firstName":"PTR","lastName":"Nayan","hobbies":["Tattoos","People Watching","Dagger Collecting","Confusing People"]}

Step 1: Define model class.

package com.self_learn.model;

import java.util.ArrayList;
import java.util.List;

import lombok.EqualsAndHashCode;
import lombok.Getter;
import lombok.Setter;
import lombok.ToString;

@EqualsAndHashCode()
@ToString
public class Employee {
 @Getter @Setter private String age;
 @Getter @Setter private String firstName;
 @Getter @Setter private String lastName;
 @Getter @Setter private List<String> hobbies = new ArrayList<>();
}


Step 2: Define TransportClientUtil class to get a transport client.

package com.self_learn.util;

import static com.self_learn.util.IPUtil.isValidHosts;
import static com.self_learn.util.IPUtil.isValidPorts;

import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Set;

import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.InetSocketTransportAddress;

import com.google.common.base.Preconditions;

public class TransportClientUtil {
 private static Map<Map<String, Integer>, Client> localMap = new HashMap<>();

 /**
  * Take machine name and port addresses as map and return transport client.
  * Key is host name, value is port number
  *
  * @throws UnknownHostException
  */
 public static Client getTransportClient(String clusterName,
   Map<String, Integer> map) throws UnknownHostException {

  Preconditions.checkNotNull(clusterName,
    "clusterName shouldn't be empty");
  Preconditions.checkNotNull(map, "Map shouldn't be empty");

  if (localMap.containsKey(map))
   return localMap.get(map);

  Preconditions.checkState(isValidHostPorts(map),
    "Map contains invalid host (or) port");

  Settings settings = ImmutableSettings.settingsBuilder()
    .put("cluster.name", clusterName)
    .put("client.transport.sniff", true).build();

  TransportClient client = new TransportClient(settings);

  InetSocketTransportAddress addresses[] = getInetSocketTransportAddresses(map);
  client.addTransportAddresses(addresses);
  localMap.put(map, client);
  return client;
 }

 private static boolean isValidHostPorts(Map<String, Integer> map) {
  Set<String> hostNames = map.keySet();
  Set<Integer> ports = new HashSet<>(map.values());

  if (!isValidHosts(hostNames.toArray(new String[hostNames.size()])))
   return false;

  if (!isValidPorts(ports.toArray(new Integer[ports.size()])))
   return false;

  return true;
 }

 private static InetSocketTransportAddress[] getInetSocketTransportAddresses(
   Map<String, Integer> map) throws UnknownHostException {
  InetSocketTransportAddress addresses[] = new InetSocketTransportAddress[map
    .size()];
  int count = 0;

  Set<String> keys = map.keySet();
  for (String key : keys) {
   InetAddress addr = InetAddress.getByName(key);
   InetSocketTransportAddress address = new InetSocketTransportAddress(
     addr, map.get(key));
   addresses[count] = address;
  }

  return addresses;
 }

 public static Client getLocalTransportClient(String clusterName, int port)
   throws UnknownHostException {
  Settings settings = ImmutableSettings.settingsBuilder()
    .put("cluster.name", clusterName)
    .put("client.transport.sniff", true).build();

  TransportClient client = new TransportClient(settings);
  InetAddress addr = InetAddress.getByName("127.0.0.1");
  InetSocketTransportAddress address = new InetSocketTransportAddress(
    addr, port);

  client.addTransportAddress(address);
  return client;
 }

}


IPUtil.java

package com.self_learn.util;

import org.apache.commons.validator.routines.InetAddressValidator;

import com.google.common.base.Preconditions;

/**
 * Validate IPaddresses
 *
 * @author harikrishna_gurram
 */
public class IPUtil {
 private static InetAddressValidator inetAddressValidator = InetAddressValidator
   .getInstance();

 public static boolean isValidIPAddress(String ip) {
  Preconditions.checkNotNull(ip, "IP address should not be null");
  return inetAddressValidator.isValid(ip);
 }

 public static boolean isValidPort(int port) {
  if (port > 0 && port < 65536)
   return true;
  return false;
 }

 public static boolean isValidHosts(String[] hostNames) {
  Preconditions.checkNotNull(hostNames, "Host names shouldn't be empty");
  for (String hostName : hostNames) {
   if (!isValidIPAddress(hostName)) {
    return false;
   }
  }
  return true;
 }

 public static boolean isValidPorts(Integer[] ports) {
  Preconditions.checkNotNull(ports, "ports shouldn't be empty");
  for (int port : ports) {
   if (!isValidPort(port)) {
    return false;
   }
  }
  return true;
 }

}


Step 3: Main.java class writes document into Elasticsearch.

package com.self_learn.test;

import java.net.UnknownHostException;

import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.client.Client;

import com.google.gson.Gson;
import com.self_learn.model.Employee;
import com.self_learn.util.TransportClientUtil;

public class Main {
 private static Gson gson = new Gson();
 private static String clusterName = "my_cluster_1";
 private static String index = "organization";
 private static String type = "employee";

 public static void main(String args[]) throws UnknownHostException {
  /* Populate model object */
  Employee emp = new Employee();
  emp.setAge("27");
  emp.setFirstName("PTR");
  emp.setLastName("Nayan");
  emp.getHobbies().add("Tattoos");
  emp.getHobbies().add("People Watching");
  emp.getHobbies().add("Dagger Collecting");
  emp.getHobbies().add("Confusing People");

  /* Get json document from model object */
  String data = gson.toJson(emp);
  System.out.println("Writing " + data
    + " to index 'organization' and type 'employee'");

  /* Get client instance for cluster */
  Client client = TransportClientUtil.getLocalTransportClient(
    clusterName, 9300);

  System.out.println("Writing data");

  /* Write data to Elasticsearch */
  IndexResponse response = client.prepareIndex(index, type, "1")
    .setSource(data).execute().actionGet();

  printResponseInfo(response);

  client.close();

  System.out.println("Writing done");
 }

 private static void printResponseInfo(IndexResponse response) {
  String _index = response.getIndex();
  String _type = response.getType();
  String _id = response.getId();
  long _version = response.getVersion();
  boolean created = response.isCreated();

  System.out.println("_index : " + _index);
  System.out.println("_type : " + _type);
  System.out.println("_id : " + _id);
  System.out.println("_version : " + _version);
  System.out.println("created : " + created);
 }
}


Run Main.java, you will get following output.

Writing {"age":"27","firstName":"PTR","lastName":"Nayan","hobbies":["Tattoos","People Watching","Dagger Collecting","Confusing People"]} to index 'organization' and type 'employee'
Sep 09, 2015 12:43:53 PM org.elasticsearch.plugins.PluginsService <init>
INFO: [Ritchie Gilmore] loaded [], sites []
Writing data
_index : organization
_type : employee
_id : 1
_version : 1
created : true
Writing done


Now open ‘sense’ and search type employee in index ‘organization’.

GET /organization/employee/_search

You will get following response.

{
   "took": 5,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hits": {
      "total": 1,
      "max_score": 1,
      "hits": [
         {
            "_index": "organization",
            "_type": "employee",
            "_id": "1",
            "_score": 1,
            "_source": {
               "age": "27",
               "firstName": "singh",
               "lastName": "rajdeo",
               "hobbies": [
                  "Tattoos",
                  "People Watching",
                  "Dagger Collecting",
                  "Confusing People"
               ]
            }
         }
      ]
   }
}