hbase batch request example – client side buffering

Hbase client uses RPC to send the data from client to server and it is recommended to enable client side write buffer so that the put operatins are batched which would reduce the number of round trips time. By default this is disabled which needs to be enabled as below

Prior to hbase version 1


HTable htable=new HTable(conf, "test");

System.out.println(htable.isAutoFlush());

htable.setAutoFlush(false);

// Put some data into the htable

// flush the data to server

htable.flushCommits();

The HTable was not threadsafe till version 2.0 and was made thread safe since 2.0.0 if we do not invoke any of the setter methods. All setters are moved into TableBuilder class after 2.0 version. Prior to 2.0 version the getWriteBuffer() was returning ArrayList<Put> which can get the in-
ternal list of buffered Put instances we have added so far calling table.put(put) which maked HTABLE not safe to use in multi threaded environment.

If we are using version above 1 we should be using the BufferedMutator class which batches the put request which is also thread safe.


import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.BufferedMutator;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.util.Bytes;

public class ClientSideBuffering {

public static void main(String[] args) throws IOException {

Configuration conf = HBaseConfiguration.create();

Connection connection = ConnectionFactory.createConnection(conf);

try {

BufferedMutator table = connection.getBufferedMutator(TableName.valueOf("test"));

Put put1 = new Put(Bytes.toBytes("row1"));

Put put2 = new Put(Bytes.toBytes("row2"));

Put put3 = new Put(Bytes.toBytes("row3"));

put1.addColumn(Bytes.toBytes("cf"), Bytes.toBytes("qual1"),
Bytes.toBytes("ValueOneForPut1Qual1"));

put2.addColumn(Bytes.toBytes("cf"), Bytes.toBytes("qual1"),
Bytes.toBytes("ValueOneForPut2Qual1"));

put3.addColumn(Bytes.toBytes("cf"), Bytes.toBytes("qual1"),
Bytes.toBytes("ValueOneForPut2Qual1"));

put1.addColumn(Bytes.toBytes("cf"), Bytes.toBytes("qual2"),
Bytes.toBytes("ValueOneForPut1Qual2"));

put2.addColumn(Bytes.toBytes("cf"), Bytes.toBytes("qual2"),
Bytes.toBytes("ValueOneForPut2Qual2"));

put3.addColumn(Bytes.toBytes("cf"), Bytes.toBytes("qual2"),
Bytes.toBytes("ValueOneForPut3Qual3"));

table.mutate(put1);
table.mutate(put2);
table.mutate(put3);

table.flush();

} finally {

connection.close();
}

}

}

Leave a Reply

Your email address will not be published. Required fields are marked *