Apache HttpClient - Multiple Threads

A multi-threaded program contains two or more parts that can run concurrently and each part can handle a different task at the same time making optimal use of the available resources.

You can execute requests from multiple threads by writing a multithreaded HttpClient program.

If you want to execute multiple client requests from threads consecutively, you need to create a ClientConnectionPoolManager. It maintains a pool of HttpClientConnections and serves multiple requests from threads.

The connections manager pools the connections based on the route. If the manager has connections for a particular route, then it serves new requests in those routes by leasing an existing connection from the pool, instead of creating a new one.

Follow the steps to execute requests from multiple threads −

Step 1 - Creating the Client Connection Pool Manager

Create the Client Connection Pool Manager by instantiating the PoolingHttpClientConnectionManager class.

PoolingHttpClientConnectionManager connManager = new
   PoolingHttpClientConnectionManager();

Step 2 - Set the maximum number of connections

Set the maximum number of connections in the pool using the setMaxTotal() method.

//Set the maximum number of connections in the pool
connManager.setMaxTotal(100);

Step 3 - Create a ClientBuilder Object

Create a ClientBuilder Object by setting the connection manager using the setConnectionManager() method as shown below −

HttpClientBuilder clientbuilder =
HttpClients.custom().setConnectionManager(connManager);

Step 4 - Create the HttpGet request objects

Instantiate the HttpGet class by passing the desired URI to its constructor as a parameter. 

HttpGet httpget1 = new HttpGet("URI1");
HttpGet httpget2 = new HttpGet("URI2");
. . . . . . . . . . . .

Step 5 - Implementing the run method

Make sure that you have created a class, made it a thread (either by extending the thread class or, by implementing the Runnable interface) and implemented the run method.

public class ClientMultiThreaded extends Thread {
   public void run() {
      //Run method implementation . . . . . . . . . .
   }
}

Step 6 - Create Thread objects

Create thread objects by instantiating the Thread class (ClientMultiThreaded) created above.

Pass a HttpClient object, respective HttpGet object and, an integer representing the ID to these threads.

ClientMultiThreaded thread1 = new ClientMultiThreaded(httpclient,httpget1, 1);
ClientMultiThreaded thread2 = new ClientMultiThreaded(httpclient,httpget2, 2);
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Step 7 - Start and join the threads

Start all the threads using start() method and join them using the join method().

thread1.start();
thread2.start();
. . . . . . . .
thread1.join();
thread2.join();
. . . . . . . . . . . .

Step 8 - Run method implementation

Within the run method, execute the request, retrieve the response and print the results.

Example

Following example demonstrates the execution of HTTP requests simultaneously from multiple threads. In this example, we are trying to execute various requests from various threads and trying to print the status, and the number of bytes read by each client.

import org.apache.http.HttpEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.util.EntityUtils;

public class ClientMultiThreaded extends Thread {
   CloseableHttpClient httpClient;
   HttpGet httpget;
   int id;
 
   public ClientMultiThreaded(CloseableHttpClient httpClient, HttpGet httpget,
   int id) {
      this.httpClient = httpClient;
      this.httpget = httpget;
      this.id = id;
   }
   @Override
   public void run() {
      try{
         //Executing the request
         CloseableHttpResponse httpresponse = httpClient.execute(httpget);

         //Displaying the status of the request.
         System.out.println("status of thread "+id+":"+httpresponse.getStatusLine());

         //Retrieving the HttpEntity and displaying the no.of bytes read
         HttpEntity entity = httpresponse.getEntity();
         if (entity != null) {
            System.out.println("Bytes read by thread thread "+id+":
               "+EntityUtils.toByteArray(entity).length);
         }
      }catch(Exception e) {
         System.out.println(e.getMessage());
      }
   }
      
   public static void main(String[] args) throws Exception {

      //Creating the Client Connection Pool Manager by instantiating the PoolingHttpClientConnectionManager class.
      PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager();

      //Set the maximum number of connections in the pool
      connManager.setMaxTotal(100);

      //Create a ClientBuilder Object by setting the connection manager
      HttpClientBuilder clientbuilder = HttpClients.custom().setConnectionManager(connManager);
 
      //Build the CloseableHttpClient object using the build() method.
      CloseableHttpClient httpclient = clientbuilder.build();

      //Creating the HttpGet requests
      HttpGet httpget1 = new HttpGet("https://www.tutorialspoint.com/");
      HttpGet httpget2 = new HttpGet("http://www.google.com/");
      HttpGet httpget3 = new HttpGet("https://www.qries.com/");
      HttpGet httpget4 = new HttpGet("https://in.yahoo.com/");
 
      //Creating the Thread objects
      ClientMultiThreaded thread1 = new ClientMultiThreaded(httpclient,httpget1, 1);
      ClientMultiThreaded thread2 = new ClientMultiThreaded(httpclient,httpget2, 2);
      ClientMultiThreaded thread3 = new ClientMultiThreaded(httpclient,httpget3, 3);
      ClientMultiThreaded thread4 = new ClientMultiThreaded(httpclient,httpget4, 4);

      //Starting all the threads
      thread1.start();
      thread2.start();
      thread3.start();
      thread4.start();

      //Joining all the threads
      thread1.join();
      thread2.join();
      thread3.join();
      thread4.join();
   }
}

Output

On executing, the above program generates the following output −

status of thread 1: HTTP/1.1 200 OK
Bytes read by thread thread 1: 36907
status of thread 2: HTTP/1.1 200 OK
Bytes read by thread thread 2: 13725
status of thread 3: HTTP/1.1 200 OK
Bytes read by thread thread 3: 17319
status of thread 4: HTTP/1.1 200 OK
Bytes read by thread thread 4: 127018
Advertisements