DynamoDB - Batch Writing

Batch writing operates on multiple items by creating or deleting several items. These operations utilize BatchWriteItem, which carries the limitations of no more than 16MB writes and 25 requests. Each item obeys a 400KB size limit. Batch writes also cannot perform item updates.

What is Batch Writing?

Batch writes can manipulate items across multiple tables. Operation invocation happens for each individual request, which means operations do not impact each other, and heterogeneous mixes are permitted; for example, one PutItem and three DeleteItem requests in a batch, with the failure of the PutItem request not impacting the others. Failed requests result in the operation returning information (keys and data) pertaining to each failed request.

Note − If DynamoDB returns any items without processing them, retry them; however, use a back-off method to avoid another request failure based on overloading.

DynamoDB rejects a batch write operation when one or more of the following statements proves to be true −

  • The request exceeds the provisioned throughput.

  • The request attempts to use BatchWriteItems to update an item.

  • The request performs several operations on a single item.

  • The request tables do not exist.

  • The item attributes in the request do not match the target.

  • The requests exceed size limits.

Batch writes require certain RequestItem parameters −

  • Deletion operations need DeleteRequest key subelements meaning an attribute name and value.

  • The PutRequest items require an Item subelement meaning an attribute and attribute value map.

Response − A successful operation results in an HTTP 200 response, which indicates characteristics like capacity units consumed, table processing metrics, and any unprocessed items.

Batch Writes with Java

Perform a batch write by creating a DynamoDB class instance, a TableWriteItems class instance describing all operations, and calling the batchWriteItem method to use the TableWriteItems object.

Note − You must create a TableWriteItems instance for every table in a batch write to multiple tables. Also, check your request response for any unprocessed requests.

You can review the following example of a batch write −

DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient( 
   new ProfileCredentialsProvider()));  

TableWriteItems forumTableWriteItems = new TableWriteItems("Forum") 
   .withItemsToPut( 
   new Item() 
   .withPrimaryKey("Title", "XYZ CRM") 
   .withNumber("Threads", 0));  

TableWriteItems threadTableWriteItems = new TableWriteItems(Thread) 
   .withItemsToPut( 
   new Item() 
   .withPrimaryKey("ForumTitle","XYZ CRM","Topic","Updates") 
   .withHashAndRangeKeysToDelete("ForumTitle","A partition key value", 
   "Product Line 1", "A sort key value"));

BatchWriteItemOutcome outcome = dynamoDB.batchWriteItem (
   forumTableWriteItems, threadTableWriteItems);

The following program is another bigger example for better understanding of how a batch writes with Java.

Note − The following example may assume a previously created data source. Before attempting to execute, acquire supporting libraries and create necessary data sources (tables with required characteristics, or other referenced sources).

This example also uses Eclipse IDE, an AWS credentials file, and the AWS Toolkit within an Eclipse AWS Java Project.

package com.amazonaws.codesamples.document;

import java.io.IOException;
import java.util.Arrays;
import java.util.HashSet;
import java.util.List;
import java.util.Map;

import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.amazonaws.services.dynamodbv2.document.BatchWriteItemOutcome;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.TableWriteItems;
import com.amazonaws.services.dynamodbv2.model.WriteRequest;

public class BatchWriteOpSample {  
   static DynamoDB dynamoDB = new DynamoDB(new AmazonDynamoDBClient( 
      new ProfileCredentialsProvider()));  
   static String forumTableName = "Forum"; 
   static String threadTableName = "Thread";  
      
   public static void main(String[] args) throws IOException { 
      batchWriteMultiItems();   
   }
   private static void batchWriteMultiItems() { 
      try {
         // Place new item in Forum 
         TableWriteItems forumTableWriteItems = new TableWriteItems(forumTableName) 
                                                                       //Forum 
            .withItemsToPut(new Item() 
            .withPrimaryKey("Name", "Amazon RDS") 
            .withNumber("Threads", 0));  
            
         // Place one item, delete another in Thread 
         // Specify partition key and range key 
         TableWriteItems threadTableWriteItems = new TableWriteItems(threadTableName) 
            .withItemsToPut(new Item() 
            .withPrimaryKey("ForumName","Product  
            Support","Subject","Support Thread 1") 
            .withString("Message", "New OS Thread 1 message")
            .withHashAndRangeKeysToDelete("ForumName","Subject", "Polymer Blaster", 
            "Support Thread 100"));  
            
         System.out.println("Processing request..."); 
         BatchWriteItemOutcome outcome = dynamoDB.batchWriteItem (
               forumTableWriteItems, threadTableWriteItems);
         do {  
            // Confirm no unprocessed items 
            Map<String, List<WriteRequest>> unprocessedItems 
               = outcome.getUnprocessedItems();  
                  
            if (outcome.getUnprocessedItems().size() == 0) { 
               System.out.println("All items processed."); 
            } else { 
               System.out.println("Gathering unprocessed items..."); 
               outcome = dynamoDB.batchWriteItemUnprocessed(unprocessedItems); 
            }  
         } while (outcome.getUnprocessedItems().size() > 0);  
      } catch (Exception e) { 
         System.err.println("Could not get items: "); 
         e.printStackTrace(System.err); 
      }   
   } 
}
Advertisements