Amazon web services – How do I migrate data from one table to another in Amazon Keyspaces?

Cassandra’s CQL does not have the equivalent of SQL’s INSERT INTO table SELECT ... command that copies data from one table into another as a single statement because this requires a full table scan that doesn’t scale for large datasets and/or large clusters.

You will need to export the contents of the source table, do the relevant transformations to the data, then import it to the destination table.

The simplest way to do this is to export the data to a CSV file using DSBulk, manually manipulate the CSV file so it only contains the data you want, then bulk-load the updated CSV into the new table using DSBulk.

Since you don’t want to deal with CSV files, the only option you have is to write a Spark or Java application to read the data one-by-one, transform it then write the transformed data to the new table. Cheers!