Hello Teradata experts,
I'm now working on optimization of loading of data into Teradata.
Our application allows to load big csv files by means of jdbc connection and executeBatch.
I've run the load with a relatively big dataset (about 6M rows and 1.2 Gb of data). I've started with our default parameters:
1) batch size = 1000 records
2) a target table had a primary key constraint
The load was very slow and could take about 3-4 hours.
When I've increased the batch size to 50000, and made a source table "multitable" and NoPI, the loading time decreased dramatically to about 10-12 min, which is already acceptable for us.
I also wanted to evaluate the "FASTLOAD" tool and set the type=FASTLOAD in our jdbc connection string.
However, when I try to load data with the fastload, it seems to hang after 50K loaded records (and it doesn't matter whether the batch size = 1000 or 50000). I also found a similar topic here: http://forums.teradata.com/forum/connectivity/teradata-jdbc-fastload-hangs-after-approximately-50-batch-insert
What could be the reason for such a behavior and how could we avoid it?
Many thanks for any hint!
Katja
