Some times it is necessary to load the records line by line from a large file and insert into the database table. There can be several ways to do this:

• Use the database provided tool to load the data into table. It can be a special sql command or a special stored procedure. This method is quite database specific and not portable.
• Write the program to load the data into the database table (Using JDBC if the program is written in Java).  This method can be applied to different databases if the insert SQL statement conforms to a level of standard.
• Use JPA to persist the object one by one. This allows for changing database more easily (under the condition that the JPA provider supports the database.

We used the third options to load the records into the database as we want to support multiple databases without  low level tweaking. However, the first implementation seemed very slow compared with the second option. And it became slower and slower when process of persisting records  progressed.

After a few checking, we realized that we didn’t call the method “clear” of the class EntityManager each time we persisted the object.  The EntityManager kept managing these objects so the resource was not released immediately, making it become slower and slower.

Once we add the “clear” method call, the performance became acceptable. So lesson learned about the use of JPA.