My problem:
1-I have a REST API that I use a lot of data to insert in an internal database (Process that lasts for hours).
2-The API dumps whole tables or filtered by an attribute (Select * from table where attribute = parameter): ex: www.apirest.com/tabla/nombreAtributo/valor/Atributo.
2-Every X time you must make a calculation with that data you need to consume again from the REST API and reload the data, but obviously 90% of the data already exists in my internal database and only I need the new ones, but I do not know how to differentiate them because I do not have a timestamp or anything like that, so I have to "delete" the previous data and fill in the new ones (or with merge, so if it exists, just modify it if necessary). to have news)
Obviously this is very inefficient, but my current knowledge makes me get stuck and I do not know how to improve this.
The application is made with Spring and hibernate as ORM.
The only thing that I found interesting is to make insertions with REPLACE INTO or with INSERT IGNORE, although I do not know if this would improve efficiency and the biggest problem: I do not know how to insert a list with hibernate without looping. p>
Is there any way to make only inserts of what does not exist in the database?
PS: I can not modify the REST API