java - Best way of performing a db load - oracle -
i want read entire schema in oracle database , copy in memory database (just once - @ startup). schema has 300 tables , overall 4gb of information. load using jdbc connections.
obviously, want memory consistent, use read mode.
i looking fastest way this.
one way: if use 1 connection , go @ every table 1 one. psuedo code:
conn = getconnection(); conn.setreadonly(true); for(string table:alltables){ ps = conn.preparestatement(string.format("select * %s", table)); ps.execute(); rs = ps.getresultset(); analyze(rs); rs.close(); ps.close(); }
the problem here not using database's computational power - parallelize above - question how:
each connection loads single table - quite easy implement, might create lot of connections , become unfair, since tables have millions of records , others mere dozens.
each connections (lets have n connections) reads part of each table - is, typical connection, indexed i, reads 1/n of each table so: "where rownum>count*i/n , rownum < count*(i+1)/n" count being number of records in table.
the same 2 using mod function - "where mod(rownum,n)=i"
the approach seems fastest right 2 - , becomes slighly faster if add in threshold - small tables loaded whole.
question: question - based on inner working of oracle - how can imporove loading time of tables?
(i mean on 1 hand seems logical oracle selects in method 1 - there little parsing, , minimizes number of commands processed - requires lot of connections , in case of big 1 table, lose efficiency.)
any suggestions?
Comments
Post a Comment