The rebalancing Algorithms can be performed in following way −
We can implement actually rebalance method using the Day-Stout-Warren Algorithm.It's linear in the number of nodes.
The following is a presentation of the basic DSW Algorithm in pseudo code.
If we can withstand lack of linearizability (i.e. we write a value but when we search for it immediately after it's not found; it will be then eventually found, but after 100ms-10s), a "copy on write" tree can be applied: all writes are performed by one thread (with rebalancing), and we periodically copy the tree into a read-only copy that can be implemented by reading threads without any concurrency control, we require to publish it atomically.
Another option is to implement a concurrent skip list: it provides logarithmic average case search/delete/insert time and is more easily parallelizable. There is a standard lock-free implementation for Java if you happen to implement it. We can obtain more information about concurrent skip lists and balanced search trees here. Particularly, we can obtain there mentions of a chromatic tree, denoted as a binary search tree that is optimized for concurrent rebalancing.