Wicket's Data Grid based on plain Iterator

In order to use Data Grid from Wicket you have to implement Data Provider:

  1 public interface IDataProvider<T> extends IDetachable {
  2  
  3     Iterator<? extends T> iterator(long first, long count);
  4  
  5     long size(); 
  6  
  7     IModel<T> model(T object); 
  8 }

It means, that you have to provide separate Iterator for each single page, so that it contains only elements required to render it. You have also to know the total amount of all elements, because it will be used to calculate number of pages.

This interface has been designed with SQL database in mind and you have to implement all methods as they are. The whole component is not meant to be customised - everything is either final or private.

But I had some different requirements:

• model should be based on plain java Iterator
• size information should not be required, not even estimated
• single iterator should be used to render all pages
• no reading in advance - Iterator should be used only to read elements that are required to render  a page for displaying
• large data sets does not have to be supported - like infinite iterator. It is legitime to cache already read elements in order to support bidirectional pager navigation  - at least for the elements that have been already displayed
• and still - implementation should be based on original Data Grid from Wicket - at least as far as possible

The reason for that was, that I wanted to display results from Cassandra, and it should support every possible query independent of its data model structure. This means that you have only one way Iterator and skipping elements means reading and rejecting. I give up size information too, firstly because it turned out that its not really needed, and most importantly, because its calculation means full table scan. Once again - you could redesign your model so that it could deliver such data in efficient way, but I wanted to have generic Data Grid which works with every possible query and model.

The component that I've developed is called IterableDataProvider and it can be found here: https://github.com/maciejmiklas/cyclop/tree/master/cyclop-wicket-components

It's based on original GridView, but I had to copy source code from Wicket in order to remove some final declarations. Here is an example:

  1 final List<String> myGridData = new ArrayList<>(); 
  2 myGridData.add("value 1"); 
  3 myGridData.add("value 2"); 
  4  
  5 IterableDataProvider<String> iterableDataProvider =  
  6    new IterableDataProvider<String>(10) { 
  7     @Override 
  8     protected Iterator<String> iterator() {
  9         return myGridData.iterator(); 
 10     } 
 11  
 12     @Override 
 13     public IModel<String> model(String s) {
 14         return Model.of(s); 
 15     } 
 16  
 17     @Override 
 18     public void detach() {
 19     } 
 20 }; 
 21  
 22 IterableGridView<String> myGrid = new IterableGridView<String>("myGrid",  
 23    iterableDataProvider) { 
 24     @Override 
 25     protected void populateEmptyItem(Item<String> item) {
 26         item.add(new Label("myValue")); 
 27     } 
 28  
 29     @Override 
 30     protected void populateItem(Item<String> item) {
 31         item.add(new Label("myValue", item.getModelObject())); 
 32     } 
 33 }; 
 34  
 35 add(myGrid); 
 36  
 37 myGrid.setItemsPerPage(10); 
 38  
 39 // you have to use custom pager and not AjaxPagingNavigator
 40 IterablePagingNavigator pager = new IterablePagingNavigator("rowNamesListPager",  
 41    rowNamesList); 
 42 resultTable.add(pager);

This code looks almost as it would have been written based on original Wicket components, so there is not much to say - I will only point out some key differences:

• GridView has been replaced by IterableGridView. This new class inherits from GridView, but not from the original one - I had to copy it in order to remove few final modifiers
• IDataProvider has been replaced with IterableDataProvider.  New provider requires only Iterator, size and iterator based on offset are gone
• AjaxPagingNavigator has been replaced with IterablePagingNavigator. It inherits from original class and replaces only behaviour for link to the last page. It's always active - I will get into that in next section

Our Iterable Data Grid has one limitation - it does not know the total amount of elements, so it cannot estimate the amount of pages, also it cannot skip results, and iterating over all elements in advance is not allowed either.  This enforces some functionality changes in paging. User have to go page by page in order to progress trough grid elements - he cannot skip pages, or jump to the last page immediately. Once particular page has been loaded, it's being stored in cache, and from now on direct access is possible.

Let's analyse practical example:
at the beginning pager looks like that:

The first page has been rendered, and there is link to the following page. In this case gird has read six elements from Iterator - five in order to render the current page, and one more to verify whether there is another page.
Clinking on "page two" has following outcome:

and again on "page three":

Now the user can go back to "page two" or "page one", but in order to progress he must go to "page four".
Once the user went trough all pages, he can navigate as usual without restrictions. Also the link to the last page is finally active:

This whole idea has one more catch: since we have simple Iterator and we would like provide bidirectional navigation, the results have to be cached somewhere. Default implementation uses memory, so I would suggest to use simple POJOs. The amount of elements read from iterator can be limited too.
You can also customise cache implementation by overwriting following factory method: IterableDataProvider#createElementsCache(). Currently it returns ArrayList, but you can replace it with off heap implementation, like BigList wich uses memory mapped files, or data base.
The whole thing is not an anti-patern as it might appear. We have only access to plain Iterator, but we have to provide bidirectional navigation - this additional information has to be stored somewhere.