Friday, May 30, 2014

Wicket's Data Grid based on plain Iterator

In order to use Data Grid from Wicket you have to implement Data Provider:
  1 public interface IDataProvider<T> extends IDetachable {
  2  
  3     Iterator<? extends T> iterator(long first, long count);
  4  
  5     long size(); 
  6  
  7     IModel<T> model(T object); 
  8 }

It means, that you have to provide separate Iterator for each single page, so that it contains only elements required to render it. You have also to know the total amount of all elements, because it will be used to calculate number of pages.

This interface has been designed with SQL database in mind and you have to implement all methods as they are. The whole component is not meant to be customised - everything is either final or private.

But I had some different requirements:
  • model should be based on plain java Iterator
  • size information should not be required, not even estimated
  • single iterator should be used to render all pages
  • no reading in advance - Iterator should be used only to read elements that are required to render  a page for displaying
  • large data sets does not have to be supported - like infinite iterator. It is legitime to cache already read elements in order to support bidirectional pager navigation  - at least for the elements that have been already displayed
  • and still - implementation should be based on original Data Grid from Wicket - at least as far as possible
The reason for that was, that I wanted to display results from Cassandra, and it should support every possible query independent of its data model structure. This means that you have only one way Iterator and skipping elements means reading and rejecting. I give up size information too, firstly because it turned out that its not really needed, and most importantly, because its calculation means full table scan. Once again - you could redesign your model so that it could deliver such data in efficient way, but I wanted to have generic Data Grid which works with every possible query and model.

The component that I've developed is called IterableDataProvider and it can be found here: https://github.com/maciejmiklas/cyclop/tree/master/cyclop-wicket-components

It's based on original GridView, but I had to copy source code from Wicket in order to remove some final declarations. Here is an example:
  1 final List<String> myGridData = new ArrayList<>(); 
  2 myGridData.add("value 1"); 
  3 myGridData.add("value 2"); 
  4  
  5 IterableDataProvider<String> iterableDataProvider =  
  6    new IterableDataProvider<String>(10) { 
  7     @Override 
  8     protected Iterator<String> iterator() {
  9         return myGridData.iterator(); 
 10     } 
 11  
 12     @Override 
 13     public IModel<String> model(String s) {
 14         return Model.of(s); 
 15     } 
 16  
 17     @Override 
 18     public void detach() {
 19     } 
 20 }; 
 21  
 22 IterableGridView<String> myGrid = new IterableGridView<String>("myGrid",  
 23    iterableDataProvider) { 
 24     @Override 
 25     protected void populateEmptyItem(Item<String> item) {
 26         item.add(new Label("myValue")); 
 27     } 
 28  
 29     @Override 
 30     protected void populateItem(Item<String> item) {
 31         item.add(new Label("myValue", item.getModelObject())); 
 32     } 
 33 }; 
 34  
 35 add(myGrid); 
 36  
 37 myGrid.setItemsPerPage(10); 
 38  
 39 // you have to use custom pager and not AjaxPagingNavigator
 40 IterablePagingNavigator pager = new IterablePagingNavigator("rowNamesListPager",  
 41    rowNamesList); 
 42 resultTable.add(pager);
This code looks almost as it would have been written based on original Wicket components, so there is not much to say - I will only point out some key differences:
  • GridView has been replaced by IterableGridView. This new class inherits from GridView, but not from the original one - I had to copy it in order to remove few final modifiers
  • IDataProvider has been replaced with IterableDataProvider.  New provider requires only Iterator, size and iterator based on offset are gone
  • AjaxPagingNavigator has been replaced with IterablePagingNavigator. It inherits from original class and replaces only behaviour for link to the last page. It's always active - I will get into that in next section
Our Iterable Data Grid has one limitation - it does not know the total amount of elements, so it cannot estimate the amount of pages, also it cannot skip results, and iterating over all elements in advance is not allowed either.  This enforces some functionality changes in paging. User have to go page by page in order to progress trough grid elements - he cannot skip pages, or jump to the last page immediately. Once particular page has been loaded, it's being stored in cache, and from now on direct access is possible.

Let's analyse practical example:
at the beginning pager looks like that:
The first page has been rendered, and there is link to the following page. In this case gird has read six elements from Iterator - five in order to render the current page, and one more to verify whether there is another page.
Clinking on "page two" has following outcome:
and again on "page three":
Now the user can go back to "page two" or "page one", but in order to progress he must go to "page four".
Once the user went trough all pages, he can navigate as usual without restrictions. Also the link to the last page is finally active:

This whole idea has one more catch: since we have simple Iterator and we would like provide bidirectional navigation, the results have to be cached somewhere. Default implementation uses memory, so I would suggest to use simple POJOs. The amount of elements read from iterator can be limited too.
You can also customise cache implementation by overwriting following factory method: IterableDataProvider#createElementsCache(). Currently it returns ArrayList, but you can replace it with off heap implementation, like BigList wich uses memory mapped files, or data base.
The whole thing is not an anti-patern as it might appear. We have only access to plain Iterator, but we have to provide bidirectional navigation - this additional information has to be stored somewhere.

Tuesday, May 6, 2014

Cyclop 1.4 has been released!

Latest Cyclop release addresses 23 issues, the full list can be found here. Here are the most important:
  • query history inclusive free text search
  • increased contrast for table headers - both row and column names
  • links showing full column content has been replaced with icons
  • mouse over does not highlight whole table row, but only column values - table header on the left has been removed from highlighting
  • blue horizontal separator between common and dynamic columns appears only of there is something to be separated
  • spell checker is disabled in query editor
  • green CQL-Completion-Hint in top right corner is being closed after query has been executed, so it does not overlap on results table. It will reappear after user starts typing again
  • test coverage has been significantly improved 
  • Findbugs is active, inclusive JSR 305 annotations
  • data validation on service level based on Bean Validation (JSR 303) and custom AOP framework - here is an example. It validates method input parameters and also returned values, so the service gets only validated data, and the client validated response from the service
  • CSV import

Tuesday, March 4, 2014

Commodore C64 emulator - playing games using laptop keyboard

VICE is finally working on my laptop and games are playable without joystick - just with keyboard !
This seems to be pretty obvious, but I had hard time configuring it, plus I did not find any tutorial besides numerous posts from people complaining that it's not working.

First you have to download the emulator itself, for example from here. After installing it, download game disk images. Clicking on image file will start emulator and load game. You will also notice, that keyboard is not working at all ..... Let's configure joystick keyboard emulation ;)

First enable joystick emulation by opening Options menu and selecting "Allow Keyset Joystick":

Now configure joystick itself by choosing Settings ->  Joystick:


  • Joystick is connected to port two (#2) because most games expect it there.
  • Not all keys are set, because they are simply not required. There is no need for dedicated right-upper key, because you can just press two arrow keys, which is natural anyway.
  • Key-codes are taken from mac book directional keys, you can also provide any key codes. Just make sure that you are not using reserved keys like esc, space or enter.
  • In order to set key code first press key that you would like to assign, and than click on corresponding button. For example to assign fire button to key X, press X and than click on button marked as "Fire" on picture above
  • I've assigned X as fire, because you will need space, for example to throw grenades ;)


Saturday, January 18, 2014

Cyclop 1.3.0 has been released !

The Cyclop version 1.3.0 addresses 12 issues:

  • Support completion for keywords with spaces
  • Select statement should support column names in "order by" when query does not contain "when" part (decision table needs another dimension)
  • Code completion suggests only first hit for upper case words
  • Horizontal scroll-bar is always visible
  • Store user preferences as cookie
  • Results table - second click on link opening "column content dialog" breaks ajax
  • Completion support for UPDATE
  • Completion support for TRUNCATE
  • Completion support for DROP KEYSPACE
  • Completion support for DROP INDEX
  • Completion support for DROP INDEX
  • Completion support for DELETE
  • Completion support for CREATE KEYSPACE
The milestone containing details can be found here.



Have a fun!

Wednesday, December 11, 2013

Cyclop - CQL3 web based editor for Cassandra

I've developed web based CQL3 editor:


The idea is, to deploy it once and have simple and comfortable CQL3 interface over web - without need to install anything.
The editor itself supports code completion, not only for CQL syntax, but also based on database content. So for example the select statement will suggest tables from active keyspace, or within where-closure only columns from table provided after "select from"

The results are displayed in reversed table - rows horizontally and columns vertically. It seems to be more natural for column oriented database.

You can also export query results to CSV, or add query as browser bookmark, send it to another developer to share results, tweeter might not be the  best idea ;)

The whole application is based on wicket + bootstrap + spring and can be deployed in any web 3.0 container.

 Here is the project (open source): https://github.com/maciejmiklas/cyclop

Thursday, November 28, 2013

Spring 3 - SpEl solution for non existing optional @Resource injection

@Resource is used to inject resources into Spring beans - like in example below: MyService uses #serverMapping to map server aliases into full names.

  1 public class MyService {
  2
  3     @Resource(name = "serverMappingConf")
  4     private Map<String, String> serverMapping;
  5
  6     public String getFullName(String alias){
  7         return serverMapping.get(alias);
  8     }
  9 }

  1 <beans xmlns=....">
  2
  3      <util:map id="serverMappingConf">
  4          <entry key="serverEU" value="my-domain.eu" />
  5          <entry key="serverUS" value="my-domain.us" />
  6      </util:map>
  7 </beans>

This is not really spectacular, but in some cases it is required to inject null if #serverMappingConf is not defined (optional functionality). This is simple in case of spring bean, because you can use @Autowired(required="false"). But you cannot inject collections of resources using @Autowired - this was discussed in this JIRA case  - autowired would collect all beans of given type and try to inject it (case describes collections, but the same rules apply to maps).

This is the solution:
  1 public class MyService {
  2
  3     @Value("#{getObject('serverMappingConf')}")
  4     private Map<String, String> dcMappingInitial;
  5 }

@Value supports SpEl expressions. SpEl's root object has method getObject(String) which returns spring bean or resource, or null if was not found. We are calling this method with name of our resource - it returns Map<String, String> or null.

Tuesday, October 15, 2013

Wicket 6 - Paging Navigator for Bootstrap 3

Bootstrap has many fine looking themes and web pages based on it's CSS look just great - this is common knowledge.  It's quiet simple to integrate Bootstrap into Wicket application, with maybe few exceptions, like for example Paging Navigator.
The usual solution to this problem would be the adoption of CSS to mach Wickets's HTML structure. I've decided to modify Wicket Paging Navigator to use unmodified Bootstrap CSS.

We will use Bootstrap 3 with Cyborg Theme, the final pager will have following look:


Before we begin, let's point out some key differences between both HTML markups.
Wicket:
<html xmlns:wicket="http://wicket.apache.org">
    <body>
        <wicket:panel>
<a wicket:id="first" class="first">&lt;&lt;</a>
<a wicket:id="prev" class="prev">&lt;</a>
<span wicket:id="navigation" class="goto">
<a wicket:id="pageLink" href="#"><span wicket:id="pageNumber">5</span></a>
</span>
<a wicket:id="next" class="next">&gt;</a>
<a wicket:id="last" class="last">&gt;&gt;</a>
        </wicket:panel>
    </body>
</html>
and corresponding Bootstrap part:
    <ul class="pagination pagination-sm">
        <li class="disabled"><a href="#">«</a></li>
        <li class="active"><a href="#">1</a></li>
        <li><a href="#">2</a></li>
        <li><a href="#">3</a></li>
        <li><a href="#">4</a></li>
        <li><a href="#">5</a></li>
        <li><a href="#">»</a></li>
    </ul>
  • HTML tree structure is completely different. Wicket layouts hyperlinks one after another. Bootstrap wraps them with bulleted list.
  • Some links in pager are not clickable - so for example it's not possible to jump to the last page, if we are already there. Both frameworks solve it differently: wicket changes rendered HTML and replaces a-tag with div-tag. Bootstrap on the other hand side does not modify HTML tree at all, it applies different CSS. But it does not change class-argument directly on a-tag, but on it's parent (li-tag), which even dose not exits in Wicket markup.
  • Current page (nr-1 on picture above, blue rectangle) has also two different indicators: Wicket solves it by replacing hyperlink with div-tag, Bootstrap sets class=active on li-tag.
  • Bootstrap additionally provides focus for onmouseover (gray box) for active links, it comes from class=pagination on ul-tag.
Let's dive into implementation details - we will modify Wicket's AjaxPagingNavigator by replacing it's HTML and corresponding java part. The new component name is: BootstrapPagingNavigator.

BootstrapPagingNavigator.html
<html xmlns:wicket="http://wicket.apache.org">
<body>
<wicket:panel>
    <ul class="pagination pagination-sm">
        <li wicket:id="firstCont" class="disabled"><a wicket:id="first" href="#">&lt;&lt;</a></li>
        <li wicket:id="prevCont" class="disabled"><a wicket:id="prev" href="#">&lt;</a></li>
        <li wicket:id="navigation"><a wicket:id="pageLink" href="#">
            <span wicket:id="pageNumber">1</span></a>
        </li>
        <li wicket:id="nextCont" class="disabled"><a wicket:id="next" href="#">&gt;</a></li>
        <li wicket:id="lastCont" class="disabled"><a wicket:id="last" href="#">&gt;&gt;</a></li>
    </ul>
</wicket:panel>
</body>
</html>
The HTML markup has structure required by Bootstrap's CSS - bulleted list with hyperlinks. All original Wicket tag names are there (red color), plus we have some extra tags in blue.

Lest move into java implementation, it will contain two key modifications: 
  • HTML hyperlinks will not get removed for inactive links - a-tag will be still generated. 
  • Inactivation and activation of links will be handled by dynamic CSS applied to li-tags (blue wicket ids)
The original AjaxPagingNavigator generates three types of links, each one created by different factory method:
  • AjaxPagingNavigator#newNavigation(...) handles: "1 | 2 | 3 | 4"
  • AjaxPagingNavigator#newPagingNavigationLink(...) handles: "first, last"
  • AjaxPagingNavigator#newPagingNavigationIncrementLink(...) handles: "prev, next"
The controller class for our pager (BootstrapPagingNavigator.java) will overwrite each of those methods in order to apply required modifications. Here are the details:

AjaxPagingNavigator#newNavigation ( "1 | 2 | 3 | 4")


  1 public class BootstrapPagingNavigator extends AjaxPagingNavigator {
  2
  3 // .......
  4
  5     @Override
  6     protected PagingNavigation newNavigation(String id, IPageable pageable, 
  7         IPagingLabelProvider labelProvider) {
  8         return new AjaxPagingNavigation(id, pageable, labelProvider) {
  9             @Override
 10             protected LoopItem newItem(int iteration) {
 11                 LoopItem item = super.newItem(iteration);
 12
 13                 // add css for enable/disable link
 14                 long pageIndex = getStartIndex() + iteration;
 15                 item.add(new AttributeModifier("class", new PageLinkCssModel(pageable, 
 16                     pageIndex, "active")));
 17
 18                 return item;
 19             }
 20         };
 21     }
 22
 23 // .......
 24 }

The newNavigation(....) method (line 6) generates links for page numbers: "1 | 2 | 3 | 4". The only thing that needs to be changed here is the indication of current page - it's a disabled link on a blue background.
This can be done by applying active class to li-tag (not the hyperlink itself):
<li wicket:id="navigation" class="active">
    <a wicket:id="pageLink" href="#"><span wicket:id="pageNumber">1</span></a>
</li>
The HTML above is the modified version for Bootstrap. When compared to original Wicket HTML, the li-tag was replaced by div (wicket:id="navigation"), and class="active" is used to indicate current page.

The replacement of li-tag with div does not require adoption of java code because wicket uses WebMarkupContainer, which is compatible with both tags.

The assignment of dynamic CSS to li-tag (wicket:id="navigation") takes place in method newItem(...)  (line 15 class above). Each link with page number is generated as LoopItem, and applied CSS will change class-argument to active for current page - this will deactivate link, remove onmouseover highlighting, and change background to blue.
Here comes  the model for our dynamic CSS:

  1 class PageLinkCssModel implements IModel<String>, Serializable {
  2
  3     private final long pageNumber;
  4
  5     protected final IPageable pageable;
  6
  7     private final String css;
  8
  9     public PageLinkCssModel(IPageable pageable, long pageNumber, String css) {
 10         this.pageNumber = pageNumber;
 11         this.pageable = pageable;
 12         this.css = css;
 13     }
 14
 15     @Override
 16     public String getObject() {
 17         return isSelected() ? css : "";
 18     }
 19
 20     @Override
 21     public void setObject(String object) {
 22     }
 23
 24     @Override
 25     public void detach() {
 26     }
 27
 28     public boolean isSelected() {
 29         return getPageNumber() == pageable.getCurrentPage();
 30     }
 31
 32     private long getPageNumber() {
 33         long idx = pageNumber;
 34         if (idx < 0) {
 35             idx = pageable.getPageCount() + idx;
 36         }
 37
 38         if (idx > (pageable.getPageCount() - 1)) {
 39             idx = pageable.getPageCount() - 1;
 40         }
 41
 42         if (idx < 0) {
 43             idx = 0;
 44         }
 45
 46         return idx;
 47     }
 48
 49 }

AjaxPagingNavigator#newPagingNavigationLink ("first, last")


  1 public class BootstrapPagingNavigator extends AjaxPagingNavigator {
  2
  3     @Override
  4     protected AbstractLink newPagingNavigationLink(String id,
  5       IPageable pageable, int pageNumber) {
  6         ExternalLink navCont = new ExternalLink(id + "Cont", (String) null);
  7
  8         // add css for enable/disable link
  9         long pageIndex = pageable.getCurrentPage() + pageNumber;
 10         navCont.add(new AttributeModifier("class", new PageLinkCssModel(pageable,
 11            pageIndex, "disabled")));
 12
 13         // change original wicket-link, so that it always generates href
 14         navCont.add(new AjaxPagingNavigationLink(id, pageable, pageNumber) {
 15             @Override
 16             protected void disableLink(ComponentTag tag) {
 17             }
 18         });
 19         return navCont;
 20     }

and corresponding markup:
<wicket:panel>
    <ul class="pagination pagination-sm">
        <li wicket:id="firstCont" class="disabled"><a wicket:id="first" href="#">&lt;&lt;</a></li>
       ........
        <li wicket:id="lastCont" class="disabled"><a wicket:id="last" href="#">&gt;&gt;</a></li>
    </ul>
</wicket:panel>
For the first- and last-links we need to change they way how they are being deactivated. The HTML above is already modified for Bootstrap - the original had only hyperlinks (red wicket ids), now they are wrapped by ul/li-tags.
From java code perspective, the AjaxPagingNavigator#newPagingNavigationLink(...) method would return AjaxPagingNavigationLink directly, and it would replace href elements with div for inactive links. We need however only to apply different CSS on li-tag (which does not exits in original wicket code), and not on hyperlink itself. This CSS will take care of deactivation and also disable highlighting for onmouseover.
The overwritten method newPagingNavigationLink(...) in line 4 returns AbstractLink, but we rather need li-tag. The solution is to return ExternalLink that points to nowhere - it will contain the original link as child. The names for our new tag:  firstCont and lastCont are dynamically generated from name of original link.  This gives us dynamic li-tag, which is required, because we want to change its style.
In line 10 we are registering dynamic CSS on it, which will set class="disabled" if pager is already on first, or respectively last page.
In line 14 we create the original link, that would be returned by unmodified newPagingNavigationLink(...) method. But of course we have to add this link to #navCont in order to mach changed HTML structure.
The method disableLink(...) remains empty, to prevent wicket from replacing a-tag with div-tag.


AjaxPagingNavigator#newPagingNavigationIncrementLink ("prev, next")


Links for next and previous page are crated analogous to first/last links described above. The only difference is the CSS model, which disables link when first  or last page is active.

  1 public class BootstrapPagingNavigator extends AjaxPagingNavigator {
  2
  3     @Override
  4     protected AbstractLink newPagingNavigationIncrementLink(String id,
  5           IPageable pageable, int increment) {
  6         ExternalLink navCont = new ExternalLink(id + "Cont", (String) null);
  7
  8         // add css for enable/disable link
  9         long pageIndex = pageable.getCurrentPage() + increment;
 10         navCont.add(new AttributeModifier("class",
 11              new PageLinkIncrementCssModel(pageable, pageIndex)));
 12
 13         // change original wicket-link, so that it always generates href
 14         navCont.add(new AjaxPagingNavigationIncrementLink(id, pageable, increment) {
 15             @Override
 16             protected void disableLink(ComponentTag tag) {
 17             }
 18         });
 19         return navCont;
 20     }
 21 }

  1 public class PageLinkIncrementCssModel implements IModel<String>, Serializable {
  2
  3     protected final IPageable pageable;
  4
  5     private final long pageNumber;
  6
  7     public PageLinkIncrementCssModel(IPageable pageable, long pageNumber) {
  8         this.pageable = pageable;
  9         this.pageNumber = pageNumber;
 10     }
 11
 12     @Override
 13     public String getObject() {
 14         return isEnabled() ? "" : "disabled";
 15     }
 16
 17     @Override
 18     public void setObject(String object) {
 19     }
 20
 21     @Override
 22     public void detach() {
 23     }
 24
 25     public boolean isEnabled() {
 26         if (pageNumber < 0) {
 27             return !isFirst();
 28         } else {
 29             return !isLast();
 30         }
 31     }
 32
 33     public boolean isFirst() {
 34         return pageable.getCurrentPage() <= 0;
 35     }
 36
 37     public boolean isLast() {
 38         return pageable.getCurrentPage() >= (pageable.getPageCount() - 1);
 39     }
 40 }



You can download source code here.