voyent
datatable performance with server push to all clients  XML
Forum Index -> Components
Author Message
akearns

Joined: 22/Nov/2006 00:00:00
Messages: 10
Offline


I am trying to display polled data in a datatable where each row is a polled device. When a poll is completed on a device the server pushes it to the client. My problem is the performance I am seeing and the amount of CPU the server is using to update the dom for each client.

Is seems my pushed data causes the datatable to update its entire model. Is there a way to limit the scope of the update (what is written to dom) to just a table cell or better just a row within the table?

Right now I am limiting when an update takes place using the delayed render manager. This helped, but will not solve the problem. As additional clients login the update will continue to use more and more of the CPU;
ie. I need to limit what is updated to what has changed. Is there any way to do this?

patrick.corless

Joined: 26/Oct/2004 00:00:00
Messages: 1982
Offline


Could give us a little more information on how you have the render groups setup. Fore example do you have a render group for each polling device? Also is a user only only added to a particular render group if the polling device is in it's view?
[Email]
akearns

Joined: 22/Nov/2006 00:00:00
Messages: 10
Offline


The initial design of my table is from a quickstart by Philip Breau.
My session data is orginized in a Tree but when rendered it is flattened into a datatable to visually represent a treeTable.

My renderable represents one pollable device the class TreeTableUserObject. This class extends from IceUserObject.
My datatable displays a collection of TreeTableUserObject ie. as returned by treeTable.linearList

Code:
   ice:dataTable value="#{treeTable.linearList}"
       var="row" cellpadding="0"
       styleClass="treeTable"  rowClasses="even,odd" columnWidths="40%,2%,2%,2%,2%,2%,2%,2%,2%,2%,2%,2%,2%,2%,2%,2%,2%,2%,2%,2%,2%"
 


TreeTableUserObject extends IceUserObject and implements Renderable
When any client is viewing that device the object is created and added to the renderManager
It is not added to the datatable until the user expands a treeNode.

Code:
 renderManager.getDelayRenderer(proId.getRenderRequestKey()).add(this);
 


proId.getRenderRequestKey return a constants string followed by the primary id of the table for that device. for example "proId.1001"

Only visible devices have JMS messages generated for them.

So yes, I have a renderGroup for each polling device. As additional clients view the same device the same renderGroup is used.

If my visible devices are polled every minute then I've used a delay of 5 seconds. The delay is setup just before I call requestRender.

Code:
     if (isBroadcastRequestNeeded()) {
         log.info(func + "requestRender " + guid.getRenderRequestKey());
         renderManager.getDelayRenderer(guid.getRenderRequestKey()).setDelay(RENDER_DELAY);
         renderManager.getDelayRenderer(guid.getRenderRequestKey()).requestRender();
     } else {
         log.info("requestRender skipping it " + guid.getRenderRequestKey());
     }
 


isBroadcaseRequestNeeded simply compares the current date to the the date the last request was made
deryk.sinotte


Joined: 26/Oct/2004 00:00:00
Messages: 1008
Offline


Would it be possible to provide some rough characteristics of your application?

Can we get an idea of the number of polling devices, clients, and typical number of rows in the table? Do all clients look at all devices? How often are the devices typically polled and updates pushed out?

The Direct-to-DOM rendering (D2D) attempts to do just what you are requesting. Take a look at what has changed on the page and only send the updates back to the client. However, to do this, it must still check the contents of the entire view to determine what has changed. In doing this, it runs the JSF lifecycle which will query the managed beans and, potentially, the model depending on its design.

So another question is, what is the shared data design for the devices. Are they all stored in a database accessed via JDBC, Hibernate,...?

Deryk Sinotte
Team Lead
ICEsoft Technologies, Inc.
akearns

Joined: 22/Nov/2006 00:00:00
Messages: 10
Offline


The application is a typical Element Management System. The main classes viewed by the user are:

Accounts
Folders
Devices

These classes are viewed by the logged in user as a tree. An account is authorized to view a specific folder and its children. When the user logs into their account the resultant tree starts at their folder. Nodes within the tree are not fetched until they are needed; on tree node expansion the children are fetched from the database

The database is accessed via hibernate. We use a postgreSQL database.

The polling of the devices is asynchronous to the rest of the application. I have a service it implements ServiceMBean which is told which devices to poll. Periodic continous polling is the responsibility of this service. When a poll is completed on a device a JMS message is sent. I only poll a device once per interval, it is independent of who many users are viewing it.

The client has an application scoped cache which holds the polled data. When the web is updated I can see my cache queried once for each client and each item in the tree. This is what I believe I need to optimize.


In testing I have 5 devices and 10 folder about 15 rows in the table. I have about 20 columns of information, only cells for the devices contain data. I usually have 3 clients looking at the data.

Right now I am polling the devices every minute and I attempt to push the data out every minute.

A client views the information they have permission to view. It depends on who they login as.


judy.guglielmin

Joined: 20/Feb/2007 00:00:00
Messages: 1396
Offline


It's hard to tell without looking at source, but are you creating your own cache or using existing cache libraries? Are you using seam-managed persistence context? The container? Have you analyzed to see which classes/methods are not performing well? Can we assume that the polled information is always different (or a delta is checked first and only pushed if there is a difference)?

If I have understood this correctly, each client is polling your application-scoped cache. Why not just push it out if it has changed or at regular intervals (rather than each client polling, have the server just push the info).

Is ServiceBean part of the same application? Do you add your view only if the item is in detail in the list or do you require any master information as well (in your treemodel?). If you only care about the details (what the EntityManager currently has), then I wonder if it's possibly to only register the view for the entity currently managed?

Correct any mis-interpretations, and we can hopefully narrow this down.
akearns

Joined: 22/Nov/2006 00:00:00
Messages: 10
Offline



I created my own cache, I am not using jboss_cache or other existing cache libraries.

I am not using seam-managed persistence.

The container is jboss 4.2.2

The profiling output I sent shows that this call heirarchy is taking most of the CPU:

87.0% - 135 s - 13 inv. edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run

86.9% - 135 s - 12 inv. com.icesoft.faces.async.render.RunnableRender.run

86.9% - 135 s - 12 inv. com.icesoft.faces.webapp.xmlhttp.PersistentFacesState.executeAndRender

86.7% - 135 s - 12 inv. com.icesoft.faces.webapp.xmlhttp.PersistentFacesState.render

This CPU snapshot was taken over 5 minutes with 7 devices being polled.

No, I have not implemented checking the previous datatable cell's value against the new to determine if I should call renderRequest. This would improve performance provided that nothing that the user is viewing has changed. I will still need to show the user that a poll has occurred, not sure how to do this without pushing data from the server.

>>If I have understood this correctly, each client is polling your >>application-scoped cache. Why not just push it out if it has changed or >>at regular intervals (rather than each client polling, have the server >>just push the info).
This is what I am doing by using the delayed renderer.

>>Is ServiceBean part of the same application?
The serviceBean is package as a sar within my overall ear file. This is all one application.

>>Do you add your view only if the item is in detail in the list or do you >>require any master information as well (in your treemodel?).
The list contains enough information for the table levels, access to the cache and a reference back to the treemodel.

>>If you only care about the details (what the EntityManager currently >>has), then I wonder if it's possibly to only register the view for the >>entity currently managed?
When the view is updated for one client it needs to be updated for all; polling is consolidated.
deryk.sinotte


Joined: 26/Oct/2004 00:00:00
Messages: 1008
Offline


I'm perplexed at the scale of the application and why it requires so much processing. Earlier in the thread it discussed 3 users, 5 devices, and polling interval of 1 minute. There shouldn't be any problem is this behaving properly given the scale of the testing.

The profiling output shows that the bulk of the processing is done in the Ajax Push cycle as suspected. The com.icesoft.faces.webapp.xmlhttp.PersistentFacesState.render call is responsible for determining the changes that need to be applied to the client. During this call, the JSF render phase is run, the changes to the client UI are calculated and sent back. It would be interesting to drill down a bit farther in the profiling output to see what is taking the render so long. If that information is available, I'd be interested in seeing it. Given that a cache is used, the information should be returned quite quickly.


Deryk Sinotte
Team Lead
ICEsoft Technologies, Inc.
 
Forum Index -> Components
Go to:   
Powered by JForum 2.1.7ice © JForum Team