katratxo on Software Development

tail -f /var/log/brain | grep -i software

Finding and fixing memory leaks in a SmartClient based application

leave a comment »

Finding memory leaks in a Web Application can be difficult, fortunately the tools for Web Development are getting better and better. You can read the post Finding memory leaks by Tony Gentilcore, where he explains how you can use Chrome Dev Tools Timeline and Heap Profiler to diagnose if your application is leaking memory.

In the case of a SmartClient application, you should know that when you create a new object, instance of a Class, the library pollutes the global namespace (window) with a global variable using the pattern: isc_TypeOfClass_Index e.g. If you execute the following code, you’ll end up with a new global variable isc_VLayout_0

var l = isc.VLayout.create({width: '100%', height: '100%'});

When you destroy an object, the global variable gets nullified:

l.destroy();
isc_VLayout_0 === null // true - still exists but its value is null

Taking this as a base, you could identify if your SmartClient application is leaking memory with the following procedure:

  • Take a snapshot of the current state of the global namespace
  • Perform the action you think causes the leak
  • Take a second snapshot of the global namespace and compare them
  • Every not null entry in the second snapshot that is not present in the first one, is a leak

Keep this strategy in mind.

Although most of the cases SmartClient handles objects lifecycle automatically, there are cases in which you need to destroy objects manually. Let’s take a look when this is required.

SmartClient handles child objects automatically

If you check the documentation, SmartClient has the concept of AutoChild that are subcomponents part of of a main object that gets automatically managed.

An example is the Window component and its subcomponent the “header” …
By default any auto-children created by canvas.addAutoChild or
canvas.createAutoChild will be destroyed when the canvas that created them is destroyed

Your child objects will be automatically destroyed if SmartClient knows about them, e.g. Using

Some code:

var l = isc.Window.create({
  width: '600',
  height: '400',
  items: [
    isc.Label.create({
      contents: 'Hello World'
    })
  ]
});

When you execute this code, you’ll have a lot of new global variables:

isc_Label_0
isc_Window_0
isc_EdgedCanvas_0
isc_Window_0_shadow
isc_Window_0_header
isc_Window_0_headerBackground
isc_Window_0_headerIcon
isc_Canvas_0
isc_Window_0_headerLabel
isc_Window_0_minimizeButton
isc_Window_0_closeButton
isc_Window_0_body

If you call the destroy method of the Window, the isc_Window_0 variable remains in the global namespace but with null value. The rest of variables are deleted.

isc_Window_0 === null // true

SmartClient doens’t handle objects that are not AutoChild

When dealing with complex composed widgets and if those subcomponets are not added using the AutoChild pattern, is quite easy to create memory leaks. An Openbravo example of this is the SelectorItem, a widget commonly used (Business Partner, Product selectors). This widget is a composite of a ComboBoxItem, a magnifier image and a ListGrid embeded in a Window that, is hidden by default, and shown when clicking the magnifier icon.

When destroying a SelectorItem you need to manually take care of destroying the associated objects.

SmartClient doesn’t destroy the dataSource object associated to a DataBound component

Another case is when SmartClient doesn’t destroy the associated dataSource object of a ListGrid.
This case is also present in the SelectorItem, since the selector Window contains a ListGrid with an associated DataSource.

The example below creates a new DataSource and a ListGrid bounded to it. The ListGrid is then added as item of a Window which is a member of a Layout that also contains a destroy Button. When the user clicks the button, the expected result is that all objects get destroyed in cascade.

As you can see the dataSource requires to get destroyed explicitly.

isc.DataSource.create({
  ID: "countryDS", // manually defining the global ID
  fields:[
    {name:"countryCode", title:"Code"},
    {name:"countryName", title:"Country"},
    {name:"capital", title:"Capital"}
  ],
  clientOnly: true,
  testData: countryData // sample data previously defined
});

isc.ListGrid.create({
  ID: "countryList",
  width: '100%', 
  height: '100%', 
  alternateRecordStyles:true, 
  showAllRecords:true,
  dataSource: countryDS,
  autoFetchData: true,
  destroy: function () { // 'overriding' destroy method
    if(this.dataSource) {
      this.dataSource.destroy(); // needs to be done manually
  }
  this.Super('destroy', arguments);
  }
});

isc.VLayout.create({
  height: 400, width: 600
})
.addMember(isc.Window.create({
  width: '100%',
  heigth: '100%',
  items: [countryList] // ListGrid gets destroyed
}))
.addMember(isc.Button.create({
  title: 'destroy',
  action: function () {
    this.parentElement.destroy();
}
}));

The Openbravo Case

In Openbravo 3 there were few places with some composed components, that child/related objects were not destroyed when destroying the main component.

Previous to the upcoming 3.0MP3 if you close a window (e.g. Sales Order), some objects were not getting destroyed, resulting in memory consumption increase. After working with the application a couple of hours (opening and closing several windows) the user got strange behavior like: slow reponse to user actions, slow repaint process, etc.

Some of the components that were not properly managed are:

  • LinkedItems section (section in Form view that shows you the related records)
  • SelectorItem (the component behind Product, Business Partner selector)
  • StatusBar (the component in the upper part of the Form view)
  • Loading Tab (temporary tab shown when requesting a View definition to the server)

Linked Items

Linked Items

The Linked Items section contains two ListGrids and a DataSource each one of them. When destroying the Form, you need to manually destroy the ListGrid and DataSource associated.

Selector

Selector

As explained before, the Selector is one of the most commonly used widget. When destroying the Form, you need to manually destroy the Window that contains a ListGrid and the associated DataSource.

Status Bar

Status Bar

The StatusBar contained some images not added using the AutoChild pattern and required to be manually destroyed when destroying the StatusBar.

Loading Tab

Loading Bar

When opening a window, a Loading Tab is shown in the TabSet. After the window gets created the Tab content is replaced using TabSet.updateTab. The documentation clearly states, “NOTE: the old pane for the tab is not destroyed”, you need to manually destroy the Loading Tab.

The Fix

Using scopeleaks

If we use the strategy described at the begining, we need to take a snapshot of the global namespace, perform the leaky action (open/close a window), take a second snapshot and compare.

We have used and modified scopeleaks, a utility tool created by Rui Lopes. The tool is intended for detecting leaks of variables to the global namespace. Since we now that SmartClient always leaks variables, we have modified it to only check for instances of a class, meaning any new global variable starting with isc_ that is not null.

Detecting leaks

The steps to detect a leak:

  • Login into the application

  • Open Web Dev Tools

  • Take a first snapshot of the global namespace

    var s1 = scopeleaks.snapshot();
  • Open and close a window (e.g. Sales Order)

  • Take a second snapshot and compare

    var leaks = scopeleaks.leaks(s1); // leaks will have an array with all the global variables that are not null

Repeating this flow, you could detect if a user interaction, creates new objects that are never released.

If you are interested in checking all the changes made on the components mentioned before, check the Issue 18227.

Conclusions

Solving simple memory leaks in a SmartClient based application is straight forward if you use this strategy. Note that we are only fixing the most obvious ones.

SmartClient is a great library, but even SmartClient code could cause memory leaks.

You need to learn and know the framework you’re using. You cannot go blindly creating objects here and there and expecting that everything will automatically collected when is not needed.

Use all tools available and measure memory consumption when using the application.

Written by katratxo

August 25, 2011 at 12:16 pm

Posted in Openbravo

Tagged with , , ,

Improved JSLint check on Mercurial

with 2 comments

We have been working with the previous version of the Mercurial JSLint precommit hook for several months now, and we pretty happy with the results, it catches the most common JavaScript pitfalls.

However this hook was not smart enough, it checked all the modified JavaScript files even if they were not part for the commit, and this situation annoyed some experienced developers:

I have unfinished JavaScript changes, but I’m committing a Java files!! Why this hook is checking files that are not part of the commit!?? I need to do this Java changes now … I’ll disable this hook.

The Solution

The solution is to run the check on the pretxncommit hook, where you have already the list of files that are part of the commit. You can check the modified JavaScript files and if JSLint complains, the transaction is rolled back.

The Code

The changes are simple. Use hg log to get the list of files from $HG_NODE and pass them through JSLint.

Update: The previous version of the script was not working on every case. The problem is that hg log –template ‘{files}’ outputs all the modified files in a single line. That output saw used as input of awk, that was not filtering properly. I found a workaround, perhaps is not the nicest script, but is working.

#!/bin/sh
# Mercurial pretxncommit hook
for i in $(hg log -r $HG_NODE --template '{files}'); do echo $i | awk '/.js$/ | xargs -rn1 /path/to/jslint || exit 1; done;

Credits

As many times, thanks to Juan Pablo and the Mercurial Community for providing the way to check only the files part of the changeset.

Written by katratxo

July 30, 2011 at 8:55 am

Posted in Openbravo

Tagged with , , ,

Openbravo 3 Developer’s Guide

with one comment

We have been working hard to finish the new Developer’s Guide targeting Openbravo 3. This guide contains information on how to change, extend and build further the application providing valuable information for starting developers, medium experienced and Openbravo experts.

The New Structure

Following best practices we are now using Categories as the way to structure our documentation. An overview of the categories is shown in the following diagram:

Developer's Guide Categories

  • Concepts: Contains documents that aim to explain core concepts behind Openbravo architecture.
  • HowTo: Step by step actions to achieve a goal, e.g. How to create and package a Module
  • Data_Model: Automatic generated documentation that contains the Database and Entity Model
  • Example: Documents that explain code present in the product that could be used as base for our developments
  • Reference:The hibernate mapping, REST XML Schema among other reference documents

Feedback

If you find a document that contains something wrong, outdated or not properly explained, please fix it, after all is a wiki. Note: If you want to make a large change in a document, please contact the original author explaining your proposed changes.

You could also log a defect or feature request in the issue tracker in the Documentation project. Remember to pick the right category: ERP Developers Guide.

I’ve also started a forum thread to collect your feedback and ideas on the Developers Guide.

That’s all folks!

Written by katratxo

July 14, 2011 at 12:41 pm

Posted in Openbravo

Tagged with ,

Development tips: JavaScript debugger statement

with 2 comments

According with the MDC the debugger statement was introduced in ECMA-262, Edition 5

Invokes any available debugging functionality. If no debugging functionality is available, this statement has no effect.

This statement is very handy when trying to debug eval’ed code like the View definition in Openbravo 3.0. It seems the only way to have an entry point to the JavaScript program running in the browser.

I’ve tested this statement in Internet Explorer 8, Firefox 4b10 and Chrome 9. And works when you have a debugger running. Note: In Firefox you need Firebug available for that page.

<html>
<head>
</head>
<body>
<script type="text/javascript">
function f() {
 var message = 'hello world';
 debugger;
 if(typeof window.console !== 'undefined') {
   console.log(message);
 }
 else {
   alert(message);
 }
}
f();
</script>
</body>
</html>

Here you have some screenshots:

Internet Explorer
Internet Explorer - JS debugger

Firefox
Firefox/Firebug - JS debugger

Chromium/Chrome
Chrome/Chromium - JS debugger

Written by katratxo

February 10, 2011 at 8:04 pm

Posted in Openbravo

Tagged with , ,

Update: Making a snapshot of your Openbravo instance

with 4 comments

I just updated the snapshot.xml script adding a new property exclude where you can define as comma separated values the patterns you want to exclude from the zip file.

If you are not familiar with this script, I suggest you read my previous post on how to make a snapshot of your Openbravo instance

I tested in some recent revision from pi, and excluding Mercurial metadata and the resulting zip file is almost 350MB smaller!

~/workspace/src/openbravo $ ls -lh openbravo-2011*
-rw-r--r-- 1 iperdomo iperdomo 254M Jan  5 11:04 openbravo-2011-01-05_11-02-07.zip
-rw-r--r-- 1 iperdomo iperdomo 603M Jan  5 11:19 openbravo-2011-01-05_11-10-58.zip

How it works?

You just need to append the property with the list of excluding patterns in the ant call, e.g. Let’s exclude all the .hg folder

~/workspace/src/openbravo/pi $ ant -f snapshot.xml -Dexclude=".hg/**"

The exclude property is passed directly to the Zip task. In order to understand how can you define exclusion patters read the Zip task documentation

Happy new year!

Written by katratxo

January 6, 2011 at 10:19 am

Running JSLint as Mercurial precommit hook

with 6 comments

JSLint is a tool that helps you identify potential problems in JavaScript code. Some experienced developers may disagree with the coding style enforced by the tool, but if you are beginning with JavaScript this tool helps you stay out of troubles.

Mercurial hooks

Mercurial offers a powerful mechanism to let you perform automated actions in response to events that occur in a repository …

Mercurial: The Definitive Guide

The idea is to automatically check all the modified JavaScript files when trying to perform a commit. If the tool (JSLint) complains on at least on file, the commit is aborted.

The components

jslint4java

jslint4java is just a Java wrapper around JSLint and gives you the possibility to run it command line, e.g.:

java -jar jslint4java-1.4.4.jar application.js
jslint:application.js:11:9:Line breaking error ')'.
jslint:application.js:11:10:Missing semicolon.

Some of the options are for the tool are:

Usage: jslint4java [options] file.js ...
Options:
--adsafe    If adsafe should be enforced
--bitwise   If bitwise operators should not be allowed
--browser   If the standard browser globals should be predefined
--cap       If upper case html should be allowed
--css       If css workarounds should be tolerated
--debug     If debugger statements should be allowed
--devel     If logging should be allowed (console, alert, etc.)
--encoding  Specify the input encoding
--eqeqeq    If === should be required
--es5       If es5 syntax should be allowed
--evil      If eval should be allowed
# Many more (...)

jslint

jslint is a shell script wrapper for calling jslint4java, e.g.

#!/bin/sh
# Note: Modify the path where the .jar is located
java -jar /home/iperdomo/tools/jslint4java/jslint4java-1.4.4.jar --evil $1

jscheck-hg

jscheck-hg is a simple shell script that checks for modified or added JavaScript files and jslint them

#!/bin/sh
# Note: Modify the path location for jslint (the previous script)
hg st -man | awk '/.js$/' | xargs -rn1 path/to/jslint || exit 1

Defining jscheck-hg as precommit hook

With all the components in place, you just need to add the hook to the repository metadata, for this modify the .hg/hgrc file, e.g.

[hooks]
precommit = /path/to/jscheck-hg

Openbravo specifics

Last Friday, the modules for 3.0 were merged into pi, so all the components required for setting this hook are already in the repository. jslint4java, jslint, jscheck-hg are part of the org.openbravo.client.kernel module.

You just need to add the precommit hook to your repository using the jscheck-hg available in org.openbravo.client.kernel/jslint. Open your .hg/hgrc and add the following lines:

[hooks]
precommit = ./modules/org.openbravo.client.kernel/jslint/jscheck-hg

That’s all. Now every commit that contains a JavaScript file modification, it needs to pass the JSLint check.

Happy coding!

Written by katratxo

December 13, 2010 at 5:57 pm

Development tips – Part II

with one comment

In Part I I explained how to:

  • Speed up Firefox by creating a different profile for Firebug and ‘vacuum’ the
    browser’s internal databases
  • Update to the latest ‘safe’ pi revision
  • Clean up your Mercurial workspace

You also know how to configure the Autologon authentication manager that allows you login the application with the same user/password skipping the login page.

Today we’ll focus in speeding up the compilation process using just some configuration properties that will result in skipping some tasks that are not required in the day-to-day development process.

Note: I assume that you are working with Eclipse IDE, you have successfully build the application and also you have experience how to compile the application. If you don’t know how to do that, please read the how to setup your development environment in the wiki.

The Openbravo.properties

The Openbravo.properties file is use to configure the application, e.g. database connection details, date format, etc. But also have properties for the build process and tweaking them allows you skip some tasks when compiling.

You can read more about the Openbravo.properties in the wiki page.

Note: Remember to always read documents in the ERP/2.50 namespace, since all other documents may have been deprecated or are no longer maintained.

minimizeJSandCSS

Some time is spent in minimizing (compressing) the JavaScript and CSS files to the context. That’s why when running the application and you try to see the
.js files everything is compressed without line breaks or spaces. The quick fix to skip this process is set to no the minimizeJSandCSS property.

# use js/css minimization (in local-context and war-file)
minimizeJSandCSS=no

deploy.mode

The deploy.mode defines where the build process needs to copy files and if it needs to generate a .war file. The default deploy.mode is class, so in every compilation call the files are sync’ed with the $CATALINA_BASE/webapps/context folder.

Since you are working with Eclipse the tomcat server you’re using is the one inside Eclipse, the $CATALIBA_BASE/webapps/context is not used at all, instead, the files are read from the WebContent and build folders.

Setting to none the deploy.mode will skip the sync of the files, and also will not generate a .war file.

#Deploy mode: valid values [class, war, none]
deploy.mode=none

Wrapping up

Setting minimizeJSandCSS=no and deploy.mode=none will speed up the day-to-day compilation process.

Remember that this changes recommended just for development environments not for production ones.

Stay tunned!

Written by katratxo

August 20, 2010 at 10:59 am

Posted in Openbravo

Tagged with ,