Imran's Tech Blog

RESTful Web Services

2011-02-07T20:18:00.000-08:00

I have come a long way in learning, understanding and imlpementing RESTful (at least RESTlike) systems in real life. From that experience of mine recently I made a technical presentation at BASIS SoftExpo 2011. The response seems to be good. The slides are as follows. Criticisms, suggestions are most welcome.

RESTful Web Services

View more presentations from imyousuf.

Using GNOME Blog

2010-06-08T08:13:00.000-07:00

I was looking for a client to work offline and post my entry, rather than writing it from a browser. I did a simple apt-cache search and found GNOME Blog (gnome-blog). So now testing it out. Lets see how it turns out; first impression is rich text edit capabilities is very limited!

RESTful Architecture

2009-07-07T21:49:00.000-07:00

The title is definitely making some readers think why is someone again writing on this topic. What inspired me to write again after a long time is the confusion between "RESTful Web Services" and "XML over HTTP". I came accross this confusion while working on a framework of mine and later I will mention how it interested me. So I would like to clear the confusion, as per my understanding, on RESTful Architecture.

First lets see how Dr. Roy Thomas Fielding in his PhD dissertation defines REST,

REST is defined by four interface constraints: identification of resources; manipulation of resources through representations; self-descriptive messages; and, hypermedia as the engine of application state.

In this writeup I would like illustrate how Dr. Fielding illustrates this and my (as a scholar of REST) understanding on the matter.

As the REST wikipedia page mentions at the center of the RESTful architecture is "Resource" and I fully agree with it. So I would like start by defining what is a resource. Dr. Fielding defines it as,

...any concept that might be the target of an author's hypertext reference must fit within the definition of a resource. A resource is a conceptual mapping to a set of entities, not the entity that corresponds to the mapping at any particular point in time.

Now lets pick a common concept to build the idea of resources. Lets take a book sales aggregator service (similar to Google Checkout); it sells books written by one or many authors and published by one or more publishing house and sold at different prices by stores and are in different state (new or old) and different descriptions to go with them.
The obvious resources here are Book, Store, Author and Publisher and I would have BookState be a resource as well, it is comprised of price and description. Cardinality would be Book has many Stores, Authors and Publishers; Store has many BookStates. I would also like to make BookExcerpt a weak entity of Book. So according to Dr. Fielding's definition and my understanding of resource all the objects in OOP (in italic) are resources.

Before going into the nature of representation lets see what Dr. Fielding has to say about constituents of a resource -

The values in the set may be resource representations and/or resource identifiers. A resource can map to the empty set, which allows references to be made to a concept before any realization of that concept exists. ....... The only thing that is required to be static for a resource is the semantics of the mapping, since the semantics is what distinguishes one resource from another.

The first part of the statement actually signifies an advantage of RESTful architecture and that one may develop individual resource components independently. The second part represents a constraint on availability of definition of how resources are inter-related . So in our example the cardinal semantics between the resources/objects should be static. Now that we know about resources next thing to visit is the representation of resources and Dr. Fielding's take on this is,

A representation consists of data, metadata describing the data, and, on occasion, metadata to describe the metadata (usually for the purpose of verifying message integrity). Metadata is in the form of name-value pairs, where the name corresponds to a standard that defines the value's structure and semantics. Response messages may include both representation metadata and resource metadata: information about the resource that is not specific to the supplied representation.

Control data defines the purpose of a message between components, such as the action being requested or the meaning of a response. It is also used to parameterize requests and override the default behavior of some connecting elements. For example, cache behavior can be modified by control data included in the request or response message.

The representation of a resource in different media type format (referred to as data, while the media type itself is a metadata) is just part of the architecture and not architecture itself. The messages should be self-descriptive, the representation format for the resource should respect other resources as they are and the manipulation of all resources and their relations to other resources should be done through transition of states. So if I want to change the cover image of a Book resource I should be able to do it using a PUT command of HTTP.

Now here is where the confusion lies - lets say I am using Java on the server to respond to the requests. If I am using Jersey+JAX-RS stack and application/xml as media type, there is a good chance that I am also using JAXB (not that its mandatory, one can use their on converters, known Producer and Consumer) for converting your resource object to XML; I will see that Book is converted to a XML which contains the representations of authors and other related resources and sub-resources within them. This creates multiple problems, e.g.,

Why do I need to get all entity sets of a resource and its sub or related resources, while I do not need them? Which will cause an additional and not-required payload to be submitted over network.
How do I manipulate any related resources of book without actually transmitting the whole book, e.g. author(s)?
How do I sent different set of control bytes for different resources linked within the Book?

IMHO, here is where the principal of REST is violated, unless the answers to the above like questions are positive. So if we are to design a RESTful API we should consider not only the representation part but also the manipulation and saying so, IMHO, using different HTTP commands (or equivalent for other protocols) is not sufficient but actually designing the resource semantics such that it can be manipulated partly and totally (depending on requirements and other factors) is a must as well.

Another important component of the architecture, which is not mentioned in the definition of REST (probably because it was reason of REST coming to existence) is Uniform Interface. The only thing it does not mention explicitly mentioned as a constraint is that the representation format also should be uniform or conform to established standards, so that its readable in a uniform manner by the clients, e.g. how resources are linked - in HTML it should be <link>, in XML it should be XLink etc.

So here is my checklist for building a RESTful system:

Identify resources and their identifier. Deduce the problem domain well to divide them to as many independent component as possible.
Choose the states well for transiting between resource states for the specific protocol. E.g. if protocol is HTTP you might map the methods as mentioned here; that is not enough, HTTP headers and response codes should be choosen appropriately; for example cache control, compression.
Resources should refer to individual resources by their URI; state transition for relation between resources might (not sure but IMHO must) be implemented as well.
The representation format should follow established norms. E.g. XML representation format should have a DTD or XSD as a metadata to the representation.

In context of our book sales aggregator example I want to elaborate my understanding of some of the above mentioned checkpoints. Firstly we have (tentatively) identified our resources; next I would want to create useful identifier for them. For Book it would be , for Author it would be email address, for Publisher it would be their name in same formatting as a blog or wiki page title and so on. In my XML representation of book, I would have a element encapsulating all authors, it would in fact be <authors>; and every <author> in would refer to the author resource using XLink. I would have the following for updating the author relation with book - /books/{ISBN}/authors/[slot/{index}]?. Now lets say I want to get the 1st Author for Book A, then my URI would be /books/A/authors/slot/0; the response message could be either a XML pointing to the Author's URI or (my choice) use HTTP response status code 302; also I would make use HTTP compression using the headers. IMHO rest of the checkpoints are trivial.

I would like to end this writeup by stating why I came up with this post . I am working on the framework stated at the beginning to make infrastructure code for common tasks, such as versioning, full text search, DAOs and representations of domain objects in various media type available out of the box. For versioning and representing domain objects I could not find a better way than REST. I am also working on another project which needs Web Services to interface with clients; my past experience with SOAP was sour enough for me to look into alternatives and I chose RESTful WS, but later realized it was nothing but RESTlike WS or XML/JSON over HTTP. Furthermore this writeup just expresses my understanding and in some case thoughts of RESTful architecture, so if you feel I have misunderstood or made any mistake, feel free to correct me :). If you are interested in more details of implement REST over HTTP please check this out.

Upgrading from Ubuntu Fiesty Fawn (7.04) to Hardy Heron (8.04)

2009-01-18T20:01:00.000-08:00

Since I have procured my laptop I have been using Ubuntu Fiesty Fawn (7.04) on it and I was more than satisfied until its support period expired on Jan 2009; I was compelled to upgrade and I did not want to take the hassle of upgrading again in the lifetime of this laptop so I decided to upgrade to Hardy Heron (8.04 LTS) (via Gutsy Gibbon). This writeup is my experience on this upgrade procedure.

My laptop is a Acer 5585WXMi, with GeForce Go 7300 VGA card, 802.11a/b/g WLAN, 5-in-1 Card Reader and Bluetooth built-in. In Fiesty all devices were recognized perfectly other than the Card Reader, did not spent much time on it since it was not much used by me.

The first hitch I faced was I found that Fiesty repository aren't there anymore so I disabled them from APT source list (/etc/apt/source.list or package manager). Then I started upgrading to Gutsy. During package installation it was trying to modify some configurations that I had earlier edited for my usage and also some which I had never touched :), so I looked at the difference (both upgrade from console and UI has it - 'd' from console) and took appropriate action. Most cases I let it override, the exception was MySQL and some other packages. The first problem I faced after upgrading to Gutsy was the display as it was stuck with 640x480 resolution and it was quite frustrating, but before spending more time on it I moved on to upgrading to Hardy.

During upgrading to Hardy all went smooth except for Slapd upgradation failed because the slapd database had data from unconfigured domain which was okey @ Fiesty and Gutsy. So after installation I manually restored the data from the backedup LDIF file and Slapd configuration was successful. But VGA was a pain as usual :(. Cutting the long story short to get the VGA to work all I had to do was enable the restricted NVidia driver and delete the /etc/X11/xorg.conf (obviously after backing up), followed by OS reboot and guess what now I can enjoy 1280x800 resolution :). Now a side note - I got the 'vesa' driver to work with 1024x768 but the problem was everything was vertically squeezed and it was depressing especially to view the photographs I took myself. Plus I also ran into problem using MSN & GTalk with Pidgin because of SSL lib issue (I am not certain why, but it could be related to my attempt to install pidgin on fiesty from source without SSL). So I uninstalled pidgin related packages and installed it from source by the help of this post; please add '--with-system-ssl-certs=/etc/ssl/certs' option during build else you will face unwanted hazards with certificate chain. Another problem faced was the Flash plugin of Firefox was not producing sound. For that I simply set all devices @ System -> Preferences -> Sound to use ALSA (by default it is PulseAudio). Plus I needed support for 'docx' and for that I needed OpenOffice 3. So I uninstalled all OpenOffice packages from my installation and downloaded DEB packages from OpenOffice.org and installed them using 'dpkg'. After main DEBs are installed check the DEB folder for another DEB @ 'desktop-integration/', that will create the menu shortcuts for you.

For photographers and users interested in graphics package you will find 'ubuntustudio-graphics' package useful, one might also find other ubuntustudio packages (apt-cache search ubuntustudio) useful - it has audio, video, desktop etc. packages for enthusiasts and professionals.

In a nutshell, now my Hardy is totally under control and I am enjoying using it :), so if anyone has not upgraded to Hardy yet please do so :).

Why Python?

2008-09-10T01:09:00.000-07:00

For long I have been wondering "Why Python?" - why is python gaining popularity? Why do many prefer python for all sorts of work? Myself being a Java fan (some of my friends say fan-atic) it makes it more interesting to me to understand the reason behind its uprising.

I am a software engineer mainly working on enterprise systems with Java. One of my hobby is to write scripts for anything that I (might) have to repeat in near future. I usually prefer shell scripting for writing scripts. But for some particular tasks, like converting seconds to ISO8601 formatted date string, multi-threaded http request and response handling with file i/o; I needed something extra for these as shell script was becoming overly complicated.

From all the buzz about Python I said myself let me give it a shot and check it out. I have to admit that I was astounded, stupefied , awed at how simple and powerful it was to achieve all that I needed for my use case. It took me just about to 3 hours to get Python 2.5, IDLE installed and write my first 2 programs which the do the following tasks-

Take seconds as input from command line argument and print its ISO8601 equivalent.
Take URLs as input and take 'n' samples for their response-time and total duration while all URLs should perform these tasks in parallel.

Not only have I learnt new languages earlier, I was also a Lab Instructor and Teaching Assistant for programming courses, from my experience to say that I will get these things done within 2 hours without having even read ANY article on Python or knowing nothing about it in the past, I have to say that its awesome (Please let me know if you feel I was slow). It was down to simplicity of the language compounded with fluent syntax and excellent documentation. Within the time mentioned I learnt and used Objects, Collections, Classes, Control flows (if, for), Exception handling, File I/O and Threads. If you are interested to checkout what I achieved you can checkout them out here.

From what I read later, I learnt that its equally simple to build UIs with Python and with some Googling I learnt of Google App Engine, which just makes developers life easy (at least thats what its there for).

I plan to do more work Python in near future and learn it and master it. If you have not yet tried Python, then I recommend to do ASAP and enjoy it.....

Setting up firewall in Ubuntu

2008-05-03T21:04:00.000-07:00

Though I am not a server or network administrator I have always been interested in learning how to secure a network. From some initial reading I learned that firewall is the starting point and trust me when I say that with Ubuntu 8.04 Server (code-named 'Hardy Heron') its seriously easy to setup a firewall. This article of mine will attempt to show beginners like myself how easy it is.

I am assuming that readers will have Hardy Heron installed before embarking on testing out the firewall. Once its installed once install the firewall front end 'ufw' using the following command -

sudo apt-get install ufw

For details reading on it one may try the Ubuntu wiki or 'man ufw'. So now I can get started in doing what I wanted to.

We have a network at my place and I want to restrict SSH from IPs other than mine and not only that I also want to ensure that pinging my servers return nothing. Being a newcomer to network firewalling to me it would be quite nice to achieve it. In general what I have seen for SSH is, there is only one gateway for the outside world to SSH into a network and from there one can SSH to the servers one is permitted to. Now SSH'ng the Gateway could be made further challening by specifying a IP to achieve which one has to be connected to the network VPN. Does it sound complicated to achieve? After using UFW I am pretty confident its not that difficult to set something up like this and hopefully you will feel the same.

I will skip the VPN part as that is a topic of it self and hopefully will have a writeup on how-to set it up sometime soon; setting a VPN server is not that difficult either thanks to OpenVPN, so interested readers if required can jump into it. My target is to block ping and block SSH from any IP other than my designated range.

Once one has UFW installed, first step would be to enable it and to that use the following command -

sudo ufw enable

Once it is enabled and one wants to check the status, one can use the following to see it -

sudo ufw status

If one wants to enable logging one can do -

sudo ufw logging on

I suppose one can easily guess how to turn logging 'off'.
The next step would be to instruct firewall to allow SSH from a particular IP or IP range. One can use the following command respectively to allow if for the 2 cases mentioned above -

sudo ufw allow from 192.168.0.104 to 192.168.0.113 port 22
sudo ufw allow from 192.168.0.0/24 to 192.168.0.113 port 22

Now the obvious question could be why mention to IP address, its because a server may have more than 1 IP address and to mention which IP address this rule would apply to the to IP address is required. If you wondering how to calculate IP range you might want to have a look at the wikipedia page and IP Address range calculator.
Now one will need to ensure that default policy is deny and to achieve that issue the following command -

sudo ufw default deny

At this point I feel that I should also mention how delete a rule; its simply just add 'delete' before the start of rule definition. For example, for the rules of SSH one can issue the following commands to delete them -

sudo ufw delete allow from 192.168.0.104 to 192.168.0.113 port 22

sudo ufw delete allow from 192.168.0.0/24 to 192.168.0.113 port 22

At this point I was thinking since default is deny and I have specified only port 22 to be open for a particular IP range then ping should not work and thinking that I pinged the server but to my astonishment I got reply. Then I started to Google and I found this. Following it I commented out the following line from /etc/ufw/before.rules -

-A ufw-before-input -p icmp --icmp-type echo-request -j ACCEPT

And pinging again returned nothing to my liking.

Now with combination off OpenVPN and UFW one can easily achieve a somewhat securer environment; saying so I actually loved the statement of Linus Trovalds when he said security is build on network of trust in his talk at Google regarding GIT. I am also a newbie to secure networking domain so please feel free to drop by your comments on the issue. If you are wondering why would I use UFW you can have a look at the small discussion in the comments section of this posting.

Code generation made easy using patterns

2008-04-18T05:44:00.000-07:00

Since starting developing some code generation plugins (toString() generator and Java Util Logger Generator) using/for the NetBeans 6.1Beta I have learnt a lot and the more I worked on it I kept asking myself how could I make it easier. Once both the plugins took rudimentary form I learnt and discovered that patterns could make this task whole lot easier for me. So I started re-implementing the Logger generator (if you are interested in getting the resources please check the logger blog) plugin.

Now I wanted to implement that the plugin will search for System.out.print(ln) and out.print(ln) if static import exists. In doing so I had to walk the PARSED tree of a Java Source file. Just to give me a flavour that I am implementing something cool I called it JavaSourceTreeParser. Basically what it does is breaks down each every Java Statement to a form which can not be further decomposed. In the initial version the conversion was done in the parser it self (Revision 21). Then it felt that I could easily use Observer pattern for it and after implementing it (Revision 31) I saw that I am right and I achieved something which every code generator can use. (As I use GIT SCM I usually work offline and make bunch of commits together so dont worry how I make multiple commits at a go :)).

Once implementing it I found that for performance improvement and communication between various listeners I felt the need of session for state information sharing and then I decided to use Composite pattern for the purpose. and it also worked like a charm and as a result improving the overall performance of the plugins. The following diagram might give an idea how to use it using the API I designed.

What I implemented in the listener simply that the listener will get notified whenever the parser come across a particular type of tree, for System.out.println it would be MethodInvocationTree (Kind.METHOD_INVOCATION), and the listener according to its implementation will handle the Tree; in my case I replaced the method invocation with a logger invocation. In case where I inserted a Log method at the beginning of every method body I listened to Kind.METHOD. Now I am implementing Log insertion for Throw, Return and End of Method block. In process I will also add parsing life cycle listener to the API so that modification to the class is done only once when the parsing is completed. I hope this API helps users interested in code generation. I am very interested to learn what users think about it.

NetBeans: Generating Java Util Logger

2008-04-15T00:47:00.000-07:00

Java Util Logger has enabled us to ship application without having dependency on any external JARs or APIs for logging purpose. After starting to use it I felt the necessity of a Logger creation tool which will create and initialize a logger for me instead of me either writing it or Copy-Pasting it for every file or worse every class. Also using a tool will enable me to maintain a coherence of naming and initialization of loggers across a project.

With the release of Java Source API with NetBeans 6.1Beta I finally got the chance to write such a tool for myself :). In sequence with my toString() generator module I started this module as well. You can download the NBM file and install it as it is free and open source. How-to of this tool is as follows.

Basically what I needed for it can be stated in 3 steps -

Parse/read the Java File
Find declared classes
Check if Logger exists if not create it and add utility methods.

With the new Java Source API and Java Compiler Tree API it is nothing more than simple tree traversing for the purpose. You can have a peek at the implementation in the addLogger method in the source file (after opening the file 'Find' might be useful). One will find these NetBeans Wiki pages being helpful.

NetBeans - Generating toString for Java classes

2008-04-02T22:29:00.000-07:00

My friend and colleague Shams developed a maven plugin to generate toString and equals method using Eclipse source code manipulation API and since then, being a NetBeans fan, I wanted to develop a similar component using NetBeans API. Though the Java Source API of NetBeans was available in the developer version, but it is finally going to be released with NetBeans 6.1 and one can have a look at it in 6.1Beta release.

So I started doing what I have been wanting to do for a long time :) - develop a NetBeans module to generate toString() for my Java classes. In this endeavour I came up with this open source project. In the generation of toString() I added iteration over array and collection over what Shams did. Users can simply download the NBM file from here and install it into your NetBeans 6.1 installation. To confirm that the installation is successful check the "Source" menu and you should see "Generate toString()" at the top of the list. To see the module in action simply open a Java Source file and choose the menu action and you will see that it will add/replace toString() methods of the classes in the file.

As future works to this plugin I have plans to add it to the context menu in project explorer for Java projects, provide a UI for user to choose what properties should be in toString and whether to replace toString or not. Additionally generate the current toString content in a separate method and invoke it from toString().

I also have plans for further plugins ; lets hope I have enough time for it :). Please create issues in the project to report issues or enhancements. Participation in development is most welcomed.

Picasa on Ubuntu

2008-03-02T20:41:00.000-08:00

I like Picasa to share photographs with my friends and relatives. I find it quiet annoying to use Picasaweb to upload pictures, Picasa (the desktop application) is just so cool. Thus I decided to install it on my Ubuntu; before starting the installation I did not except it to be a walk in the park, believe it or not it was.

Install Wine as specified in this blog; Then type winecfg in your console and you will see a GUI popping up. Ensure that your Windows Version in the Applications tab is Windows XP. Download Picasa and double click the exe it should start the installer in the Wine emulator; if not use the following command -
wine (Path-to-exe)/picasaweb-current-setup.exe
Now proceed with the installation as you would in Windows. If you choose to have a desktop shortcut you will see one in your desktop double clicking it would start picasa in your wine emulator. You can also start Picasa from your Applications -> Wine -> Programs -> Picasa2 -> Picasa2; you will also find the uninstall action there (which I did not use).
Now you can enjoy using Picasa on your Ubuntu; Happy Image Sharing!

Finding files in jars by their content

2008-02-27T18:59:00.000-08:00

While doing a bug hunting we needed to find a file containing a certain string and we did not know where to look for it. At one stage of "file hunting" we needed to be able to search for the file in jars (Java Archives). Thus in searching for it we came up with this shell script; using this script not only can we find jars containing certain filename but also search file content to get results. Lets take three use cases - "I want to find jars that contain .properties file", "I want to find jars containing .properties file containing the word com.smartitengineering" and "I want to find jars, whose name has smart, containing .properties file containing the word com.smartitengineering". Using the script all these use cases can be achieved by using the following commands respectively:

./search-jar.sh - /home/user/ .properties -
./search-jar.sh - /home/user/ .properties com.smartitengineering
./search-jar.sh *smart*.jar /home/user/ .properties com.smartitengineering

The command takes exactly 4 arguments - jar_filename_pattern, search_dir_path, search_file_name and content_to_search. If jar_filename is "-" than it will search within all jars within search_dir_path. If content_to_search is "-" it will not search for content, but rather jars with certain filename.

As the script is in open domain I would appreciate if users would make their own edit and submit or request for features through this post's comments.

Ubuntu: Getting Java Applets to work with Firefox

2008-01-11T02:15:00.000-08:00

I was having problem to get Applet working in Firefox, primarily because I work with manually installed JDKs (that is not downloaded using apt-get). I did the following to get Applets working going through the following procedure.

locate libjavaplugin_oji.so

After I do this I will get the list of this file present in my machine. In my case one of the paths was -

/opt/jdk1.6.0_01/jre/plugin/i386/ns7/libjavaplugin_oji.so

Now I did the following and restarted my machine -

sudo ln -s /opt/jdk1.6.0_01/jre/plugin/i386/ns7/libjavaplugin_oji.so /usr/lib/mozilla-firefox/plugins/libjavaplugin_oji.so

After this I got applets to work in Mozilla Firefox. Just for fun delete the symlink and without restarting the browser check whether you can see the applet or not :).

Propagating GIT commands to its submodules

2008-01-02T23:01:00.000-08:00

GIT submodules is an idea (of many) that actually played an extremely important role in me moving to GIT. As a new user I started playing around with GIT and I noticed that GIT commands executed on the parent module is not propagated to the child modules. In some use cases it would be extremely useful (at least for me) to be able to be propagate a command from the master module to all its child at all depth. I wrote this bash shell script to simply propagate commands from parent to its child. To use this script you can simply do the following (I am assuming that the TXT will have the name git-modules and will be an executable in $PATH):

for: git-pull
do: git-modules pull

for: git-status
do: git-modules status

for: git-commit -a -m "This is a test commit"
do: git-modules commit -a -m "This is a test commit"

for: git-checkout master
do: git-modules checkout master

Basically any git-X command can be simply be done as "git-modules X args-as-usual".

I would really appreciate/welcome criticism, feedback and addition to the script. I will be publishing it to the repo.or.cz tomorrow after a little bit more testing.

GIT: Starting using GIT as your SCM

2007-12-31T20:37:00.000-08:00

I am a user of SVN and was a fan of it as well. But the following video demonstrated some requirements of SCM which I liked a lot Especially as I want to work distributed, offline, modular and essentially fast GIT came to me as a gift and I want to share it with everyone. This Tech talk from Linus actually convinced me to take a deep look at GIT.

Then this video (which for some reason YouTube does not allow to embed) actually helped me understand more about how GIT works and some tips to work better with GIT.

This image could be helpful enough to have it beside your desk and it also has a larger resolution one if you want it. These images and other documentation are available at the GIT Wiki site. If you are looking for a extended cheat sheet you can find it here. If you are looking for the manual its here and you might also find the "Everyday GIT ..." page helpful. Some useful How-To s can be found here. If you are looking for how to setup a hook, have a look here; you can also simply find those scripts doing locate hooks in your linux box.

I have developed such likings towards GIT to the extend that I have started writing a Maven2 SCM Provider for GIT (using the Mercurial provider as example). Once I am done with it I will also be writing a NetBeans VCS Plugin again in light of the Mercurial plugin (if no VCS plugin for GIT is available in NetBeans prior to that).

Retreiving all classes in a package

2007-12-27T19:44:00.000-08:00

Straight to the point - I had to list all ReadOnly attributes for my domain objects as I had to list them in the office Wiki. I had several of them and doing that manually would be tiring (not to mention that I actually do not like documentation that much), I had to find a shortcut and only way is to use reflection; the next question was - how can I just simply mention the package and it will do the rest for me, so after Googling I could not find a solution to my liking so I thought of writing one myself and it took 10~15 minutes to come up with this piece of code.
I did the following with it and it worked nicely for me -

javac GenerateProperties.java
java -cp /opt/jdk1.6.0_01/jre/lib/rt.jar:./ GenerateProperties

If you are running your program from IDE, if you have several modules in your classpath (i.e. their target/classes/) and/or zip/jar(s) it will also search through them to list the methods. I am thinking of making a small framework, or API, out of it. I would like some enhancements on this code, please feel free to send them to me.

SMTP setting for GMail in Evolution

2007-12-27T19:05:00.000-08:00

This post of mine involves the least technical depth probably :), it is mainly about configuring SMTP for GMail with Evolution mail client (not that it could not be figured from the image). As GMail users know GMail does not use the default SMTP port (25), the custom port number has to be mentioned in the configuration and though there is no field labelled as Port, but users can easily set the port number as I did. Please see the red border region for it in the image - server-address:port. If required port number can also be provided in similar fashion for incoming mail server.

Modular project development - with Maven and GIT

2007-12-23T19:34:00.001-08:00

For sometime I have been searching for a combination for Multi-Module project development with SCM and Project Build tool. After using and testing some tools I have settled GIT and Maven; actually honestly speaking I have quite impressed by GIT. Though the main repository for my projects will be either CVS and SVN because dev.java.net provides those only but the developers will be requested/encouraged to use GIT (through online collab) as their local/team SCM/VCS. As result of my conviction I have decided to start a plugin for GIT-NetBeans integration as well; Almighty willing. At first I will use it for the DatabaseReport and JAIN SIP adapter projects. Everyone interested can join in any of the 3 projects (including the GIT-NetBeans Plugin).

NetBeans RCP development with Maven

2007-12-17T18:34:00.000-08:00

I have been wanting to work with NetBeans RCP for quite some time now, but I felt that resources were lacking; then came along API changes for NetBeans 6.0 Platform and I felt there is a lot of improvement in it, along with its platform home, the excellent RCP book and Maven integration for plugin/suite/Rich Client Application development (I personally tested it whether it works or not and it does smoothly except that when I was reading it Figure 4 and Figure 6 were in opposite places), now I reckon NetBeans RCP as a force. The biggest advantage of NetBeans RCP that actually initially dragged me to NetBeans was the JavaHelp integration.
I just wish NetBeans also support(s)/ed GIT. Actually I am planning to add support for it in NetBeans.

Killing a process in Linux

2007-12-12T18:39:00.000-08:00

As a software developer one thing I have several times is killing a process, some people will term is as ugly or bad, but general software developers often come across situation where killing a process is handy or necessary. I will mention some use cases later in the blog.

Killing a process usually involves 2 steps. Finding the process ID that is intended to be terminated and terminating it with a kill statement. Step one for general users (like myself) again involves a ps command with some grepping and then manual lookup. I have to these almost every time I kill a process. I have simply put this steps into this script so that I can do them in a single command. If someone has any improvements please inform me about it.

The shell script expects 3 arguments, of which 2 are mandatory and 1 is optional. The first 2 are grep params to identify the process. For example, to identify a Tomcat running Escenic Content Engine I use java and escenic.root to identify the process. The third param is the user to search it with. If it is not provided then whoami will be used to determine the user executing the shell.

This shell script has enabled me to sync, build, deploy my projects all with a single command and this helps me save some sustainable amount of time. Now without a build server I can still achieve what a build server like Hudson can do. Actually I use this script from within Hudson and thus I almost "never" have to be concerned about deploying applications to my demo server. I hope this is useful to all.

Guarding against softwares with memory leak

2007-12-12T01:26:00.000-08:00

I have been using IntelliJ IDEA as the IDE for my project development. It has been satisfying but sometimes it annoys me a lot. For example, when I am typing in the code editor and all of a sudden I cant hear the music playing (which I always play while coding), the IDE stopped responding and the mouse pointer hardly moves. The only way to start working normally again is to restart my laptop and on a laptop it is not a happy scenario to force reboot and moreover I do not like force reboot as a solution. So first I had to detect what caused the catastrophe and I find that the Java VM running IDEA keeps taking up memory linearly at 45 Degrees angle in the resource monitor. Furthermore this is not a infrequent event it usually occurs 2-3 times a day.

As I could not live with the reboot I had too make sure it did not occur and for that I thought of the a solution - I will have a cron job that will monitor the used memory by IDEA and will kill it if it crosses a certain amount. So I wrote a shell script and created a cron job following the the Ubuntu help. That solved my problem. The shell script can actually be used to kill any Java process that leaks memory and might disrupt the normal flow of PC usage experience.

If anyone has any alternate solution please feel free to share it.

WinRAR on Ubuntu

2007-11-12T03:57:00.000-08:00

As a novice Linux and Ubuntu Fiesty Fawn user I have faced some issues related to RAR Archives. WinRAR is the Windows tool widely used for this purpose and it is also a powerful archiver. If one has wine installed as mentioned in one of my earlier blogs they can install WinRAR in no time following this.

I am just mentioning the steps in short.

Download WinRAR 3.7
wine {PATH_TO_EXE}. E.g. wine wrar371.exe. Having Desktop emulation configured using Wine is necessary. winecfg -> Graphics -> Emulate.... checkbox
Run WinRAR with the following command: wine {PATH_TO_INSTALLATION}/WinRAR.exe

I hope all enjoys WinRAR on Ubuntu.

Developing projects that support multiple Databases

2007-11-05T01:10:00.000-08:00

It is a common scenario in product/project development that there is a requirement to support multiple databases. This actually leads to 3 challenges - generating DDLs for the DBMS, keeping the queries in Data Access Layer portable across the DBMS platforms and optimizing the read SQLs for the specific DBMS platform. This blog entry will cover only introduce the probable solution to the first of the three challenges.

DDL usually differs from one DBMS to the other. When it comes to generating DDLs there is a issue of creating or altering a table. i.e. when the DDLs are being executed for the first time it should create the database, whereas on consecutive execution it should only execute the alter table table columns to ensure that only the changes are taking effect. There is also the issue of being able to generate the DDLs when required - both the whole DB script and the alter table script. We would also like the script to be generated and executed only on specific cycles of build process.

After searching the internet the tool that caught my eye is the DDL Utils project of the Apache DB Project. This tool fulfils the requirements described above when used in conjunction with Maven. Have a look at the following sample project to see how it can be achieved.
Sample Project
After decompressing the sample project please go to the maven-plugin folder and execute the shell script. Windows users can simply use the same command in it :). As the example uses MySQL please check the connection parameters in the /src/main/resources/com/smartitengineering/.../schema/
db-connection-params.properties.

The project is build around maven and thus the DB operations are also attached with the Maven commands compile, install and deploy. Their outcomes are as follows:

mvn compile - Generates the DDL for the whole schema regardless of the current DB state
mvn install - Generates the DDL required to sync the current schema to the DB; that is generates the minimum possible alter table statements
mvn deploy - Execute the alter table ddls on the DB to sync with the latest state

The first 2 only generates the DDLs but does not execute them. You can find the DDLs in the target folder. Another thing to note is that the POM file takes the DB parameters as properties so they can lie anywhere. Currently I am reading it from a properties file in the classpath and the additional plugin is required for this purpose. Important links of the tool are -
DDL-Utils Ant Tasks
Schema Documentation
In order to get the deploy working you will need a valid repository to deploy. So that is the only thing that needs to be edited in the POM file.

All in all the tools is very useful handy when it comes to DDL generation and history maintenance. The only thing I found missing in the tool's release is support of ON UPDATE/DELETE CASCADE/RESTRICT/.. support. Though the schema supports it, the implementation does not support it; but considering the whole this is quite a useful tool. The good news is that the source code in the SVN has support for it during creation only and not in altering, so we can hope to see it soon as well

Getting Intel 82801G to work in Fiesty Fawn

2007-10-25T00:33:00.000-07:00

My new laptop is a Acer Aspire 5585WXMi. It ships with ALC883 Intel HDA chipset. I could get everything to work with Ubuntu Fiesty Fawn except for my sound card and it was so sad that I could basically work with everything excepting for sound.
After trying a lot of things all of a sudden it started to work. So I started looking into why and how. I noticed that I have ALSA 1.0.15rc1 installed from the source code; I did that basically as instruction provided from the Ubuntu help but the wiki alone did not solve the problem then. Later I have installed alsaplayer-alsa, alsaplayer-gtk, alsa-tools and alsa-tools-gui packages and ran alsaconf again. Used the ALSA mixer to turn the PCM channel on and set full volume and then all of a sudden hurrah! there is sound in my laptop. But I do not think that upgrading to ALSA 1.0.15 is required it could be achieved in 1.0.13 as well if the additional packages are installed but that is my hypothesis, I am not absolutely certain. I hope this works for others as well. If it does let me know.

Bridging SIP Servlets and JAIN SIP

2007-07-26T19:47:00.000-07:00

As discussed in one of my earlier blogs, there are 2 SIP stacks that can be used to build SIP applications - SIP Servlet API and JAIN SIP API. Both have their advantages and disadvantages. For example, JAIN SIP API provides more structured way of building SIP Applications and could be used for any of the SIP entities. But SIP Servlet is more suitable for server side SIP entities; saying this SIP Servlet has the advantage of leveraging enterprise services more directly as it is integrated with Java EE containers.
As I am planning to develop a VVoIP framework I would like to leverage strength of both. So I have decided to develop a bridge (or more appropriately adapter) between the APIs and that initiative have resulted JAIN SIP API Adapter project. The project's aim is to let JAIN SIP API based Server side applications benefit from JAVA EE Services by using SIP Servlet API. I hope to complete this project in 3 months to come and release a beta version. Initially the system will be tested on SailFin with a IM server I developed based on JAIN SIP API.

Video Recording using Flash

2007-07-05T02:53:00.000-07:00

Recording Video using Flash is a trend on the high. YouTube Quick Capture is one that has caught our eyes on this. So I was trying to get something done in the same way. I am basically Java developer and have experience of developing Video Recorder using Java Media Framework. Though I am a Java fanatic I feel that at present Flash is the way to Video on Internet. Its adaptation in video technology is very wide spread and well acclaimed. This blog is about is about video recording.

The first thing that is required for Flash video recording is a media server for streaming; because Flash unlike signed applet can store on local disk, it in fact streams the captured video to media server. There is Adobe Media Server or Flash Communication Server that we can use after procuring or there is Red5; thanks to the open source community. Now with Red5 comes a sample simple video recorder, using which you can simply stream video to the server. If we look at a customized Simple Recorder (it is only the recorder and there is no attached streaming server :) if anyone can help please let me know) we will find that for sole recording purpose there is a video container that shows the video. Initially it shows the currently being captured video and once the recording is stopped it shows the streamed video of the just recorded stream. This events are triggered by the buttons on SWF video. It uses the oflaDemo Red5 Application provided as demo with the server. All streamed videos are stored in {RED5_HOME}/webapps/oflaDemo/streams/.

Now lets look at the code used for this purpose. The first task is to open an stream when we intend to record something.

var nc:NetConnection = new NetConnection();

// connect to the local media server
// rtmp://{host}:{port}/{Red5_application_context}
// default port is 1935
nc.connect("rtmp://localhost/oflaDemo");

// create the netStream object and pass the netConnection object in the
// constructor
var ns:NetStream = new NetStream(nc);

The second thing that we can do is detect and select the capture devices.

// get references to the camera and mic
var cam:Camera = Camera.get();
var mic:Microphone = Microphone.get();

Now above we selected the default devices but we could Camera.names to get the names of the capture devices and select the one that we want; this is something that YouTube does :). Next we need to configure the video capture stream.

// setting dimensions and framerate
cam.setMode(320, 240, 30);
// set to minimum of 70% quality
cam.setQuality(0,70);
//Setting sampling rate to 8kHz
mic.setRate(8);

Next we need to add the device streams to the capture stream.

// This FLV is recorded to webapps/oflaDemo/streams/ directory
// attach cam and mic to the NetStream Object
ns.attachVideo(cam);
ns.attachAudio(mic);

After that we need to add the camera to the container and publish the video.

// attach the cam to the videoContainer on stage so we can see ourselves
videoContainer.attachVideo(cam);
// Publish the video and mention record
// lastVideoName is the name of the video and it will be saved as ]
// lastVideoName.flv
ns.publish(lastVideoName, "record");

Now once the recording is just clear the video container and the captured stream to the container and specify the stream to play the just recorded video

// attach the netStream object to the video object
videoContainer.attachVideo(ns);
// play back the recorded stream
ns.play(lastVideoName);

Hopefully with these steps a video recording and stream playback should be possible.

Stickies on Ubuntu

2007-07-03T20:21:00.000-07:00

It seems that Wine is an excellent tool to run Windows application on Ubuntu. Recently I had to use Stickies and tried it on Ubuntu. So first I had to install Ubuntu. I executed the following commands to install Wine on Ubuntu 7.04 Fiesty.

sudo wget http://wine.budgetdedicated.com/apt/sources.list.d/feisty.list -O /etc/apt/sources.list.d/winehq.list

sudo apt-get update

sudo apt-get install wine

For other platforms you can have a look at http://www.winehq.org/site/download. Then you have to simply download the stickies.exe and then execute

wine {PATH-TO-STICKIES}/stickies.exe

But hold on the installation is not done, you need get the following the installation steps mentioned in

http://appdb.winehq.org/appview.php?iVersionId=5169&iTestingId=4356

scroll down to Install Note and follow that. The only change will be installing the MSI file, instead of the one mentioned there use the following to install the MSI file

wine msiexec /i/{PATH-TO-MSI}MSISetup.MSI

Now try to execute

wine /.wine/{PATH-TO-STICKIES-INSTALLATION}/stickies.exe

It might say that MFC42.dll is missing, if it says so please get it from a Windows installation and copy it to /.wine/drive_c/windows/system32/ and try to execute it again.

Hopefully that will be enough for Stickies Installation :).
Now to get the Stickies running you can execute the following command:

sudo wine ~/.wine/drive_c/Program\ Files/stickies/stickies.exe > {PATH-TO-LOG}
2>&1 &

Though this is for Ubuntu but I guess excepting the installation procedure rest should be same across Linux platform. Let me know if any further changes were required.

SIP application structure

2007-06-10T19:58:00.000-07:00

As mentioned in my previous blog I feel that SIP applications are the next generation of applications for Desktop, Web and Mobile. By the will of Almighty, today I will be discussing the structure of SIP applications.

We will see the main layers of the application and discuss them in bottom up approach. Any SIP application will contain 3 layers. Topmost is the Application Layer - where all the logics of the application resides and which takes care of the SIP packets. Then comes SIP Stack it self, which is responsible for abstracting the SIP packets from the application developer by providing an API to do the low level SIP stuffs. Lastly comes the Transport layer that is responsible for carrying the SIP packets and sending them to the intended destination; the SIP stack is very closely binded with this layer as it will provide transport for common protocols and developers has to extends the transport API to add more transport layer protocols.

Almost any transport layer protocol can be used with SIP as long as the SIP stack recognizes it. The most commonly used transport layer protocol is UDP; on some occasions TCP and TLS are also used. We all know that all networks are converging towards IP backbone and as those protocols are IP based it should be more convenient for developers in future as they will not have to write SIP transport layers for different protocols. One challenge would be from Cell Phone to BTS; so SIP stack providers for Cell phone will have to implement transport layer for GSM and CDMA phones.

IMHO the most important layer of the application structure is SIP stack. This will abstract the SIP packets and Transport layer hassles from the developers and will provide them API's to call and use them to make an SIP application. There are quite a few SIP stacks available such as reSIProcate, Jain SIP (JSR-32) and SIP Servlet (JSR-000116). Among them SIP Servlet Application (archives with sar extensions) is usually deployed in Java EE Container (BEA Weblogic, IBM Websphere, GlassFish-SailFin) or SIP Container implementing SIP Servlet. Personally I have used Jain SIP and its simplicity in usage and good API structure has created liking in me for it. In my blogs related to SIP I will be using this stack to develop examples. I will also discuss about SailFin.

Application layer is where everybody gets to exercise their innovations. Now is the time to generates SIP centric ideas and implementing them. All SIP application can be categorized in four categories -

Registrar Server - A server responsible to register a SIP user and its location
Proxy Server - A server responsible to act as the main server where as it is nothing but a proxy and knows which is the real server
Redirect Server - A server that redirects request to another server; an example of this functionality would be load balancing of requests
User Agent - This is a client side entity responsible for communicating with the server and server the Client's request. Anything could be user agent such as PC Software, Cell Phone, Refrigerator, Television, Microwave Owen etc.

Developers can develop applications that serve any of these purposes.

By the will of Almighty, I will soon make some postings about what could be SIP applications.

SIP - For next generation communication

2007-06-10T00:03:00.000-07:00

I have been introduced to SIP (Session Initiation Protocol) from my college courses. From then it appealed a lot to me. As timed passed by I came to realize that with the rate in which everything is getting globalized it is a matter of time when VoIP will rule the world of communication. As internet is already accepted as the global information highway, communication is also bound to be built on that. SIP is IETF engineered protocol and open for everybody's suggestion.

Recently SIP related activities are very visible; especially, with reSIProcate, Jain-SIP protocol stacks popularity and major Java EE Container vendors supporting SIP Servlets it is easily comprehensable that industry is expecting a increase in use of SIP in business applications, as more and more applications want to include web based voice/video chat.

Recently, GlassFish project has introduced the SailFin project, which is a SIP Servlet container. Hopefully, by the will of Almighty, it will also include support for easy portability for Jain-SIP based server applications into its container as well.

If I were to bet on a technology of next generation I would place my bet on SIP based applications. Hopefully, by the will of Almighty, I hope to bring more writeups about SIP applications - mostly server side.