Project

General

Profile

Where is your code executed?

An in-depth look at the Servoy architecture

By Patrick Talbot - November 2011

More and more on the Servoy forum, I see questions that clearly indicates how people need a better understanding of the Servoy architecture and especially of the relations between the various components involved … Whether this has to do with the increasing usage of client-side JavaScript (with the injection of jQuery scripting in their pages), streaming to and from the server using the file plugin, using web services, client and server plugins, batch processors, headless clients, more often than not, these questions originate from some level of confusion about the exact context of the executed code.

This is indeed one of the most misunderstood parts of Servoy, where newbies haven’t got a clue, and even most long time users would be hard pressed to give a clear answer about it. Still, it is essential to know where and how your code is executed, and even if this can be confusing at times, it should help you becoming a better Servoy developer.

As simple as it seems at first glance, Servoy is not one monolithic environment: there are numerous threads running, in Developer, on the server-side, the client-side, in your web browser, there are numerous clients and ways your code can be run and called...
Servoy is made to abstract you of these concepts, which is a good thing, but knowing a little bit of what’s under the hood can help sometimes, especially when something is going wrong….
For example, knowing the difference between the JavaScript executed in a browser within a web client and the Servoy JavaScript code executed on the server might be a difficult thing to grasp at first, but once you get it, it all starts to make sense...

The trick is also to understand that each of these environments is indeed very different and that your code is executed by different classes of the Servoy code, within very different contexts.

Also understanding about the Servoy architecture and the underlying technologies will help you grasp what is where, when and how your code is executed.
Once you get to know all of this, you will know what RMI/Tunnel is about, what the purpose of JDBC, what's about headless clients, client and server plugins, batch processors, web services, servlets and JSP, Swing and Wicket, because all of these things are what makes the Servoy environment.

Oh! and another thing: you might never notice it, because most of the time all you do is write your code in JavaScript, but in the end all the code executed in Servoy is actually Java code :)
 
First, let’s have an overview of the different components involved:

Servoy client/server simplified communication view

I called this a ‘simplified communication view’ because not all paths from clients to server and back have been represented here; I focused on the main channels between the ‘client-side’ and the ‘server-side’ and listed the possible ‘actors’ you might encounter when you work and use Servoy.

The real detailed view would have more connections, and most of them would point to and from the main actor here: the Servoy Application server.

The basis of this architecture is often referred as a “three-tier” architecture, the three tiers being the database server, the application server and the client.

Now let’s look at each actor more closely and see what it is made of and its main function.

The Servoy Application server

One first thing you need to notice about the Servoy Application server is that it is the only part of Servoy with direct communication channels to the database(s). No clients or components share this privilege.

The server alone is holding the connections (actually a pool1 of JDBC2 connections). This is a good thing in terms of security (no need to open your database server port to the outside world) and this is also what allows Servoy to be database-agnostic3.

Another thing you might want to know about the Servoy Application server is that it is, in essence, a set of Servlets and Java components working together and running within the Apache Tomcat4 Servlet container. Just note also that Tomcat IS the main context of all the server-side code.

The Servoy Application server’s main function is to serve clients and give them access to the database. It also serves http requests to static resources located in /server/webapps/ROOT/ (just like any Tomcat application), but also any JSP files you might have at this location (handled by the Jasper JSP5 Engine of Tomcat), The Servoy Application also installs a set of Servlets dedicated to various serving tasks.

A good way of discovering them is to have a look at the /server/webapps/ROOT/WEB-INF/web.xml file. You will see that in there a set of servlets is configured for various request paths:
  1. /tunnel this is the tunnel, which allows Smart client to communicate using RMI6 over plain HTTP (on a port 80 for example) allowing method invocation to happen even through firewalls.
  2. /servoy-client this is the servlet which will serve the JNLP files used to install the client’s jar and launch the client using Java Web Start7 - once launched, the client will communicate directly with the server using RMI on the RMI port or via the tunnel on the http port.
  3. /servoy.properties as its name implies, this serves a bunch of properties to the Smart client once it is launched, if you navigate to this URL, you will see a list of plugins, the available LAF classes, and information about the port and method used to connect to the server.
  4. /servoy-admin this is the servlet that serves all the pages of the servoy admin, which is mainly a web UI to manage the properties of your servoy.properties file.
  5. /servoy-rmi-portserver this one is returning the RMI port to use (incremented for each new client).
  6. /servoy-service is a generic servlet for plugins to expose services. For example the RESTful plugin uses this servlet to expose RESTful web-services, that support both JSON and XML format.
  7. /servoy_messages this one is serving properties files (your i18n messages) to the Smart clients, thes files are generated on the fly from the i18n database on the server, but cached on the client side in your USER_HOME/.servoy folder.
  8. /servoy-webclient you know about this one of course, it is the main point of call for all the Wicket generated pages.
  9. /servoy-webclient/templates this is a WebDav Servlet allowing read/write access to the Servoy html templates of your application server.

The servoy-webclient servlet is the main entry point for the Wicket application framework, which in itself is capable of handling many clients, and managing their sessions, using the build in Tomcat session mechanism.
Apart from the database connection pool, the Web client Wicket framework and these various servlets, the Servoy Application server is also the place where various ‘headless’ clients are handled. They will each run in their own separate thread, but these threads will all be part of the same JVM8 instance (and thus will share the same available memory the server has been allocated at startup).

You know these headless clients by the names of:
  • Headless client (of course!), this is a client instantiated by a batch processor, or the headless_client plugin, or by a JSP or any other Servlet and even by the web service plugin.
  • Batch processor, this is a client which you can instantiate from the Servoy server ‘Batch processors’ admin page, it is a long running thread running on the server (as a daemon thread) as long as the server is running (or until you stop or disable it)
  • Authenticator module: this one is a short lived client used to authenticate a user as part of the ‘enhanced security’ scheme introduced with Servoy 5.2
    No more true! The Authenticator is no longer using a client to work.

Finally the Server Application server is holding instances of server plugins. They can also add their own new RMI services and Servlets (or any other kind of services actually).

The main context of all these clients, Servlets and various processes, is the Servoy Application server context, which in turn is also the Tomcat Application server context.

So now, you should be able understand that each time you are running a headless client, a server plugin, a batch processor, a servlet, a JSP page, a web client, a web service, your code is run on the server side in the same JVM as the Servoy Application server.

The other place where your code can run is of course the Smart client.

The Servoy Smart client

You launch a Smart client from a JNLP9 file retrieved from a URL, served by the Servoy Application server. This JNLP file contains a reference to all the jars the client needs to download and to the main class of the Smart client, the com.servoy.j2db.smart.J2DBClient class, which is instantiated by Java Web Start (javaws executable in short).

So Java Web Start is the context of the Smart client. It runs under a specific security manager, but with the security constraints introduced by Oracle with Java 6 update 19, the Servoy Smart client and all the related client plugins and beans are now forced to run with an All-Permission level, meaning they can access the local resources, once the user has granted access of course.

The Servoy Smart client is a classical Java Swing10 application, but you might wonder how, when all you do is writing JavaScript, you actually execute Java code?

That’s one nice trick which is performed by Servoy thanks to the Java Rhino11 library, an Open Source implementation of the JavaScript language in Java. What this means is that each of your Servoy scripts is running inside the context of the Rhino engine (it is compiled at runtime into Java code).

The Smart client is communicating with the server for every access to the database (retrieving foundsets and datasets), but when it comes to local resources, it is important to note that these are located on the client itself. So when you access the file system, for example using the file plugin, the files you have access to are those on the client-side, a call to plugins.file.getDesktopFolder() will always give you the path to the client local Desktop.

Of course there are also special cases where you can run code on the server-side from a Smart client, for example when you use the headless_client plugin (the client is run on the server-side), or when you stream files to and from the server using the file plugin, or when you index or query with the SmartDoc plugin, or when you use the file browser of the HtmlEditor of the BrowserSuite, this is true because each of these plugins have a server-side plugin which is accessed from the client-side plugin, the client-side plugin communicates using RMI with its server-side counterpart.

But typically, just remember that in Smart client, your code is usually executed on the client-side, within a Swing application inside the Java Web Start context.

The Servoy Web client

The Servoy Web client is another case because we can consider that it has 2 sides of code execution.

The Servoy JavaScript code you use in your globals/forms methods and calculations is executed on the server, within the context of the Wicket application Servlet, so inside the same JVM as the server.

But a Web client also runs client-side JavaScript in your browser. This is the classic JavaScript that every web application is using, manipulating the DOM12 in the browser, with or without the help of JavaScript frameworks like jQuery, yahoo-UI, or any others.

The Servoy web client is using Ajax polling in the background to update the forms (it is important to note that it is not the server that is ‘pushing’ things to the client, it only appears so because the web page is calling the server every x seconds to retrieve possible changes (the exact amount of time for this can configured in the servoy-admin with the servoy.webclient.datanotify.frequency property). This polling is done using Ajax Get or Post requests, and it allows the client to always display the latest version of the data and of the forms components.

Each time you click on a button or trigger an event which is linked to a Servoy method, Servoy is calling the server to call that method on the server using the Wicket13 framework, within the related session of your client and waiting for the response from the server: this is the little red ‘loading…’ square box that you will often see in Web client when the page triggers a callback to the server. It all happens without the page needing to be completely refreshed, but still it is less responsive that a Smart client, where the code is executed on the client itself, because a round-trip to the server is needed to trigger the method and get the result of that method back, and because the Smart client is holding the connection opened, something that the http protocol doesn’t allow.

The result returned by the server is usually some JavaScript code or JSON/JSONP14 which will in turn refresh some portions of the page.

You can also inject browser related JavaScripts from a Servoy method or from a field content, for example using a field of type HTML_AREA where put some script tags, or using the WebClientUtils plugin to inject JavaScript in the page (and optionally get a result back to the Servoy method). These are special cases, where the injection is triggered from a server-side Servoy script, injected in the page to be executed on the client side in the browser, then sent back again (when asked to) to the initial Servoy method for further processing…

But for the most part, just remember that your Servoy code is executed server-side and your browser JavaScript code is executed in the browser on the client-side.

Servoy Headless clients

We’ve seen that numerous clients instantiated by various means can all be called ‘headless’ clients. Understand that headless in that case just mean ‘a client without UI’ (headless is a special flag you can pass to the JVM: java.awt.headless – typically servers with no display will use this flag to tell the JVM to avoid initializing AWT and Swing components) – if you have a look at the application_server/servoy_server.bat or servoy_server.sh, you will see that the server is indeed launched with this flag:

java -Djava.awt.headless=true …

Meaning the whole Servoy Application server (and everything that runs within it) is run in a headless JVM, it is not meant to have a Java (Swing/AWT/SWT) UI.

This is the main reason why you cannot access form’s elements within a headless client: they haven’t been properly initialized, they have no graphic context (in Java this is the Graphics – or Graphics2D - class) associated with them: they are not displayed anywhere, it really makes no sense to call a UI property when there’s no UI.

So, what headless clients are there? How do you instantiate (create) one?
  • Batch processor: it is a headless client launched from the servoy-admin web page and kept alive as long as the server is up (or until you stop or disable it). You usually combine it with some cron enabled calls to Servoy methods (using the scheduler15 plugin)
  • Headless client: you can create a headless client using the headless_client plugin (either called from the client-side or, using a web client or batch processor for example, from the server-side: in all cases, the headless client’s code is executed on the server-side)
  • Web services: the rest_ws plugin is instantiating a headless client to call the methods on Servoy forms
  • JSP or Servlets: you can instantiate a headless client, using the com.servoy.j2db.server.headlessclient.HeadlessFactory class (see the Servoy public API16) inside the Java code of your JSP pages or Servlets

Note that each of these headless clients is a client in itself and as such, consumes a Servoy client license. Which is why for example the Authenticator is using a license for the time it is run (usually a very short time during the authentication process).
The Authenticator module doesn't consume a license anymore.

And of course remember that all these clients are run on the server-side within the context of the Servoy Application server.

Servoy Developer

There is one context which supersedes all of the others and which is a special case in itself; where the distinction between client-side and server-side is irrelevant: Servoy developer takes this distinction away and merges client and server inside one big design/debug context: the Eclipse17 environment.

Servoy is an Eclipse feature, packaged into a distribution, but it is still Eclipse, and as such it has all features of the Eclipse environment.

You can use Eclipse in lots of ways, one of them being the design of Servoy applications.
Just note that you can also use Eclipse to build Java applications, or even PHP, JavaScript, Ruby, Python, C++, you name it!
You can use the Eclipse IDE to design databases, to create UML diagrams, to design UI, work with XML and schemas, etc. There are plugins around for all sorts of development oriented tasks.

So, Servoy "Developer" being really Eclipse, you are actually using specialized Eclipse plugins (build into what's called a "feature" = a group of plugins working together) to design your forms, relations, value-lists and so on, and write the code that goes with it.

This is the famous design-time mode that you will often hear Servoy creators talk about on the forum, as opposed to the runtime mode which is when you launch a client and run your code.

At design-time, you are using the Eclipse tools available, plus the one tailored for the Servoy environment: you create forms and elements and all the pieces that will constitute your application. This is also when you write your Servoy JavaScript code, and of course it is not yet executed, although it is checked for errors and parsed for annotations (the JSDocs). This is the time when obvious errors in your code will be found (think typo) by the JavaScript parser (the Servoy-customized DTLK18 editor you might have heard about).

At runtime, you launch a client (either a Smart client or a Web client), within the context of the Eclipse debugger. All the clients in debug mode are communicating with the embedded Tomcat server hosting the Servoy Application server (also launched under the surveillance of the Eclipse debugger).

The Eclipse debugger is capable of watching external processes, but in the case of Servoy, the Servoy Application server is launched inside the same JVM (when Servoy Developer is started, it also launches the Tomcat application server), and the Smart client as well (also that the class of the Smart client run in developer is not the same as the one used in a real Smart client, in this case it is com.servoy.j2db.debug.DebugJ2DBClient, extending the usual com.servoy.j2db.smart.J2DBClient).

So in Developer, wherever the code is executed, it ends up being in the context of the Eclipse environment and watched by the Eclipse debugger.

Comments

Add your comments here

Footnotes

1 Like many other Java applications, Servoy is using the Apache commons DBCP project to manage a pool of connections to the databases, you can learn about this project here: http://commons.apache.org/dbcp/

2 JDBC stands for Java DataBase Connectivity; it is the common SQL Database driver standard interface for all Java applications. There is one (and sometimes more than one) JDBC driver for practically every Database that matters. Usually it is a simple jar that you put in your ‘drivers’ folder inside the Servoy ‘application_server’ folder.See http://en.wikipedia.org/wiki/Java_Database_Connectivity for more information on JDBC

3 What allows Servoy to be database-agnostic is actually its reliance on the Hibernate library to communicate with any databases. Hibernate is an ORM (Object Relational Mapping) framework, one of the most widely used in the Java world (and also in .Net actually). One of the neat features of Hibernate is that it uses different ‘dialects’ to talk to different databases, and has abstracted all the difference s between all the various SQL database vendor inside these dialects. So when you change your database source, all you need to do is to use a different dialect, which is what Servoy is doing under the hood depending on the database server you have selected. If you are interested in Hibernate, have a look at http://www.hibernate.org/

4 Apache Tomcat Application server (not to be confused with the Apache Httpd web server) is a Servlet container (it is even the reference Servlet container, upon which many Java Application server are based on). Servlet is a Java technology which was developed primarily to serve dynamic web pages, but they can do more... If you want to know more about Tomcat, have a look at: http://tomcat.apache.org/

5 JSP stands for Java Server Pages, it is the equivalent of ASP and PHP pages, a JSP page is actually compiled into a Servlet before it can be executed, but this is done just in time by the Tomcat Jasper JSP engine (not to be confused with JasperReports), to know more about JSP, have a look at http://www.oracle.com/technetwork/java/javaee/jsp/index.html to know more about Tomcat Jasper, see http://tomcat.apache.org/tomcat-6.0-doc/jasper-howto.html

6 RMI stands for Remote Method Invocation, it is a Java protocol to invoke distant methods on a server, it is relatively simple compared to the CORBA protocol for example. To know more about RMI, have a look at http://www.oracle.com/technetwork/java/javase/tech/index-jsp-136424.html

7 Java Web Start is a method of deployment for applications (and applets), it is part of the standard JRE. The Smart client context is Java Web Start. To know more about it, have a look at http://www.oracle.com/technetwork/java/javase/tech/index-jsp-136112.html

8 JVM stands for Java Virtual Machine, often referred as the JRE, the Java Runtime Environment, this is the special virtual machine capable of running Java byte code, as opposed to native code. Have a look at http://en.wikipedia.org/wiki/Java_Virtual_Machine to know more of the underlying technology that makes the JVM

9 JNLP stands for Java Network Launching Protocol, a JNLP file is the descriptor of an application to be launched by Java Web start, you can have a look at the JNLP syntax here: http://download.oracle.com/javase/1,5.0/docs/guide/javaws/developersguide/syntax.html

10 Swing is a Java UI widget toolkit, it provides a set of components, like JFrame, JPanel, JLabel, JButton etc. which are the base elements the Servoy elements you put on your forms are derived from. Swing is part of the Java Foundation Classes, have a look here to know more: http://java.sun.com/javase/technologies/desktop/

11 Rhino is a Mozilla Open Source project (Mozilla is the team that has produced the Firefox browser), see http://www.mozilla.org/rhino/ for more information about Rhino.

12 DOM stands for Document Object Model, which is the standard JavaScript model to access HTML elements, have a look at http://en.wikipedia.org/wiki/Document_Object_Model for more info about it

13 Wicket is an Apache Open Source Java Web framework, it is component oriented (compared to MVC oriented frameworks like Struts) and has built-in support for Ajax requests and responses. See http://wicket.apache.org/ for more information on the Wicket framework.

14 JSON stands for JavaScript Object Notation, a light-weight interchange format widely used to serialize JavaScript objects. The P in JSONP stands for ‘with Padding’ (a simple way to encapsulate a JavaScript Object inside a function). See http://en.wikipedia.org/wiki/JSON

15 The scheduler plugin is using the Quartz Open Source library, an enterprise level job scheduler, see http://www.quartz-scheduler.org/ for more information about it.

16 The Servoy Public API javadocs can be found at http://www.servoy.com/docs/ (for various versions).

17 The Eclipse platform is the platform that Servoy Developer extends. One can consider Servoy Developer as a special ‘distribution’ of the Eclipse platform, a specialized package (a feature) of dedicated plugin made to help you design your forms in a way which is suitable and compatible with the Servoy Application server. The Eclipse platform is a vast eco-system of various projects made into a platform: the Eclipse IDE. one of the most widely used IDE (Integrated Development Environment). To know more about Eclipse, have a look at http://www.eclipse.org/ Note also that Eclipse is using a special Java UI widget toolkit: SWT (Standard Widget Toolkit), which is not quite the same as AWT/Swing, to know more about SWT, have a look at http://www.eclipse.org/swt/

18 DLTK stands for Dynamic Languages ToolKit, it is an Eclipse project as well, and you can learn about it at http://www.eclipse.org/dltk/ Servoy ships a customized version of the DLTK project tailored for Servoy JavaScript.

Images