Technology For Human

JSF 1.2 (myFaces) + RichFaces 3.3.x + IE 9 = troubles

2011-12-08T17:53:00.001+01:00

Requirements:

working JSF 1.2 sample application from here
latest myFaces for JSF 1.2 - version 1.2.11, latest RichFaces for JSF 1.2 - version 3.3.3 final

You will learn:

how to fix problems with RichFaces 3.3.x for IE 9

Let's assume that You develop JSF 1.2 web application. It does not matter if Your application uses .jsp pages or Facelets with .xhtml pages. I assume that .jsp pages are in use. Then You decided to add some Ajax capabilities, new components and so on. So You simply take RichFaces and add them to the project. This is simple thing:
a) download latest RichFaces 3.3.3 (final), take those jars: richfaces-api-3.3.3.Final.jar, richfaces-impl-3.3.3.Final.jar, richfaces-ui-3.3.3.Final.jar and place them in WEB-INF\lib folder of Your web application.
b) modify your faces-config.xml file by adding there:

 <filter>
  <display-name>RichFaces Filter</display-name>
  <filter-name>richfaces</filter-name>
  <filter-class>org.ajax4jsf.Filter</filter-class>
 </filter>

 <filter-mapping>
  <filter-name>richfaces</filter-name>
  <servlet-name>Faces Servlet</servlet-name>
  <dispatcher>REQUEST</dispatcher>
  <dispatcher>FORWARD</dispatcher>
  <dispatcher>INCLUDE</dispatcher>
 </filter-mapping>

c) add proper taglib directives to each Your jsp page (note that for .xhtml pages syntax is different):

<%@ taglib uri="http://richfaces.org/a4j" prefix="a4j"%>
<%@ taglib uri="http://richfaces.org/rich" prefix="rich"%>

Now You can use RichFaces components. Let's try to modify our sample JSF 1.2 application by creating new web project named "JSFRichFaces". Then let's add ajax button (<a4j:commanButton>) and expandable toggle panel (<rich:simpleTogglePanel>). All modifications are shown below:

Everything works OK under Firefox, Chrome, IE 7, IE 8 - unless You use Internet Explorer 9. You can observe two problems: with RichFaces ajax components and RichFaces components layout.
RichFaces 3.3.x does not fully support IE 9. RichFaces developers encourage everyone to migrate into RichFaces 4.x which are IE 9 compatible. The main problem is that RichFaces 4.x works only with JSF 2.x application, and this is not so easy to migrate If You have many clients where You already delivered Your application...

Problem 1: ajax components do not work (some JavaScript errors).

When You try to press ajax button named "Go (RichFaces) " under IE 9, You can observe following error (open IE Developer's Toolbar Java Script console before):

Solution: You have to force IE 9 to act as IE 8, where RichFaces 3.3.x works OK. In order to do it You have to define X-UA-Compatible header for each page. You can do it in two ways:
a) add a special <meta> element for each page or
b) write a special filter which adds this header to the HTTP response

In our simple case it is sufficient to add a <meta> into the page <head> section:

<meta http-equiv="X-UA-Compatible" content="IE=EmulateIE8" />

Remember that this <meta> must be placed as first element in <head> section!

Problem 2: RichFaces components have broken layout (problems with CSS).

When You open our sample application under IE 9, toggle panel has no CSS styles set at all. Under Chrome or Firefox everything looks OK. Checking HTTP request/response headers shows that for IE 9 proper CSS for RichFaces component (CSS located in the .jar file) is not loaded and server returns error:

Console under Eclipse shows that exception is thrown:

Under Firefox everything looks OK:

Why it happens? Accroding to that article, there was a change in IE 9 for MIME type handling. IE 9 ignores CSS styles if they are not delivered with a text/css MIME type.
Why it works for previous IE 7 and IE 8? Because also HTTP Request Accept header sent by IE 9 is different than sent from IE 7 or 8. IE 7/8 sent text/css, */* as Accept, where IE 9 sends only text/css as Accept.
Knowing that it is time to look inside myFaces source, to find the code which produces mentioned exception. Everything is located in HtmlRendererUtils.java (located in myfaces-impl-1.2.11.jar) in the selectContentType() method. It works for IE 7/8 because */* sent in Accept by IE 7/8 is on supported content type list, where text/css is not recognized at all.

Solution: add text/css support, then recompile HtmlRendererUtils.java and replace it in myfaces-impl-1.2.11.jar (or build complete myfaces-impl-1.2.11.jar). In order to avoid this mumbo-jumbo with Maven to build whole .jar, just create simple Java Project, create a HtmlRendererUtils.java in certain package, change it and compile, then replace compiled class in myfaces-impl-1.2.11.jar ;-)

Or download recompiled myfaces-impl-1.2.11.jar from here.

That's all. IE 9 works again without need to add Your web page to its compatibility view list or something. And You have time to migrate to JSF 2.x and RichFaces 4.x before IE 10 comes to market...

Tomcat - using Realm to protect access to JSF application (part 2 of 2)

2011-09-21T21:47:00.001+02:00

Requirements:

understanding basics of Tomcat's Realm

You will learn:

how to define users and roles in own web applications (JSF 2.0 as example)
how to use encrypted passwords for realm users instead od plain text

In the previous post I described how to configure access to Tomcat's server administration application using Tomcat's Realm elements. The whole thing boiled down to write a few lines of text in server.xml file. But how does it look like when we have to do it from scratch in our application?

Step 1. Let's create very simple JSF application which has only 3 pages:

/index.html - main page with two buttons navigating into two pages:
/unrestrictedPage.xhtml - page always accessible and visible to anyone
/restricted/restrictedPage.xhtml - page and directory available only for valid users

The complete application structure will look like:

Step 2: we have to add a privilleged user named admin with password adminpass who belongs to role privillegedUsers (You can choose any user login, password and role name). Just simply add this line:

<user username="admin" password="adminpass" roles="privillegedUsers" />

into [tomcat directory]\conf\tomcat-users.xml file.

Step 3: configuring application. We have to modify web.xml of our sample application by adding following text:

<web-app>
...
    <security-role>
        <description>
              All persons belong to that role have access to restricted application area.
        </description>
        <role-name>privillegedUsers</role-name>
    </security-role>

    <security-constraint>
        <web-resource-collection>
            <web-resource-name>Restricted</web-resource-name>
            <url-pattern>/faces/restricted/*</url-pattern>
            <url-pattern>/restricted/*</url-pattern>
        </web-resource-collection>
        <auth-constraint>
            <role-name>privillegedUsers</role-name>
        </auth-constraint>
    </security-constraint>

    <login-config>
        <auth-method>BASIC</auth-method>
        <realm-name>Restricted access</realm-name>
    </login-config>
...
</web-app>

First we defined a role named privillegedUsers - the same we put in tomcat-users.xml file above. Second we have to defined prottected parts of our application, by providing proper URL pattern. Please note that we have two entries here: /restricted/* - everything in this directory is protected in case someone typed this part manually in browser's address bar, /faces/restricted/* - JSF adds "faces" prefix when navigating between pages, so the URL is a little bit different and we have to watch for this also. After that we have to define a role which is allowed to access to the protected parts. This is privillegedUsers role defined earlier. At the end we have to define the way how the authentication is done - the simplest way is BASIC, which displays predefined form with credentials.

Now we are ready to deploy our application on Tomcat and run it. It should look like this:

when we try to access to restricted area. In order to do this we have to put credentials for user defined in step 2 (admin, adminpass).

Note: there is a little trick here, for the button "Restricted area" - see index.xhtml page source code:

<h:commandButton value="Resricted area" action="/restricted/restrictedPage.xhtml?faces-redirect=true" />
<h:commandButton value="Unrestricted area" action="/unrestrictedPage.xhtml" />

Because JSF internally by default makes forward to other page, browser is unaware what has happened and display URL from one step back. In such case our URL security patterns will not match anything, and restrictedPage.xhtml will be displayed without asking for login and password! In order to make protection work we have to perform full redirection for that action in order to force browser to fetch target URL and display it. For JSF applications it is better to use filters or Spring Security in order to avoid such dirty tricks.

Encrypted passwords.

As You probably have seen, passwords vor valid users are stored inside tomcat-users.xml file as a plain text. There is a possibility to store them in encrypted form, using MD5. Here is what needs to be done:

Step 1: using Eclipse modify Tomcat server.xml and its <Realm> atrribute into:

<Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase" digest="MD5"/>

Step 2: in web.xml change <auth-method> from BASIC to DIGEST

Step 3: for user admin and his previous password adminpass we have to generate its md5 equivalent using Tomcat's digest.bat file (located in [tomcat directory]\bin folder) :

digest.bat -a md5 admin:"Restricted access":adminpass

"Restricted access" is a realm name taken from <auth-method> tag from web.xml. Those names must match, if name contains spaces, it must be surrounded with "".

Step 4: using Eclipse modify Tomcat's tomcat-users.xml file:

<user username="admin" password="db7bc05adcf611fc779f32a4e680cc01" roles="privillegedUsers" />

where password was taken as a result of executing command from step 3.

-------------------------------------------

Download source files:

Note: make sure that Java, Eclipse and Tomcat are properly installed and configured for running the project (additional configuration may be required if different directories are used).

Eclipse complete sample project is here (with all required libraries).

Tomcat - change default application, protecting access to web applications using Realm (part 1 of 2)

2011-09-13T22:01:00.000+02:00

Requirements:

installed Java (description here)
installed and configured Eclipse (description here)
installed and confiured Tomcat for the Eclipse (description here)

You will learn:

how to change default Tomcat application
how to allow users access to Tomcat's server administration application using Tomcat's Realm

Let's assume that You have Tomcat which is configured to work in Eclipse. When You start Tomcat (using Eclipse or standard scripts from Tomcat's distribution) and type in the browser URL http://localhost:8080, You will see Tomcat's default application:

This application is located in the [tomcat directory]\webapps\ROOT folder. If You want to change this page, just edit content of file index.jsp and index.html. If You want to completely remove this application, just remove complete ROOT folder.

Removing is not the best choice, because Tomcat is shipped with a special application for server administration. You can access this application using links (in red square) from left menu of a default page: Status and Tomcat Manager. Try to click on that links - You should see screen with login and password prompt:

Those parts are protected using Tomcat Realm. What is Realm? Realm is a set of valid users (defined by user name and password) and roles where those users belong. The idea is to configure access to web application only for valid users from certain roles. But where are those users and roles stored? It depends on a implementation of a realm - they can be stored in database, in LDAP, or in xml file. Tomcat's server administration application uses xml file to define users' access. Let's try to set up some users able to start that application.

Step 1: using Eclipse open server.xml from the Servers, and make sure that entry <Realm classname="org.apache.catalina.realm.UserDatabaseRealm" resourcename="UserDatabase"></Realm> exsits between <Engine> tags, outside <Host&gt tags - it means that this realm will be used for all hosts and all applications on that hosts:

Step 2: using Eclipse open tomcat-users.xml from the Servers and add an entry for the user who will be able to access Tomcat's server administration application:

By default UserDatabaseRealm uses [tomcat directory]\conf\tomcat-users.xml file to load users and their roles into memory on server startup. In order to allow defined users to access administration application, they need to belong to roles named manager-status and manager-gui. After changes made, restart Tomcat server and try to access Tomcat's administration application giving username and password from tomcat-users.xml file. If everything was set up OK, You should see manager application:

You may wonder why we used here roles named manager-status and manager-gui. Those role names come from Tomcat's server administration application specific settings, stored in the web.xml file. In the next post I will show how to protect own application (JSF2 application will be used as an example) and how to define own roles.

Sending emails using Java Mail and Google Mail account

2011-08-04T22:15:00.001+02:00

Requirements:

installed Java (description here)
installed and configured Eclipse (description here)
valid Google Mail account

You will learn:

how to send e-mails from Java application using Java Mail and Google Mail account.

Sometimes You need to send an email from Your Java application. In order to do this You have to use valid e-mail account, and You have to know the address of SMTP server for this account. And of course You have to write some code which will connect to the SMTP server (using Your account credentials) and then create and send email. A Google Mail account will be used as an example.

Step 1: create clean Java Eclipse Project

Step 2: download JavaMail API 1.4.4 (javamail_1_4_4.zip) from here.

After downloading JavaMail, extract it and go into its /lib folder. Copy all .jar files from that directory into Your project and add them to the build path.

Step 3: write a simple code for sending email using Google Mail. You can use TLS or SSL connection to Google SMTP server.

a) using TLS connection to SMTP server:

// ...
import java.util.Properties;
import javax.mail.Message;
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
// ...
Properties propsTLS = new Properties();
propsTLS.put("mail.transport.protocol", "smtp");
propsTLS.put("mail.smtp.host", "smtp.gmail.com");
propsTLS.put("mail.smtp.auth", "true");
propsTLS.put("mail.smtp.starttls.enable", "true"); // GMail requires STARTTLS

Session sessionTLS = Session.getInstance(propsTLS);
sessionTLS.setDebug(true);

Message messageTLS = new MimeMessage(sessionTLS);
messageTLS.setFrom(new InternetAddress("random_sender_4568744122@gmail.com", "John Smith"));
messageTLS.setRecipients(Message.RecipientType.TO, InternetAddress.parse("random_recipient_4568744122@gmail.com")); // real recipient
messageTLS.setSubject("Test mail using TLS");
messageTLS.setText("This is test email sent to Your account using TLS.");

Transport transportTLS = sessionTLS.getTransport();
transportTLS.connect("smtp.gmail.com", 587, "random_sender_4568744122@gmail.com", "sender_account_pass"); // account used
transportTLS.sendMessage(messageTLS, messageTLS.getAllRecipients());
transportTLS.close();

b) using SSL connection to SMTP server:

// ...
import java.util.Properties;
import javax.mail.Message;
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
// ...
Properties propsSSL = new Properties();
propsSSL.put("mail.transport.protocol", "smtps");
propsSSL.put("mail.smtps.host", "smtp.gmail.com");
propsSSL.put("mail.smtps.auth", "true");

Session sessionSSL = Session.getInstance(propsSSL);
sessionSSL.setDebug(true);

Message messageSSL = new MimeMessage(sessionSSL);
messageSSL.setFrom(new InternetAddress("random_sender_4568744122@gmail.com", "John Smith"));
messageSSL.setRecipients(Message.RecipientType.TO, InternetAddress.parse("random_recipient_4568744122@gmail.com")); // real recipient
messageSSL.setSubject("Test mail using SSL");
messageSSL.setText("This is test email sent to Your account using SSL.");

Transport transportSSL = sessionSSL.getTransport();
transportSSL.connect("smtp.gmail.com", 465, "random_sender_4568744122@gmail.com", "sender_account_pass"); // account used
transportSSL.sendMessage(messageSSL, messageSSL.getAllRecipients());
transportSSL.close();

Note: in both cases You for the connect() method, You have to pass a real login and password of the Google Mail account used. The main differences between those two methods are: different protocol (smtp for TLS and smtps for SSL), different ports (587 for TLS and 465 for SSL). In addition TLS requires STARTTLS.

-------------------------------------------

Download source files:

Note: make sure that Java and Eclipse are properly installed and configured for running the project (additional configuration may be required if different directories are used).

Eclipse complete sample project is here (with all required libraries).

SSL in Tomcat under Eclipse (part 2 - certificate from CA)

2011-08-03T23:38:00.001+02:00

Requirements:

You should be able to generate self signed SSL certificate and integrate it into Tomcat, as described in previous post.

You will learn:

how to obtain and install a real SSL certificate from well known Certificate Authority (CA)

In the previous post I described the complete procedure of generating self signed SSL certificate and integrating it into Tomcat. In this post I would like to focus on the example with real certificate obtained from CA - I will use Thawte as example.

Note: let's assume that You created an application which is going to be visible under the following URL: http://www.myapp.com.

Step 1: generating self signed certificate for domain www.myapp.com.

This is exactly the same step to step 1 in previous post. You have to execute the following command:

keytool -genkey -alias myappcert -keyalg RSA -keystore myapp.keystore

Step 2: Generate Certificate Signing Request (CSR).

You have to generate a special request, which will be send to the CA. You have to execute command:

keytool -certreq -keyalg RSA -alias myappcert -file certreq.csr -keystore myapp.keystore

Generated request is saved as a certreq.csr file, which will be send to CA. CA will use this file to generate certificate signed by them.

Important: You have to use exactly the same alias (in this example: myappcert) for step 1 and step 2.

Step 3: Getting certificate from CA.

Usually certificate are delivered in PKCS#7 or X.509 format. For the first one, the file with certificate will have .p7b extension, for the second one - .cer. Sometimes You can get such certificate also by email as a pure text - for the X.509 format, certificate will be placed between tags -----BEGIN CERTIFICATE----- and -----END CERTIFICATE-----, for PKCS#7 certificate will be placed between tags -----BEGIN PKCS7----- and -----END PKCS7-----. Then You have to copy the certificate content (including those begin/end tags!) and save this as a .cer or .p7b file (You can use Notepad for that). Now You are ready to import Your signed certificate.

Step 4: importing signed certificate.

a) as a .p7b (PKCS#7) format:

Acording to the Thawte, when You obtained Your certificate as a .p7b file, You need only one command to import and install this certificate in Your keystore:

keytool -import -alias myappcert -trustcacerts -file signed_cert.p7b  -keystore myapp.keystore

where signed_cert.p7b is a signed certificate obtained from CA.

b) as a .cer (X.509) format:

According to the Thawte, when You obtained Your certificate as .cer file, You need to download Primary and Secondary Intermediate CAs and import them. They are delivered as .p7b format (i.e. intermediate.p7b). I should import them first, using command:

keytool -import -trustcacerts -alias intermediatecerts  -file intermediate.p7b -keystore myapp.keystore

and then import my certificate (it is in .cer file signed_cert.cer) using this command:

keytool -import -trustcacerts -alias myappcert -file signed_cert.cer -keystore myapp.keystore

where signed_cert.cer is a signed certificate obtained from CA.

Problem and workaround:

Both above ways seems to be simple, but... when I was buying some time ago a web SSL certificate from Thawte, I receive certificate as a pure text in e-mail. Certificate was in X.509 format (I had tags ----BEGIN/END CERTIFICATE-----). I created a signed_cert.cer file from that e-mail content. Then I tried to install intermediate certificates like it was described in Thawte documentation. Unfortunately it didn't work. The first command for importing intermediate certificates failed with error:

keytool error: java.lang.Exception: Input not an X.509 certificate

It seems that keytool does not work with .p7b format (I used JDK 1.6.0_16), so I expect that even if I get my certificate as .p7b file (complete, without need to import intermediate certificates as in a) subpoint) it will also not work. I took a .p7b file with intermediate certificates, opened it under Windows, and for each certificate found inside i exported it as a X.509 DER certificate, giving each file .cer extension. So I had three .cer (X.509) files: signed_cert.cer, thawte_primary.cer, thawte_secondary.cer. Now I had to import intermediate certificates and after that import my signed certificate with those commands:

keytool -import -trustcacerts -alias primary -file thawte_primary.cer -keystore myapp.keystore
keytool -import -trustcacerts -alias secondary -file thawte_secondary.cer.cer -keystore myapp.keystore

keytool -import -trustcacerts -alias myappcert -file signed_cert.cer -keystore myapp.keystore

Now import was OK. So the general workaround rule (if there are any troubles) is: get all needed certificates in X.509 form, import intermediate certificates (if any required) and import Your signed certificate at the end.

Important: You have to use exactly the same alias (in this example: myappcert) like You used in step 1 and step 2, except for importing intermediate certificates - You can use whatever alias You like.

Step 5: copy Your myapp.keystore file into Tomcat's /conf directory

Step 6: modify Tomcat configuration to use SSL certificate

It is exactly the same as step 3 in previous post.

That's all.

SSL in Tomcat under Eclipse (part 1 - self signed certificate)

2011-08-01T23:21:00.001+02:00

Requirements:

installed Java (description here)
installed and configured Eclipse (description here)
installed and configured Tomcat for the Eclipse (description here)

You will learn:

how to set up Tomcat for SSL connection using self signed certificate under Eclipse

When You want to secure Your application SSL is the most natural choice. In order to make Your application be recognized as trusted by browser, You can by a buy a certificate from well known certificate authority (CA), which will be generated for the domain Your application uses. If You do not care about being recognized as trusted service, but instead You just want to encrypt the data exchanged between server and client's browser, You can use Your own self signed SSL certificate. In this post I will show how to generate such SSL certificate and how to set up Tomcat (from the Eclipse level) to use generated certificate.

Note: let's assume that You created an application which is going to be visible under the following URL: http://www.myapp.com.

Step 1: generating self signed certificate for domain www.myapp.com.

Go into Your Java bin directory (i.e C:\Development\Java\bin). Then open Windows console (under Windows Vista/7 open the console with Administrator right) and type in command:

keytool -genkey -alias myappcert -keyalg RSA -keystore myapp.keystore 

where myappcert is the name of the certificate being generated and myapp.keystore is a file where certificate will be stored. You will be asked about the password for created the myapp.keystore file. Type in "mypass", press Enter and type it again and press Enter again. Then You will be asked for some details about You:

Please note that for the first question about first and last name I gave answer www.myapp.com. This is very important - this name (known as CN - Common Name) will be used for the checking if certificate on the page we visit was generated for the same URL which we typed in in the browser.

At the end You will be asked for the password for the newly created certificate. The password must be the same as one used for myapp.keystore file ("mypass"). Do not type antyhing, just press Enter to use the same password. Your myapp.keystore file containing myappcert certificate is ready.

Step 2: copy Your myapp.keystore file into Tomcat's /conf directory

Step 3: modify Tomcat configuration to use SSL certificate.

Make sure that Your Tomcat is configured with Eclipse and works OK without SSL (start Tomcat from Eclipse and type in http://localhost:8080 in the browser). Stop Tomcat if it is running. Then open server.xml file form the "Servers" view:

and locate default element for standard HTTP connections (marked red). Then add additional element for the SSL connection (next to existing one):

<Connector
        SSLEnabled="true"
        clientAuth="false"
        keyAlias="myappcert"
        keystoreFile="conf/myapp.keystore"
        keystorePass="mypass"
        maxThreads="200"
        port="8081"
        scheme="https"
        secure="true"
        sslProtocol="TLS"
 />

Please note that next to some specific SSL settings, we set the location of the keystore file, the password for that file and certificate name to be use. SSL connection uses port 8081, where normal HTTP connection

uses port 8080.

Note 1: all modification of server.xml file were done from Eclipse level, using "Servers" view. If You want to use Eclipse WTP for starting Tomcat (like I do so far) You can't edit this file from elsewhere. Eclipse WTP overrides original Tomcat configuration files by files visible under "Servers" - changes done outside Eclipse will not be visible for WTP.

Note 2: I added element specific for SSL next to existing element for standard non encrypted HTTPS connections. This is second issue directly connected with Eclipse WTP - when You remove standard connector for HTTP, Eclipse will close Tomcat after a time set in "Timeouts" section in the Tomcat settings:

Some people try to extend the timeout time into long period, but still after this period Eclipse kills Tomcat process, as if Tomcat was not properly started in the required time. Unfortunately there is no way to set timeout into ifinite time - the only way to fix this under WTP is to leave standard HTTP element in Tomcat's server.xml file.

Step 4: that's all. Tomcat is configured to work with SSL.

Try to enter the URL: https://localhost:8081. If You see Tomcat's page, everything works OK (You can see certificate warning - You Need to add security exception for that certificate). Of course You can still open the same Tomcat page by standard HTTP (You have two elements), just enter http://localhost:8080 URL.

Tomcat OutOfMemoryError: Heap space/PermGen space

2011-07-27T19:52:00.004+02:00

Have You ever seen such error in Your logs? Probably yes. It simply means that available memory for Tomcat was consumed and nothing left. To be precise: available memory for JVM which Tomcat uses to run.

What causes this error? It can be caused by memory leaks in the application or the applications has big requirements "by design", even if there are no memory leaks. For the first case You should use some profiling tools to find and fix memory leaks - perhaps this might help without need to change memory settings for Tomcat JVM. If You are sure that Your application has no memory leaks, the only way is to increase memory used by Tomcat JVM. See below how to do that - please note that I described modifying Tomcat memory settings when it is installed as a service under Windows OS.

32-bit Windows

The main problem here is that 32-bit OS is able to see no more than 3,2GB of RAM, even if You have 4GB or more physically installed. This is upper limit of memory that can be use - in theory. However, in practise You will not be able to use more than 1 to 1,5GB of RAM for Tomcat's JVM - the rest of memory is used by OS itself and installed software.

64-bit Windows

Let's assume that You need to assign more than 1,5GB of RAM for Tomcat JVM. Therefore You need 64-bit OS with at least 4GB of RAM (for example 64-bit Windows Professional supports up to 192GB of RAM). Of course You must also use 64-bit JDK - when You use 32-bit JDK under 64-bit OS, You will again face the limit of 3,2GB RAM. You also must use 64-bit version of Tomcat (as it contains Windows service wrapper to use with 64-bit JVMs on 64-bit Windows platforms). Below You will find complete list of all needed steps:

Step 1: make sure You are using 64-bit Windows OS.

Step 2: install 64-bit JDK and set $JAVA_HOME to the installation directory.

Step 3: download .zip file with 64-bit Tomcat for Windows (file: apache-tomcat-[version]-windows-x64.zip).

Step 4: extract Tomcat and go into Tomcat's /bin directory.

Step 5: open service.bat file and locate the line with --JvmMS and --JvmMX parameters, and modify it with new values memory, eg:

%EXECUTABLE%" //US//%SERVICE_NAME% ++JvmOptions "-Djava.io.tmpdir=%CATALINA_BASE%\temp" ++JvmOptions "-XX:MaxPermSize=1024m" --JvmMs 2048 --JvmMx 4096

Note: PermSize is set in different way than heap which has predefined JvmMS and JvmMX flags.

Step 6: open Windows console with administrator rights, then go into Tomcat /bin and execute command:

service.bat install tomcat6

Step 7: start service with command:

net start tomcat6

Note 1: You can skip point 5 and execute directly point 6. After that just start tomcat6w.exe which is a GUI tool for managing the service. You can set memory values there.

Note 2: as an alternative to above steps, You can download Tomcat service installer (file: apache-tomcat-[version].exe) which performs service installation, and then use tomcat6w.exe tool for tune memory settings.

That's all. Your Tomcat service should use provided memory settings.

PostgreSQL - strange database server errors under Windows

2011-05-22T17:41:00.005+02:00

I often use PostgreSQL as a database server for my web applications which are usually deployed under Tomcat. PostgreSQL and Tomcat are registered as system services which starts automatically when Windows starts.

In one of my application I have about 50 different databases with connection pools for each database, where each pool has about 10 connections opened. So I have about 500 connections opened for requests. Each opened connection is visible as a separate postgres.exe process in the system next to the own PostgreSQL processes also visible as postgres.exe (usually 5 processes). Sometimes under Windows XP Prof. SP 3 I saw PostgreSQL stop working and in the server logs there were errors like:

java.net.SocketException: No buffer space available (maximum connections reached?): recv failed

or:

server process (PID XXXX) was terminated by exception 0xC0000142 HINT: See C include file "ntstatus.h" for a description of the hexadecimal value.

--------------------------------------------

No buffer space available error:

"Maximum connection reached" may suggest that we set too low value of max_connections in postgresql.conf file. Let's assume that settings for connections amount and required memory are OK for the server, but the problem is still visible from time to time.

The first thing to be checked is amount of short-lived ports (ephemeral). Those ports are used by applications during their normal work. They are opened and live until certain application works, in comparison to server application ports that are typically open for the entire time that the server computer is running. They are assigned automatically from predefined range. Range differs between operating systems:

a) Windows 2000 -> from 1024 to 5000
b) Windows 2003 Server -> from 1024 to 5000
c) Windows XP (all service packs) -> from 1024 to 5000
d) Windows Vista -> from 49152 to 65535
e) Windows 2008 Server -> from 49152 to 65535
f) Windows 7 -> from 49152 to 65535

g) Linux kernels -> from 32768 to 61000 (see /proc/sys/net/ipv4/ip_local_port_range)

If the available pool of ephemeral ports is exahsuted we see Windows WSAENOBUFS error (no buffer space, code 10055). In order to check for exceeded range, we have to see how many processes are running in the system and how many ports they are opening - we can use free TCPView application. We can kill the process that causes exceeding or we can increase the range of ephemeral ports. If we want to increase it, we have to change a registry entry named maxUserPort in:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\maxUserPort

If that registry entry does not exist it means that system uses default value (see above) and we have to create this entry. Then we have to set a proper value for that entry (REG_DWORD) from the range 5000 to 65534.

The second thing to be checked is limit of half-opened TCP/IP connections (connection with state SYN-RCVD). When this limit is exceeded, TCP/IP starts up SYN flooding attack protection (when SynAttackProtect is enabled). This limit differs between operating systems:

a) Windows 2000 Workstation -> 500

b) Windows 2000 Server -> 100

c) Windows Server 2003 -> 500

d) Windows XP SP 1 -> 100

e) Windows XP SP 2, SP3 -> 10

f) Windows Vista SP 1 -> 10

g) Windows Vista SP 2 -> limit removed (not implmented = unlimited)
h) Windows Server 2008 SP 1 -> 10

i) Windows Server 2008 SP 2 -> limit removed (not implemented = unlimited)

f) Windows 7 -> limit removed (not implemented = unlimited)

We can use mentioned TCPView application to check for half-opened TCP connections. This limit can be increased by changing the registry entry named TcpMaxHalfOpen in:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\TcpMaxHalfOpen

If that registry entry does not exist it means that system uses default value (see above) and we have to create this entry and then manualy assign new value.

Under Windows Vista SP2, Windows 7 or Linux I did not see that error.

--------------------------------------------

0xC0000142 error:

This problem is not so easy to explain, although I was able to fix it. The error code means that "A dynamic link library (DLL) initialization routine failed." I am not sure what exacly causes this...

In my case under Windows XP Prof. SP 3, when I started my application and number of postgres.exe processes was about 100 (it should be about 500, see description at the top) the whole database server process was suddenly terminated and I saw second mentioned error (with error code 0xC0000142) in the logs. The most important part here is that PostgreSQL service, which spawned many postgres.exe child processes, was registered under LOCAL SYSTEM account, without "Allow Service to Interact with Desktop" option set.

It seems that this problem is not only connected with PostgreSQL. It is general problem with services and their subprocesses (see this and that article). From the information found the problem my be connected with so-called desktop heaps (see more here and here). Some people having similar problems with many subprocesses did the workaround by setting "Allow Service to Interact with Desktop" to true for their LOCAL SYSTEM services. I was able to fix it under Windows XP Prof. SP 3, by registering PostgreSQL service for regular user account instead of LOCAL SYSTEM (see "Long Version" from my article about manual installation of PostgreSQL how to do this). I also checked this under Windows Vista SP2 and Windows 7 - it works OK there, even under LOCAL SYSTEM account. Under Linux I did not see that error.

I did not try to assign more desktop heap from default for Windows XP SharedSection=1024,3072,512 (2nd or 3rd value needs to be changed) and then try it again to run under LOCAL SYSTEM without "Allow Service to Interact with Desktop". Perhaps this might also help.

JSF 2, Spring 3, JPA (Hibernate 3), PostgreSQL, c3p0 - everything together (part 2 of 2)

2011-04-26T19:36:00.002+02:00

Requirements:

running database and complete project with all required libraries integrated, described in previous post (can be found here)

Step 4 (continued): configuration files - faces-config.xml.

Comparing to a JSF 2.0 project with Spring integrated, only applicationContext.xml file will be changed - web.xml and faces-config.xml files will be the same. We need to add below code into applicationContext.xml:

<bean id="pooledDataSource" 
 class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close"> 
 <property name="driverClass" value="org.postgresql.Driver" />
 <property name="jdbcUrl" value="jdbc:postgresql://localhost:5432/bikes"/>  
    <property name="user" value="postgres"/>  
    <property name="password" value="pgpass"/> 
    <property name="maxPoolSize" value="10" />
 <property name="maxStatements" value="0" />
 <property name="minPoolSize" value="5" />  
</bean>
 
<bean id="JDBCDataSource" 
 class="org.springframework.jdbc.datasource.DriverManagerDataSource" >
 <property name="driverClassName" value="org.postgresql.Driver"/>
 <property name="url" value="jdbc:postgresql://localhost:5432/bikes"/>
 <property name="username" value="postgres"/>
 <property name="password" value="pgpass"/>
</bean>
 
<bean id="entityManagerFactory"
 class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
 <property name="jpaVendorAdapter">
  <bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter" >
   <!-- 
   Specific properties for Hibernate are in persistence.xml file, 
   but also can be placed here and removed from persistence.xml file.  
   -->
  </bean>
 </property>
 <property name="dataSource" ref="pooledDataSource" />
 <property name="persistenceUnitName" value="persistenceUnit"/> 
</bean>

<bean 
 class="org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor" />

<bean name="transactionManager" 
 class="org.springframework.orm.jpa.JpaTransactionManager">
 <property name="entityManagerFactory" ref="entityManagerFactory" />  
</bean>

<tx:annotation-driven />

Note that this approach is similar to situation when plain Hibernate API is used with Spring: JPA's EntityManagerFactory is like Hibernate's SessionFactory, JPA's EntityManager (obtained from EntityManagerFactory) is like Hibernate's Session (obtained from SessionFactory). JPA's EntityManager is injected into Repositories (with the help of Spring's PersistenceAnnotationBeanPostProcessor class), while Hibernate's Session was injected into DAO's (with the help of Spring's HibernateDaoSupport class exteneded by each DAO). JPA's TransactionManager (built on EntityManagerFactory) is like Hibernate's HibernateTransactionManager (built on SessionFactory).

JPA's Repositories and Services are similar to oldschool DAOs and Facades. Repositories are injected into Services, while DAOs are injected into Facades.

Please note that I defined here two datasources: a basic one without connection pool (JDBCDataSource) and second one using c3p0 as a connection pool (pooledDataSource).

Step 5: configuration files - persistence.xml

In addition JPA uses a special configuration file named persistence.xml. This file contains additional settings for JPA vendor. The location of this file is described in this article. In order to have this file inside WEB-INF/classes/META-INF/persistence.xml under Eclipse, I created a directory "config" where I put META-INF subdirectory with persistence.xml file inside. Then I select the directory "config" to be used as a source directory. This means that content of "config" directory will be on classpath, and for a web application during a build time it will be moved into WEB-INF/classes. Note that I also added there log4j config file:

Persistence.xml file contains some specific settings for Hibernate (as JPA vendor):

<?xml version="1.0" encoding="UTF-8" ?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
        http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd"
 version="1.0">

 <persistence-unit name="persistenceUnit" transaction-type="RESOURCE_LOCAL">
  <properties>   
     <property name="hibernate.dialect" value="org.hibernate.dialect.PostgreSQLDialect" />
     <property name="hibernate.show_sql" value="true" />
    </properties> 
 </persistence-unit>
</persistence>

Step 6: configuration files - log4j.properties

As I wrote before we will use Log4j instead of Hibernate's default SLF4j. Lets's log everything we can (we can see how Hibernate opens/closes connections, how connections are taken from the pool, how Spring manages transactions and so on):

log4j.appender.A1=org.apache.log4j.ConsoleAppender
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%d{HH:mm:ss,SSS} %-5p [%c] - %m%n
log4j.rootLogger=info,A1

log4j.logger.com.mchange.v2=DEBUG 
log4j.logger.org.hibernate=DEBUG
log4j.logger.org.springframework=DEBUG

Step 7: database entities.

As shown at the beginning of previous post, we had only one entity represented by the class Bike.java. Instances of Bike.java where created inside Spring service named bikeDataProvider (BikeDataProviderImp.java). Now we have a database with tables Bike, Bike_Category, Account and Role. For each table we should create one class (note: this is the simplest mapping one table to one class - there are other way to map tables to classes).

Bike entity:

package com.jsfsample.model;

@Entity
@Table(name="bike")
public class Bike implements Serializable {

 private Integer id;
 private BikeCategory category; 
 
 @Id
 @GeneratedValue(strategy=GenerationType.IDENTITY)     
 @Column(name="bike_id")
 public Integer getId() {
  return id;
 }
 public void setId(Integer id) {
  this.id = id;
 }
 
 @ManyToOne
 @JoinColumn(name="bike_category_id") 
 public BikeCategory getCategory() {
  return category;
 }
 public void setCategory(BikeCategory category) {
  this.category = category;
 }
 
 // rest of columns
}

Bike_category entity:

package com.jsfsample.model;

@Entity
@Table(name="bike_category")
public class BikeCategory {
 
 private Integer categoryId;
 
 @Id
 @GeneratedValue(strategy=GenerationType.IDENTITY)
    @Column(name="bike_category_id")
 public Integer getCategoryId() {
  return categoryId;
 }
 public void setCategoryId(Integer categoryId) {
  this.categoryId = categoryId;
 }
 // rest of columns
}

Account entity:

package com.jsfsample.model;

@Entity
@Table(name="account")
public class Person {
 
 private Integer accountId;
 private Role currentRole;
 
 @Id
 @GeneratedValue(strategy=GenerationType.IDENTITY)
    @Column(name="account_id")
 public Integer getAccountId() {
  return accountId;
 }
 public void setAccountId(Integer accountId) {
  this.accountId = accountId;
 }
 
 @ManyToOne
 @JoinColumn(name="role_id") 
 public Role getCurrentRole() {
  return currentRole;
 }
 public void setCurrentRole(Role currentRole) {
  this.currentRole = currentRole;
 }
 // rest of columns 
}

Role entity:

package com.jsfsample.model;

@Entity
@Table(name="role")
public class Role {

 private Integer roleId;
 
 @Id
 @GeneratedValue(strategy=GenerationType.IDENTITY)
    @Column(name="role_id")
 public Integer getRoleId() {
  return roleId;
 }
 public void setRoleId(Integer roleId) {
  this.roleId = roleId;
 }
 // rest of columns
}

Note how classes are mapped to tables (using @Table(name="...")) and how fields are mapped to columns (using @Column(name="...")). We also mapped many-to-one relations between tables using @ManyToOne annotation with proper joim column name. Note also how primary keys are generated - we do not use PostgreSQL sequence (GenerationType.SEQUENCE) - PostgreSQL supports SERIAL data type for primary keys, which actually hides sequences (see this link).

Step 8: repositories and services.

Spring repositories are similar to DAO. All database interactions is done insde them. As an example, let's look inside repository resposnible for loading and saving bikes - BikesDAOImpl.java:

package com.jsfsample.repositories.impl;

@Repository("BikesRepository")
public class BikesDAOImpl implements BikesDAO {
 
 private EntityManager em = null;
 
 @PersistenceContext
 public void setEntityManager(EntityManager em) {
  this.em = em;
 }
 
 @Override
 public Bike loadSelectedBike(Integer bikeId) {
  return em.find(Bike.class, bikeId);
 }
 // other methods for saving bike and loading bike for given category

}

This repository is used in Spring's service named bikeDataProvider (BikeDataProviderImpl.java):

package com.jsfsample.services.impl;
...
@Service("bikeDataProvider")
public class BikeDataProviderImpl implements BikeDataProvider {

 @Resource(name="BikesRepository")
 private BikesDAO bikesRepository;
 
 @Resource(name="DictionaryRepository")
 private DictionariesDAO dictionaryRepository;

 public Bike getBikeById(Integer id){
  return bikesRepository.loadSelectedBike(id);
 }
 
 @Transactional
 public void add(Bike newBike, Integer categoryId) {
  BikeCategory categorySelected = dictionaryRepository.loadBikeCategroryById(categoryId);
  newBike.setCategory(categorySelected);
  bikesRepository.saveBike(newBike);
 } 
 
 // other methods using repositories
}

Similar solution is used for account (person) repository which is then used in Spring's Security service named userDetailsService (UserDetailsServiceImpl.java). Note how easy is to execute a method within database transaction - just use @Transactional annotation and voila.

That's all about adding JPA into Spring-based JSF2 sample application.

The rest of application code (web pages, JSF managed beans) are the same as in pure Spring example project - we only modified services by providing repositories.

How to test it? After deploying application on the server and starting the server, we have to open a browser and type in URL:

http://localhost:8080/JSF2FeaturesSpringJPA

-------------------------------------------

Download source files:

Note: make sure that Java, Eclipse and Tomcat are properly installed and configured for running the project (additional configuration may be required if different directories are used).

Eclipse complete sample project is here (with all required libraries). The sample project is a ready to run application which contains JPA (Hibernate), Spring (with Spring Security). You can also download a war file located here (just copy it inside webapps folder in Your Tomcat and start Tomcat with the script startup.bat). Make sure that before doing this You created required database and PostgreSQL server is up and running (You can check access using pgAdmin III).

JSF 2, Spring 3, JPA (Hibernate 3), PostgreSQL, c3p0 - everything together (part 1 of 2)

2011-03-09T21:28:00.002+01:00

Requirements:

a working example of JSF 2.0 application with Spring Framework and Spring Security integrated (can be found here)
installed PostgreSQL database (simple installation variant described here)

You will learn:

how to use database for loading and storing the data in the sample JSF 2.0 application
how to access the database from the application using JPA (Hibernate) and Spring
how to optimize database access by using connection pool (c3p0)

It is a high time to create a real working example with a database. Sample application used in previous posts use no database. We had Spring's managed service named bikeDataProvider acting as a database - all bikes data were created in that service during startup, all bikes data were retrieved from that service, even new bike data were stored internally in the service:

package com.jsfsample.services.impl;
...
@Service("bikeDataProvider")
public class BikeDataProviderImpl implements BikeDataProvider {

 private List<Bike> bikes;

 @PostConstruct
 private void prepareData(){
  bikes = new ArrayList<Bike>();
  
  // MTB
  Bike mtb1 = new Bike();
  mtb1.setId(1);
  mtb1.setName("Kellys Mobster");
  mtb1.setDescription("Kellys Mobster, lorem ipsut...");
  mtb1.setPrice(6500);
  mtb1.setCategory(1);
  
  // ... rest of mock bikes created here
 }
 
 public List<Bike> getBikesByCategory(Integer categoryId, boolean onlyWithDiscount) {
  // returns bikes by given category
 }
 
 public Bike getBikeById(Integer id){
  // returns certain bike for its details
 }

 @Override
 public void add(Bike newBike) {
  // add new bike 
  bikes.add(newBike);
 }
}

The same method of creating mock data was used in Spring's managed service named userDetailsService - users and their authorities were created directly inside the service:

package com.jsfsample.application.impl;
...
@Service("userDetailsService")
public class UserDetailsServiceImpl implements UserDetailsService {

 private HashMap<String, org.springframework.security.core.userdetails.User> users = new HashMap<String, org.springframework.security.core.userdetails.User>();
 
 @Override
 public UserDetails loadUserByUsername(String username) {
  // returns user
 }

 @PostConstruct
 public void init() {
  
  // mocked roles  
  Collection<GrantedAuthority> adminAuthorities = new ArrayList<GrantedAuthority>();
  adminAuthorities.add(new GrantedAuthorityImpl("ROLE_ADMIN"));
  
  Collection<GrantedAuthority> userAuthorities = new ArrayList<GrantedAuthority>();
  userAuthorities.add(new GrantedAuthorityImpl("ROLE_REGISTERED"));
  
  boolean enabled = true;
  boolean accountNonExpired = true;
  boolean credentialsNonExpired = true;
  boolean accountNonLocked = true;
  
  // mocked users with roles
  users.put("admin", new org.springframework.security.core.userdetails.User("admin", "admin", enabled, accountNonExpired,
    credentialsNonExpired, accountNonLocked, adminAuthorities));
  
  users.put("user", new org.springframework.security.core.userdetails.User("user", "user", enabled, accountNonExpired,
    credentialsNonExpired, accountNonLocked, userAuthorities));
 }
}

In both cases all data are created in the method annotated with @PostConstruct. That annotation means, that method will be invoked first after creating an instance of a class (in this case Spring creates instances of those service classes).

Our objective is to remove that mocked data created explicite in the services classes and replace them with data coming from real database.

Step 1: we need database structure:

CREATE TABLE "role" (
 role_id SERIAL NOT NULL,
 name VARCHAR(32),  
 CONSTRAINT role_pk PRIMARY KEY (role_id)
);

CREATE TABLE "account" (
 account_id SERIAL NOT NULL,
 role_id INTEGER NOT NULL,
 login VARCHAR(32),
 password VARCHAR(32),
 CONSTRAINT account_pk PRIMARY KEY (account_id),
 CONSTRAINT role_id_fk FOREIGN KEY (role_id) REFERENCES role(role_id)
);

CREATE TABLE "bike_category" (
 bike_category_id SERIAL NOT NULL,
 name VARCHAR(32),  
 CONSTRAINT bike_category_pk PRIMARY KEY (bike_category_id)
);

CREATE TABLE "bike" (
 bike_id SERIAL NOT NULL,
 bike_category_id INTEGER NOT NULL,
 name VARCHAR(32),
 description TEXT,
 price numeric(10,2),
 discount_price numeric(10,2),    
 CONSTRAINT bike_pk PRIMARY KEY (bike_id),
 CONSTRAINT bike_category_fk FOREIGN KEY (bike_category_id) REFERENCES bike_category(bike_category_id)
);

This is very simple database with two many-to-one relations: many users belongs to (have) one role and many bikes belongs to (have) one category. Of course in real world user would have many roles, so we would use many-to-many relation, but I decided to use many-to-one to simplify it.

Tables account and role will be used by Spring Security - they store users and their authorities (roles). Tables bike and bike_category are the "heart" of bike store.

Step 2: when structure is ready, it is time to insert some sample data:

INSERT INTO role (name) values ('ROLE_ADMIN'); -- id 1
INSERT INTO role (name) values ('ROLE_REGISTERED'); -- id 2

INSERT INTO account (role_id, login, password) values (1, 'admin', 'admin'); -- ROLE_ADMIN
INSERT INTO account (role_id, login, password) values (2, 'user', 'user'); -- ROLE_REGISTERED

INSERT INTO bike_category (name) values ('Mountain'); -- id 1
INSERT INTO bike_category (name) values ('Trekking'); -- id 2
INSERT INTO bike_category (name) values ('Cross'); -- id 3

INSERT INTO bike (bike_category_id, name, description, price, discount_price) values (1, 'Kellys Mobster', 'Kellys Mobster, lorem ipsut...', 6500, null); -- Mountain
INSERT INTO bike (bike_category_id, name, description, price, discount_price) values (1, 'Scott Scale', 'Scott Scale, lorem ipsut...', 18900, null); -- Mountain
INSERT INTO bike (bike_category_id, name, description, price, discount_price) values (1, 'Author Magnum', 'Author Magnum, lorem ipsut...', 17200, 15500); -- Mountain

INSERT INTO bike (bike_category_id, name, description, price, discount_price) values (2, 'Giant Accend', 'Giant Accend, lorem ipsut...', 5000, 4600); -- Trekking
INSERT INTO bike (bike_category_id, name, description, price, discount_price) values (2, 'Merida Freeway', 'Merida Freeway, lorem ipsut...', 2400, 2100); -- Trekking
INSERT INTO bike (bike_category_id, name, description, price, discount_price) values (2, 'Mbike Massive', 'Mbike Massive, lorem ipsut...', 1900, null); -- Trekking

INSERT INTO bike (bike_category_id, name, description, price, discount_price) values (3, 'Giant Roam XR 1', 'Giant Roam XR 1, lorem ipsut...', 3900, null); -- Cross
INSERT INTO bike (bike_category_id, name, description, price, discount_price) values (3, 'Cannondale Quick Cx', 'Cannondale Quick Cx, lorem ipsut...', 4999, null); -- Cross
INSERT INTO bike (bike_category_id, name, description, price, discount_price) values (3, 'Cube Cross', 'Cube Cross, lorem ipsut...', 4500, 4200); -- Cross

Step 3: adding required libraries into project.

Comparing those libraries with libraries from clean JSF 2.0 project (look here) or libraries from JSF 2.0 project with Spring (look here) we have extra libraries here: Hibernate libraries marked green, Spring JPA libraries marked red and some third party libraries marked blue. Those blue libraries are:

c3p0 libraries for connection pool
log4j library and bridge library from sl4j to log4j - Hibernate by default uses sl4j, but we would like to use log4j
JDBC driver for PostgreSQL

Step 4: configuration files - will be continued in next post.

-------------------------------------------

Download source files:

The complete working example of mentioned application which will contain all described issues, will be available in the last (second) article of this serie.

JSF 2 with Spring 3 - protection with Spring Security (part 2 of 2)

2011-01-28T21:37:00.006+01:00

Requirements:

a working example of JSF 2.0 application with Spring Framework integrated (can be found here)

You will learn:

how to use Spring Security Framework in order to protect web application

From the previous post we know what Spring Framework is, and what advantages it gives us when used in web application. Business logic managed by Spring is not the only one advantage coming from Spring - we can use Spring's embedded mechanisms to secure our web application. This post will show the basic usage of Spring Security for securing our sample JSF 2.0 webapp.
What parts of our application will be protected? Consider those scenarios:
1. Only registered user (or page administrator) can see details of a selected bike.
2. Only page administrator can add a new bike to the shop offer.

We need two user roles which will determine the privilleges which user has: registered users role and admin users role. Moreover, for the scenario 2, we have to add a new function: adding new bike. For this function we will create a page addBike.xhtml and a JSF managed bean for that page, named addBike.java.

addBike.xhtml source code is shown below:

<?xml version='1.0' encoding='UTF-8' ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:ui="http://java.sun.com/jsf/facelets"
      xmlns:h="http://java.sun.com/jsf/html"
      xmlns:f="http://java.sun.com/jsf/core">
  <ui:composition template="../shopTemplate.xhtml">
       
        <ui:define name="content">
        <h:form>
         <h:outputText value="#{msg['bikes.list.name']}: "/><h:inputText value="#{addBike.name}" /><br/>
         <h:outputText value="#{msg['bikes.list.price']}: "/><h:inputText value="#{addBike.price}" /><br/>
         <h:outputText value="#{msg['bikes.list.discountprice']}: "/><h:inputText value="#{addBike.discountPrice}" /><br/>         
         <h:outputText value="#{msg['bikes.list.description']}: "/><h:inputText value="#{addBike.description}" /><br/>
         <h:commandButton action="#{addBike.addNewBike}" value="#{msg['bikes.add.button']}" />
        </h:form>
        </ui:define>

  </ui:composition>
</html>

Nothing special - standard form for entering the data.
addBike.java source code is also simple:

package com.jsfsample.managedbeans;
...
@ManagedBean(name="addBike")
@SessionScoped
public class AddBike implements Serializable {

 private static final long serialVersionUID = -2155913853431899821L;
 
 
 @ManagedProperty("#{bikeDataProvider}")
 private BikeDataProvider bikeDataProvider; // injected Spring defined service for bikes
 
 private String name;
 private String description;
 private String price;
 private String discountPrice;
 private Integer categoryId;
 
 public String addNewBike(){

  Bike newBike = new Bike();
  newBike.setName(getName());
  newBike.setDescription(getDescription());
  newBike.setPrice(Integer.parseInt(getPrice()));
  newBike.setDiscountPrice(Integer.parseInt(getDiscountPrice()));
  newBike.setCategory(categoryId);
  
  // save new bike and return to the shop
  bikeDataProvider.add(newBike);  
  return "/bikesShop.xhtml";
 }; 
 ...
}

Please note that we use here BikeDataProvider.java class, which is Spring managed service, the same we used for loading bikes list and loading a certain bike details in previous post.

Now it is time for protected parts of application. I will show two ways of protecting webapp: protecting resources (like access to certain page) and protecting business logic methods execution. Scenario 1 will be an example of protecting business logic and scenario 2 will be an example of protecting resources.
When user tries to access the protected area (resource or invoke protected method), application will check user roles and based on them will decide if let the user go further or force him to log in. Log in - that's right - a login page will be displayed where user will enter his credentials. Based on them Spring Security will decide what roles user has and depends on assigned roles further action will be continued or not. Let's modify our application to use Spring Security:

Step 1. Modify configuration files:
applicationContext.xml source:

...
 <!-- 
 resource security  
 -->
 <sec:http auto-config="true" access-denied-page="/faces/accessDenied.xhtml">
  <sec:form-login login-page="/faces/login.xhtml" />
  <sec:intercept-url pattern="/faces/admin/**" access="ROLE_ADMIN" />     
 </sec:http>
 <!-- 
 business logic (method) security 
 -->
 <sec:global-method-security
  secured-annotations="enabled" jsr250-annotations="enabled" >  
 </sec:global-method-security>
 <!-- 
 manager responsible for loading user account with assigned roles 
 -->
 <sec:authentication-manager alias="authenticationManager">
  <sec:authentication-provider
   user-service-ref="userDetailsService" />
 </sec:authentication-manager>
 ...

Access-denied-page is invoked when user is authenticated but is not authorized to access protected resources. When user is not authenticated, he is moved into form-login instead of access-denied-page.
web.xml source:

...
 <filter>
  <filter-name>springSecurityFilterChain</filter-name>
  <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
 </filter>
 <filter-mapping>
  <filter-name>springSecurityFilterChain</filter-name>
  <url-pattern>/*</url-pattern>
 </filter-mapping>
 ...

Step 2. Additional pages: login.xhtml and accessDenied.xhtml.
accessDenied.xhtml is simple page displaying only a message saying that the user is authenticated but still is not authorized to go further.
login.xhtml is a simple page with login form where user enters his credentials (login and password). The more interesting part is corresponding managed bean LoginBean.java which uses a Spring service for authenticating users:

package com.jsfsample.managedbeans;
...
@ManagedBean(name = "loginBean")
@SessionScoped
public class LoginBean implements Serializable {
 private static final long serialVersionUID = 1L;

 private String login;
 private String password;

 @ManagedProperty(value = "#{authenticationService}")
 private AuthenticationService authenticationService; // injected Spring defined service for bikes


 public String login() {

  boolean success = authenticationService.login(login, password);
  
  if (success){
   return "bikesShop.xhtml"; // return to application but being logged now 
  }
  else{
   FacesContext.getCurrentInstance().addMessage(null, new FacesMessage("Login or password incorrect."));   
   return "login.xhtml";
  }
 }
 ...
}

When login was successful and user is authenticated, he is moved to the shop. If not a proper message is displayed and user can re-enter his credentials or go back to the shop without login. Let's look inside AuthenticationService.java class which is a service deciding if user is authenticated or not.

Step 3. AuthenticationService implementation:

package com.jsfsample.application.impl;
...
@Service("authenticationService")
public class AuthenticationServiceImpl implements com.jsfsample.application.AuthenticationService {


 @Resource(name = "authenticationManager")
 private AuthenticationManager authenticationManager; // specific for Spring Security

 @Override
 public boolean login(String username, String password) {
  try {
   Authentication authenticate = authenticationManager
     .authenticate(new UsernamePasswordAuthenticationToken(
       username, password));
   if (authenticate.isAuthenticated()) {
    SecurityContextHolder.getContext().setAuthentication(
      authenticate);    
    return true;
   }
  } catch (AuthenticationException e) {   
  }
  return false;
 }
 ...
}

This Spring managed service uses internally a class AuthenticationManager, which comes from Spring Security and was defined as a manager in applicationContext.xml file:

...
 <!-- 
 manager responsible for loading user account with assigned roles 
 -->
 <sec:authentication-manager alias="authenticationManager">
  <sec:authentication-provider
   user-service-ref="userDetailsService" />
 </sec:authentication-manager>
 ...

Note that we do not explicit define AuthenticationManager! It is a ready to use object. But AuthenticationManager has helper service named userDetailService defined in applicationContext.xml file - this service must be written by our own.

Step 4. Implementation of userDetailService.
userDetailsService source code is shown below:

package com.jsfsample.application.impl;
...
@Service("userDetailsService")
public class UserDetailsServiceImpl implements UserDetailsService {

 private HashMap users = new HashMap();
 
 @Override
 public UserDetails loadUserByUsername(String username)
   throws UsernameNotFoundException{
  
  org.springframework.security.core.userdetails.User user = users.get(username);
  
  if (user == null) {
   throw new UsernameNotFoundException("UserAccount for name \""
     + username + "\" not found.");
  }
  
  return user;
 }

 @PostConstruct
 public void init() {
  
  // sample roles  
  Collection adminAuthorities = new ArrayList();
  adminAuthorities.add(new GrantedAuthorityImpl("ROLE_ADMIN"));
  
  Collection userAuthorities = new ArrayList();
  userAuthorities.add(new GrantedAuthorityImpl("ROLE_REGISTERED"));
  
  boolean enabled = true;
  boolean accountNonExpired = true;
  boolean credentialsNonExpired = true;
  boolean accountNonLocked = true;
  
  // sample users with roles set
  users.put("admin", new org.springframework.security.core.userdetails.User("admin", "admin", enabled, accountNonExpired,
    credentialsNonExpired, accountNonLocked, adminAuthorities));
  
  users.put("user", new org.springframework.security.core.userdetails.User("user", "user", enabled, accountNonExpired,
    credentialsNonExpired, accountNonLocked, userAuthorities));
 }
}

We have here a Spring Security specific objects representing users and roles. In the init() method I created some mocked data two roles representing page administrators and registered users - ROLE_ADMIN and ROLE_REGISTERED. For each role I created a single account: admin (password: admin) for the ROLE_ADMIN and user (password: user) for the ROLE_REGISTERED. That's all - it is time to protect applicartion.

Step 5. Protecting application.
5 a) Scenario 1: protecting business logic. We have to protect invoking a method which allows to see bike details. This method is placed inside BikeDataProvider.java service class. In order to protect the method we have to add an annotation defining roles allowed to execute this method:

package com.jsfsample.services;
...
public interface BikeDataProvider {
 ...
 
 @RolesAllowed({"ROLE_ADMIN","ROLE_REGISTERED"}) 
 public abstract Bike getBikeById(Integer id);
 
 public abstract void add(Bike newBike);
}

This simply means that only registered users or admin users can see bike details.
Why we do not protect the add(...) method? Because we protect the whole page access where this method is executed - of course in addition we can also protect this method by annotating it with @RolesAllowed({"ROLE_ADMIN"}).
5 b) Scenario 2: protecting resource. According to rules of protecting resources defined in applicationContext.xml, we protect all resources which are located inside /admin directory. So we have to create a directory /admin under /WebContent directory and move addBike.xhtml page there. It should look like this:

Note: there is a little trick in the protecting resources like pages in JSF. Spring Security tries to match exact URL address to apply the rule. But in JSF there is a "old URL" issue - after navigtation from page A to page B, URL address in browser still points to page A. In order to make the rule working we have to force the browser to show the current URL instead of old one. It is done by adding a special command into the navigation string returning a page for adding a bike:

public String showForm(){  
 ...  
 return "/admin/addBike.xhtml?faces-redirect=true";
}

That's all about Spring Security in our sample application.

How to test it? After deploying application on the server and starting the server, we have to open a browser and type in URL:

http://localhost:8080/JSF2FeaturesSpring

Then try to display some bike details. When promped for login, enter credentials: user, user and try again. Then try to add a new bike - You should see access denied page. The close the application and clean the browser cache and try the same with the user admin, admin.

-------------------------------------------

Download source files:

Note: make sure that Java, Eclipse and Tomcat are properly installed and configured for running the project (additional configuration may be required if different directories are used).

Eclipse complete sample project is here (with all required libraries). The sample project is a ready to run application which contains all described Spring Security issues in this post. You can also download a war file located here (just copy it inside webapps folder in Your Tomcat and start Tomcat with the script startup.bat)

JSF 2 with Spring 3 - basics (part 1 of 2)

2010-12-14T22:45:00.003+01:00

Requirements:

a working example from the last article of a serie introducing new JSF 2.0 features.

You will learn:

how to integrate Spring Framework into JSF 2.0 application

In the previous three posts I described some interesting JSF 2.0 features and I put them together in the sample application. Those parts were mostly focused on web content, a GUI and its behaviour.
What about the server side? What about the business logic executed underneath? Do we had a business logic in the sample application mentioned above? Of course we had. Presenting bikes list or a certain bike under some condition - this is business logic resposniblity. Filtering bikes and presenting only those with the discount - this is also business logic. Don't think of it as a naive filtering of presented data - the business logic decides what does it mean that the certain bike has discount - it can be lower price but it can be also more complicated. In the previous example we had a class named BikeDataProvider.java which represented business logic. It was a singleton invoked with the help of static method getInstance() anywhere where needed.
When the application starts to grow up, we have more business logic accomplishing some business cases. We need something that will help us to manage the whole business logic in an elegant way - this is where Spring comes to play.
We will change the BikeDataProvider class into object created and managed by Spring. This object will be called service. A service which serves business logic. Then the service will be used by JSF managed beans. Let's add Spring to our sample web apllication.

Step 1: adding required Spring libraries into the project.

Step 2: creating Spring's configuration file named applicationContext.xml. The file has to be located inside /WEB-INF directory. This is full content of that file:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:context="http://www.springframework.org/schema/context"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop"
 xmlns:faces="http://www.springframework.org/schema/faces"
 xmlns:int-security="http://www.springframework.org/schema/integration/security"
 xmlns:tx="http://www.springframework.org/schema/tx" xmlns:sec="http://www.springframework.org/schema/security"
 xsi:schemaLocation="http://www.springframework.org/schema/integration/security http://www.springframework.org/schema/integration/security/spring-integration-security-2.0.xsd
  http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
  http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.0.xsd
  http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
  http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
  http://www.springframework.org/schema/faces http://www.springframework.org/schema/faces/spring-faces-2.0.xsd
  http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
  http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">

 <context:component-scan base-package="com.jsfsample" />

</beans>

Step 3: modifying web.xml by registering listener resposnible for loading Spring in web application.

<listener>
  <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
 </listener>
 <context-param>
  <param-name>contextConfigLocation</param-name>
  <param-value>/WEB-INF/applicationContext.xml</param-value>
 </context-param>

Step 4: modifying faces-config.xml file for allow JSF components use Spring components.

<application>
... <el-resolver>org.springframework.web.jsf.el.SpringBeanFacesELResolver</el-resolver>
...
 </application>

Step 5: modifying BikeDataProvider.java toward Spring managed service.

First of all we will add a new functionality to the application - possibility to add a new bike to the selected category. We need to create an .xhtml page with the form, then managed bean for that page and at the end a business logic method in class BikeDataProvider.java responsible for adding new bike to bikes' list. That's easy part - it will be visible in the attached complete example.
Assuming that we have this new funcitonality, we can modify .java files for Spring integration. First we create an interface:

package com.jsfsample.services;
//imports
public interface BikeDataProvider {

 public List<Bike> getBikesByCategory(Integer categoryId,
   boolean onlyWithDiscount);

 public Bike getBikeById(Integer id);

 public void add(Bike newBike); // new function

}

Then we have to create an implementation named BikeDataProviderImpl.java (yes, I know that this naming convention is bad) which will have the source code from previous BikeDataProvider.java class. This implementation will have Spring specific annotation defining the service:

package com.jsfsample.services.impl;
//imports
@Service("bikeDataProvider")
public class BikeDataProviderImpl implements BikeDataProvider {

 private List<Bike> bikes;
 private Integer currentBikeId;
 
 @PostConstruct
 private void prepareData(){
  bikes = new ArrayList<Bike>();
  
  // MTB
  Bike mtb1 = new Bike();
  mtb1.setId(1);
  mtb1.setName("Kellys Mobster");
  mtb1.setDescription("Kellys Mobster, lorem ipsut...");
  mtb1.setPrice(6500);
  mtb1.setCategory(1); 
  bikes.add(mtb1);
  // other bikes are mocked up the same way
  
 }
 
 public List<Bike> getBikesByCategory(Integer categoryId, boolean onlyWithDiscount) {
  // implementation 
 }
 public Bike getBikeById(Integer id){
  // implementation 
 }

 public void add(Bike newBike) {
  bikes.add(newBike);
 }
}

The annotation @Service("bikeDataProvider") means that this is Spring managed object (created by Spring) and is visible in the Spring context under the name "bikeDataProvider". @PostConstruct is a little trick here - when object of this class is instantiated by Spring, the method is invoked right after the object is created. I used this for preparing demo data of bikes. The whole BikeDataProviderImpl.java class acts as a simple data source for the application - in the future we will use a real database instead.

Step 6: modifying JSF managed beans to use Spring service inside.

We will use registered Spring service inside JSF managed-beans. For example consider BikeDetails.java managed bean. Previously we loaded certain bike in this way:

package com.jsfsample.managedbeans;
// imports
@ManagedBean(name="bikeDetails")
@RequestScoped
public class BikeDetails {

 private Integer bikeId;
 private Bike bike; 
 
 public void loadBike(){
  bike = BikeDataProvider.getInstance().getBikeById(bikeId);
 }
        ...
}

Now BikeDataProvider.java is not a singleton - it is a Spring service. BikeDetails.java managed bean is changed:

package com.jsfsample.managedbeans;
// imports
@ManagedBean(name="bikeDetails")
@RequestScoped
public class BikeDetails {

 private Integer bikeId;
 private Bike bike; 
 
 @ManagedProperty("#{bikeDataProvider}")
 private BikeDataProvider bikeDataProvider; // injected Spring service

 public void loadBike(){
  bike = bikeDataProvider.getBikeById(bikeId);
 }
        ...
}

We used the name of registered service ("bikeDataProvider") to inject it into JSF managed bean (please note that we used here an interface as an instance variable, Spring injects its concrete implementation). This injection is possible thanks to modifications from step 4.

That's all. Other JSF managed beans responsible for displaying bikes' lists or adding a bike use mentioned Spring service in the same way.

-------------------------------------------

Download source files:

The complete working example of mentioned application which will contain all described issues, will be available in the last (second) article of this serie.

JSF 2.0 - New features (part 3 of 3)

2010-10-21T21:19:00.004+02:00

Requirements:

working example from previous post as a basis

You will learn:

support for Ajax
support for GET and creating bookmarkable URLs

This is the third post about new features that could be found in JSF 2.0 comparing to JSF 1.x. In the previous post (part 1), I described templating mechanism and simplified navigation model. In the second post (part 2), I described resource loading mechanism. In this post I would like to focus on the built in Ajax support and on possibility to create bookmarkable URLs.

Ajax support.
Ajax is a very wide issue. I would like to show only one of the simplest example - refreshing part of the page after some action. The purpose of this example is to show that using simple Ajax does not require any additional configuration or libraries.
In the sample application I have a page bikesList.xhtml. This page shows the bikes list in selected category (category is chosen in main menu on left). Some of the bikes have discount price. I have two buttons (above the displayed bikes list) acting as bike list filters. Pressing those buttons causes filtering the list and reloading it on the view. For example: if I press a button "Discount bikes" (in red box), only one bike will be shown on a list (Magnum bike with doscount price, marked by red box). If I press "All bikes" - all bikes from the category will be shown:

The idea is to reload the bike list after pressing the button, without reloading the whole page. This is done by Ajax call which is "hooked" to the buttons:

<ui:define name="content">
        <h:commandButton actionListener="#{bikesListBean.showAllBikes}" value="#{msg['bikes.filter.all']}">
            <f:ajax render="bikesTable"  />
          </h:commandButton>
              
          <h:commandButton actionListener="#{bikesListBean.showDiscountBikes}" value="#{msg['bikes.filter.discount']}">
            <f:ajax render="bikesTable"  />
          </h:commandButton>
           
        <h:dataTable id="bikesTable" value="#{bikesListBean.bikesList}" var="b">
...
</ui:define>

Ajax call defined by <f:ajax render="bikesTable" /> causes reloading the corresponding part of thw page with id=bikesTable. In our case this is the list <h:dataTable id="bikesTable" ... > which shows all or filtered bikes. Simple - isn't it?

Creating bookmarkable URLs (GET support).
Why do I need GET? Consider this situation: I found interesting bike in the Bike Shop. I would like to get the URL and send it to someone in order to show what I found. Quite normal thing isn't it? Unfortunately not possible in JSF 1.x because of lack of GET support. JSF 1.x is POST-centric, so passing parameters in the URL wiht GET was not possible - so I have no way to save the URL.
JSF 2.0 provided GET support by introducing so-called "view parameters". A page which uses a special component for view parameters, is able to catch incoming view parameters (they are in the URL) and update page model (i.e. fields in managed bean) with their values. Moreover standard conversion and validation of the incoming parameters is possible - just like for the POST data.

How it work in real example? Let's go back to our sample application. We have a page named bikesList.xhtml which shows all bikes. We also have a page named bikeDetails.xhtml which shows the detail of selected bike. I would like to see the particular bike and take the URL with it and send it to someone. So the bikeDetails.xhtml page will contain view parameter component:

<f:metadata>
  <f:viewParam name="bikeId" value="#{bikeDetails.bikeId}"/>
  <f:event type="preRenderView" listener="#{bikeDetails.loadBike}"/>
</f:metadata>

Page expects the URL parameter named bikeId, then value of this parameter is copied into the model to the field #{bikeDetails.bikeId}. For now skip <f:event .../> component - I will back to it at the end.
Where those parameters are created and passed to the URL? JSF 2.0 provides two components allowing to add GET parameters to the target URL: <h:button /> and <h:link />(they work similar to the POST-centric <h:commandButton /> and <h:commandLink). In our example bikesList.xhtml page uses <h:button /> for each presented bike to construct the button which navigates us to bikeDetails.xhtml and sets bikeId as a parameter:

<h:dataTable id="bikesTable" value="#{bikesListBean.bikesList}" var="b">
...
<h:button outcome="bikeDetails" value="#{msg['bikes.list.seebike']}">
  <f:param name="bikeId" value="#{b.id}"/>
</h:button>
...
</h:dataTable>

Notice outcome attribute and its value - it points to the target page where view parameters component is used. Simplified navigation is used here. Generated URL looks like this:

http://localhost:8080/JSF2Features/faces/bikeDetails.xhtml?bikeId=5

We have bokkmarkable URL which can be saved and sent to other people and used later.

Solution with view parameters on bikeDetails.xhtml uses another new feature in JSF 2.0: system events. We register a special listener using the tag

<f:event type="preRenderView" listener="#{bikeDetails.loadBike}"/>

The listener is executed before the view is rendered. What is the benefit? When a view is reached, view parameter is copied into the managed bean and listener is executed and after that the whole view (page) is rendered. Our listener uses passed bikeId to load the selected bike from repository like database. When the page is rendered, the proper bike is loaded from repository and ready to be displayed - we do not have to wait for loading the data.

That's all. We are ready to test the application. After deploying application on the server and starting the server, we have to open a browser and type in URL:

http://localhost:8080/JSF2Features

-------------------------------------------

Download source files:

Note: make sure that Java, Eclipse and Tomcat are properly installed and configured for running the project (additional configuration may be required if different directories are used).

Eclipse complete sample project is here (with all required libraries). The sample project is a ready to run application which contains all described JSF 2.0 features in first, second and this (third) post. You can also download a war file located here (just copy it inside webapps folder in Your Tomcat and start Tomcat with the script startup.bat)

JSF 2.0 - New features (part 2 of 3)

2010-10-20T20:28:00.003+02:00

Requirements:

working example from previous post as a basis

You will learn:

resource loading

This is the second post about new features that could be found in JSF 2.0 comparing to JSF 1.x. In the previous post (part 1), I described templating mechanism and simplified navigation model. In this post I would like to focus on the improved resource loading and its capabilities in JSF 2.0.

Resource loading.

What are resources in terms of a web application? It can be images files, JavaScript script files or CSS files used on web pages in the application - so some external files accessible by special tags in the page source. In the JSF application we also name as resources message bundle files.

Just as a short reminder for message bundles:

1. We create message bundle files for each language we would like to support. Then we add some lines into the configuration in faces-config.xml where we define what language will be supported (<locale-config> section) and we define EL access object (<var>msg</var>) for the created message bundle files:

2. We use message from the bundle files by calling defined EL accessor in the page source, for example:

<h:outputText value="#{msg['top.name']}" />

This approach gives us the most important benefit: localization of the application. Based on the browser's language, suitable message bundle file is used (EN or PL in above example).

So far so good - nothing new is here when comparing to JSF 1.x. But what about other resources like JS, CSS or image files? JSF 2.0 can handle those resources in a more intelligent way than before - a special ResourceHandler is used for load resources from some predefined locations. According to the documentation resources are expected to be placed in:

WebRoot/resources/<resourceIdentifier> 

WebRoot is a root of web application - in the Eclipse generated web projects it is the directory named "WebContent". Part resourceIdentifier have this structure:

[localePrefix/][libraryName/][libraryVersion/]resourceName[/resourceVersion]

resourceIdentifier defines the subdirectory structure inside WebRoot/resources/directory. And the most important part: localePrefix means that resources can be localized based on the browser's language, like message bundles. Part libraryName can be use to group resources by their type in different directories, for example "css", "images", "scripts".

Let' see how it work in a sample application. As I wrote in previous post, we used shopTemplate.xhtml file as a template page for all pages in application. The second purpose of this file (next to defining reusable parts) is to separate the structure from the presentation - template page will include all CSS files and common images files. We will use new ResourceHandler in order to load some CSS and images suitable for current browser language:

images: we have two pictures (bike_logo.jpg and flag.gif) in sample application visible in the header - they will change depending on current browser language
CSS: depending on current browser language, different CSS will be loaded. The only difference between loaded CSS's is footer color (just to show that CSS is really changed)

According to the documentation we have to create directory structure for mentioned resources and put different resource there:

How to load such resources on the page? First we have to perform additional configuration to enable loading resources using proper localePrefix ("pl" and "en" on the screenshot above). We have to create a special localized entry named javax.faces.resource.localePrefix in a resource-bundle file. This file must be configured as message-bundle in faces-config.xml - we can not use resource bundles (used for localized messages as shown above) to put entry there:

Now we can use the resources on the shopTemplate.xhtml page:

Please note that we use here a new JSF 2.0 tag:

<h:outputStylesheet library="css" name="layout.css" />

Attributes library and name correspond to the libraryName and resourceName from <resourceIdentifier> respectively.

If we want to load JS scripts we have to use additional JSF 2.0 tag: <h:outputScript ... /> wich has additional attribute named target - specifying where to render script link.

Loading images with standard tag <h:graphicImage ... /> is possible in two ways:

using attributes library and name correspond to the libraryName and resourceName from <resourceIdentifier> respectively (like for CSS)
using special EL expression "#{resource['images:bike_logo.jpg']}" (wartch out for letters: EL has resource word but in the WebContent there is a resources directory!)

Note about testing: in order to test if images and CSS (footer color) changes when browser language is changed, I recommend clean the browser cache and restart it completely before testing (sometimes image cache prevents from changing the image). The test result should be:

PL version:

EN version:

-------------------------------------------

Download source files:

The complete working example of mentioned application which will contain all described features, will be available in the last (third) article of this serie.

JSF 2.0 - New features (part 1 of 3)

2010-10-17T20:39:00.007+02:00

Requirements:

working "Hello World" example in JSF 2.0 as a basis (from here)

You will learn:

templating with Facelets
simplified page navigation

In the previous post I showed how to create classic "Hello World" application in JSF 2.0. The example was created in the simplest possible way. I focused on generating and setting up working and ready to use project in Eclipse. Created application did not differ much from similar "Hello World" applications in JSF 1.2, because I did not use JSF 2.0 built-in featues. Now it is a time to show what new features are in JSF 2.0 and how they can improve development web applications.

Sample application:

In order to go higher than "Hello World" level we will create fully functional mini application which will serve as an example for mentioned JSF 2.0 features.

Let's assume that we are creating a web application for bicycle shop. Our application should present a list of available bikes. Bikes will be divided because of their type like MTB, Trekking and Cross bikes. In addition bike list will have a filter allowing to display only bikes with a special discount price. User should be able to display detailed information about selected bike from the presented list. We will use those pages in application:

bikesShop.xhtml - start page presenting information about bike shop
bikesList.xhtml - page presenting bikes of given type
bikeDetails.xhtml - page presenting detailed information about bike displayed on bikesList.xhtml

Under the hood we will use those classes:

BikeDataProvider.java - singleton, acts as service which belongs to business logic. Responsible for loading bikes of certain type and loading single bike with its details. For simplicity all bikes instances are created and stored inside that class.
Bike.java - a class from the model representing single bike instance. A list of bike instances is created and used inside BikeDataProvider.java.
BikeDetails.java, BikesList.java - managed beans used for bikeDetails.xhtml and bikesList.xhtml respectively. They call BikeDataProvider.java for loading bikes list or single bike.

Web application will have popular and standard layout - header on top, menu on left, content on righ, footer on bottom. Header will have shop's logo and name, menu will have bike types listed, content will have some information about the shop and will display bikes list for given type, footer will be empty with some background color. The whole application will look like this:

bikesShop.xhtml:

bikesList.xhtml:

bikesDetails.xhtml:

Facelets - templating.

One of the key feature of Facelets is ability to create page templates. Have a look at our sample applications screenshot above - all of them have common content like header, left menu and footer. If we had JSF application based on JSP pages without using Facelets, those elements would be included separately into source code of each web page. Imagine small change in the header - it may become a maintenance nightmare because we have to change source code of all pages where header is visible. With the Facelets is it possible to extract common content and put it into one page template. The template has special sections where variable content will be displayed - pages which use the template "injects" to those sections their specific content.

For our application we will create a template page named shopTemplate.xhtml which will act as a template for three pages visible above (bikesShop.xhtml, bikesList.xhtml, bikeDetails.xhtml). The whole page layout common for all pages will be defined inside the single template page. This gives us another advantage - we can follow one of the best practises in designing web pages here - separate the structure from the presentation. The template will contain only the pages structure while the whole presentation will be placed in separate CSS file used by template. The source code for the template will look like this:

<?xml version='1.0' encoding='UTF-8' ?> 
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:ui="http://java.sun.com/jsf/facelets"
      xmlns:h="http://java.sun.com/jsf/html"
      xmlns:f="http://java.sun.com/jsf/core">
    
    <h:head>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
        <title>Bike Shop 2.0</title>
    </h:head>
    
    <h:body>
        <div id="container">
            <div id="top">
                <!-- logo and name goes here -->
            </div>
            <div id="leftnav">
                <h:form>
                    <!-- links goes here -->
                </h:form>
            </div>
            <div id="content">
                <ui:insert name="content" />
            </div>
            <div id="footer">
                ####
            </div>
        </div>
     </h:body>   
</html>

Please note the element <ui:insert name="content" /> inside the div named "content". It represents variable content which will be displayed in this place. Pages bikesShop.xhtml, bikesList.xhtml and bikeDetails.xhtml will "inject" here their content using special syntax. Let's have a look how bikesShop.xhtml page display its content using the template. Here is the code for bikesShop.xhtml:

<?xml version='1.0' encoding='UTF-8' ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:ui="http://java.sun.com/jsf/facelets"
      xmlns:h="http://java.sun.com/jsf/html"
      xmlns:f="http://java.sun.com/jsf/core">
  <ui:composition template="shopTemplate.xhtml">
       
        <ui:define name="content">
         Lorem ipsum dolor sit amet...
        </ui:define>

  </ui:composition>
</html>

First note that bikesShop.xhtml page have no <head> or <body> tags - they are defined in the template. A special element <ui:composition template="shopTemplate.xhtml"> tells the page to use the mentioned template.

As I wrote, the template has a special section for inserting the variable content (<ui:insert name="content" />). The page bikesShop.xhtml defines content to be placed into that section by using the tag <ui:define name="content"> ... </ui:define> - note that attribute name="content" is the same for <ui:insert/> and <ui:define />. Pages bikesList.xhtml and bikeDetails.xhtml use the same mechanism (they have the same <ui:define /> tag in their source).

That's all about templates.

Simplified page navigation.

Every page navigation in JSF 1.x required a proper entry in faces-config.xml file. It was something like this:

<navigation-rule>
     <description>Welcome page to message page</description>
     <from-view-id>/index.jsp</from-view-id>
     <navigation-case>
       <from-outcome>helloMessage</from-outcome>
       <to-view-id>/message.jsp</to-view-id>
    </navigation-case>
   </navigation-rule>

JSF 2.0 provides a simplified navigation model - we do not need any entries (navigation rules) in the faces-config.xml file. Why?

Suppose we have a method used for navigation which returns some string value. This string value (known as outcome) is taken by a navigation handler and the handler checks for navigation rules in faces-config.xml which have defined the same from-outcome value. When the rule is found, it is applied and a proper navigation is done. This is how it worked in JSF 1.x and how it works in JSF 2.0. But JSF 2.0 navigation handler does additional operation here: if no matching from-outcome value is found in faces-config.xml (in other words: there is no navigation rule to apply), handler checks also existing pages names (view identifiers). If existing page name matches returned outcome value, the navigation is done to that page.

For example if we have a navigation method in some backing bean:

public String getBikes(){
   bikesList = ... // load some bikes
   return "bikesList";
}

and there is a page named bikesList.xhtml, invoking this method will cause the navigation to that page - without defining proper navigation rule in faces-config.xml.

-------------------------------------------

Download source files:

The complete working example of mentioned application which will contain all described features, will be available in the last (third) article of this serie.

JSF 2.0 - "Hello World" in Eclipse (with *.xhtml pages)

2010-09-15T19:39:00.009+02:00

Requirements:

installed Java (description here)
installed and configured Eclipse (description here)
installed and configured Tomcat for the Eclipse (description here)

You will learn:

how to create "Hello World" application in JSF 2.0 using Eclipse

Note 1: The following post and example show classic JSF 2.0 application with a view technology as .xhtml pages. I do not use any additional libraries and extensions for the standard JSF libraries. I don't use (for the time being) templates. JSF implementation used comes from MyFaces.

Note 2: I use standard capabilities of IDE platform to generate the project, without those thousands automated plugins which do everything for the user. Sometimes configuration and usage of a plugin is a "pain in the ..." - in my opinion it is worth to know how to deal with a standard set of tools, and after that there is always a time to faciliate our life ;-)

In this post I will show how to create sample JSF 2.0 application using Eclipse. It will be similar application like in the post about creating sample JSF 1.2 application. But there will be some small differences, especially during generating the project and adding JSF capabilities to it. Let's do it.

Step 1: create new dynamic web project for the Tomcat. It is available in "web" menu section under option "File->New->Project":

Press "Next" to go further and set some basic project's properties:

Step 2: adding JSF capabilities for the generated project (JSF and JSTL libraries).

This step is the same as step 2 in previous JSF 1.2 example under Eclipse. The main difference prior to version 1.2 is that JSTL libraries are not necessary, unless application uses *.jsp pages with some standard JSP tags, and we have to choose JSF 2.0 MyFaces. JSF 2.0 support for Project Facets was added in Eclipse Helios and above. For previous Eclipse version (like Galileo) only JSF 1.2 was supported and configuration for JSF 2.0 needs to be done in alternative way (see below).

Note: this configuration is recommended in order to let the Code Assist for *.xhtml pages work properly. Code Assist allows automatically complete JSF tags when typying by pressing CTRL + Space. Unfortunately access to beans and their methods and fields from .xhtml page is not working (see bug: https://bugs.eclipse.org/bugs/show_bug.cgi?id=323709) unless beans are directly declared in faces-config.xml instead of annotations.

--------------------------------------------
Steps 2 in an alternative way:

We do not have to download requires .jar files for JSF, make user libraries from them in Eclipse and add those libraries to the generated project with "Project Facets" screen. The alternative way is to download all required .jar files and copy them into the generated "WebContent\WEB-INF\lib" directory. After that just refresh the whole project by pressing "F5" on the project's root and all libraries should appear in the section named "Web App Libraries" in the project's structure.

Note: this configuration (despite the fact it is faster) is not recommended, because of Code Assist which does not completely.

Step 3: the project structure is ready. Creating sample application.

This will be the same application as for version 1.2 with some differences coming from JSF 2.0:

- pages extenstion will be .xhtml instead of .jsp

- we use annotations in backing beans

- there will be no faces-config.xml file (now)

Everything should look like this:

File web.xml has to be changed like shown on screenshot below:

Step 4: deployment and running.

This step is similar to steps 6 and 7 from JSF 1.2 version. After deploying application on the server and starting the server, we have to open a browser and type in URL:

http://localhost:8080/FirstJSF2

That's all. We should have working JSF 2.0 application.

-------------------------------------------

Download source files:

Note: make sure that Java, Eclipse and Tomcat are properly installed and configured for running the project (additional configuration may be required if different directories are used).

Eclipse complete sample project is here (with all required libraries). You can also download a war file located here (just copy it inside webapps folder in Your Tomcat and start Tomcat with the script startup.bat)

JSF 2.0 - "Hello World" in NetBeans (with *.xhtml pages)

2010-09-15T18:26:00.004+02:00

Requirements:

installed Java (description here)
installed and configured NetBeans (description here)
installed and configured Tomcat for the NetBeans (description here)

You will learn:

how to create "Hello World" application in JSF 2.0 using NetBeans

In a previous post I showed how to create a sample JSF 1.2 application using NetBeans. Creating JSF 2.0 application under NetBeans is almost the same as creating JSF 1.2 application - the main difference is just selecting JSF 2.0 version instead of 1.2.

I will show how to create JSF 2.0 sample application similar to JSF 1.2 application.

Step 1: creating and configuring web application.

Now we have to define project name and its location:

After pressing "Next" we have to configure target runtime and context name:

Press "Next" in order to go to the JSF settings screen. Here we have to select JSF 2.0 version:

Step 2: project is configured. This is what NetBeans generated for us:

Just like for version 1.2, project contains all requires libraries and is ready to be deployed on the server.

Step 3: creating application.

This will be the same application as for version 1.2 with some differences coming from JSF 2.0:

- pages extenstion will be .xhtml instead of .jsp

- we use annotations in backing beans

- there will be no faces-config.xml file (now)

Everything should look like this:

Step 4: deploying on server and running.

This step is exactly the same as step 4 from previous post about sample JSF 1.2 application. We also don't have to start the browser manually to see the application. Browser will be started automatically with proper URL pointing to our application.

What next? This simple working project can be used as a base for further learning about JSF. For example we can change the view technology from .jsp to facelets. We can add libraries which extend our standard tags by adding tags with ajax support, i.e RichFaces.

JSF 1.2 - "Hello World" in NetBeans (with *.jsp pages)

2010-09-02T22:25:00.002+02:00

Requirements:

installed Java (description here)
installed and configured NetBeans (description here)
installed and configured Tomcat for the NetBeans (description here)

You will learn:

how to create "Hello World" application in JSF 1.2 using NetBeans

Note 1: The following post and example show classic JSF 1.2 application with a view technology as .jsp pages. I do not use any additional libraries and extensions for the standard JSF libraries.

In the previous post I described how to generate working sample of JSF 1.2 application using Eclipse and its capabilities. I decided to check how it looks like using NetBeans. And this was a nice surprise for me - it is easier than in Eclipse. I will show how to generate a working JSF application similar to application created under Eclipse.

Assumptions:

1. Installed Java (description here).

2. Installed and configured NetBeans (description here).

3. Installed and configured Tomcat for the NetBeans (description here).

Step 1: creating and configuring web application.

Press "Next" and on the next screen configure some basic options like name and directory for the project.

Press "Next" and on the next screen configure runtime for the project.

Press "Next" and on the next screen add JSF capabilities to the project. Select "JavaServer Faces" option and set up JSF 1.2 libraries.

Step 2: project is configured.

Screenshot below shows the project structure generated by NetBeans.

That's all. Project is ready to deploy on the server. You don't need to add and configure JSF or JSTL libraries like under Eclipse. Those libraries are already in the project. Anyway in order to show that everything works the same we will modify generated project and write some piece of an application like under Eclipse.

Step 3: creating application.

Now we are ready to implement some sample JSF application. First we should create a package named com.firstjsf.backingbeans in the "Source Packages" directory. Second we should create two classes inside the package named HelloMessageBean and WelcomeBean. Third we should modify generated faces-config.xml file by adding there our classes and navigation rules. Everything should look like this:

In the faces-config.xml file we created entries for the navigation rules for not existing yet web pages. Let's create those web pages (index.jsp and message.jsp) in the Web Pages directory:

Step 4: deploying on server and running the application.

We have to click only one icon - NetBeans will start the Tomcat with our application deployed:

Results of application running can be visible in the console. We don't have to directly tell the Tomcat what application to deploy like under Eclipse - this step is done automatically. Moreover we don't have to start the browser manually to see the application. Browser will be also started automatically with proper URL pointing to our application.

JSF 1.2 - "Hello World" in Eclipse (with *.jsp pages)

2010-09-01T20:45:00.018+02:00

Requirements:

installed Java (description here)
installed and configured Eclipse (description here)
installed and configured Tomcat for the Eclipse (description here)

You will learn:

how to create "Hello World" application in JSF 1.2 using Eclipse

Note 1: The following post and example show classic JSF 1.2 application with a view technology as .jsp pages. I do not use any additional libraries and extensions for the standard JSF libraries.

Many publications and tutorials about JSF (Java Server Faces) very briefly descibe the configuration of development environment for the described examples. Many times I saw such situation: after few sentences about "how amazing is JSF" there was a simple list of JSF tags with their attributes described. Or at the beginning there was an example of "Hello World" application and every line of code was described and explained.

I think that is what is missing is the lack of an example of complete project with its directory structure and an information how to create this project and deploy on the web server. I saw some examples where the project structure was shown, but for the building and deploying application user had to use Ant and type some commands. Great, but we need Ant, when we have Ant, we need to be familiar with it to write build.xml file, then we need something else and so on and so on.

This is of course OK if we want to write everything from scratch by hand. But wouldn't it better just to start our IDE, make some clicks and have a complete and working project ready for further development?

Let's see how it looks like in Eclipse IDE.

Step 1: creating dynamic web project for the Tomcat.

Open menu "File->New->Project" and in the "web" section find and choose "Dynamic Web Project" and press "Next":

Press "Next" to go further and set some basic project's properties:

After that we should have generated project structure for the web application.

Step 2: adding JSF capabilities for the generated project.

In order to do this just right click on the project and from the opened context menu choose "Properties" option. Locate the section "Project Facets" and check the option "JavaServer Faces".

Choose "Further configuration required..." option below to configure JSF libraries:

Choose "User Library" type and press marked icon to download JSF libraries. Eclipse Galileo allows to download two implementations of JSF: MyFaces anf from the Sun. Let's choose MyFaces:

Please note that we created a special directory for libraries in the Eclipse workspace - libraries will be downloaded to this directory. After pressing "Next" we have to follow the library installer instructions:

After pressing "Finish", JSF libraries are visible and ready to use in the project.

Step 3: JSTL libraries.

If JSP tags will be used in JSF application, JSTL libraries are required next to automatically downloaded JSF libraries. Unfortunately above automated solution for downloading JSF libraries does not download JSTL libraries, so we have to download them manually. I recommend to create another directory for JSTL libraries next to the JSF downloaded libraries in Eclipse's workspace directory. So create directory "libraries\JSTL" in Eclipse workspace directory, download JSTL libraries from there and put them into that directory. It should look like this:

Now we should create library from the downloaded JSTL files in the Eclipse in order to be able to use them in the project. It will be the similar operation to the creating JSF library - the difference is that we will do it manually. Just open the context menu for the whole project (right click) and go to the "Properties" screen. Then locate "Project Facets" section and find the "Add library" button.

Please note that library list contains JSF libraries added in step 2. Now create new library named "JSTL" like it is shown above, and add .jar files from JSTL directory:

Then make sure that those two libraries (JSF i JSTL) are added to the project (application):

Note: this configuration is recommended in order to let the Code Assist for *.jsp pages work properly. Code Assist allows automatically complete JSF tags when typying by pressing CTRL + Space. Also access to beans, their methods or fields is possible directly from JSP page (by click on bean/field name).

--------------------------------------------

Steps 2 and 3 in an alternative way:

We do not have to download requires .jar files for JSF and JSTL, make user libraries from them in Eclipse and add those libraries to the generated project with "Project Facets" screen. The alternative way is to download all required .jar files and copy them into the generated "WebContent\WEB-INF\lib" directory. After that just refresh the whole project by pressing "F5" on the project's root and all libraries should appear in the section named "Web App Libraries" in the project's structure.

Note: this configuration (despite the fact it is faster) is not recommended, because of Code Assist which does not work at all.

Step 4: the overview of a project.

When steps 1-3 where done as described, the project structure should look like this:

Please note that JSF and JSTL .jar files are visible as user libraries in created in steps 2 and 3.

Step 5: "Hello World" application.

Now we are ready to implement some sample JSF application. First we should create a package named com.firstjsf.backingbeans in the "src" directory. Second we should create two classes inside the package named HelloMessageBean and WelcomeBean. Third we should modify generated faces-config.xml file by adding there our classes and navigation rules. Everything should look like this:

Be careful about small and big letters - names should be exactly as shown above. In the faces-config.xml file we created entries for the navigation rules for not existing yet web pages. Let's create those web pages (index.jsp and message.jsp) in the WebContent directory:

Application is ready.

Step 6: deploying on Tomcat.

At the bottom of a screen locate "Severs" tab. We should have already Tomcat there. Right click on server name to open its context menu, ten choose option "Add and Remove...":

Choose our project and add it to the server by pressing "Add" button:

Step 7: running the application.

Go back to the "Server" tab again and make sure that our project is visible on the server. Then click the marked icon to start the server:

After starting the server we should see that the server state was changed:

After deploying application on the server and starting the server, we have to open a browser and type in URL:

http://localhost:8080/FirstJSF

-------------------------------------------

Download source files:

Note: make sure that Java, Eclipse and Tomcat are properly installed and configured for running the project (additional configuration may be required if different directories are used).

Eclipse complete sample project is here. External libraries folder (with JSF and JSTL) is here. You can also download a war file located here (just copy it inside webapps folder in Your Tomcat and start Tomcat with the script startup.bat)

"My databse is slow" - part 3: some technical stuff

2010-02-24T23:50:00.267+01:00

In the previous two posts I tried to show what can affect the overall performance of the database and how important are indexes.

If we assume that the database is normalized, memory settings for the server are ok and we have indexes, the only way to check the performance is monitoring the query plans. Therefore I would like to return once again to the query plans and look at them with more details.

From the previous post, we know that we can generate plans for queries using two commands. These commands are:

EXPLAIN SQL_command
EXPLAIN ANALYZE SQL_command - this command is a variant of the first with small difference: examined query is executed and EXPLAIN ANALYZE returns the real execuion time.

Using the test database from the previous post, let's see the query plan for the following SQL command:

select * from customers where lname = 'Cervantes';

To see the query plan, execute those two commands:

vacuum analyze;
explain select * from customers where lname = 'Cervantes';

Query plan that has been generated for this query is as follows:

Always we are most interested in "the highest" line of displayed plan - it shows the total cost of the query execution, after all other internal operations for more complex queries (eg joins, sorts, etc.).

In our case we have: (cost: 4.51 .. 59.47 rows = 33 width = 18). What does it mean?

cost: 4.51 .. 59.47 - this is an interval defining total execution time measured in special units called disk page fetches - these units are not seconds nor other units of time. First number estimates time needed to retrieve first row in the result, second estimates total cost of returning all rows in the result. The calculation of this value is "higher mathematics", especially if the query uses more techniques than primitive table sequential scanning - which is the easiest and can be easily calculated. Unfortunately, but only for primitive examples like "select * from some_table" (without the use of indices to be 100% sure that the scan will be performed sequentially and without any query conditions "where"), You can find over the Web some examples of calculating these values. Those who are interested in more details and technical stuff, should visit this page
rows = 33 - this is estimated number of rows returned by a query.
width = 18 - this is estimated size (in bytes) of all the returned rows

Above data are obtained on the basis of statistics located in a special system tables. There are two tables of statistics which are crucial for the query planner:

pg_class - contains information about certain database relations (tables, indexes, etc.), i.e. their location and size.
pg_stats - contains information about the data stored in tables, i.e. the most common values, the frequency of common values occurrence.

Let's see what are the statistics for the table "customers", and what is their impact on some elements of the query plan for above query about user "Cervantes".

Returned data mean that the table "cutomers" has 10.000 records and is located on 62 "pages".

Returned data are based on some statistical data for all the columns in the table "cusotmers" and their content. It is specified which values are most popular, what is their frequency, and amount of values which are different from each other.

What is the meaning of this data in the context of our query about "Cervantes" user?

pg_class - is the basis in calculating the cost of the query execution. For our "Cervantes" query in the case of sequential scan (if there were no index on the "lname" field in the Customers table), we would calculate the cost from the formula:

(relpages * SEQ_PAGE_COST) + (reltuples * CPU_TUPLE_COST) + (relpages * CPU_OPERATOR_COST)

Capitalized names are costs of arbitrary operation - the are defined in postgresql.conf. Unfortunately (or "fortunately"), our query uses indexes, so the calculation of the cost is a bit more complicated (task for volunteers, formulas are here, You only have to to be familiar with the concepts of "last recently used buffers" (LRU) and Mackert-Lohman formula ;-)).

pg_stats - allows to estimate the number of rows returned in a query using given logical condition (in our case: "where lname = 'Cervantes'"). In our case for the "Cervantes" query we are lucky - "Cervantes" occurs directly in the list of most common surnames, and also is known for its occurrence. The number of rows returned is calculated using the formula:

relpages * selectivity

where:

selectivity = most_common_freqs["Cervantes"]

Hence we get:

10000 * 0,00333333 = 33

And what if we are looking for value which is not in the column "most_common_freqs"? Then the formula has a different form:

selectivity = (1 - sum(most_common_freqs))/(num_distinct - num_most_common_vals)

Calculating selectivity value for the name "Smith", I leave for volunteers as usually ;-). Remember that first You have to execute "vacuum analyze" and then view statistics and then the query plan.

--------------------------------------------

Some notes about the statistics:

The content of statistics tables is generated based on random samples taken from the tables in the database. Generating (and therefore refreshing) full statistics takes place during the execution of the VACUUM ANALYZE command. Hence, it is very important to run this command periodically to ensure that statistics for the planner are up to date. Therefore, before showing the above example the first command executed by me was VACUUM ANALYZE in order to query plan be based on the most reliable statistics.

It should be noted that next execution of this command can generate a slightly different statistics, so query plan for the same query may be slightly different (in terms of cost and estimated number of rows). This is because of mentioned random samples taken for calculations.

--------------------------------------------

A few final notes about performance

I can say that the general rule to optimize query execution time is as follows:

"There is no certain way to increase performance in all cases. The best method is trial and error."

In practice it means to try to generate different query plans (in addition to that which is generated by default) and select those with the best execution time. These attempts usually involve forcing the query planner to behave in different way which leads to a different query plan. This can be achieved through:

1. Forcing the query planner to use an index (often the index exists but is not used by the planner). This can be done for example by:

switching off a global sequential scans by setting enable_seqscan = off in postgresql.conf file
partial indexes (conditional) - You can create a conditional index for the most frequently returned data in column
indexes using multiple fields (watch out for the order - the order of fields in the index should be the same as the order of fields in queries)
some SQL tricks related to change the query selectivity, for example using LIMIT, ORDER BY or trailing condition AND TABLE_ID = TABLE_ID (where TABLE_ID is a master key in the table)

2. Changing the order of joining multiple tables, for example executing "(A join B) join C" can be different than executing "A join (B join C)" - this is similar to a problem of "optmial matrix-chain multiplication".
3. Using stored procedures, views, breaking huge query into a number of smaller queries .

Interesting fact (found over the Web):

In order to monitor the progress of executing a long query to the database (a really long time, not a few seconds) you can create a sequence with the value set to 1 (eg, the sequence "monitor_seq") and add it to executing query:

select nextval('monitor_seq'), column1, column2....

Executing:

select currval('monitor_seq');

at the time of running the long query will show us the progress.

"My database is slow" - part 2: indexes.

2010-01-05T21:25:00.124+01:00

This post is a continuation of previous entry about database performance.

Suppose we have a database that is normalized to our needs, the memory of the server was also quite reasonably allocated, we are using a connection pool and we did not change the default transaction isolation level for our database - but the database still is not a "speed daemon". What can You do in this situation? Did we miss something?

You can look inside the database giva a closer look on specific operations performed on the data to determine which operations cause performance degradation and possibly "help" them perform faster. As the sample database I will use PostgreSQL - I will examine its specific mechanisms to check and improve performance. This will be only an outline of what can be done (based on crude example), because the topic is so large that it may well be placed on a separate dedicated blog.

Requirements:

installed PostgreSQL (description here)
a test database with sample data - just to show how PostgreSQL performs a very simple SQL opeartions

Let's consider a very simple database for some online store with two tables: customer table (called Customer) and customers' orders table (called Order). Just open pgAdmin and execute this SQL command:

CREATE DATABASE store ENCODING = 'UTF8';

Then connect to database "store" and run this SQL commands:

CREATE TABLE customers
(
customer_id INTEGER,    
fname VARCHAR(32),
lname VARCHAR(32) NOT NULL,
CONSTRAINT customer_pk PRIMARY KEY(customer_id)
);

CREATE TABLE orders
(
order_id INTEGER,
customer_id INTEGRE NOT NULL CONSTRAINT customer_order_fk REFERENCES customers(customer_id),
description TEXT,
price NUMERIC(7,2),    
CONSTRAINT order_pk PRIMARY KEY(order_id)    
);

For simplicity, here I do not use a SERIAL field to hold primary keys - just ordinary integer with constraint. Now we should fill the database with some sample data: lets put about 10 000 records to Customers table and 30 000 records to Orders table. That is rather huge amount of data. How can we do it?

The best way is to write a Java program that generates a file with proper SQL INSERT statements for each table - this is homework for You ;-)

I wrote simple program which takes popular first names for males and popular first names for females, then takes popular surnames from here and combines them together to produce 10.000 "people". Having those records and knowing their customer_id (that's why I used ordinary integer field - just to write a program) my program generated random 30 000 orders for those customers.

And now suppose we want to retrieve all customers named Smith and we want to see all orders for each customer Smith. We need two queries:

select * from customers where lname = 'Smith';
select * from orders join customers on orders.customer_id=customers.customer_id where customers.lname = 'Smith';

In my case, the first query returns just 9 customers named Smith (from total of 10 000) and 30 orders (from total of 30 000) for all 9 "Smiths".

Okay, we have SQL commands - but how can we check their execution time on a database? PostgreSQL provides two commands that show details about executed commands:

a) EXPLAIN SQL_command

b) EXPLAIN ANALYZE SQL_command - this command is a variant of the first with small difference: examined query is executed and EXPLAIN ANALYZE returns the real execuion time.

Before we show an example of above commands execution, some technical information about the process of executing the query by PostgreSQL: when the PostgreSQL server receives SQL command from client application, the query string is passed to the query parser that checks for syntax errors. If everything is ok, parser based on the query creates a structure called the query tree. Then, this structure is passed to the query planner which generates all possible query plans and chooses the shortest one from them. This plan is passed to the executor who performs the query and returns the records in the manner described in query plan.

Mentioned EXPLAIN command displays the details of the plan chosen by the planner as the shortest one. Let's see the result of executing EXPLAIN ANALYZE for the queries about users Smith:

Because reading query plans and their details is a separate art, we will focus on the actual run time (visible as a Total Runtime, respectively 1,482 ms for the first question, and 16,552 ms for the second question).

Lets provide indexes for the fields we are searching for and for the fields where the join operation is performed. In our case it will be:

a) the lname column of the Customers table - create index with the command:

CREATE INDEX lname_index ON customers USING btree (lname);

b) the foreign key column in Orders table (the column cutomer_id) - create an index using the command:

CREATE INDEX customer_fk_index ON orders USING btree (customer_id);

Note:

After You create indexes, You have to refresh the statistics used by database planner to calculate the optimal query plan. This is done by running VACUUM ANALYZE command. After executing this command, once again execute the EXPLAIN ANALYZE command for our "Smith" queries:

The results of the queries which are several times faster than previously. They are respectively: 0,123 ms (1,482 ms previously), and 0,507 ms (16,552 ms previously).

The advantage is obvious. But whether the index will solve all our problems? Not always. There is no guarantee that the planner will benefit from the index. Sometimes it appears that the sequential table scan can be faster than using an index (such behavior may depend on the "selectivity" degree of certain query). In most cases it is good to have indexes - they can only help.

"My database is slow" - part 1

2009-10-12T20:18:00.256+02:00

Perhaps You often heard complaints that some application is veeeery sloooow. Despite the technology used, bought another X GB of RAM and so on the problem still exists. Sometimes the cause can be very simple - interaction with the database can be a bottleneck and a source of all problems. This can be checked and possibly improved.

Often the developer itself must change something in the database - do not be afraid to say it - hire a database magician cost a lot of money for the company and often the developer becomes a database specialist ;-). Therefore, anyone who changes something in the database must have some knowledge. Otherwise, some ad-hoc changes can cause a lot of performance problems.

What can you do then? Having the database, You can take a closer look on some points:

1. Database normalization (decomposition).

Simply said - making a database to be in the n-th normal form. You may naively believe that "the database engine can handle" somehow our SQL query, without worrying ourselves about performance. Theoretically yes, even nightmarish database schema will work somehow - after all, a bit smarter (than the rest) people formed the principles of relational databases. The problem starts when our application is no longer a primitive data browser which executes the same "select" SQL query all the time. That's why the other smarter people have invented a normalization. For example this allows the database engine not to retrieve 20 columns from one table just to show us only those 2 which we are interested.

For example let's assume that we have a company and we store all the employees in a one big table:

CREATE TABLE employees ( 
id                int4 PRIMARY KEY,
active            boolean, 
first_name        varchar, 
middle_name       varchar, 
last_name         varchar, 
ssn               varchar, 
address1          varchar, 
address2          varchar, 
city              varchar, 
state             varchar(2), 
zip               varchar, 
home_phone        varchar, 
work_phone        varchar, 
cell_phone        varchar, 
fax_phone         varchar, 
pager_number      varchar, 
business_email    varchar, 
personal_email    varchar, 
salary            int4, 
vacation_days     int2, 
sick_days         int2, 
employee_number   int4, 
office_addr_1     varchar,
office_addr_2     varchar, 
office_city       varchar, 
office_state      varchar(2), 
office_zip        varchar, 
department        varchar,
title             varchar,
supervisor_id     int4
);

And now some Secretary wants to find those people who earn more than 5000 USD:

SELECT first_name, last_name from employees where salary > 5000 ;

Let's assume that the company is a big corporation which employs thousands of people. Database engine must scan entire table, retrieve all columns for people who earn more than 5000 and at the end pefrorm a projection and return only 2 columns - first name and last name.

Note: do not kid ourselves - normalization is not an ideal remedy for performance problems, and sometimes may even contribute to decrease a performance. Why? For example let's take huge reporting queries that collect a lot of cross-sectional data. In the fully normalized database, the amount of join operations between tables can be very large and could have significant impact on performance. Sometimes it is better to have non-normalized parts of database schema, or even have non-normalized redundant data tables (for generating reports) next to normalized parts. There is no silver bullet solution for that - the approach depends on specific problem.

2. Memory configuration for the server.

The fact that Your hardware allows You to play latest games, does not mean that the database will work fast. Even 10-th normal form of a database schema and super-optimal queries will not help when server has total memory similar to required by old DOS games. Sooner or later because of lack of memory many I/O disk operation will be performed to compensate memory request. System resources are important. Unfortunately, at this point databases vary widely in terms of available configuration options. "No silver bullet" also applies here - the only way to discover best performance is to tune database memory settings and test it against SQL queries.

3. Opening connection to the database.

Although the authors of JDBC drivers maintain that connection cost is small and well optimized, we must not forget that after all this is some expense. As an example, consider a typical web application where in every moment a user clicks on different screens, which show him various data. We have to approaches here:

maintaining an open connection during user's working time
opening connection for each request, performing some database query, returns the result (which is presented to the user) and closing the connection.

First approach is expensive (in terms of resources and performance), second is much better. But when we take into account the large number of users and their actions, total cost of opening and closing connections may have considerable importance. It is worth here to consider using the so-called connection pool - such as Apache DBCP or C3P0. Responsibility for creating/closing of connections goes to connection pool. The pool itself has various configuration options that can increase the efficiency of cooperation with the database application. For example some connections may have been opened after startup and ready to immediate use and so on.

4. Transaction isolation levels and locks.

Transaction isolation levels have a direct impact on the types of locks used. If You are not sure how to choose the level of transaction isolation, it is better to remain at the default level for a particular database. You have to remember one important thing: the higher isolation level you choose the more subtle and complex lock must be use by the database engine to provide concurrent access to data. Using higher levels of isolation can also cause deadlock occurrence.

Note: You may not agree with mentioned points. They are only my personal subjective choice based on problems and observations during my work. Database performance is a very broad issue dependent on many factors - I pointed only four general issues, but You may find another four or more which will be just as important and necessary.

Installing and configuring Tomcat for Eclipse and NetBeans

2009-05-04T20:37:00.174+02:00

Probably You always wanted to write your own web-based application in Java (ok, read: writing another Facebook-like portal and live in luxury in Dubai to the end of Your life ;-)). Of course it is possible, but before that inevitable moment occurs, You have to do much more simple thing - configure the environment for developing web applications (unless You are headmaster in Your own company and someone else does it for you...).

So let's look at how to configure Apache Tomcat which is professionally defined as the servlet container (i.e. for JSP). I will show how to configure it under Eclipse and NetBeans.

--------------------------------------------

Eclipse:

Recall that so far we have managed to install Java and set up Eclipse to work with it. It is time for Tomcat.

Step 1: download the Tomcat binary distribution (core version as a .zip file) from here (use current Tomcat 6.x version, 6.0.32 - edited on Tuesday, July 26, 2011)

Step 2: Unpack the .zip from step 1 to the C:\Development\Tomcat_Eclipse directory. We should get something like this:

Step 3: Open Eclipse IDE and the Java EE perspective (Window menu -> Open Perspective -> Other -> Java EE). This will be the default perspective for our work. Using this perspective, at the bottom in the "Servers" tab we add a new server:

Step 4: Configure the new server. Select the type of the server in accordance with step 1 (Tomcat 6), and leave the name set to localhost:

Step 5: Further configuration of the new server. Select the server's installation directory and Java virtual machine which will be used - in this case it will be the same virtual machine which we configured to work with Eclipse (description here).

Step 6: Basic configuration of a server

First we allow the Eclipse to manage Tomcat installation (so we can start and stop the server directly from Eclipse). Second we change the "deploy-path" from proposed to the Tomcat default webapps directory (in our case C:\Development\Tomcat\webapps) - it is the default directory where Tomcat keeps web applications. The third and final thing is to change the way Tomcat publish applications on the server - each application will have a separate xml configuration file in Tomcat (so-called "context" file).

Tomcat is ready and configured to work. We can start and stop it using the icons on the right side in the "Servers" tab. The results of these operations (logs), are visible in the "Console" tab.

--------------------------------------------

NetBeans:

We do not need to install Tomcat under NetBeans - provided that we followed instructions from this post. We should have Tomcat installed.

If we did not install Tomcat during NetBeans installation, we can do it similar like for Eclipse. Just unpack the Tomcat to specified directory, start NetBeans and go to the Tools -> Servers -> Add Server:

The rest of the installation steps are similar to that of Eclipse and it should not cause trouble.

Installing PostgreSQL (hard version)

2009-04-22T00:39:00.027+02:00

In a previous post I showed how to install PostgreSQL using Windows installer. It was quite easy. But ...

There are people that do not play with such ease, what is more, they can not stand when the system is doing something for them in the background, when something is installed - some services, user accounts... They feel insecure, they are suspicious - only the execution of all activities from the command line calms their nerves.

This time, I prepared the description for the people who love technical masturbation and who love to do everything manually and have everything under control. We'll become the PostgreSQL installer and manually execute all the installation steps to get an effect similar to using Windows Installer. So, let's get to work.

Requirements:

we are working on the account with Administrator rights (or we belong to the Administrators group)
database will be installed to C:\Development\PostgreSQL (the same as when using Windows Installer)
disable all antivirus/firewall (this one from Windows too)

Step 1: download the version of the installation file (.zip) of PostgreSQL from here. Download the file with the "binaries-no-installer" part in its name.

Step 2: extract the contents of the file from step 1, to C:\Development\PostgreSQL (zip has the pgsql directory and rest of the folders inside it - I recommend to get rid of it and move all subfolders to C:\Development\PostgreSQL - we will have a shorter path). We should get something like this:

Step 3: we need to create an empty directory named "data" in the directory C:\Development\PostgreSQL (along with the rest of the directories from the picture above).

Step 4: create and initialize the database cluster (remember that we are working on the account with Administrator rights). To do this go to the console to the directory C:\Development\PostgreSQL\bin and execute the command:

Desription:

option "-U root" means the owner of the database named root - such user account will be created in database (as a superuser)
option "-W" means that You will be prompted for a password for this user
option "-D .. \ data" means the location of the files for generated database
option "- encoding = UTF8" means that the server will have UTF8 encoding by default (each newly created database too!)

After executing this command, we should see sometling like this:

At the bottom, PostgreSQL offers us to run a newly created database using two different commands. It will not work (You can execute those command to see what error is generated) - remember that we are working on account with Administrator rights, and for safety reasons, we can not start and stop the PostgreSQL server on that account. We should create a system service which will run PostgreSQL server process.

--------------------------------------------

The short version:

Step 5: Perform the following command in the console:

That's all. You're done. The service will be created and registered for the Local System Account. This is a special account which can run services, but it is not a typical account with administrator privileges (more info here).

Do not run the service yet.

--------------------------------------------

Long version:

Step 5: continue to imitate the installer. PostgreSQL installer creates a special account with no administrator privileges - so we will also create such account. Acount login and password will be both set to "postgres". We go to Computer Management, then Users:

Now we add new user with login and password postgres:

For the security reason we do not allow to change the password.

We would like now to make "postgres" account able to run the PostgreSQL server. So we need to create and register a special service for this account, which will carry out these operations. Before we do that we need to assign to this account the appropriate permissions to directories where database server is installed.

Step 6: Change the access permissions for the directory server for the postgres account.

And so:

1. Directory C:\Development should only have read permission.

2. Directory C:\Development\PostgreSQL (and its subdirectories) should have all possible permissions.

Step 7: Registering the service.

Using the console go to the directory C:\Development\PostgreSQL\bin and execute there this command:

Description:

option -N "PostgreSQL Service" defines the service name
option -U postgres says on what account the service is launched
option -P postgres gives the password for service account
option -D "C:\Development\PostgreSQL\data" tells where database cluster is located (be careful it is very important here to give the full path to the directory "data")

We do not run the service.

--------------------------------------------

Configuration:

Before starting the service must change the basic settings. Move to the directory C:\Development\PostgreSQL\database and open the file postgresql.conf. In this file, You must find and uncomment the following lines:

listen_addresses = 'localhost'
port = 5432

Now You can start a service in the panel management services.

If everything was done properly, the service should start. If we chose the longer version, in case of trouble with starting the service we should go to the configuration of a service, then select an account for this service and retype password for that service (in our case: postgres)

Theoretically we have the ability to log from the console to the database using the command line (marked red colour):

Thus we should get exactly the same result as using the Windows Installer.

Notes: a short way works ok under Windows XP Prof., should also works ok under Windows Vista - in both systems there is the Local System Account for services. For Windows 2000 You must perform the installation in a long way. For safety, PostgreSQL installer always chooses longer way - it creates an account for the service, then creates sevice itself and registers it to the created account.

Installing PostgreSQL (soft version)

2009-04-19T19:17:00.016+02:00

Installing PostgreSQL is theoretically piece of cake, but in my opinion there are some aspects worth saying a little more.

Those who installed MySQL database using the graphic installer on Windows, know that the installation process goes something like this: "Next, next, next, yes I want to have root access from remote machines, quit." This is almost the same as installing any program on Windows. PostgreSQL installer is not much more different in this case.

So what's the point? In a slightly different philosophy of security model - different than in MySQL. Okay, enough of this pseudo-technological gibberish - let's install database.

Step 1: Download the PostgreSQL from here. Download a file named postgresql-8.3.7-1.zip.

Step 2: After unpacking the zip file, run the installer and choose English

Step 3: after the splash screen and license information installer will ask us about the target installation directory. I suggest using directory C:\Development\PostgreSQL.

Step 4: Configure account for the PostgreSQL service.

It is convenient to install PostgreSQL as a service in Windows - the database server will always run at startup. We have something new in this point: PostgreSQL requires a special account in the system that does not have administrator rights in order to start the database service and initialize the database cluster. It is important to set a well-known password for this account, because in case of problems with PostgreSQL we will be able to use this account to peform some fixes on database files.

Step 5: configure the settings of the database server, super user account (root) and access from remote machines.

This is a step similar to step in the installation of MySQL, I recommend setting the server and client encoding to UTF-8 - this is the standard that should be use instead of some bizarre national encoding. Why? Because it gives us the flexibility - perhaps we want to store in our database accented Polish and German umlauts. In addition we will avoid potential problems with conversions etc.

Step 6: Installation.

Warning: before installation make sure that You have enabled and running Windows service named "Secondary Logon". Our system account "postgres" from step 4 needs that service. When it is not enabled, the installer will report an error:

Just turn on the service and the trouble is gone. Then on next screens just click "Next" without changing anything until You get to the last screen - the final step in our installation process.

Step 7: The end of the installation.

The selected checkbox allows You to install additional software and extensions for PostgreSQL. At the moment we do not need them so we can safely uncheck it and complete the installation - voila, we have PostgresSQL :-)

Okay, once we have database installed, let's take a look inside - just to write simple SELECT and be proud of Yourself that You still remember some SQL syntax ;-) Nothing could be more simple. Along with PostgreSQL was installed pgAdminIII - an advanced graphical client. Just find it in the windows start menu and run:

When You double-click the server, You will be asked for Your root password from step 5. After typing the correct password, You will be logged to the server as root (superuser):

PostgreSQL creates a default database named "postgres". We can log in to this default database using command line. Of course, we can also delete this database. Then we have available two more so-called "service" databases that are hidden (they are not visible by default in pgAdmin tree view). The names of those databases are "template0" and "template1". PgAdmin allows us to configure the server, manage users, roles, databases, etc. For details, refer to the documentation.

IDE (Eclipse & NetBeans) installation, configuring Java for both

2009-04-18T00:40:00.030+02:00

Well ... There is really no big deal here. But in order to follow the rules from initial post I will describe how to "install" (in this case is a strong word ;-)) Eclipse Galileo and NetBeans 6.8 and configure it to work with previously installed JDK (JDK install as described here). So:

-------------------------------------------

Eclipse:

Step 1: Download the latest Eclipse. Download exactly the Eclipse IDE for Java EE Developers (use current Eclipse Indigo version - edited on Tuesday, July 26, 2011).

Step 2: Once we have the Eclipse, unpack the package to C:\Development\Eclipse - and that's all Eclipse is installed :-)

Now it's time for a basis configuration:

Configuration Step 1: After first startup Eclipse will ask us where we want to put the workspace directory. I suggest using this directory C:\Development\Eclipse_Workspace.

Configuration Step 2: After closing welcome screen, Eclipse will stay with default Java EE view perspective. It is OK, now it's the time to configure JDK. go to the Window -> Preferences menu and select:

Configuration Step 3: adding new standard virtual machine (JVM).

Configuration Step 4: we show the location of our previously installed JDK (description of the installation here).

Configuration Step 5: switch to the installed JDK.

Configuration Step 6: Configure Eclipse startup parameters.

Search for the eclipse.ini file and at the beginning of the file add these two (yes two!) lines:

-vm

C:\Development\Java\bin

By doing this we set the default virtual machine which Eclipse uses. Please note that this is exactly the same Java JDK which was configured in the above step 4 (here in addition we point into directory "bin").

Eclipse is installed and ready for use.

-------------------------------------------

NetBeans:

NetBeans (unlike the Eclipse) has the graphic installer which allows to configure this IDE during installation process. Let's see how it looks like:

Step 1: Download the latest NetBeans. Download the exact version for Java (not for Java SE).

Step 2: Run the installer:

The installer shows us what we want to install, along with predefined servers. Let's change a bit this list by selecting "Customize":

We throw support for Groovy, and we add Tomcat 6.0.20 (included with the NetBeans as GlassFish).

Step 3: Setting target installation directory for NetBeans, and setting Java installation directory which denotes what Java will be used to work with NetBeans - please note that this is the same Java JDK like for Eclipse above. NetBeans automatically finds this JDK based on the variable JAVA_HOME (we created this variable during JDK installation).

Step 4: Configure the installation directory for the installed servers (Tomcat 6.0.20 and GlassFish v3)

An that's all. The installer should install the IDE and notify us about successful installation at the end. After first startup it is worth to check installed plugins and if they are enabled or not (especially Java Web and EE plugin). In order to do this, go to Tools -> Plugins -> Installed. Make sure that all plugins are enabled.

Installing Java

2009-04-17T23:55:00.031+02:00

It may seem that installing Java is a piece of cake. Really? Let's see ...

Step 1: Download the Java JDK from here. Download exactly Java SE Development Kit (JDK), without any additives inside (bundles) - just regular JDK.

Step 2: We start the installation. After two (or maybe three) screens we come to the screen where we choose the installation location. Let's change the target directory to C:\Development\Java:

Step 3: throw away the Java DB database - we will use our database (it saves about 25MB on disk - better than nothing

Step 4: Install the virtual machine (JRE) for the browser - leave the default target unchanged (C:\Program Files\...).

So far so good and suddenly ... I got the magic error message at the last stage of installation (JRE) - something about a broken installer. I ignored it by clicking OK. In theory, the installation is completed. But I looked inside "Add/remove programs" under Windows just to check my new JRE - and I see no JRE (although JDK was present)...

I uninstalled Java and I moved installer file from the desktop directly to the C drive. I repeated the whole installation procedure - this time without any problems. Perhaps it was something with white spaces in my path to the Desktop (C:\Documents and Settings) where I run the JDK installer. Like I wrote here - it's better not to use paths with whitespaces.

Step 5:

We create new system variable named JAVA_HOME and we set its value to C:\Development\Java

Gather all hammers inside toolbox.

2009-04-17T22:44:00.014+02:00

As I wrote in my first post, we will start from installing the necessary tools. What are we going to install? For today, it will be (when I get something new this post will be changed):

Java JDK
Eclipse IDE and NetBeans IDE
PostgreSQL database
Apache Tomcat and GlassFish

Some comments why we need those tools. Well, for Java JDK comment is probably redundant. I will use certain IDE I depending on the problem and offered improvements - and sometimes depending on my mood ;-).

PostgreSQL - why just this database ("everyone knows that MySQL is the best, fastest and for free")? It is my personal choice - I used those two databases, but it seems to me that PostgreSQL is more polished and more meets my needs. And most important: it is completely free (I am not sure if MySQL is free for every case). Okay it is clear, but why do we need a database? Just in case we would like to write some sophisticated data storing application. Database will be very useful in such case ;-).

Okay, but what about Tomcat or GlassFish? Do we need them? Yes we do - just in case we would like to move our super data storing application into the internet (meaning: create another online store and earn a lot of money ;-)). For the most of examples (connected with web applications) Tomcat will be sufficient, but when it's time for something complicated and we need a real application server, GlassFish will be used (or maybe even JBoss).

Okay okay, but where should we install those tools? In order to have consistent solution I suggest put everything in one directory, and create some subdirectory structure inside.

As You see I decided to put everything inside directory named "Development" on my C drive - C:\Development.

Good advice for the future: do not use paths with whitespaces and any national characters. This will save our time for searching and fixing some X-Files problems ;-).

The Beginning

2009-04-17T22:03:00.008+02:00

First of all I would like to apologize for my english. I decided to write this blog in english for one simple reason - more people will find it and read it ;-)

You should already now what is the idea of this blog (if not, You can read about it on the left). I assume that You did something using Java and You have a minimum knowledge of the language itself. But even if You do not, You may find some articles useful for understanding the Java language and other technologies.

How will it look like? At the beginning we need a workshop and tools. I intend to show You how to install everything from scratch in order to be able to do anything with some frameworks, libraries, etc. In other words: I will start with well-known "Hello World". After that I will add some bricks to build the whole wall: extend existing threads by adding new issues, wrote about the potential problems and so on - towards the complete solution.

Why do all of this on such elementary level? Because sometimes the basics are the worst problem to solve - it is often enough to discourage someone and cause him to give up. Perhaps we loose potential genius... who knows?

My goal is that all examples shown here are possible to execute by Yourself. Their purpose is to serve as a quick-start for something more serious.

That's all. Let the mortal combat begin...