Roll your own Continuous Integration System (C.I.S.): Artifactory installation, Linux, MySQL and Tomcat 6.x

Roll your own Continuous Integration System (C.I.S.)

Content:

Abstract

Install Tomcat

Basic Tomcat configuration - Memory

Basic Tomcat configuration - JMX

Basic Tomcat configuration - Application Manager and permissions

Apache and uSVN

Installing Artifactory from WAR

Configure Artifactory and MySQL

Configuring Artifactory security and repositories

War installation

There are numerous well-formed tutorials about how to install Artifactory as the chosen Maven Dependencies Repository of our Continuous Integration System.

I will summarize their steps as concisely as possible:

• Create a directory with owner tomcat.tomcat (user/group) where Artifactory is meant to put all its information and artifacts. Let's say... /var/artifactory
• Edit /usr/share/tomcat6/conf/tomcat6.conf and add the variable anywhere in the file. I prefer at top. ARTIFACTORY_HOME="/var/artifactory"
• Download last version of OSS Artifactory (Or pay the license if you find it worthy, of course).
• Unzip and install artifactory.war into your Tomcat6, remember how?
• you may copy artifactory.war into /usr/share/tomcat6/webapps/ if you have direct control over the filesystem, or
• you may use the Manager (/manager/html) application if you installed it previously
• Once artifactory is copied, automatically is run, and you should see in your ARTIFACTORY_HOME how some directories have appeared. Now you have a secureless, derby-managed Artifactory installation not ready for production, but perfect for testing.

A trip to a default configured Artifactory

Some default features:
- It supports Maven, Ivy and Gradle systems. We will only use Maven here.
- Artifactory provide several remote dependency repositories, and it will act as a proxy downloading and storing from those repositories for you.
- A dead-simple security schema. It is possible only for admins to upload files. User:admin, Pass: password. Shhh, it's a secret ;)
- Search engine for classes, packages and other files.


Welcome Page:

Do you need more explanations?

Maven settings:

As said before, Artifactory can serve to Maven, Gradle and Ivy.
See the Home tab -> Left menu -> Client settings -> Maven Settings. (You are allowed to nose around Ivy and Gradle, but don't tell me :P )
Next, you have some sections to declare and put in your settings.xml configuration file. It's a bit early for this, don't worry, only keep it in mind.

Artifacts:

Some layouts to improve your feeling in searches (What the fuck, so much time reading Microsoft marketing), mainly they are offered to you both tree and list structure.

The tree layout is created through the groupId + artifactId, being the default repositories the roots of the trees, one for each repository.

Notice that if you click over a repository, you will get a Maven snippet for your pom.xml, for example:


<distributionManagement>
    <repository>
        <id>linux-wdx0</id>
        <name>linux-wdx0-releases</name>
        <url>http://localhost:8080/artifactory/libs-3rd-party</url>
    </repository>
</distributionManagement>


If you paste the snippet in your pom.xml (remember, only one distributionManagement per pom.xml, so, if you already have one, you will have to merge them), you will be able to upload (if you are authorized, of course) generated artifacts to that repository with "mvn deploy" in deploy phase.

Default repositories in Artifactory:
libs-release-local: Your libraries and products go here, only releases.
libs-snapshots-local: Your libraries and products go here, snapshots.
plugins-release-local: Your plugins, or plugins needed by you, releases, here.
plugins-snapshots-local: Your plugins, or plugins needed by you, releases, here.
ext-releases-local: 3rd party libraries, releases, needed by your projects. (i.e. Spring, ojdbc driver.. )
ext-snapshot-local: 3rd party libraries, snapshots, needed by your projects.

Artifact resolution in Artifactory:
I do not like to simply repeat what is already written and it is not improvable by me:
http://wiki.jfrog.org/confluence/display/RTF/Understanding+Repositories

Here finishes first chapter about Artifactory. Coming soon:  Simple security and customization of repositories.

Roll your own Continuous Integration System (C.I.S.): Basic Tomcat configuration - Memory

Roll your own Continuous Integration System (C.I.S.)

Content:

Abstract

Install Tomcat

Basic Tomcat configuration - Memory

Basic Tomcat configuration - JMX

Basic Tomcat configuration - Application Manager and permissions

Apache and uSVN

Installing Artifactory from WAR

Configure Artifactory and MySQL

Configuring Artifactory security and repositories

In previous post, you installed Tomcat and let it running in your computer. Now it is time for a first configuration. We are going to focus in three usual needs.

1) We need more memory, always.
2) We need to monitor our system, we will use built-in JMX server.
3) We need permissions to get the manager application run. That manager application will make it easier the deploy task in near future.

Enlarge your memory
In spite of the amount of memory your system has, Java limits some thresholds for its applications. Default size for Sun's JVM PermGen space are 64M, and this is little memory for Artifactory, Hudson or Sonar. I do not remember which gives the problem :P

Objectives:
- Avoid PermGen exceptions by classloaders or application leaks, at least, in development environment.
- Increase available memory, some of the applications we will install are really hungry. Our Continuous Integration System will do some compilations which take much memory and we will foresee that need.

• For any Linux Tomcat6: Open /usr/share/tomcat6/conf/tomcat6.conf (or $TOMCAT_HOME/conf/tomcat6.conf if you installed it in other folder).
• Check that JAVA_OPTS is not set, I mean, there is a "#" in front of every JAVA_OPTS= line or any JAVA_OPTS line exists. If it is really not set, add this line:
  
  JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -XX:PermSize=256m -XX:MaxPermSize=256m"
  
  else, just merge available with new parameters, but if the system is just installed, it shouldn't exist.
   
• For Windows Tomcat6: Easier to find, Start->All programs->Apache Tomcat 6.0->Configure Tomcat. Then click "Java" tab.

•  Check if any of next parameters are set, I mean, if that line exists. No "#" possibility. If the parameter exists, replace with new value, else simply add.
• -Djava.awt.headless=true 
• -Dfile.encoding=UTF-8 
• -XX:PermSize=256m 
• -XX:MaxPermSize=256m

What have I added?:
All these are system parameters, they do not do anything by themselves, but that information is available for all applications and JVM. This way, when an application or the JVM itself is running, they can read system parameters and change their behavior.

java.awt.headless=true:
Headless mode is a system configuration in which the display device, keyboard, or mouse is lacking. Sounds unexpected, but actually you can perform different operations in this mode, even with graphic data.
http://java.sun.com/developer/technicalArticles/J2SE/Desktop/headless/

-Dfile.encoding=UTF-8:
You are specifying the default encoding for this JVM. From now on, if anything different is said, your applications will read and write and String will be built using that internal encoding.
By specifying this, you do not have to speculate with customer OS-dependent configuration.

-XX:PermSize=256m and -XX:MaxPermSize=256m modifies structure and behavior of JVM memory. I owe you a better explanation.


More options:
http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html

Tomcat6 performance guide:
http://tomcat.apache.org/articles/performance.pdf 

Roll your own Continuous Integration System (C.I.S.): Install Tomcat

Roll your own Continuous Integration System (C.I.S.)

Content:

Abstract

Install Tomcat

Basic Tomcat configuration - Memory

Basic Tomcat configuration - JMX

Basic Tomcat configuration - Application Manager and permissions

Apache and uSVN

Installing Artifactory from WAR

Configure Artifactory and MySQL

Configuring Artifactory security and repositories

As we said in the previous post, we are building our C.I.S. in the top of a Tomcat 6.x, it is not the best server in the world, but it is open, free and widely supported by a vast community and thousand of different running project all over the world, so, it is not that bad either.

There is no point to explain deeply how to install Tomcat, I will write my own installation in order to reproduce all steps in the future if necessary. You may also use this for a easy set-up of your system.

Pre-Requisites: Java JRE 1.5 (No matter what implementation, I have proofed both Sun and OpenJDK virtual machines with the same problems :P ).


For Windows XP:
Download: http://apache.rediris.es/tomcat/tomcat-6/v6.0.30/bin/apache-tomcat-6.0.30.exe
Execute it!

1) Accept Welcome
2) Accept License
3) Minimum installation: Tomcat and Manager
4) Select execution port (8080 in my case), user and pass manager (tomcat/tomcat, of course), and role (let manager-gui).
5) Select your JRE 1.5 installation path.
6) Select your desired installation path for Tomcat.
7) Confirm install.

If you want to see your Tomcat running, you have to get "Apache Tomcat 6.0" menu. (Start->Applications in Windows XP), then execute "Configure Tomcat".

And press "Start" button. Then you will be able to see Tomcat running at http://localhost:8080
Change "Startup type" from "Manual" to "Automatic" in order to get it as a service. Windows will run Tomcat every time you boot Windows.


For CentOS 5.5
Easier than Windows :P

(log as root)
cd /etc/yum.repos.d/
wget http://www.jpackage.org/jpackage50.repo
yum --disablerepo=jpackage* install jpackage-utils
yum install tomcat6 tomcat6-webapps tomcat6-admin-webapps

Beware with permissions, due to your TOMCAT_HOME will be /usr/share/tomcat6, so you will need to change the owner of that directory if you want to write into, next post I will explain a good configuration.

For OpenSuse 11.3
Easier impossible:

(log as root)
zypper install tomcat6 tomcat6-admin-webapps tomcat6-webapps.

Beware with permissions, due to your TOMCAT_HOME will be /usr/share/tomcat6, so you will need to change the owner of that directory if you want to write into, next post I will explain a good configuration.


We will use and modify this installation a few times before finishing this tutorial. Be patient, the result will be worthy it.

CheckStyle Eclipse plugin. Installation

Content:

CheckStyle Eclipse plugin. Installation

Tunning your CheckStyle and Formatter

Every programmer working as only one

What if... I get a "Fileset from project [projectname] has no valid check configuration" Error?

CheckStyle configuration distribution

In the lollipop world, your code and my code are completely equal, it is also well formatted, well documented and optimized for every executer machine. In that world, bugs does not exist, clients only do what they are supposed to do and the big boss never downplays your suggestions.

In a perfect world, your code and mine differs but not so much, your cat never walk over your keyboard and your pretty workmate is in love for you.

In this very real world, the stupid mate will not format a line correctly unless Damocles insert his sword in his back-door, clients bring up over your user manual and, probably, over the inexistent specification. Ups, I almost forget it, your ugly workmate is sexually harassing you.


In order to pacify this war field, the first tool I bring to you is CheckStyle Eclipse Plugin.


Description from its webpage: Checkstyle is a development tool to help programmers write Java code that adheres to a coding standard. It automates the process of checking Java code to spare humans of this boring (but important) task. This makes it ideal for projects that want to enforce a coding standard.

The installation is quite easy, as said here, in your Eclipse/SpringSource just

1. In Eclipse go to Help->Software Updates...

2. Add a new installation site and providehttp://eclipse-cs.sf.net/update/ as location URL.

3. Mark the plugin version you would like to install, as well as future optional features, then press Install...

4. Review and confirm the plugins to install

5. Restart Eclipse



Afterwards, right click over your project -> CheckStyle -> Activate CheckStyle. Then you might find something like this:




You are able to see my best code ;) (I am sorry for Spanish description).

A bit summary of warnings in the code window:

-Javadoc is missing
-Missing package-info java file
-Comment is like to-do comment
- { should be in previous line
...
among others.


Firstly, with a Source -> Format (Ctrl + Shift + F) we will keep aesthetic warnings low.

I pass from 34 to 26 warnings, check how "{" have been moved and now they pass CheckStyle filter.

The Formatter and the Checkstyle Plugin must be synchronized, otherwise every formatting you do, and you should delegate format in tools like this, you will gain CheckStyle warnings in an endless war.

Benefits for a single programmers are limited, but benefits for a entire team that must work together are incalculable. If not this, tools like CheckStyle (and an automatic code formatter) are obligatory in every development department.


An article about Tunning CheckStyle in the oven...

Basic Artifactory installation

On Un*x:

http://wiki.jfrog.org/confluence/display/RTF/Installing+on+Un*x

On Windows (I really like this one)

http://www.theserverside.com/news/1364121/Setting-Up-a-Maven-Repository