Roll your own Continuous Integration System (C.I.S.)
Content:
Understand that it is necessary, you need it, you deserve it, you want Subversion to control your code revisions. But you are clumsy enough (just as me) to be able to build your own repository at home with svn and Apache.
Thank God I found some brilliant tutorials and products, all of them, of couse, open source. I was intended to use only Tomcat6 and war packaged software, but both Apache and uSvn really deserve some Megas in the hard drive of your heart (and in the Continuous Integration Server's one).
I liked it too much because, mainly, because when I read about an easy tutorial, it takes me some days or weeks to accomplish its goal, ufff, I am not exaggerating, I am often silly. But it didn't happen this time with this uSVN tutorial and product.
By the way, what is uSVN?
"User-Friendly USVN is a web interface written in PHP used to configure Subversion repositories.It's goal is to ease the creation of new projects without having to use the command line interface, therefor, maybe without privileged access to the server. USVN will then generate the list of users allowed to access your source code. This enable the delegation of rights to administrate Subversion repositories."
From its official web:
https://trac.usvn.info/wiki/Documentation/Install
Another point of view from one of my favourites and abandoned blogs:
http://killerconfigurations.blogspot.com/2009/08/easy-svn-web-administration.html
Enjoy :)
lunes, 21 de febrero de 2011
domingo, 20 de febrero de 2011
Roll your own Continuous Integration System (C.I.S.): Basic Tomcat configuration - Basic application manager and user permissions
Roll your own Continuous Integration System (C.I.S.)
Content:
Our tomcat installation provided an application manager, with all basic function to administrate the life cycle of our applications.
Top advantages:
1) No need to leave applications in a folder to deploy them, this application will copy there for us.
2) Manage start, stop, redeploy and undeploy of every application comfortably.
3) Secured, only users with "manager" or "manager" role can reach this application.
4) Web and REST interface.
As said in the third point, this application is secured, using basic tomcat configuration. Take a look to your conf/tomcat-users.xml, read the useful comments inside <tomcat-users> tags:
<!--
NOTE: By default, no user is included in the "manager" role required
to operate the "/manager" web application. If you wish to use this app,
you must define such a user - the username and password are arbitrary.
-->
<!--
NOTE: The sample user and role entries below are wrapped in a comment
and thus are ignored when reading this file. Do not forget to remove
<!.. ..> that surrounds them.
-->
Now uncomment the next block of xml user declarations, although I rather keeping only one line like this:
<user username="tomcat" password="tomcat" roles="manager"/>
and I usually erase the others, in order to keep it clean.
Once done this, restart the server (service tomcat6 restart) and try to enter to your server:port/manager/html (usually localhost:8080/manager/html). type your user and password for a manager user (in my example, "tomcat" and "tomcat") and discover one of the most useful tools in tomcat administration.
The first half of the page is devoted to list your application and their status.
Scroll down a bit, you will find two ways to deploy an application, by local address in tomcat's computer, or by uploading a war file from your computer, that might not be always the same.
Now you are ready to deploy easily all the applications for your Continuous Integration System :)
pd: A stopped application will start if Tomcat is rebooted. That is important and not very intuitive.
Content:
Our tomcat installation provided an application manager, with all basic function to administrate the life cycle of our applications.
Top advantages:
1) No need to leave applications in a folder to deploy them, this application will copy there for us.
2) Manage start, stop, redeploy and undeploy of every application comfortably.
3) Secured, only users with "manager" or "manager" role can reach this application.
4) Web and REST interface.
As said in the third point, this application is secured, using basic tomcat configuration. Take a look to your conf/tomcat-users.xml, read the useful comments inside <tomcat-users> tags:
<!--
NOTE: By default, no user is included in the "manager" role required
to operate the "/manager" web application. If you wish to use this app,
you must define such a user - the username and password are arbitrary.
-->
<!--
NOTE: The sample user and role entries below are wrapped in a comment
and thus are ignored when reading this file. Do not forget to remove
<!.. ..> that surrounds them.
-->
Now uncomment the next block of xml user declarations, although I rather keeping only one line like this:
<user username="tomcat" password="tomcat" roles="manager"/>
and I usually erase the others, in order to keep it clean.
Once done this, restart the server (service tomcat6 restart) and try to enter to your server:port/manager/html (usually localhost:8080/manager/html). type your user and password for a manager user (in my example, "tomcat" and "tomcat") and discover one of the most useful tools in tomcat administration.
The first half of the page is devoted to list your application and their status.
Scroll down a bit, you will find two ways to deploy an application, by local address in tomcat's computer, or by uploading a war file from your computer, that might not be always the same.
Now you are ready to deploy easily all the applications for your Continuous Integration System :)
pd: A stopped application will start if Tomcat is rebooted. That is important and not very intuitive.
viernes, 18 de febrero de 2011
Roll your own Continuous Integration System (C.I.S.): Basic Tomcat configuration - JMX
Roll your own Continuous Integration System (C.I.S.)
Content:
About the content of jmxremote.password, a simply plain text with pairs of name password like this:
monitorRole tomcat
controlRole tomcat
About the content of jmxremote.access, a simply plain text with pairs of name policy like this:
monitorRole readonly
controlRole readwrite
Now you are able to see Tomcat with codename Catalina in your JConsole panel if you are running it locally, or throw remote connection.
Sorry for the delay, it really got me on my nerves to find the way of populating the jmx service!
Some help for Windows:
http://www.askproductions.no/blogg/2009/02/add-jmx-support-to-tomcat-6-on-windows/
Content:
I continue with this tutorial, this time I will assume you are working with Linux, in order to simplify the installation and configuration explanations. I do not have any license of Windows (no, Verbatim is not a Windows licenser).
JMX is one of those fabulous tools+protocol that almost anybody knows and everybody should know.
JMX is one of those fabulous tools+protocol that almost anybody knows and everybody should know.
It will help you to monitor any application since Java 1.5, it is based on reflexive calls and that guaranties uniform calling to unknown classes.
"The Java Management Extensions (JMX) API is a standard API for management and monitoring of resources such as applications, devices, services, and the Java virtual machine. The JMX technology was developed through the Java Community Process (JCP) as Java Specification Request (JSR) 3, Java Management Extensions, and JSR 160, JMX Remote API.
Typical uses of the JMX technology include:
Consulting and changing application configuration
Accumulating statistics about application behavior and making them available
Notifying of state changes and erroneous conditions.
The JMX API includes remote access, so a remote management program can interact with a running application for these purposes."
(from here.)
There's an easy-to-use tool for newbies called JConsole (jconsole.exe in Windows or jconsole in Linux), it is brought by JDK >1.5, you will find it in the same folder than java.exe or javac.exe. Find it, execute it, right now, do not wait one more minute.
The first you discover is the connection panel, first half for local applications, second half for remote ones. Local connections will never be empty due to the JConsole application itself is JMX-monitorizable. A monitorizable monitor!! Yehaaa!!!
Close to a dream, data, data, dataaaaaaaa :)
All about JConsole, it shows cpu consumption, memory usage, vm parameters, and MBeans...
MBeans are some special entities in java allowed to be exported like statistics and methods to be run. Indeed, cpu and memory statistic data are extracted from some well-known MBeans, and JConsole takes advance of knowing those MBeans for plotting some graphs. But it could could any metric you define.
Some day, in near future, I will describe how to populate application metrics and methods for being executed from JConsole or your own JMX protocol consumer.
Do not be shy, press that button, you will provoke a Garbage Collector call, and some memory will be freed. Change to memory tab to confirm it.
Summarizing, I find it quite important to monitor our Tomcat server in order to see memory consumption, CPU, and our application exported beans.
How to get it? It is not difficult.
http://tomcat.apache.org/tomcat-6.0-doc/monitoring.html#Enabling_JMX_Remote
It has taught me longer than expected to make it run, and the cause is that a line is missing for empty Tomcat6 installation.
The final configuration for Tomcat6 JMX enabled and simply secured is:
It has taught me longer than expected to make it run, and the cause is that a line is missing for empty Tomcat6 installation.
The final configuration for Tomcat6 JMX enabled and simply secured is:
# You can pass some parameters to java here if you wish toJAVA_OPTS="$JAVA_OPTS -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Xms1536m -Xmx1536m -XX:NewSize=256m -XX:MaxNewSize=256m -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+DisableExplicitGC"
# Enabling JMX CATALINA_OPTS="$CATALINA_OPTS -Dcom.sun.management.jmxremote.port=8091 -Dcom.sun.management.jmxremote.password.file=$CATALINA_BASE/conf/jmxremote.password -Dcom.sun.management.jmxremote.access.file=$CATALINA_BASE/conf/jmxremote.access -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=192.168.1.106"
About the content of jmxremote.password, a simply plain text with pairs of name password like this:
monitorRole tomcat
controlRole tomcat
About the content of jmxremote.access, a simply plain text with pairs of name policy like this:
monitorRole readonly
controlRole readwrite
Note some changes:
1) JAVA_OPTS is redefined with $JAVA_OPTS, in order to preserve some possible variable preset. The same for CATALINA_OPTS.
2) -Djava.rmi.server.hostname is defined in CATALINA_OPTS, with the server address. If not present, JMX module will only handle local calls, even though you set com.sun.management.jmxremote.port.
3) "com.sun.management.jmxremote" is not needed if "com.sun.management.jmxremote.port is defined."
4) Every variable set in a single line.
5) jmxremote.password and jmxremote.access should be only readable (and only readable) by tomcat user.
Of course, you can apply these instructions to the tomcat6.conf file in Linux, or Start->All programs->Apache Tomcat 6.0->Configure Tomcat. "Java" tab, for Windows, as we did before for memory configuration.
1) JAVA_OPTS is redefined with $JAVA_OPTS, in order to preserve some possible variable preset. The same for CATALINA_OPTS.
2) -Djava.rmi.server.hostname is defined in CATALINA_OPTS, with the server address. If not present, JMX module will only handle local calls, even though you set com.sun.management.jmxremote.port.
3) "com.sun.management.jmxremote" is not needed if "com.sun.management.jmxremote.port is defined."
4) Every variable set in a single line.
5) jmxremote.password and jmxremote.access should be only readable (and only readable) by tomcat user.
Of course, you can apply these instructions to the tomcat6.conf file in Linux, or Start->All programs->Apache Tomcat 6.0->Configure Tomcat. "Java" tab, for Windows, as we did before for memory configuration.
Remember, open Firewall for your JMX port if necessary!
Now you are able to see Tomcat with codename Catalina in your JConsole panel if you are running it locally, or throw remote connection.
Sorry for the delay, it really got me on my nerves to find the way of populating the jmx service!
Some help for Windows:
http://www.askproductions.no/blogg/2009/02/add-jmx-support-to-tomcat-6-on-windows/
lunes, 14 de febrero de 2011
Half a Thousand Thanks
Two month after beginning this blog, I have a modest visit rate, fair enough to feed my ego and make me very happy. Thanks!
By the way, Who are you? :)
pd: Javier del Palacio was here (to correct my English).
domingo, 6 de febrero de 2011
Roll your own Continuous Integration System (C.I.S.): Basic Tomcat configuration - Memory
Roll your own Continuous Integration System (C.I.S.)
Content:
In previous post, you installed Tomcat and let it running in your computer. Now it is time for a first configuration. We are going to focus in three usual needs.
1) We need more memory, always.
2) We need to monitor our system, we will use built-in JMX server.
3) We need permissions to get the manager application run. That manager application will make it easier the deploy task in near future.
Enlarge your memory
In spite of the amount of memory your system has, Java limits some thresholds for its applications. Default size for Sun's JVM PermGen space are 64M, and this is little memory for Artifactory, Hudson or Sonar. I do not remember which gives the problem :P
Objectives:
- Avoid PermGen exceptions by classloaders or application leaks, at least, in development environment.
- Increase available memory, some of the applications we will install are really hungry. Our Continuous Integration System will do some compilations which take much memory and we will foresee that need.
What have I added?:
All these are system parameters, they do not do anything by themselves, but that information is available for all applications and JVM. This way, when an application or the JVM itself is running, they can read system parameters and change their behavior.
java.awt.headless=true:
Headless mode is a system configuration in which the display device, keyboard, or mouse is lacking. Sounds unexpected, but actually you can perform different operations in this mode, even with graphic data.
http://java.sun.com/developer/technicalArticles/J2SE/Desktop/headless/
-Dfile.encoding=UTF-8:
You are specifying the default encoding for this JVM. From now on, if anything different is said, your applications will read and write and String will be built using that internal encoding.
By specifying this, you do not have to speculate with customer OS-dependent configuration.
-XX:PermSize=256m and -XX:MaxPermSize=256m modifies structure and behavior of JVM memory. I owe you a better explanation.
More options:
http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html
Tomcat6 performance guide:
http://tomcat.apache.org/articles/performance.pdf
Content:
In previous post, you installed Tomcat and let it running in your computer. Now it is time for a first configuration. We are going to focus in three usual needs.
1) We need more memory, always.
2) We need to monitor our system, we will use built-in JMX server.
3) We need permissions to get the manager application run. That manager application will make it easier the deploy task in near future.
Enlarge your memory
In spite of the amount of memory your system has, Java limits some thresholds for its applications. Default size for Sun's JVM PermGen space are 64M, and this is little memory for Artifactory, Hudson or Sonar. I do not remember which gives the problem :P
Objectives:
- Avoid PermGen exceptions by classloaders or application leaks, at least, in development environment.
- Increase available memory, some of the applications we will install are really hungry. Our Continuous Integration System will do some compilations which take much memory and we will foresee that need.
- For any Linux Tomcat6: Open /usr/share/tomcat6/conf/tomcat6.conf (or $TOMCAT_HOME/conf/tomcat6.conf if you installed it in other folder).
- Check that JAVA_OPTS is not set, I mean, there is a "#" in front of every JAVA_OPTS= line or any JAVA_OPTS line exists. If it is really not set, add this line:
JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -XX:PermSize=256m -XX:MaxPermSize=256m"
else, just merge available with new parameters, but if the system is just installed, it shouldn't exist.
- For Windows Tomcat6: Easier to find, Start->All programs->Apache Tomcat 6.0->Configure Tomcat. Then click "Java" tab.
- Check if any of next parameters are set, I mean, if that line exists. No "#" possibility. If the parameter exists, replace with new value, else simply add.
-Djava.awt.headless=true-Dfile.encoding=UTF-8-XX:PermSize=256m-XX:MaxPermSize=256m
What have I added?:
All these are system parameters, they do not do anything by themselves, but that information is available for all applications and JVM. This way, when an application or the JVM itself is running, they can read system parameters and change their behavior.
java.awt.headless=true:
Headless mode is a system configuration in which the display device, keyboard, or mouse is lacking. Sounds unexpected, but actually you can perform different operations in this mode, even with graphic data.
http://java.sun.com/developer/technicalArticles/J2SE/Desktop/headless/
-Dfile.encoding=UTF-8:
You are specifying the default encoding for this JVM. From now on, if anything different is said, your applications will read and write and String will be built using that internal encoding.
By specifying this, you do not have to speculate with customer OS-dependent configuration.
-XX:PermSize=256m and -XX:MaxPermSize=256m modifies structure and behavior of JVM memory. I owe you a better explanation.
More options:
http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html
Tomcat6 performance guide:
http://tomcat.apache.org/articles/performance.pdf
Suscribirse a:
Entradas (Atom)