miércoles, 18 de mayo de 2011

What if I want to deploy/upload several files to artifactory or nexus with Maven?

Regardless of the repository you are using to store generated artifacts, the default Maven configuration for, let's say, WAR file, is upload a WAR file.. interesting..

A common configuration commented here also uploads the source code as a jar file, and that is very recommendable for debugging.

Another uploaded or auto-generated file is a "pom.xml" project descriptor file for your artifact.

But we want to go further, we want to upload the sql file needed to generate our initial database, or the pdf with the documentation of this release.. how to get this?

<plugin>   
  <groupId>org.codehaus.mojo</groupId>   
  <artifactId>build-helper-maven-plugin</artifactId>
  <executions>
    <execution>

      <id>attach-artifacts</id>
      <phase>package</phase>
      <goals> 

        <goal>attach-artifact</goal>
      </goals> 
      <configuration>
        <artifacts> 
          <artifact> 
            <file>src\main\sql\initdb.sql</file>
            <type>sql</type>
            <classifier>create</classifier>
          </artifact>
          <artifact>
            <file>target\site\docs\document.pdf</file>   
            <type>pdf</type>
            <classifier>doc</classifier>
          </artifact>
        </artifacts>
      </configuration>
    </execution>
  </executions>
</plugin>


With this plugin, you are associating with the same artifact the "document.pdf" file, the "initdb.sql" file, the war, the sources jar, the pom.xml file... all them with the same version, and with the possibility of declaring any of them as a dependency.

It will be executed in package phase, beware because you may mistake the phases and attach something not yet created, for example, the pdf may be created in pre-site phase instead of .. compile, for example.

In the case of multiple configurations, you can configure it like this:


<plugin>   
  <groupId>org.codehaus.mojo</groupId>   
  <artifactId>build-helper-maven-plugin</artifactId>
  <executions>
    <execution>

      <id>attach-artifacts1</id>
      <phase>package</phase>
      <goals> 

        <goal>attach-artifact</goal>
      </goals> 
      <configuration>
        <artifacts> 
          <artifact> 
            <file>src\main\sql\initdb.sql</file>
            <type>sql</type>
            <classifier>create</classifier>
          </artifact>
        </artifacts>
      </configuration>
    </execution>

    <execution>
      <id>attach-artifacts2</id>
      <phase>site</phase>
      <goals> 

        <goal>attach-artifact</goal>
      </goals> 
      <configuration>
        <artifacts> 
          <artifact>
            <file>target\site\docs\document.pdf</file>   
            <type>pdf</type>
            <classifier>doc</classifier>
          </artifact>
        </artifacts>
      </configuration>
    </execution>

  </executions>
</plugin>

In this case, if we execute "mvn deploy", only "initdb.sql" will be attached, if we execute "mvn site-deploy", it will be only attached "document.pdf" (because it is supposed to be created in pre-site phase, but I am not sure if it will be deployed in "artifactory" or "nexus", because you don't pass through "deploy" phase, I would say it won't.

The best, if you execute "mvn site-deploy deploy", firstly "document.pdf" is attached in "site" phase, then "initbd.sql" is attached in "package" phase, and finally all them are deployed in "deploy" phase. Really useful!

This is the solution for a problem we wanted to solve in my job, the solution was taken here, thanks!

lunes, 9 de mayo de 2011

Opinion: Simulated annealing algorithm, the ignored biological algorithm

This is a personal opinion article, based on my own experience and alienation.

I knew this algorithm for first time in Artificial Intelligence subject of my Master Degree in Computer Sciences in 2005. I liked it so much that I repeated one more year, only to increment my marks, a FAIL was not enough for me. I know, sometimes my ambition scares me the hell out too.

This algorithm models a universe where the solution to a problem does not try to be perfect, only good (my bro sometimes says that something good is the best enemy of the perfection). The approach is highly random at first, and, little by little, people (ups, I meant the search of the solution) go behaving as they are supposed to, and finally, it is like any other algorithm that reach a similar solution in a fashion way (faster, yes, but not better).

What I really love of this algorithm is not my implementation 5 years ago, I do not really remember what I solved, but it has not gone down in history. What I loved about that algorithm is the ease of application to the psychology of human life:

Example 1:

A child, with all that potential, could be anything in life, and, according with chaos theory, little changes makes big differences, an opinion, an idea or an image could make a child empathize or left aside a sport, a hobby, a passion...
Years later, with less  of that potential, it is not possible for a teenager to be the best in several fields, it is too late for him/her, but it is still headed for other disciplines.
Finally, the child grew adult, and the possibilities are within a cone that gets narrower and narrower. Of course, changes are always possible, my own mother got a job when she was 52 in something she never excelled. When she tried to poison my bro and me several times with her "cuisine", now she cooks very well in a restaurant.

Example 2:

A project, in its first design phase, it is a flower and fruit garden, a unexplored world full of possibilities, the childhood.
Only defining the base language, initial technology and a framework, most possibilities really disappear, and other grow strong. Java, Spring, AOP for Tomcat deploying weakens the possibilities of being portlet and destroys the possibilities of being a C++ desktop application. There are still many things to define and create with patterns and algorithms.
A month to deliver, what can we really change?

Those are my arguments for consider this algorithm essential for a global understanding of any creation, specially for engineers that makes of creation their primary task. Personally, I consider it a biological algorithm, until a God, Creator or BigBang algorithm category were created.


==========================================
Simulated annealing (SA) is a generic probabilistic metaheuristic for the global optimization problem of locating a good approximation to the global optimum of a given function in a large search space. It is often used when the search space is discrete (e.g., all tours that visit a given set of cities). For certain problems, simulated annealing may be more effective than exhaustive enumeration — provided that the goal is merely to find an acceptably good solution in a fixed amount of time, rather than the best possible solution.

The name and inspiration come from annealing in metallurgy, a technique involving heating and controlled cooling of a material to increase the size of its crystals and reduce their defects. The heat causes the atoms to become unstuck from their initial positions (a local minimum of the internal energy) and wander randomly through states of higher energy; the slow cooling gives them more chances of finding configurations with lower internal energy than the initial one.

By analogy with this physical process, each step of the SA algorithm replaces the current solution by a random "nearby" solution, chosen with a probability that depends both on the difference between the corresponding function values and also on a global parameter T (called the temperature), that is gradually decreased during the process. The dependency is such that the current solution changes almost randomly when T is large, but increasingly "downhill" as T goes to zero. The allowance for "uphill" moves potentially saves the method from becoming stuck at local optima—which are the bane of greedier methods.

The method was independently described by Scott Kirkpatrick, C. Daniel Gelatt and Mario P. Vecchi in 1983,[1] and by Vlado Černý in 1985.[2] The method is an adaptation of the Metropolis-Hastings algorithm, a Monte Carlo method to generate sample states of a thermodynamic system, invented by M.N. Rosenbluth in a paper by N. Metropolis et al. in 1953.[3]

From wikipedia

jueves, 5 de mayo de 2011

Roll your own Continuous Integration System (C.I.S.): Artifactory repositories configuration: snapshot, releases and security.

Roll your own Continuous Integration System (C.I.S.)

Content:
Abstract
Install Tomcat
Basic Tomcat configuration - Memory
Basic Tomcat configuration - JMX
Basic Tomcat configuration - Application Manager and permissions
Apache and uSVN 
Installing Artifactory from WAR
Configure Artifactory and MySQL
Configuring Artifactory security and repositories 


In order to ease the explanation about security and maven deploy, lets create the simplest Maven project possible.

Create a directory
Create a "pom.xml" file with the following content:

<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemalocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelversion>4.0.0</modelversion>

<groupid>com.mycompany</groupid>
<artifactid>mavenproject</artifactid>
<version>1.0-SNAPSHOT</version>
<packaging>pom</packaging>

</project>


Now that you have the out-of-the-box Artifactory installation and a Maven project, try to upload your project to the Artifactory repository manager.

Remember: You must have a Maven installation in order to run those examples. As far as you have followed this blog, you should have installed SpringSource Tool Suite, that brings with Maven. Maybe you need insert "mvn" executable in the PATH.

Try to execute "mvn deploy" in the same directory than "pom.xml".


[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building mavenproject 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ mavenproject ---
[INFO] Installing /home/***/NetBeansProjects/mavenproject10/pom.xml to /home/***/.m2/repository/com/mycompany/mavenproject/1.0-SNAPSHOT/mavenproject-1.0-SNAPSHOT.pom
[INFO]
[INFO] --- maven-deploy-plugin:2.5:deploy (default-deploy) @ mavenproject ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.586s
[INFO] Finished at: Tue May 03 22:36:23 CEST 2011
[INFO] Final Memory: 3M/74M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.5:deploy (default-deploy) on project mavenproject: Deployment failed: repository element was not specified in the POM inside distributionManagement element or in -DaltDeploymentRepository=id::layout::url parameter -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException


Read the bold, it means that either in your project or in other possible settings file, you are not specifying where are you willing to deploy your project.

Note: not explained yet, but there is a lifecycle somewhere in Internet that says "deploying requires compiling first" for instance. So, when you ask Maven to do "deploy", you are also asking it to compile, testing and some other actions.

Note: The examples are based in basic configurations and no network troubles. Everything in the same host and so on. Take it as it is, an example.

Go to Artifactory, "Artifacts" tabs, and click in the second repository, "libs-snapshot-local"


By clicking the repository, its information is displayed at the right side. We want "Distribution Management" section. 


Copy it to your "pom.xml" (your project, remember?)


<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>com.mycompany</groupId>
<artifactId>mavenproject</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>pom</packaging>

<distributionManagement>
<repository>
<id>dhcppc24</id>
<name>dhcppc24-releases</name>
<url>http://localhost:8080/artifactory/libs-release-local</url>
</repository>
</distributionManagement>

</project>


Now execute again "mvn deploy", let's check the response: (Yes, it fails again, I do not like suspense)

[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building mavenproject 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ mavenproject ---
[INFO] Installing /home/***/NetBeansProjects/mavenproject10/pom.xml to /home/***/.m2/repository/com/mycompany/mavenproject/1.0-SNAPSHOT/mavenproject-1.0-SNAPSHOT.pom
[INFO]
[INFO] --- maven-deploy-plugin:2.5:deploy (default-deploy) @ mavenproject ---
Downloading: http://localhost:8080/artifactory/libs-release-local/com/mycompany/mavenproject/1.0-SNAPSHOT/maven-metadata.xml
Uploading: http://localhost:8080/artifactory/libs-release-local/com/mycompany/mavenproject/1.0-SNAPSHOT/mavenproject-1.0-20110503.214739-1.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.660s
[INFO] Finished at: Tue May 03 23:47:39 CEST 2011
[INFO] Final Memory: 3M/74M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.5:deploy (default-deploy) on project mavenproject: Failed to deploy artifacts: Could not transfer artifact com.mycompany:mavenproject:pom:1.0-20110503.214739-1 from/to dhcppc24 (http://localhost:8080/artifactory/libs-release-local): Failed to transfer file: http://localhost:8080/artifactory/libs-release-local/com/mycompany/mavenproject/1.0-SNAPSHOT/mavenproject-1.0-20110503.214739-1.pom. Return code is: 409 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException


This error means that you are not allowed to do the upload. That is because you are trying to upload an SNAPSHOT in a RELEASE repository. WTF?

See your pom.xml, <version>1.0-SNAPSHOT</version>. That -SNAPSHOT is being read by Artifactory in order to determine whether it is a release or not.
Now travel to Artifactory. Log in (user "admin", password "password"), and go to Admin tab.



Inside Admin tab, Repositories menu and put your mouse over "libs-release-local", click "Edit".




There it is, the guts of the repository configuration. See how out-of-the-box "libs-release-local" repository can only handle with RELEASES, and you are trying to upload a SNAPSHOT. It is not possible!!.


First option, change <version>1.0-SNAPSHOT</version> for <version>1.0</version>. Come on! do it! it will fail xD. Why? You do not have permissions yet :P
It is not a good idea anyway. Just do not release anything until you know what are really doing.

Second option, in that window, you can tick "Handle Snapshots" and it will fail, because of permissions. It is a matter of time.

Third option, see above now to copy the "distribution management" information of a repository, and take the information from "libs-snapshot-local". This is the best option right now. Your "pom.xml" should look like:


<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>com.mycompany</groupId>
<artifactId>mavenproject</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>pom</packaging>

<distributionManagement>
<repository>
<id>dhcppc24</id>
<name>dhcppc24-releases</name>
<url>http://localhost:8080/artifactory/libs-release-local</url>
</repository>
<snapshotRepository>
<id>dhcppc24</id>
<name>dhcppc24-snapshots</name>
<url>http://localhost:8080/artifactory/libs-snapshot-local</url>
</snapshotRepository>

</distributionManagement>

</project>


Now try "mvn deploy" in order to confirm that I was right, permissions are not set yet.


[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building mavenproject 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ mavenproject ---
[INFO] Installing /home/***/NetBeansProjects/mavenproject10/pom.xml to /home/***/.m2/repository/com/mycompany/mavenproject/1.0-SNAPSHOT/mavenproject-1.0-SNAPSHOT.pom
[INFO]
[INFO] --- maven-deploy-plugin:2.5:deploy (default-deploy) @ mavenproject ---
Downloading: http://localhost:8080/artifactory/libs-snapshot-local/com/mycompany/mavenproject/1.0-SNAPSHOT/maven-metadata.xml
Downloaded: http://localhost:8080/artifactory/libs-snapshot-local/com/mycompany/mavenproject/1.0-SNAPSHOT/maven-metadata.xml (359 B at 7.5 KB/sec)
Uploading: http://localhost:8080/artifactory/libs-snapshot-local/com/mycompany/mavenproject/1.0-SNAPSHOT/mavenproject-1.0-20110505.213434-2.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.637s
[INFO] Finished at: Thu May 05 23:34:34 CEST 2011
[INFO] Final Memory: 3M/74M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.5:deploy (default-deploy) on project mavenproject: Failed to deploy artifacts: Could not transfer artifact com.mycompany:mavenproject:pom:1.0-20110505.213434-2 from/to dhcppc24 (http://localhost:8080/artifactory/libs-snapshot-local): Failed to transfer file: http://localhost:8080/artifactory/libs-snapshot-local/com/mycompany/mavenproject/1.0-SNAPSHOT/mavenproject-1.0-20110505.213434-2.pom. Return code is: 401 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException


Now let's set the damn permissions!

"Admin" tab, menu "Permissions"


New (in the right side)



We are going to configure the permissions for all our local repositories:

Name: Local repositories
Include Patterns: Any
Exclude Patterns: None
Repositories: Any Local Repository


Users tab: Anonymous users must be able to Deploy, automatically will be able to Annotate and Read.


And "Create" it, it is done.

Repeat the operation, "mvn deploy", and it should be able to compile and deploy your project to Artifactory repository :)


[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building mavenproject 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ mavenproject ---
[INFO] Installing /home/***/NetBeansProjects/mavenproject10/pom.xml to /home/***/.m2/repository/com/mycompany/mavenproject/1.0-SNAPSHOT/mavenproject-1.0-SNAPSHOT.pom
[INFO]
[INFO] --- maven-deploy-plugin:2.5:deploy (default-deploy) @ mavenproject ---
Downloading: http://localhost:8080/artifactory/libs-snapshot-local/com/mycompany/mavenproject/1.0-SNAPSHOT/maven-metadata.xml
Downloaded: http://localhost:8080/artifactory/libs-snapshot-local/com/mycompany/mavenproject/1.0-SNAPSHOT/maven-metadata.xml (359 B at 5.7 KB/sec)
Uploading: http://localhost:8080/artifactory/libs-snapshot-local/com/mycompany/mavenproject/1.0-SNAPSHOT/mavenproject-1.0-20110505.214558-2.pom
Uploaded: http://localhost:8080/artifactory/libs-snapshot-local/com/mycompany/mavenproject/1.0-SNAPSHOT/mavenproject-1.0-20110505.214558-2.pom (823 B at 3.4 KB/sec)
Downloading: http://localhost:8080/artifactory/libs-snapshot-local/com/mycompany/mavenproject/maven-metadata.xml
Downloaded: http://localhost:8080/artifactory/libs-snapshot-local/com/mycompany/mavenproject/maven-metadata.xml (351 B at 8.6 KB/sec)
Uploading: http://localhost:8080/artifactory/libs-snapshot-local/com/mycompany/mavenproject/1.0-SNAPSHOT/maven-metadata.xml
Uploaded: http://localhost:8080/artifactory/libs-snapshot-local/com/mycompany/mavenproject/1.0-SNAPSHOT/maven-metadata.xml (598 B at 20.1 KB/sec)
Uploading: http://localhost:8080/artifactory/libs-snapshot-local/com/mycompany/mavenproject/maven-metadata.xml
Uploaded: http://localhost:8080/artifactory/libs-snapshot-local/com/mycompany/mavenproject/maven-metadata.xml (317 B at 4.1 KB/sec)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.034s
[INFO] Finished at: Thu May 05 23:45:58 CEST 2011
[INFO] Final Memory: 3M/74M
[INFO] ------------------------------------------------------------------------




Last travel to Artifactory, let's find the uploaded artifact.


In the "Artifacts" tab, navigate through "libs-snapshot-local" and you will arrive to your project.

By clicking in the artifact itself, you will get the Maven code for declaring it as dependency for other projects.


<dependency>
<groupId>com.mycompany</groupId>
<artifactId>mavenproject</artifactId>
<version>1.0-20110505.214856-1</version>
<type>pom</type>
</dependency>



The end... quite long, isn't it?