Advanced repository - step 1
On the road to a professional repository
We will study in this section how to build a professionnal repository. What is a professionnal dependency resolver ? Our vision is to say that a good quality repository must follow clear rules about projects naming and must offer corrects, usuables, configurables and verified project descriptors. In order to achieve those goals, we think that you have to build your own repository.
We have seen in the previous example, that we could use some public repositories to begin to build our own repository.
Nevertheless, the result is not at all the one that was excepected. Indeed there is a problem with public repositories : their partial incompatibility.
For example, in ivyrep all commons-* projects belong to the apache organisation. In both ibiblio versions, it is not the case. The same problem could appear for other projects with other repositories, it is not the debate here.
To resolve some of the incompatibilities, we will use a new feature of ivy 1.3 the namespaces.
Using namespaces
In order to use namespaces, we first need to see what's happening when none are used on repositories that do not fit our needs.
Let's take commons-lang 1.0 from ibiblio with a maven2 pom.
First clean your cache and repository.
Z:\ivy-repository>ant clean-cache clean-repo
Then call the good ant target : ant commons-lang-1-0-ibiblio-no-namespace
Z:\ivy-repository>ant commons-lang-1-0-ibiblio-no-namespace Buildfile: build.xml init-advanced: :: Ivy 20060125070719 - 20060125070719 :: http://ivy.jayasoft.org/ :: :: configuring :: file = Z:\ivy-repository\ivy-conf-advanced.xml commons-lang-1-0-ibiblio-no-namespace: :: installing [ commons-lang | commons-lang | 1.0 ] :: :: resolving dependencies :: found [ commons-lang | commons-lang | 1.0 ] in ibiblio-maven2-nonamespace found [ junit | junit | 3.7 ] in ibiblio-maven2-nonamespace :: downloading artifacts to cache :: downloading http://www.ibiblio.org/maven2/commons-lang/commons-lang/1.0/commons-lang... ... .............. (62kB) [SUCCESSFUL ] [ commons-lang | commons-lang | 1.0 ]/commons-lang.jar[jar] (1313ms) downloading http://www.ibiblio.org/maven2/junit/junit/3.7/junit-3.7.jar ... ............. .............. (114kB) [SUCCESSFUL ] [ junit | junit | 3.7 ]/junit.jar[jar] (2360ms) :: installing in local-repository :: published commons-lang to Z:\ivy-repository/ivy-local-repository/commons-lang/commons-lang/jars/commons-lang-1.0.jar published ivy to Z:\ivy-repository/ivy-local-repository/commons-lang/commons-lang/ivys/ivy-1.0.xml published junit to Z:\ivy-repository/ivy-local-repository/junit/junit/jars/junit-3.7.jar published ivy to Z:\ivy-repository/ivy-local-repository/junit/junit/ivys/ivy-3.7.xml :: install resolution report :: --------------------------------------------------------------------- | | modules || artifacts | | conf | number| search|dwnlded|evicted|| number|dwnlded| --------------------------------------------------------------------- | default | 2 | 2 | 2 | 0 || 2 | 2 | --------------------------------------------------------------------- BUILD SUCCESSFUL Total time: 6 seconds Z:\ivy-repository>
If we take a look at the repository, we can see that we respect the ibiblio maven2 architecture, the organisation apache does not appear.
Z:\ivy-repository>dir /s /B /A:-D ivy-local-repository Z:\ivy-repository\ivy-local-repository\commons-lang\commons-lang\ivys\ivy-1.0.xml Z:\ivy-repository\ivy-local-repository\commons-lang\commons-lang\jars\commons-lang-1.0.jar Z:\ivy-repository\ivy-local-repository\junit\junit\ivys\ivy-3.7.xml Z:\ivy-repository\ivy-local-repository\junit\junit\jars\junit-3.7.jar
If you take a look at the ivy descriptor for commons-lang, you will see that the organisation is still commons-lang. It could not be another thing as we did not do anything for it.
<ivy-module version="1.0">
<info organisation="commons-lang"
module="commons-lang"
revision="1.0"
status="integration"
publication="20051124062021"
/>Introduction to namespaces
Let's see directly the result, we will have some explanations after.
Clean your repo and cache, and call : ant commons-lang-1-0-ibiblio-with-namespace
Z:\ivy-repository>ant commons-lang-1-0-ibiblio-with-namespace Buildfile: build.xml init-advanced: :: Ivy non official version :: http://ivy.jayasoft.org/ :: :: configuring :: file = Z:\ivy-repository\ivy-conf-advanced.xml commons-lang-1-0-ibiblio-with-namespace: :: installing [ apache | commons-lang | 1.0 ] :: :: resolving dependencies :: found [ apache | commons-lang | 1.0 ] in ibiblio-maven2 found [ junit | junit | 3.7 ] in ibiblio-maven2 :: downloading artifacts to cache :: downloading http://www.ibiblio.org/maven2/commons-lang/commons-lang/1.0/commons-lang... ... ............. (62kB) [SUCCESSFUL ] [ apache | commons-lang | 1.0 ]/commons-lang.jar[jar] (1094ms) downloading http://www.ibiblio.org/maven2/junit/junit/3.7/junit-3.7.jar ... ............................ (114kB) [SUCCESSFUL ] [ junit | junit | 3.7 ]/junit.jar[jar] (1641ms) :: installing in local-repository :: published commons-lang to Z:\ivy-repository/ivy-local-repository/apache/commons-lang/jars/commons-lang-1.0.jar published ivy to Z:\ivy-repository/ivy-local-repository/apache/commons-lang/ivys/ivy-1.0.xml published junit to Z:\ivy-repository/ivy-local-repository/junit/junit/jars/junit-3.7.jar published ivy to Z:\ivy-repository/ivy-local-repository/junit/junit/ivys/ivy-3.7.xml :: install resolution report :: --------------------------------------------------------------------- | | modules || artifacts | | conf | number| search|dwnlded|evicted|| number|dwnlded| --------------------------------------------------------------------- | default | 2 | 2 | 2 | 0 || 2 | 2 | --------------------------------------------------------------------- BUILD SUCCESSFUL Total time: 5 seconds
Now if we look at our repository, it seems to look fine.
Z:\ivy-repository>dir /s /B /A:-D ivy-local-repository Z:\ivy-repository\ivy-local-repository\apache\commons-lang\ivys\ivy-1.0.xml Z:\ivy-repository\ivy-local-repository\apache\commons-lang\jars\commons-lang-1.0.jar Z:\ivy-repository\ivy-local-repository\junit\junit\ivys\ivy-3.7.xml Z:\ivy-repository\ivy-local-repository\junit\junit\jars\junit-3.7.jar
Have a look at the ivy file to see it it looks better than before. Ok, we have now our apache organisation.
<ivy-module version="1.0">
<info organisation="apache"
module="commons-lang"
revision="1.0"
status="integration"
publication="20051124062021"
/>How does this work ?
If we look at the ant target commons-lang-1-0-ibiblio-with-namespace, we can see that it uses a resolver called ibiblio-maven2.
Let's find it... The configuration file used for this test is ivy-conf-advanced.xml. This one includes ivy-maven2-ivyconf.xml where the required resolver is defined.
Let's see it's definition :
<ibiblio name="ibiblio-maven2"
root="${ibiblio-maven2-root}"
pattern="${ibiblio-maven2-pattern}"
m2compatible="true"
namespace="ibiblio-maven2"
/>Ok, i see it, it is a ibiblio resolver for which we specify the root and the pattern. The important things here are the 2 other parameters.
- m2compatible is a flag telling that we allow reading POMs file and make some transformations on URLs regarding the organisation name. Indeed, maven transforms organisations like "org.apache" into "some_url/org/apache" to retrieve information on ibiblio repositories.
- namespace this attribute defines a domain in which same projects (meanning organsiation, module or revision) can be nammed whith differents kinds.
A namespace is defined by a set of rule, for ibiblio-maven2, whe have declared some rules :
rule handling imported apache maven1 projects
<rule> <!-- imported apache maven1 projects -->
<fromsystem>
<src org="apache" module=".+"/>
<dest org="$m0" module="$m0"/>
</fromsystem>
<tosystem>
<src org="commons-.+" module="commons-.+" />
<src org="ant.*" module="ant.*" />
...
<src org="xmlrpc" module="xmlrpc" />
<dest org="apache" module="$m0"/>
</tosystem>
</rule>
In order to distinguish matching regular expressions found in organisation, module & revision the notation used prefixes the matching regular expression with the letters 'o', 'm' & 'r'.
$o0 : the whole regular expression term in the organisation attribute
$o1 : the first matching expression term that was marked in the organisation attribute
...
The same applies for modules : $m0, $m1, ...
and for revisions : $r0, $r1, ...
- fromsystem : we define here that the projects defined in the system under the organisation called "apache" are transformed into the destination namespace (whose resolver it applies) into projects whose organisation is nammed with the module name, we don't care here about the revision. For example, the project ['apache', 'commons-lang', '1.0'] in the namespace system will be translated into ['commons-lang', 'commons-lang', '1.0'] in the ibiblio-maven2 resolver namespace.
- tosystem : we define here the reverse mapping, ie how to translate apache projects from ibiblio into real apache projects in the namespace system. The rule here, is telling that all projects matching commons-.+ (see it as java regular expression) for their organisation name and module name are transformed into projects whose organisation is apache with the module name as it was found. The same kind of rule is applied for others apache projects like ant, etc. For example, ['ant','ant','1.6.2'] in ibiblio-maven2 namespace will become ['apache','ant','1.6.2'] int the system namespace.
rule handling new apache projects
<rule> <!-- new apache projects -->
<fromsystem>
<src org="apache" />
<dest org="org.apache"/>
</fromsystem>
<tosystem>
<src org="org.apache" />
<dest org="apache" />
</tosystem>
</rule>The mapping adds or removes the package 'org' before the organisation name to conform to maven2 choices.
More rules
<?xml version="1.0" encoding="UTF-8"?><ivyconf>
<!-- you can override this property to use mirrors
http://mirrors.dotsrc.org/maven2
http://ftp.ggi-project.org/pub/packages/maven2
http://mirrors.sunsite.dk/maven2
http://public.planetmirror.com/pub/maven2
http://ibiblio.lsu.edu/main/pub/packages/maven2
http://www.ibiblio.net/pub/packages/maven2
-->
<property name="ibiblio-maven2-root"
value="http://www.ibiblio.org/maven2/" override="false" />
<property name="ibiblio-maven2-pattern"
value="[organisation]/[module]/[revision]/[module]-[revision].[ext]"
override="false" />
<namespaces>
<namespace name="ibiblio-maven2">
<rule><!-- commons-jelly -->
<fromsystem>
<src org="apache" module="commons-jelly-.+"/>
<dest org="commons-jelly" module="$m0"/>
</fromsystem>
<tosystem>
<src org="commons-jelly" module=".+"/>
<dest org="apache" module="$m0"/>
</tosystem>
</rule>
<rule><!-- imported apache maven1 projects -->
<fromsystem>
<src org="apache" module=".+" />
<dest org="$m0" module="$m0" />
</fromsystem>
<tosystem>
<src org="commons-.+" module="commons-.+" />
<src org="ant.*" module="ant.*" />
<src org="avalon-.+" module="avalon-.+" />
<src org="avalon" module="avalon" />
<src org="axis" module="axis" />
<src org="axis2" module="axis2" />
<src org="batik" module="batik" />
<src org="bcel" module="bcel" />
<src org="beehive" module="beehive" />
<src org="bsf" module="bsf" />
<src org="cactus" module="cactus" />
<src org="cocoon" module="cocoon" />
<src org="cornerstone-.+" module="cornerstone-.+" />
<src org="create-checksums"
module="create-checksums" />
<src org="crimson" module="crimson" />
<src org="directory-.+" module="directory-.+" />
<src org="directory" module="directory" />
<src org="ecs" module="ecs" />
<src org="excalibur-.+" module="excalibur-.+" />
<src org="excalibur" module="excalibur" />
<src org="fop" module="fop" />
<src org="fulcrum" module="fulcrum" />
<src org="geronimo-.+" module="geronimo-.+" />
<src org="geronimo" module="geronimo" />
<src org="hivemind" module="hivemind" />
<src org="jakarta-regexp" module="jakarta-regexp" />
<src org="james" module="james" />
<src org="jaxme" module="jaxme" />
<src org="jcs-javagroups" module="jcs-javagroups" />
<src org="jcs" module="jcs" />
<src org="jspapi" module="jspapi" />
<src org="jstl" module="jstl" />
<src org="juddi" module="juddi" />
<src org="log4j" module="log4j" />
<src org="logkit" module="logkit" />
<src org="lucene" module="lucene" />
<src org="magicGball" module="magicGball" />
<src org="maven" module="maven" />
<src org="merlin-developer"
module="merlin-developer" />
<src org="merlin" module="merlin" />
<src org="muse" module="muse" />
<src org="myfaces" module="myfaces" />
<src org="nekohtml" module="nekohtml" />
<src org="ojb" module="ojb" />
<src org="oro" module="oro" />
<src org="pluto-container" module="pluto-container" />
<src org="poi" module="poi" />
<src org="pubscribe" module="pubscribe" />
<src org="sandesha" module="sandesha" />
<src org="servletapi" module="servletapi" />
<src org="slide" module="slide" />
<src org="stratum" module="stratum" />
<src org="struts" module="struts" />
<src org="taglibs" module="taglibs" />
<src org="tapestry" module="tapestry" />
<src org="tomcat-util" module="tomcat-util" />
<src org="tomcat" module="tomcat" />
<src org="torque" module="torque" />
<src org="turbine" module="turbine" />
<src org="velocity-.+" module="velocity-.+" />
<src org="velocity" module="velocity" />
<src org="ws-commons.*" module="ws-commons.*" />
<src org="wsdl4j" module="wsdl4j" />
<src org="wsrf" module="wsrf" />
<src org="xalan" module="xalan" />
<src org="xerces" module="xerces" />
<src org="xercesjarv" module="xercesjarv" />
<src org="xml-apis" module="xml-apis" />
<src org="xml-resolver" module="xml-resolver" />
<src org="xml-security" module="xml-security" />
<src org="xmlbeans" module="xmlbeans" />
<src org="xmlrpc" module="xmlrpc" />
<dest org="apache" module="$m0" />
</tosystem>
</rule>
<rule><!-- new apache projects -->
<fromsystem>
<src org="apache" />
<dest org="org.apache" />
</fromsystem>
<tosystem>
<src org="org.apache" />
<dest org="apache" />
</tosystem>
</rule>
<rule><!-- new hibernate projects -->
<fromsystem>
<src org="hibernate" />
<dest org="org.hibernate" />
</fromsystem>
<tosystem>
<src org="org.hibernate" />
<dest org="hibernate" />
</tosystem>
</rule>
<!-- codehaus projects -->
<rule>
<fromsystem>
<src org="codehaus" module="nanocontainer.+" />
<dest org="nanocontainer" module="$m0" />
</fromsystem>
<tosystem>
<src org="nanocontainer" module=".+" />
<dest org="codehaus" module="$m0" />
</tosystem>
</rule>
<!-- codehaus nanocontainer projects -->
<rule>
<fromsystem>
<src org="codehaus" module=".+" />
<dest org="$m0" module="$m0" />
</fromsystem>
<tosystem>
<src org="picocontainer" module=".+" />
<src org="groovy" module=".+" />
<dest org="codehaus" module="$m0" />
</tosystem>
</rule>
</namespace>
</namespaces>
<resolvers>
<ibiblio name="ibiblio-maven2" root="${ibiblio-maven2-root}"
pattern="${ibiblio-maven2-pattern}" m2compatible="true"
namespace="ibiblio-maven2" />
</resolvers>
</ivyconf>