Jonathan Lalou's Blog

Archive for the ‘en-US’ Category

Going on learning Groovy, I faced the following case: how to call an external shell (or MS-DOS) command from a Groovy script?

It is very easy. The following example show how to display the Subversion status of the current folder:
[java]def proc = "svn st"
def sb = new StringBuffer()
proc.consumeProcessErrorStream(sb)

println proc.text + "\n" + sb.toString()
[/java]

I have learnt Groovy since the beginning of the week.
I had to import some classes from Oracle JDBC driver. As a newbie, I wrote the following annotation:
[java]@Grab(group = ‘com.oracle’, module = ‘ojdbc14’, version = ‘10.2.0.4.0’)[/java]

I got the following error:
[java]Error grabbing Grapes — [unresolved dependency: com.oracle#ojdbc14;10.2.0.4.0: not found][/java]

To sum up, adding a dependency to Oracle driver raises two issues:

• adding any external dependency
• adding a dependency to any JDBC driver

Indeed, Grab requires a kind of Maven repository, and you can catch the link between the @Grab annotation below and classic Maven 2 <dependency> tag. As I understood, Grab instructions are closer to Ivy than to Maven system. In my situation, having no “Grab repo” (I don’t know how to call that), I had to download the jars from a Maven repo. To fix it, two operations are needed: hint at Maven (or Ivy) repo, and allow Groovy to access the network.

Regarding the second issue, it is answered by added a @GrabConfig annotation.
Therefore, to fix the original issue:

• replace [java]@Grab(group = ‘com.oracle’, module = ‘ojdbc14’, version = ‘10.2.0.4.0’)[/java]
  with[java]@GrabConfig(systemClassLoader=true)
  @GrabResolver(name=’nameOfYourMavenRepo, root=’http://url-of-your-maven-repo:port’)
  @Grab(group = ‘com.oracle’, module = ‘ojdbc14’, version = ‘10.2.0.4.0’)
  [/java]
• at first run, if needed, hint at the proxy to allow Grab to download the Jars, eg: [java]groovy -Dhttp.proxyHost=myProxyHost -Dhttp.proxyPort=8080[/java]

You may notice the needed JAR will be written in the $USER_HOME/.groovy/grapes folder.

From now and then, I am certified Scrum Master.

I attended the training lead by Jeff Sutherland (co-founder of Scrum method, CEO of Scrum Inc.).
Jeff gathers both practical knowledge from his personnal experience, and a deep work of research and reflexion on IT and processes around IT. As s skilled manager, engineer and orator, his training brings a lot of information and added value. Many thanks to him!

I’d like to thank Xebia France for their welcome, and, of course, Sungard Global Services and the BKN “Lean and Agile” for investing in its architects continuous improvement.

Case

From an Agile and TDD viewpoint, performing uni tests on DAO is a requirement. Sometimes, instead of using DBUnit datasets “out of the box”, the developper need test on actual data. In the same vein, when a bug appears on production, isolating and reproducing the issue is a smart way to investigate, and, along the way, fix it.
Therefore, how to export actual data from Oracle DB (or even MySQL, Sybase, DB2, etc.) to a DBUnit dataset as a flat XML file?

Here is a Runtime Test I wrote on this subject:

Fix

Spring

Edit the following Spring context file, setting the login, password, etc.
[xml]
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">

<!– don’t forget to write this, otherwise the application will miss the driver class name, and therfore the test will fail–>
<bean id="driverClassForName" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetClass" value="java.lang.Class"/>
<property name="targetMethod" value="forName"/>
<property name="arguments">
<list>
<value>oracle.jdbc.driver.OracleDriver</value>
</list>
</property>
</bean>
<bean id="connexion" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean"
depends-on="driverClassForName">
<property name="targetClass" value="java.sql.DriverManager"/>
<property name="targetMethod" value="getConnection"/>
<property name="arguments">
<list>
<value>jdbc:oracle:thin:@host:1234:SCHEMA</value>
<value>myLogin</value>
<value>myPassword</value>
</list>
</property>
</bean>

<bean id="databaseConnection" class="org.dbunit.database.DatabaseConnection">
<constructor-arg ref="connexion"/>
</bean>
<bean id="queryDataSet" class="org.dbunit.database.QueryDataSet">
<constructor-arg ref="databaseConnection"/>
</bean>
</beans>[/xml]

The bean driverClassForName does not look to be used ; anyway, if Class.forName("oracle.jdbc.driver.OracleDriver") is not called, then the test will raise an exception.
To ensure driverClassForName is created before the bean connexion, I added a attribute depends-on="driverClassForName". The other beans will be created after connexion, since Spring will deduce the needed order of creation via the explicit dependency tree.

Java

[java]public class Oracle2DBUnitExtractor extends TestCase {
private QueryDataSet queryDataSet;

@Before
public void setUp() throws Exception {
final ApplicationContext applicationContext;

applicationContext = new ClassPathXmlApplicationContext(
"lalou/jonathan/Oracle2DBUnitExtractor-applicationContext.xml");
assertNotNull(applicationContext);

queryDataSet = (QueryDataSet) applicationContext.getBean("queryDataSet");

}

@Test
public void testExportTablesInFile() throws DataSetException, IOException {
// add all the needed tables ; take care to write them in the right order, so that you don’t happen to fall on dependencies issues, such as ones related to foreign keys

queryDataSet.addTable("MYTABLE");
queryDataSet.addTable("MYOTHERTABLE");
queryDataSet.addTable("YETANOTHERTABLE");

// Destination XML file into which data needs to be extracted
FlatXmlDataSet.write(queryDataSet, new FileOutputStream("myProject/src/test/runtime/lalou/jonathan/output-dataset.xml"));

}
}[/java]

I wrote the following notice on Jean-Pierre ISOARD‘s profile on LinkedIn:

I have worked with Jean-Pierre for almost two years. Jean-Pierre has a wide knowledge of Java technologies and frameworks, plus a large experience of project management and team leading. Very committed to his tasks, Jean-Pierre releases a high quality work. Open-minded, he does not hesitate to challenge his ideas and positions, spotting at one point: provide the technical solution that fits the best the functional need. Therefore, I warmly recommend Jean-Pierre.

I wrote the following notice on Vincent NAEGELEN‘s profile on LinkedIn:

Vincent joined the Prime Brokerage team as an intern. He quickly integrated himself to the new project “from scratch” I was setting up, and learned various technologies of JEE world, such as GWT, Hibernate, Spring, etc.Vincent kept on working within my team after his internship, on a brand new and innovative application based on Mule ESB, giving satisfaction to all his colleagues. I strongly recommand Vincent for his technical skills and functionnal knowledge.

Case

I have the following block in my Mule config file:

[xml]<servlet:inbound-endpoint path="authenticationService" address="http://localhost:1234">[/xml]

. Even though this block matches XSD constraints, it is illegal from a functionnal point of view.

I get the following stacktrace:

[java]Offending resource: mule-config.xml; nested exception is org.springframework.beans.factory.BeanDefinitionStoreException: Unexpected exception parsing XML document from class path resource [lalou/jonathan/mule-config.xml]; nested exception is org.mule.config.spring.parsers.processors.CheckExclusiveAttributes$CheckExclusiveAttributesException: The attributes of Element servlet:inbound-endpoint{address=http://localhost:1234, name=.authenticationService:inbound.157:inbound-endpoint.158, path=authenticationService} do not match the exclusive groups [address] [ref] [path][/java]

Explanation and Fix

The exception is meaningful: for the tag <servlet:inbound-endpoint>, only one among the three following attributes is needed: path, ref and address.

To fix the issue, keep only the relevant attribute.

Case

I try to deploy a Mule ESB configuration, using this XML:

[xml]<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:pattern="http://www.mulesoft.org/schema/mule/pattern"
xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core
http://www.mulesoft.org/schema/mule/core/3.1/mule.xsd
http://www.mulesoft.org/schema/mule/pattern
http://www.mulesoft.org/schema/mule/pattern/3.1/mule-pattern.xsd
">
<pattern:simple-service name="authenticationService"
address="http://localhost:1234/authenticationService"
component-class="lalou.jonathan.esb.components.AuthenticationComponent"
type="direct" />
</mule>[/xml]

I get the following error:

[java]Ignored XML validation warning
org.xml.sax.SAXParseException: schema_reference.4: Failed to read schema document ‘http://www.mulesoft.org/schema/mule/pattern/3.1/mule-pattern.xsd'[/java]

Extended Stacktrace

[java]2011-11-22 16:10:25,375 WARN  xml.XmlBeanDefinitionReader         – Ignored XML validation warning
org.xml.sax.SAXParseException: schema_reference.4: Failed to read schema document ‘http://www.mulesoft.org/schema/mule/pattern/3.1/mule-pattern.xsd’, because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:195)
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.warning(ErrorHandlerWrapper.java:96)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:380)[/java]

Notice that Mule ESB files are similar to classic Spring files. Of course, I first checked the pointed XSD was actually reachable.
Anyway, this error should be raised when your application, for any reason -firewall, proxies, network interruption-, cannot access the remote site where the XSD is hosted.

Fix

• Copy the XSD to a local folder
• Create a file spring.schemas
• Make it available in the classpath in META-INF.
• Add the following line
• [java]http\://www.mulesoft.org/schema/mule/pattern/3.1/mule-pattern.xsd=WEB-INF/classes/mule-pattern.xsd[/java]

  The pattern is: missing resource (beware of escaping colon) = path in classpath of the local XSD

• Rebuild, pack and run!

A Short String

I have a table of which one field is a CLOB. Let’s say I have to insert one record with a short text. The following command is allowed:

[sql]INSERT INTO jonathan_table VALUES (1, ‘hello world!’);[/sql]

A Bigger Text

Error ORA-01704

Now, my text is larger, let’s say 5000 characters. When I launch the same query, I get the following error:

[sql]ORA-01704: string literal too long[/sql]

Indeed, Oracle/SQL*Plus have a limit on CLOB inserts: 4000 bytes.

Workaround

To pass through the limit on canonical SQL, you’ll have to use a PL/SQL procedure. The following command will be successful for any text larger than 4000 bytes, but shorter than 32767:

[sql]DECLARE
bigtext1   VARCHAR2 (32767);
BEGIN
bigtext1 := lpad(‘X’, 32000, ‘X’)
INSERT INTO jonathan_table VALUES (1, bigtext1);
END;[/sql]

An Even Bigger Text

Errors ORA-06550 and PLS-00103

You guess it: beyond this limit of 32 KB, an error occurs. So the following script:

[sql]DECLARE
bigtext1 VARCHAR2 (42000);
BEGIN
bigtext1 := lpad(‘X’, 42000, ‘X’)
INSERT INTO jonathan_table
VALUES (1, bigtext1);
END;
[/sql]

raises such an error:

[sql]Error at line 1
ORA-06550: line 5, column 4:
PLS-00103: Encountered the symbol "INSERT" when expecting one of the following:

. ( * % & = – + ; < / > at in is mod remainder not rem
<an exponent (**)> <> or != or ~= >= <= <> and or like like2
like4 likec between || multiset member submultiset
The symbol ";" was substituted for "INSERT" to continue.[/sql]

Fix this issue

I searched a lot to find an easy fix to go beyond the limit of 32KB. My point was that with Java for instance there is no limit of 32KB. In the same way, with TOAD I was able to update the record with many mega bytes of text, via the clipboard. After further search, I learnt that the 32KB barrier was a SQL*Plus limitation on actual strings, but the patterns insert into ... select ... from were not affected.
Here is the idea:

• create a temporary table
• split the text into blocks shorter than 32KB
• insert the blocks into the temporary table
• perform a first insert with a null CLOB
• update the record using a select on the temporary table (yet you can insert the actual value since previous step)

Here is an example:

[sql]DROP TABLE tt_jonathan_table;
CREATE GLOBAL TEMPORARY TABLE tt_jonathan_table
(
ID NUMBER(10),
pdlsuffix CLOB
) ON COMMIT PRESERVE ROWS;

TRUNCATE TABLE tt_jonathan_table;

DECLARE
bigtext1 VARCHAR2 (32767);
bigtext2 VARCHAR2 (32767);
BEGIN
bigtext1 := lpad(‘X’, 32000, ‘X’)
bigtext2 := lpad(‘Y’, 32000, ‘Y’)
INSERT INTO tt_jonathan_table
VALUES (1, bigtext1);

INSERT INTO tt_jonathan_table
VALUES (2, bigtext2);

INSERT INTO jonathan_table
(id, myClobField)
VALUES (jonathan_seq.NEXTVAL, NULL);

UPDATE jonathan_table
SET myClobField = (SELECT CONCAT (rls1.myClobField, rls2.myClobField)
FROM tt_jonathan_table rls1, tt_jonathan_table rls2
WHERE rls1.ID = 1 AND rls2.ID = 2)
WHERE myClobField is null;
END;
/

TRUNCATE TABLE tt_jonathan_table;
[/sql]

Case

Sometimes, you need test the Log4J’s loggers are called with the right parameters. How to perform these tests from with JUnit?

Let’s take an example: how to test these simple class and method?

[java]public class ClassWithLogger {
private static final Logger LOGGER = Logger.getLogger(ClassWithLogger.class);

public void printMessage(Integer foo){
LOGGER.warn(&quot;this is the message#&quot; + foo);
}
}[/java]

Example

Define an almost empty Log4J appender, such as:

[java]public class TestAimedAppender extends ArrayList&lt;String&gt; implements Appender {
private final Class clazz;

public TestAimedAppender(Class clazz) {
super();
this.clazz = clazz;
}

@Override
public void addFilter(Filter newFilter) {
}

@Override
public Filter getFilter() {
return null;
}

@Override
public void clearFilters() {
}

public void close() {
}

@Override
public void doAppend(LoggingEvent event) {
add(event.getRenderedMessage());
}

@Override
public String getName() {
return &quot;TestAppender for &quot; + clazz.getSimpleName();
}

@Override
public void setErrorHandler(ErrorHandler errorHandler) {
}

@Override
public ErrorHandler getErrorHandler() {
return null;
}

@Override
public void setLayout(Layout layout) {
}

@Override
public Layout getLayout() {
return null;
}

@Override
public void setName(String name) {
}

public boolean requiresLayout() {
return false;
}
}[/java]

Then create a TestCase with two fields:

[java]public class ClassWithLoggerUnitTest {
private ClassWithLogger classWithLogger;
private TestAimedAppender appender;
…
}
[/java]

In the setup, remove all appenders, create an instance of our appender, and then add it to the logger related to the class which we want to test:

[java] @Before
public void setUp() throws Exception {
final Logger classWithLoggerLogger = Logger.getLogger(ClassWithLogger.class);
classWithLoggerLogger.removeAllAppenders();
appender = new TestAimedAppender(ClassWithLogger.class);
classWithLoggerLogger.addAppender(appender);
appender.clear();

classWithLogger = new ClassWithLogger();
}[/java]

Then write the following test. The code is documented:

[java] @Test
public void testPrintMessage() throws Exception {
final String expectedMessage = &quot;this is the message#18&quot;;

// empty the appender
appender.clear();
// check it is actually empty before any call to the tested class
assertTrue(appender.isEmpty());

// call to the tested class
classWithLogger.printMessage(18);

// check the appender is no more empty
assertFalse(appender.isEmpty());
assertEquals(1, appender.size());
// check the content of the appender
assertEquals(expectedMessage, appender.get(0));

}[/java]

Conclusion

This basic example shows how to perform tests on logger, without overriding the original code or using mocks. Of course, you can improve this basic example, for instance in discriminating owing to the log level (INFO, WARN, ERROR, etc.), use generics, and even any other fantasy ;-).