Jpa 2.0 performing a count on a query with joins

The Ninja's Apprentice

It’s been too long since I’ve blogged. Here is an interesting one. I recently had to modify old code. They were using jpa 2.0 in more of a 1.0 syntax. There was a very complex query with joins used for searching. There was pagination, sort of. They were not returning the total record count. So I decided to do a count on the same query as a happy medium until we can upgrade to a newer way of doing things.

I tried using the same query and predicates to do the count. I tried following several examples from stack overflows and forums. But, all the examples were with simple queries that did not do joins.

The biggest thing here is that I had to use different query objects and different predicate objects, while using the same root when creating the count predicates and the count query. It appears that query.from()…

View original post 57 more words

Posted in Uncategorized | Leave a comment

Hibernate Logging (e.g. JBOSS logging)

I was getting the following debug logging, even though my logging config said info and above.

15:14:02.541|DEBUG|RdbmsOperation with SQL [p_select_validate_pin_person_token] compiled||org.springframework.jdbc.object.RdbmsOperation:344  
15:14:02.541|DEBUG|Compiled stored procedure. Call string is [{call p_select_validate_sso_person_token(?, ?)}]||org.springframework.jdbc.object.SqlCall:154  
15:14:02.541|DEBUG|RdbmsOperation with SQL [p_select_validate_sso_person_token] compiled||org.springframework.jdbc.object.RdbmsOperation:344  
15:14:02.542|DEBUG|Compiled stored procedure. Call string is [{call p_delete_truncate(?)}]||org.springframework.jdbc.object.SqlCall:154  
15:14:02.542|DEBUG|RdbmsOperation with SQL [p_delete_truncate] compiled||org.springframework.jdbc.object.RdbmsOperation:344  
15:14:02.543|DEBUG|Compiled stored procedure. Call string is [{call p_update_setting_inprogress(?)}]||org.springframework.jdbc.object.SqlCall:154  

During my research, I learned that Hibernate is logged via JBoss Logging. It used to require a properties file that denoted which logging implementation to use. It now will try to use several logging frameworks, such as commons-logging, log4j and logback. I use Logback.

One common problem is that if you are pulling in lots of dependent libraries, it isn’t uncommon to have multiple logging APIs preset. That was the case that I had.

I checked my dependencies and discovered that I had libraries for:

  • logback-classic – logback is my preferred logging framework.
  • commons-logging – this is the culprit. The JBoss logging API is finding this framework, but isn’t finding a configuration file so everything is being logged.
  • jboss-logging – this is the jboss logging api and is required
  • log4j-over-slf4j – this is the slf4j implementation of the log4j api. In essence, hijacking log4j and funnelling the log4j logging events through slf4j.

To fix this, I had to find every Maven dependency that has a transitive dependency on commons-logging and exclude commons-logging. An example of something that had commons-logging as a transitive dependency was spring-webmvc. I modified the dependency by adding exclusions.

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-webmvc</artifactId>
            <exclusions>
                <exclusion>
                    <artifactId>commons-logging</artifactId>
                    <groupId>commons-logging</groupId>
                </exclusion>
            </exclusions>
        </dependency>

Once I removed all commons-logging transitive dependencies, I had to provide a suitable substitute. This is where another SLF4j dependency comes in handy. This is a SLF4J implementation of the commons-logging api which funnels all of the commons-logging events to my slf4J implementation.

        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>jcl-over-slf4j</artifactId>
            <version>1.7.7</version>
        </dependency>
Posted in hibernate, hibernate logging, jboss, jboss logging, jcl-over-slf4j, log4j-over-slf4j, logback, Logging, slf4j | Tagged , , , , , , , , | 1 Comment

Logback log filtering

We wanted to default logging of all messages to warning and above for libraries that we used. We wanted our code to be logged at the info level and above. There was one specific library class that was logging a warning message that was causing confusion to the junior members of the team. Here is what my logback.xml contains.

  • Line 9 – log classes under the com.cdi package at the info level and above
  • Line 11 – log this specific class at the error level and above
  • Line 13 – default logging of the standard out appender to warnings and above
<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true">
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <layout class="ch.qos.logback.classic.PatternLayout">
            <Pattern>%d{HH:mm:ss.SSS}|%-5level|%msg||%class:%line %xException{full} %n</Pattern>
        </layout>
    </appender>

    <logger name="com.cdi" level="INFO"/>
    <!--Don't show the warning message for no ViewResolvers-->
    <logger name="org.springframework.web.servlet.view.ContentNegotiatingViewResolver" level="ERROR"/>

    <root level="WARN">
        <appender-ref ref="STDOUT"/>
    </root>
</configuration>
Posted in appender, appender configuration, logback, Logging, Logging configuration, slf4j | Tagged , , , , , | Leave a comment

Spring C-notation

In a previous post, I talked about Spring P-notation.  In this blog post, I will give an example of the Spring C-notation for constructor arguments.

The C-notation namespace needs to be added to your configuration file.

xmlns:c="http://www.springframework.org/schema/c"

All that is left is to use it for a constructor argument

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:task="http://www.springframework.org/schema/task"
       xmlns:util="http://www.springframework.org/schema/util"
       xmlns:p="http://www.springframework.org/schema/p"
       xmlns:c="http://www.springframework.org/schema/c"
       xsi:schemaLocation="http://www.springframework.org/schema/beans   http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
                           http://www.springframework.org/schema/task    http://www.springframework.org/schema/task/spring-task.xsd
                           http://www.springframework.org/schema/util    http://www.springframework.org/schema/util/spring-util.xsd">

    <!-- removed unnecessary configurations -->

    <bean id="requestFactory" class="org.springframework.http.client.HttpComponentsClientHttpRequestFactory"
          p:readTimeout="250"
          p:connectTimeout="250"/>

    <bean id="restTemplate" class="org.springframework.web.client.RestTemplate"
          c:requestFactory-ref="requestFactory"/>
</beans>
Posted in Java | Tagged , , , , , , , , , | Leave a comment

Random Number Generator provider abstract class

I had a need to support multiple hardware random number generators (RNG) in a project. I started with an RngProvider abstract class. This class defines the functionality that will be the same and defines a method that will contain the functionality that is unique per RNG.

Queue size
We set the desired queue length to be 80k integers per hardware RNG which will allow us to cache enough integers to make over 3k decks of cards. 80k integers per integration doesn’t take up any space worth worrying about. 80k * 4 / 1024 = 312.5kb. That is practically free. Cache away. A queue fill operation is going to be triggered when the queue is 1/4 full.

LinkedBlockingQueue
I chose a linked blocking queue because I needed synchronized access to prevent the random integer from being selected more than once. The blocking operations are within the BlockingQueue interface. I used put() and take() to queue and dequeue with blocking until the queue is available.

Available
I created this property as a way to add or remove the device without having to stop/start the server, if the device supports that kind of functionality. The thinking was that the constructor would set it to available, the nextInt() would check to see if it was empty and set it to unavailable, if necessary, and the implementation of the getRandomIntegerData() method would set it to available as it adds integers.

getRandomIntegerData()
This method defines the method that would be filled in by the specific implementation needed by the various hardware devices.

nextInt()
This method is used to retrieve an integer from the specific integer queue and also start the thread that will manage adding integers to the queue, if necessary.

FillQueue
This thread is created when a number is requested to be removed from the queue. If the size of the queue has dropped below a specified number, then the getRandomIntegerData() method is called to replenish the integer data. The block of code that determines if the integer data is to be replenished is synchronized to prevent multiple thread executions from trying to replenish the queue at the same time.

package com.cdi.rng;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;

/**
 * Interface to define random number generator providers.
 * @author norris.shelton
 */
public abstract class RngProvider {
    /** SLF4J logger object. */
    protected Logger logger = LoggerFactory.getLogger(getClass());
    /** The desired size of the integer queue. */
    public static final int DESIRED_QUEUE_LENGTH = 80000;
    /** The minimum integer queue size. */
    public static final int MIN_QUEUE_LENGTH = DESIRED_QUEUE_LENGTH / 4;
    /** Is this RNG provider available. */
    boolean available = false;
    /** Queue of integers for consumption. */
    protected BlockingQueue<Integer> integerQueue = new LinkedBlockingQueue<>(DESIRED_QUEUE_LENGTH);

    /**
     * Is this random number generator provider available.
     * @return true if available for use, else false
     */
    public boolean isAvailable() {
        return available;
    }

    /**
     * Sets that this random number generator is available.
     * @param available true if available, else false
     */
    public void setAvailable(boolean available) {
        this.available = available;
    }

    /**
     * Fills the integer queue with integers.
     * @throws Exception
     */
    public abstract void getRandomIntegerData() throws Exception;

    /**
     * Return the next random integer.
     * @return random signed integer
     */
    public int nextInt() throws InterruptedException {
        logger.debug("Using {} provider", getClass());
        FillQueue fillQueue = new FillQueue();
        fillQueue.run();

        // if taking the last one
        if (integerQueue.size() <= 1) {
            setAvailable(false);
            logger.warn("Pulling the last integer from {} queue", getClass());
        }

        return integerQueue.take();
    }

    /**
     * Thread used to fill the queue asynchronously.
     */
    public class FillQueue extends Thread {
        /**
         * When an object implementing interface <code>Runnable</code> is used to create a thread, starting the thread
         * causes the object's <code>run</code> method to be called in that separately executing thread.
         * <p/>
         * The general contract of the method <code>run</code> is that it may take any action whatsoever.
         * @see Thread#run()
         */
        @Override
        public void run() {
            synchronized (this) {
                if (integerQueue.size() < MIN_QUEUE_LENGTH) {
                    try {
                        getRandomIntegerData();
                    } catch (Exception e) {
                        logger.error("Unable to add integers to queue by thread", e);
                    }
                }
            }
        }
    }
}
Posted in Java | Tagged , , , , , , , , , , | Leave a comment

Filling a queue asynchronously via a thread

I was working with a hardware random number generator (RNG) and what it came down to was we needed a cache of numbers that were available, but also needed the fill the numbers without slowing the process of giving out numbers. I settled on storing the data in a queue. I used a blocking queue to prevent the consumers from ever getting the same number.

private Queue<Integer> values = new LinkedBlockingQueue<>();

When a number was pulled off the queue, I also started a thread that would ensure that the queue was never empty.

    @Override
    public int nextInt(int min, int max) {
        FillQueue fillQueue = new FillQueue();
        fillQueue.run();

        return values.poll();
    }

The thread that filled the queue was an inner-class.

    /**
     * Thread used to fill the queue asynchronously.
     */
    private class FillQueue extends Thread {

        /**
         * When an object implementing interface <code>Runnable</code> is used to create a thread, starting the thread
         * causes the object's <code>run</code> method to be called in that separately executing thread.
         * <p/>
         * The general contract of the method <code>run</code> is that it may take any action whatsoever.
         * @see Thread#run()
         */
        @Override
        public void run() {
            if (values.size() < MIN_QUEUE_LENGTH) {
                try {
                    logger.info("Retrieving {} integers from Comscire", START_QUEUE_LENGTH - MIN_QUEUE_LENGTH);
                    getIntegersFromConscireProcess(START_QUEUE_LENGTH - MIN_QUEUE_LENGTH);
                } catch (IOException e) {
                    logger.error("Unable to add integers to queue by thread", e);
                }
            }
        }
    }
Posted in Java | Tagged , , , , , , | Leave a comment

Apache HttpClient 4 byte handling

After I wrote my own HttpClient ResponseHandler in Apache HttpClient 4 Custom Bytes ResponseHandler, I discovered that there might be a better way. I had used the following utility to get the bytes. I found that one of their classes also used it.

EntityUtils.toByteArray(entity);

Here is what I came up with…

    /**
     * Get the weather radar bytes image.
     * @return byte[] of weather radar data
     */
    @Nullable
    public BufferedHttpEntity getWeatherRadarBytes() {
        HttpClient httpClient = new DefaultHttpClient();
        HttpGet httpGet = new HttpGet("http://services.intellicast.com/200904-01/387694749/Image/Radar/Winter%20Storm%20Mosaic/Loop/SectorName/bwg");


        BufferedHttpEntity bufferedHttpEntity = null;
        try {
            HttpResponse httpResponse = httpClient.execute(httpGet);
            HttpEntity httpEntity = httpResponse.getEntity();
            bufferedHttpEntity = new BufferedHttpEntity(httpEntity);
        } catch (IOException e) {
            log.error("IOException", e);
        } finally {
            // When HttpClient instance is no longer needed,
            // shut down the connection manager to ensure
            // immediate deallocation of all system resources
            httpClient.getConnectionManager().shutdown();
        }
        return bufferedHttpEntity;
    }

An example of using it would be something like the following. NOTE: I never ran this code. Use at your own risk.

    @Test
    public void testGetWeatherRadar() {
        BufferedHttpEntity bufferedHttpEntity = httpService.getWeatherRadarBytes();
        assertNotNull(bufferedHttpEntity);
        assertTrue(bufferedHttpEntity.getContentLength() > 0);

        FileOutputStream fileOutputStream = null;
        try {
            File file = new File("/staticfiles/weather-portlet/kyRadarImage.gif");
            fileOutputStream = new FileOutputStream(file);
            byte[] bytes = new byte[bufferedHttpEntity.getContent().available()];
            bufferedHttpEntity.getContent().read(bytes);
            fileOutputStream.write(bytes);
            fileOutputStream.flush();
            fileOutputStream.close();
        } catch (FileNotFoundException e) {
            fail(e.getMessage());
        } catch (IOException e) {
            fail(e.getMessage());
        } finally {
            if (fileOutputStream != null) {
                try {
                    fileOutputStream.close();
                } catch (IOException e) {
                    fail(e.getMessage());
                }
            }
        }
    }
Posted in Java | Tagged , , , , | Leave a comment