Tuesday, September 9, 2014

How to make an asynchronous processor in Apache Camel for generating and streaming files

We've had a case where we needed to generate a PDF file and stream it back to the client from Camel. The PDF generating library expects an OutputStream to write to, and as a file can grow big, it was important not to buffer an entire stream in memory. While Camel already provides many options when it comes to concurrency, they are centered on invoking a route asynchronously, which was not (or at least not nicely) applicable to our case as we want only a part of the logic to run in a separate thread. We needed an asynchronous processor. An interface AsyncProcessor already exists, but it only provides a callback to notify the caller of job done, but no actual mechanism to execute logic asynchronously. This needs to be done manually, but luckily, Camel does offer an easy way to create thread pools and use them to submit asynchronous tasks. To create a thread pool, add a stanza similar to the following into Camel's Spring configuration, inside the camelContext:

<threadPool id="smallPool" threadName="pdfWriter"
 poolSize="5" maxPoolSize="30" maxQueueSize="50"/>

This pool can then be used as normal to invoke routes asynchronously:

    <from uri="direct:start"/>
    <to uri="log:start"/>
    <threads executorServiceRef="smallPool">
        <to uri="log:hello"/>

but this is not what we're after. Instead, we can inject this pool into an AsyncProcessor where we can submit a task to it. In the following example, the processor with create an instance of a CircularByteBuffer and give its OutputStream to the PDF generation process running in a separate thread to write to, and immediately return the associated InputStream as a response (without blocking or buffering everything) to be consumed by the client.

public class PdfGenerationProcessor implements AsyncProcessor {
    private ExecutorService smallPool;
        public void process(Exchange exchange) throws Exception {
        AsyncProcessorHelper.process(this, exchange); //Wrap synchronous invocations
    public boolean process(Exchange exchange, AsyncCallback callback) {
       CircularByteBuffer buffer = new CircularByteBuffer(1024 * 100); //Buffer a 100K
       smallPool.submit(new PDFGenerator(exchange, callback, buffer.getOutputStream())); //Generate the PDF in a separate thread
       exchange.getIn().setBody(buffer.getInputStream()); //Immediately return the stream for the client to read from
       return false; //Signifies that the job will be performed asynchronously
   private class PDFGenerator implements Runnable {
       private Exchange exchange;
       private AsyncCallback callback;
       private OutputStream out;
       private PDFGenerator(Exchange exchange, AsyncCallback callback, OutputStream out) {
           this.exchange = exchange;
           this.callback = callback;
           this.out = out;
       public void run() {
           try {
               generatePDF(out); //Actual logic for generating the file and writing it to the stream is here
           } catch (Exception e) {
               exchange.setException(e); //Async processors must not throw exceptions and must add them to the Exchange instead
           } finally {
               // callback must be invoked
               //Cleanup, close streams etc.
               try{out.flush(); out.close();} catch(Exception e) {/*ignore*/}


  • The library used here (CircularByteBuffer) is a convenient implementation of piped streams (see more here). To use it, add this to Maven:
  • Camel in Action, chapter 10 has very good guides for concurrency in Camel

Saturday, July 5, 2014

Spring Security Run-As example using annotations and namespace configuration

Spring Security offers an authentication replacement feature, often referred to as Run-As, that can replace the current user's authentication (and thus permissions) during a single secured object invocation. Using this feature makes sense when a backend system invoked during request processing requires different privileges than the current application.
For example, an application might want to expose a financial transaction log to the currently logged in user, but the backend system that provides it only permits this action to the members of a special "auditor" role. The application can not simply assign this role to the user as that would potentially permit them to execute other restricted actions. Instead, the user can be given this right exclusively for viewing their transaction log.

Only two classes are used to implement this feature. Instances of RunAsManager are tasked with producing the actual replacement authentication tokens. A sensible default implementation is already provided by Spring Security. As with other types of authentication, it is also necessary to register an instance of an appropriate AuthenticationProvider.
<bean id="runAsManager"
  <property name="key" value="my_run_as_key"/>

<bean id="runAsAuthenticationProvider"
  <property name="key" value="my_run_as_key"/>
Tokens produced by runAsManager are signed with the provided key (my_run_as_key in the example above) and are later checked against the same key by runAsAuthenticationProvider, in order to mitigate the risk of fake tokens being provided. These keys can have any value, but need to be the same in both objects. Otherwise, runAsAuthenticationProvider will reject the produced tokens as invalid.

If an instance is registered, RunAsManager will be invoked by AbstractSecurityInterceptor for every intercepted object invocation for which the user has already been given access. If RunAsManager returns a token, this token will be used be used instead of the original one for the duration of the invocation, thus granting the user different privileges. There are two key points here. In order for the authentication replacement feature to do anything, the call has to actually be secured (and thus intercepted), and the user has to already have been granted access.

To register a RunAsManager instance with the method security interceptor, something similar to the following is needed:
<global-method-security secured-annotations="enabled" run-as-manager-ref="runAsManager"/>
Now, all methods secured by the @Secured annotation will be able to trigger RunAsManager. One important point here is that global-method-security will only work in the Spring context in which it is defined. In Spring MVC applications, there usually are two Spring contexts: the parent context, attached to ContextLoaderListener, and the child context, attached to DispatcherServlet. To secure Controller methods in this way, global-method-security must be added to DispatcherServlet's context. To secure methods in beans not in this context, global-method-security should also be added to ContextLoaderListener's context. Otherwise, security annotations will be ignored.

The default implementation of RunAsManager (RunAsManagerImpl) will inspect the secured object's configuration and if it finds any attributes prefixed with RUN_AS_, it will create a token identical to the original, with the addition of one new GrantedAuthorty per RUN_AS_ attribute found. The new GrantedAuthority will be a role (prefixed by ROLE_ by default) named like the found attribute without the RUN_AS_ prefix.

So, if a user with a role ROLE_REGISTERED_USER invokes a method annotated with @Secured({"ROLE_REGISTERED_USER","RUN_AS_AUDITOR"}), e.g.

public class TransactionLogController {

 @Secured({"ROLE_REGISTERED_USER","RUN_AS_AUDITOR"}) //Authorities needed for method access and authorities added by RunAsManager prefixed with RUN_AS_
 @RequestMapping(value = "/transactions",  method = RequestMethod.GET) //Spring MVC configuration. Not related to security
 @ResponseBody //Spring MVC configuration. Not related to security
 public List<Transaction> getTransactionLog(...) {
  ... //Invoke something in the backend requiring ROLE_AUDITOR

 ...  //User does not have ROLE_AUDITOR here
the resulting token created by RunAsManagerImpl with be granted ROLE_REGISTERED_USER and ROLE_AUDITOR. Thus, the user will also be allowed actions, normally reserved for ROLE_AUDITOR members, during the current invocation, permitting them, in this case, to access the transaction log.

To enable runAsAuthenticationProvider, register it as usual:
<authentication-manager alias="authenticationManager">
    <authentication-provider ref="runAsAuthenticationProvider"/>
    ... other authentication-providers used by the application ...
This is all that is necessary to have the default implementation activated.

Still, this setting will not work for methods secured by @PreAuthorize and @PostAuthorize annotations as their configuration attributes are differently evaluated (they are SpEL expressions and not a simple list or required authorities like with @Secured) and will not be recognized by RunAsManagerImpl. For this scenario to work, a custom RunAsManager implementation is required, as, at least at the time of writing, no applicable implementation is provided by Spring.

A custom RunAsManager implementation for use with @PreAuthorize/@PostAuthorize

A convenient implementation relying on a custom annotation is provided below:

public class AnnotationDrivenRunAsManager extends RunAsManagerImpl {

    public Authentication buildRunAs(Authentication authentication, Object object, Collection<ConfigAttribute> attributes) {
        if(!(object instanceof ReflectiveMethodInvocation) || ((ReflectiveMethodInvocation)object).getMethod().getAnnotation(RunAsRole.class) == null) {
            return super.buildRunAs(authentication, object, attributes);

        String roleName = ((ReflectiveMethodInvocation)object).getMethod().getAnnotation(RunAsRole.class).value();
        if (roleName == null || roleName.isEmpty()) {
            return null;

        GrantedAuthority runAsAuthority = new SimpleGrantedAuthority(roleName);
        List<GrantedAuthority> newAuthorities = new ArrayList<GrantedAuthority>();
        // Add existing authorities
        // Add the new run-as authority

        return new RunAsUserToken(getKey(), authentication.getPrincipal(), authentication.getCredentials(),
                newAuthorities, authentication.getClass());

This implementation will look for a custom @RunAsRole annotation on a protected method (e.g. @RunAsRole("ROLE_AUDITOR")) and, if found, will add the given authority (ROLE_AUDITOR in this case) to the list of granted authorities. RunAsRole itself is just a simple custom annotation:
public @interface RunAsRole {
    String value();
This new implementation would be instantiated in the same way as before:
<bean id="runAsManager"
  <property name="key" value="my_run_as_key"/>
And registered in a similar fashion:
<global-method-security pre-post-annotations="enabled" run-as-manager-ref="runAsManager">
    <expression-handler ref="expressionHandler"/>
The expression-handler is always required for pre-post-annotations to work. It is a part of the standard Spring Security configuration, and not related to the topic described here. Both pre-post-annotations and secured-annotations can be enabled at the same time, but should never be used in the same class. The protected controller method from above could now look like this:
public class TransactionLogController {

 @PreAuthorize("hasRole('ROLE_REGISTERED_USER')") //Authority needed to access the method
 @RunAsRole("ROLE_AUDITOR") //Authority added by RunAsManager
 @RequestMapping(value = "/transactions",  method = RequestMethod.GET) //Spring MVC configuration. Not related to security
 @ResponseBody //Spring MVC configuration. Not related to security
 public List<Transaction> getTransactionLog(...) {
  ... //Invoke something in the backend requiring ROLE_AUDITOR

 ... //User does not have ROLE_AUDITOR here