Xor News

JNA is surely deserving of all the praise it has been getting. It’s being used on some pretty high profile projects like JRuby with great success. After having done JNI the hard way, the painful, tortuous, despicable, bang-head-on-keyboard-while-wondering-if-ReleaseStringUTFChars-applies-here and why-the-jvm-is-segfaulting-again way, well I have a deep appreciation for JNA.

Still who wants to go write a bunch of useless Java interfaces for stuff that already exists in built into the native library itself? Not me. So that’s where Jython and JRuby come in.

This week I needed Jython/Python access to some native modules, namely ssdeep for fuzzy hashing. There’s already a pretty nice solution for connecting pure python to native libraries — you can either use swig or pyrex. The pyrex piece for ssdeep has been mostly written here. Needed to add the fuzzy_hash_buf method into that mix but it was nice and easy. From inside pure python with ssdeepmodule.so (via pyrex) and libfuzzy.so (from ssdeep) (or .dylib for Mac or .dll on Windows) sitting there on your LD_LIBRARY_PATH, you get to do this coolness:

  from ssdeep import ssdeep
  import sys, os
  f = open("/bin/ls","rb")
  data = f.read()
  f.close()
  ss = ssdeep()
  fuzzy_hash = ss.fuzzy_hash_buf(data)

Pretty nice you have to admit. But from my pure java p2p data-driven workflow framework, I really wanted to do this from Jython to keep from having to start the interpreter up in a subprocess over and over. Pyrex extensions do not work in Jython. Makes perfect sense. M’kay. I could write a whole bunch of lame JNi code to hook libfuzzy.so in there. Or I could use JNA and write some non-dry interface in java and figure out all the details of the types and so forth. Or … I could just push all that code down into the python module that I’m going to call.

from com.sun.jna import NativeLibrary, Function, Memory
import sys,os

class ssdeep:
    fuzlib = None
    hash_func = None
    FUZZY_HASH_SIZE = 116

    def __init__(self):
        self.fuzlib = NativeLibrary.getInstance('fuzzy')
        self.hash_func = fuzlib.getFunction('fuzzy_hash_buf')
        pass

    def hash_data(self,data):
        ptr = Memory(self.FUZZY_HASH_SIZE)
        i = self.hash_func.invokeInt([data,len(data),ptr])
        return ptr.getString(0,False)

With a class and method conveniently named exactly the same as the pyrex module I can make it all flexible enough to work either way:

try:
   # try the pyrex extension module
   from ssdeep import ssdeep
except ImportError:
   try:
     # try the jna wrapper when in jython
     from ssdeepjna import ssdeep
    except ImportError:
      # write tmp files and just exec the dumb thing

Which is all pretty nice I think. Not too many worries about creating interfaces or other crazy things. Seems very efficient, a little extra packaging and we are good to go.

Well there was one problem in getting different values from the hashes when the data was binary. Turns out the JNA layer needs to be told how to convert data with -Djna.encoding=8859_1 on the JVM command line. Since I usually run with -Dfile.encoding=UTF-8 and in a UTF-8 locale, this made all the difference.

If that is inconvenient or you want to encode things differently only sometimes, the extra steps in the python layer would be something like

  from java.lang import String
  def wrap_hash_buf(self,data):
    javastr = String(data,"8859_1")
    jbytes = javastr.getBytes("8859_1")
    return hash_buf(jbytes)

The same type of thing would work just as well from JRuby.

Welcome to the sweet spot. Code on, baby!

In a Rails system with a pretty standard user model, views, and controller, I recently found that the users_controller was getting pretty large. There was a pretty significant amount of code that I wanted to add to deal with the users using a different set of views under some circumstances — and I didn’t want to add all of that to the users_controller.rb. What I wanted was to move these functions to a staff_controller.rb, use restful routing on it and just treat the user object differently.

This didn’t work out so well initially. Trying to use form_for in a view always reflects on the type of object you have and sends the results right back to the users_controller — not what I wanted. I tried a bunch of stuff to force the form_for to do what I wanted but no joy. What I wanted I thought was a way to alias the user model.

class Staff < User
end

Problem solved.

The User model had some nice named scopes for finders, for example:

  named_scope
          :recent,
          :conditions => ["created_at > ?", 2.weeks.ago]

And now I can do all of the following

  Staff.first
  Staff.recent
  Staff.find(params[:id])
  @staff = Staff.recent.find(:all,
              :conditions => ['name = ?',"jim"])
  @staff.last_login_at = Time.now
  @staff.save!

.. which since reflection now says they are staff objects, causes
all the view helpers that reflect on stuff to work properly and doesn’t
break any of the ActiveRecord stuff.

The typical Ruby on Rails flash area is only drawn when there is a message to flash. I think something like this is fairly common (this example is haml, but that’s not the point) usually done in a partial, something like app/views/shared/_flash.html.haml:

- %w(info notice error).each do |type|
  - if flash[type.to_sym]
    .warning.flash_message{:class => "#{type.to_s}"}
      %span= h(flash[type.to_sym])
      %small
        %a{:href=>"#", :onclick=>"$$('div.flash_message')[0].hide();return false"} Close this message
 

The problem with this is that if you are Ajaxing a bunch of stuff you could use the flash area for user interaction and highlighting messages, but if it isn’t there, that could be a problem. So how about doing this instead so that the flash area is always drawn, just not displayed:

- %w(info notice error).each do |type|
  .warning.flash_message{ :class => "#{type}", :id => "flash_#{type}", :style => "display: #{flash[type.to_sym] ? 'block' : 'none'}" }
    %span
      - if flash[type.to_sym]
        = h(flash[type.to_sym])
    %small
      %a{:href=>"#", :onclick=>"$$('div.#{type.to_s}')[0].hide();return false"} Close this message

The flash closer is still in there so we don’t have to worry about the complexity of building that again every time, and the message is in a separate span element so it is easy to fill in. So then we can add a little bitty application helper (app/helpers/application_helper.rb)

  def flashnow(page,msg)
    page.select("#flash_notice span").first.replace("#{msg}")
    page.select("#flash_notice").first.show
  end

You might need to change the replace method for replaceHtml
depending on the version of prototype you are working with.
and call it from any view that is building an rjs response:

  flashnow(page,'Yeeha, cowboy, it worked!')