Computers


In a Rails system with a pretty standard user model, views, and controller, I recently found that the users_controller was getting pretty large. There was a pretty significant amount of code that I wanted to add to deal with the users using a different set of views under some circumstances — and I didn’t want to add all of that to the users_controller.rb. What I wanted was to move these functions to a staff_controller.rb, use restful routing on it and just treat the user object differently.

This didn’t work out so well initially. Trying to use form_for in a view always reflects on the type of object you have and sends the results right back to the users_controller — not what I wanted. I tried a bunch of stuff to force the form_for to do what I wanted but no joy. What I wanted I thought was a way to alias the user model.

class Staff < User
end

Problem solved.

The User model had some nice named scopes for finders, for example:

  named_scope
          :recent,
          :conditions => ["created_at > ?", 2.weeks.ago]

And now I can do all of the following

  Staff.first
  Staff.recent
  Staff.find(params[:id])
  @staff = Staff.recent.find(:all,
              :conditions => ['name = ?',"jim"])
  @staff.last_login_at = Time.now
  @staff.save!

.. which since reflection now says they are staff objects, causes
all the view helpers that reflect on stuff to work properly and doesn’t
break any of the ActiveRecord stuff.

I may write up a better review of the new Canon A650 that I got for my birthday once I figure out a little more about how to use all of the fine features and get a little more experience. I’ve been too busy to take very many pictures recently, but I did experiment with the panorama stitching feature while on vacation in Waves, NC.
Outer Banks, Waves, NC

The image is large (9MB) so don’t click if you are on dial-up. Does dial-up still exist?

Upon putting the camera into panorama mode, the next step is to indicate whether you will shoot moving left or moving right. The camera then lets you take up to 26 pictures and keeps showing the last 1/3 of the previous picture on the large LCD screen as an aid to lining up the next shot.

Having taken all the pictures they should then be downloaded to a folder on the computer. This panorama comprises 13 images shot at 3 MP resolution (the camera does 12.1 MP) and then stitched using the Canon software on Mac OS X. The stitching process took about 4 seconds. I think it did a pretty reasonable job and I’m excited to try it out some more.

I got sucked into a trap. In a Rails migration it’s so easy to use an ActiveRecord::Base model object to update some stuff, and it looks nice and DRY. But it is extremely fragile. For example suppose your domain has a user and sponsor model. Each user has a sponsor. You start out this way:

   def self.up
      create table :users do |t|
         t.string :name
         t.string :password
         t.string :sponsor_name
         t.timestamps
      end
   end

Later you want to insist that every user in the system have a sponsor. You already have a bunch of test data that you are intellectually invested in, so you whip up a quick migration to make sure.

  def self.up
     User.find(:all, :conditions => 'sponsor_name IS NULL').each do |u|
         u.sponsor_name = "The Default"
         u.save!
      end
  end

Cool. Works good. You can go up and down. But some time later you realize the supreme error of your ways. You should have had a sponsor model and the user should have a sponsor_id, not a sponsor_name. Doh! What was I thinking. Time for another migration:

   def self.up
      add_column :users, :sponsor_id, :integer
      drop_column :users, :sponsor_name
   end

and the corresponding self.down method too, and the necessary belongs_to, has_one in the user and sponsor model classes. But now you can’t migrate the database up from bootstrap any more because your use of the the User model with u.sponsor_name above will not work. The user no longer has a sponsor_name.

The root of the issue is that the model objects are not versioned along with the database as it goes up and down. Since fixtures just use the same model objects that doesn’t help one bit.

The only reliable way we have come up with is to inject data using raw sql. So step two above
could change to

  def self.up
     execute 'update users set sponsor_name = "The Default" where sponsor_name IS NULL'
  end

That works, up and down the line, because the database sql IS in sync with the database at every step.

Have a better way to write less brittle migrations? Let me know in the comments.

The typical Ruby on Rails flash area is only drawn when there is a message to flash. I think something like this is fairly common (this example is haml, but that’s not the point) usually done in a partial, something like app/views/shared/_flash.html.haml:

- %w(info notice error).each do |type|
  - if flash[type.to_sym]
    .warning.flash_message{:class => “#{type.to_s}”}
      %span= h(flash[type.to_sym])
      %small
        %a{:href=>”#”, :onclick=>”$$(’div.flash_message’)[0].hide();return false”} Close this message
 

The problem with this is that if you are Ajaxing a bunch of stuff you could use the flash area for user interaction and highlighting messages, but if it isn’t there, that could be a problem. So how about doing this instead so that the flash area is always drawn, just not displayed:

- %w(info notice error).each do |type|
  .warning.flash_message{ :class => "#{type}", :id => "flash_#{type}", :style => "display: #{flash[type.to_sym] ? ‘block’ : ‘none’}” }
    %span
      - if flash[type.to_sym]
        = h(flash[type.to_sym])
    %small
      %a{:href=>”#”, :onclick=>”$$(’div.#{type.to_s}’)[0].hide();return false”} Close this message

The flash closer is still in there so we don’t have to worry about the complexity of building that again every time, and the message is in a separate span element so it is easy to fill in. So then we can add a little bitty application helper (app/helpers/application_helper.rb)

  def flashnow(page,msg)
    page.select("#flash_notice span").first.replace("#{msg}“)
    page.select(”#flash_notice”).first.show
  end

You might need to change the replace method for replaceHtml
depending on the version of prototype you are working with.
and call it from any view that is building an rjs response:

  flashnow(page,'Yeeha, cowboy, it worked!')

I guess this would be in the “You know you have a problem when…” category, but I don’t want to make one of those because, well, uh, too many things might end up there.

But when you first see one of these, a chisel bladed jack-hammer bit, and the first though is “Man, XML is even pervasive in *that* industry too!”, well, I don’t know what other category it could go in.
Asynchronous JackHammer and XML

I can’t believe this never occurred to me before. Use GNU date(1) to convert between various date formats


   mike% date '+%Y%j' -d 20080430
   2008121

And for moving a date around in time


  mike% date '+%Y%j' -d yesterday
  2008121

Sweet.

Cloud Computing might be a big waste of time. It sure has
a lot of hype to it. Google does this, google does that.
Whatever. What do the numbers say? Here is cloud by the
numbers.

  1. Cloud One Flying CLub - mkay http://cloudone.org/
  2. A dead-end line drawing of a cloud - boring http://cloud2.com/
  3. The theatrical portfolio of Senda ED Shallow - whatever http://cloudthree.net
  4. CloudFour.com - Expert Web and Mobile design, development and strategy - blah blah blah http://cloudfour.com/
  5. Five computer clouds are all we need - http://gigaom.com/2007/11/22/five-computer-clouds-are-all-we-need/
  6. Unit 4 Cloud6, why? http://ocw.usu.edu/Forest__Range__and_Wildlife_Sciences/Wildland_Fire_Management_and_Planning/cloud6.jpg/view
  7. Dedicated to 100 years of underachievement by the Hull City Football Club - ’nuff saidhttp://www.oncloudseven.com/
  8. Cloudeight - Home of the Web’s Best and Most Popular Email Stationery - right, gotta have it http://www.cloudeight.net/
  9. Cloud Nine Super Shuttle - Cumulonimbus - http://cloudnineshuttle.com/
  10. Cloud Ten Pictures - Home of the Left Behind series - blech http://cloudtenpictures.com/
  11. Cloud Eleven, a band in L.A. - http://cloudeleven.com/
  12. Cloud Twelve mood music - that’s so last century http://www.cloudtwelve.com/
  13. More film stuff - http://cloudthirteenproductions.com/
  14. A web comic whose author didn’t even have time to color it in - http://cloudfourteen.net/
  15. A USPS series of 15 types of clouds - http://www.cloudappreciationsociety.org/cloudspotter-and-stamp-collectors-unite/
  16. A personal 16mm film collection. You must be kidding. - http://cloud16.com/
  17. Last years secret Jet Blue promo code - worhtless http://www.debonairmag.com/cloud_seventeen_-_jet_blue_promo_code.htm
  18. A domain parked for over two years with nothing to show for it - http://cloud18.com/
  19. Some type of film/design thing whose navigation flash cruft is broken - http://cloud19.com/
  20. A scalable database appliance early stage startup (a.k.a. ppt-ware) - http://cloud20.com/
  21. DVD Authoring software - another coaster maker http://cloud21.com/

What a wasteland. The numbers do not lie, my friend.

And then there’s hadoop. But they don’t even have their
curly braces in the right spots so they are destined to fail I guess.

Mkay, everyone is doing it. So here’s mine.

[mike@happy ~]$ history|awk ‘{a[$2]++} END{for(i in a){printf “%5d\t%s \n”,a[i],i}}’|sort -rn|head
  235   ls
  147   svn
   92   cd
   46   rm
   34   ssh
   34   ant
   24   for
   23   script/generate
   22   grep
   18   vi

Most of the work happens in emacs and I click that from the desktop usually. “ls” is kind of like a nervous twitch more than getting anything done. Sort of like hitting Control-s every two sentence or close-curly in emacs. I’ve tried to stop, but it’s in muscle memory now. And about that “svn”, yes I know about Control-x v v vc-mode, but it doesn’t really suit trying to keep commits as logical atomic units for me.

The Java JVM has the -Xnoclassgc argument to inhibit class garbage collection. That is so 2006. If you have a long-running server JVM, this is most likely going to leak memory.

Specifically, if your process

  • Uses serialization
  • Uses reflection
  • Can have a remote debugger or JConsole attached
  • Uses any dynamically generated classes
  • Uses any 3rd party jars that do any of the above

then you have a slow, psssssssssssssssssssss sound coming from your JVM.

Do you seriously think that in 2008 you know that none of the jars on your classpath use reflection or serialization?

We live in a world now where JRuby comes along and may generate holder and wrapper classes during runtime and do all sorts of stuff to the internals trying to get things to byte-compile down. Only geniuses can truly understand this stuff.

Instead of trying to save a few millis over the course of a month of server JVM run time by turning off the class GC, make it ConcurrentMarkSweep enabled instead:


java -server -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled

and let’s relegate -Xnoclassgc to the scrap-heap. It probably doesn’t do what you think it does.

It all started with Log4j — Java, log4j and I sort of grew up together. I loved log4j. But along came j.c.l, jdk1.4 logging and I had to try them all out. When building a library that other people are supposed to embed into something of theirs, though, turning the logging choice over to them makes a lot of sense. As our little project grew and needed to bring in other components we always appreciated those that did logging the way we wanted it done.

Jetty was a standout in that area — I first learned the coolness of dependency injection from studying Jetty. Jetty provided our first introduction to Slf4j. Slf4j is sort of the ultimately flexible logging system. You build to the Slf4j API and drop in the statically bound jar file for the back end implementation you want (e.g. log4j, logback, log4juli). This is a fantastic idea and thanks are due to Ceki Gülcü for log4j, logback and slf4j.

Quite some time back we felt the pain of just trying out a new logging system. Since every file has imports and a factory method call to get the logger, it is painful to switch. So we insulated ourselves from that by creating our own Logger API, an abstract base class that pretty closely matched what we needed and were using from log4j. We had a Logger.getLogger(String) method, debug, info, warn, error and fatal levels. And we had a LogManager that acted as the factory for it all, returning a wrapper around log4j loggers that met our API. Life was good.

We added Slf4j into the mix at first only to get logging from some Jakarta components that were using j.c.l out into the log4j backend where we wanted them. Slf4j was fantastic at that.

Then the haunting question — why aren’t we doing in our own library the very thing that we enjoy about the logging capabilities of Jetty? Jetty has a soft dependency on Slf4j and falls back to some type of logging to stdout when it isn’t present. Cool. We can do that too.

The friction came when I realized that our nice little log4j-based API abstraction needed NDC. The Slf4j API does not support NDC. Major bummer. We find this incredibly helpful in debugging a complex multi-threaded system. We have pushContext(String), popContext() and clearContext() methods in our logging abstraction and I don’t really want to give them up.

But hey! Why not make a soft dependency on those features just like we are doing for Slf4j? So in the new Slf4jLogger that we use to encapsulate our Slf4j capability, we can check for the presence of org.apache.log4j.Logger and grab those methods we need for NDC. If they aren’t there, ok, we drop the NDC information, but if they are there, then we have it all!

Here’s some code to add the soft dependency on NDC. First, in the static initializer of our Slf4jLogger class:

 // Support methods for ndc capability if log4j is available
 private static Method ndcClear;
 private static Method ndcPop;
 private static Method ndcPush;

   static {
            /* Not shown - Slf4j soft dependency loading */
            Class ndcz = null;
            String NDC = "org.apache.log4j.NDC"
            try
            {
                ClassLoader cl = Thread.currentThread().getContextClassLoader();
                ndcz = cl==null?Class.forName(NDC):cl.loadClass(NDC);
                ndcClear = ndcz.getMethod("clear",new Class[]{});
                ndcPop = ndcz.getMethod("pop",new Class[]{});
                ndcPush = ndcz.getMethod("push",new Class[]{String.class});
            } catch (Throwable t) {}
   }

So that grabs the underlying methods we need when they are present and doesn’t carp when they aren’t.
We can wrap that up in a simple function to let us know when NDC capability is present:

    protected static boolean isNDCEnabled ()
    {
         return ndcPush != null;
    }

And protect the actual methods that are the NDC part of the logger API:

    /**
     * Push the string on the ndc context stack if the capability is supported
     * @param s the new context
     */
    public void pushContext(String s)
    {
        if (isNDCEnabled())
        {
            try
            {
                ndcPush.invoke(null,s);
            }
            catch (Exception e) { /* e.printStackTrace(); */}
        }
    }

Since we are pushing the context into the actual Log4j NDC object, we didn’t have to reinvent anything. If Log4j is on the runtime classpath the features will be found. If Slf4j is configured to actually use Log4j as the backend for logging, then the NDC will actually pop out of the log and we still don’t need Log4j to be on the classpath at compile time!

So have your Slf4j and your NDC too!

Next Page »