life.events.each { |event| learn_from(event) } - Home

awesome_fields Gets A FormBuilder, New headerize Plugin

2008-06-16T22:58:00Z

I haven’t been able to work on Quill too much lately, in part because I lost my stylus somewhere along the way (new one should be forthcoming today, actually) and in part because my day job involves Rails at the moment, so that’s the world my mind has been in. What I have been working on is my awesome_fields plugin and a new plugin, headerize. Actually, the `work’ I have done on these has been extracting some stuff that I commonly do in my Rails projects into plugins. There’s very little that is technically new code from me there, though the headerize plugin is well spec’ed-out.

Anyway, so awesome_fields has received a LinedBuilder for creating better marked-up forms with less effort, and headerize was created to facilitate putting all your stylesheets and scripts in one place (typically the document head) while still being able to include them from the view. That way, the stylesheet that needs to be included by the _form partial is included there, so that it is spatially near to the place where it is used, while the actual markup for it goes in the head, where it belongs. So I’ll hit these two separately…

`awesome_fields` and the `LinedBuilder`

Let’s face it, one thing Rails sucks at is facilitating the creation of labeled form fields. Outputting errors for form fields is also a bit repetitive out of the box. It seems like this begs for a bit of DRY goodness that can live in a custom FormBuilder. Enter the LinedBuilder, whose purpose is to make explicit labels and errors as much a part of the turbulent past as possible.

Basically, the LinedBuilder gives you the usual methods, such as text_field, text_area, etc, but it gives them labels, wraps them in a div, and automatically includes errors in them if necessary. So let’s say I had my form:

<% form_for @model do |f| %>
  <p>
    <%= error_message_on 'model', 'name' %>
    <label for="model_name">Name: </label>
    <%= f.text_field 'name' %>
  </p>

  <p>
    <%= error_message_on 'model', 'author' %>
    <label for="model_author">Author: </label>
    <%= f.text_field 'author' %>
  </p>
<% end %>

That’s a lot of typing, plus it’s repetitive. With awesome_fields (and therefore LinedBuilder):

<% lined_form_for @model do |f| %>
  <%= f.text_field 'name' %>
  <%= f.text_field 'author' %>
<% end %>

Dearie me, that does look cleaner. Behind the scenes, this means we will get:

<form ...>
  <div class="form_line">
    <label for="model_name">Name: </label>
    <input type="text" id="model_name" name="model[name]" value="..." />
  </div>
  <div class="form_line">
    <label for="model_author">Author: </label>
    <input type="text" id="model_author" name="model[author]" value="..." />
  </div>
</form>

And, if there is an error on the name field from validates_presence_of, we get:

<form ...>
  <div class="form_line_with_errors">
    <div class="field_error">Name should not be blank.</div>
    <label for="model_name">Name: </label>
    <input type="text" id="model_name" name="model[name]" value="..." />
  </div>
  <div class="form_line">
    <label for="model_author">Author: </label>
    <input type="text" id="model_author" name="model[author]" value="..." />
  </div>
</form>

As an additional bonus, say validates_presence_of and validates_numericality_of both fired, the error becomes:

  <div class="form_line_with_errors">
    <div class="field_error">Name should not be blank and should be a number.</div>
    <label for="model_name">Name: </label>
    <input type="text" id="model_name" name="model[name]" value="..." />
  </div>

This is because, if the object returned by errors.[] responds to to_sentence, then it is converted into a sentence. Since Rails defines an excellent such method for arrays, we get the above pretty output.

So that’s the LinedBuilder in a nutshell. There are a few options you can pass. If using the automatic label (which is the humanized name of the field) doesn’t cut it, you can pass in a :label option with the text you want to use. If you don’t want a label, you can pass in a :no_label option set to true. And finally, if you need to distinguish between labels for long and short fields, you can pass in a :long option set to true. If you do this with a call to text_area, it will also adjust the default number of rows and columns.

For more information, see the RDocs in the source file . To install the plugin on Rails <=2.0:

git clone git://github.com/Shadowfiend/awesome_fields.git vendor/plugins/awesome_fields

If you are using git, make sure to:

rm -R vendor/plugins/awesome_fields/.git

So that you can check in the code. If you’re running Rails 2.1, then you can hit up:

script/plugin install git://github.com/Shadowfiend/awesome_fields.git

`headerize`—Javascript and Stylesheets Where They Belong

In addition to the changes to awesome_fields, headerize is a new plugin that lets you put your JS and stylesheet tags where they belong (in the head of your document, technically, though some people put JS at the bottom as a loading speed optimization) while letting you include them wherever you want.

When would you use this? For Javascript, if you’re trying to write unobtrusive Javascript, you usually want to include a file for your particular view. So say you’ve got a form, and you want to make it remote. But, you want this not to clutter up your HTML. So, you including a JS file called remote_forms.js that has the unobtrusive javascript to make things work.

Trouble is, you don’t want to include it on every page. One way to do this is to include it anyway and just use a CSS class or somesuch to ensure that only forms with the `remote’ class get the behavior. But another way is just to include the script only on pages that need it. The same goes for stylesheets: if there’s no form on a page, why include the styles for it?

Since the layout in Rails is separate from the view you are rendering, however, it is annoying to include anything per-view. headerize takes care of this problem. After installing it, it makes two helpers available: add_javascript and add_stylesheet (both exist in pluralized forms for readability when taking care of multiple items). These each behave exactly like javascript_include_tag and stylesheet_link_tag, only they don’t drop their links into the HTML right where they are called. Instead, wherever you want your stylesheets and Javascript, you insert the output of javascript_includes and stylesheet_links.

For example, to deal with the above situation, I could have a layout application.html.haml (in HAML):

!!! XML
!!!
%html
  %head
    %title My Site
    = stylesheet_links
    = javascript_includes
  %body
    = yield

And, in your new view, say new.html.erb (in ERB):

<% add_stylesheet 'forms'
  add_javascript 'remote_forms' %>
<% form_for @model do |f| %>
  <%= f.field_for :name %>
  <%= f.field_for :date %>
<% end %>

That’s it. When the view is rendered, the layout will have the right stylesheets and Javascript includes right there, easy as pie. Note that this has some implications on fragment caching (since the links and includes are built every time, if you fragment cache the portion including the stylesheet calls and it is not re-evaluated, then the stylesheets and includes will not function correctly). Also note that these functions can be called from partials, so for example an _form partial can always include the relevant stylesheets and forms and no matter where it is rendered from, the correct stuff will be included.

Also worth mentioning, as presented before, the calls to add_stylesheet and add_javascript are exactly the same as their inline counterparts, so you could easily do any of these:

<% add_stylesheet 'print', :media => 'print'
   add_javascripts 'remote_forms', 'remote_links', :cache => 'remote_stuff' %>

As before, for more information, see the RDocs in the source . To install the plugin on Rails <=2.0:

git clone git://github.com/Shadowfiend/headerize.git vendor/plugins/headerize

Again, if you are using git, make sure to:

rm -R vendor/plugins/awesome_fields/.git

So that you can check in the code. If you’re running Rails 2.1, then you can hit up:

script/plugin install git://github.com/Shadowfiend/headerize.git

And that’s all folks! Hope some of you get some use out of these plugins!

magic_fields Moves to GitHub, Renamed to awesome_fields

2008-04-30T21:49:00Z

So Software Without Incident (the site) kind of disappeared a little while ago due to some accidents while I was toying around on the server. The SVN repository’s still sitting there on my server, though, so my code’s still around. And I still pull magic_fields into my new projects. I am a particular fan of the collection handling, if I do say so myself. The name, however, clashes with a term given to Rails’s built-in `magic fields’ for ActiveRecord, things like updated_at and created_at. So I took advantage of the situation to rename the plugin awesome_fields, too, so it’s a little more distinguishable.

Anyway, I went ahead and used git-svn to pull the code from the SWI repository into a git repository and then pushed it off to GitHub. I’m more a fan of Bazaar myself (the documentation for it is amazing, and the commandset is readable; plus, there’s no required -a parameter for committing everything, which is the way I happen to work), but I know the Rails community is on git-mania now, so git it is. For the interested, it’s under awesome_fields at GitHub.

Documentation and a README await there.

Qt Radial Menu for Quill

2008-04-04T05:03:00Z

In my forays into the world of creating a Qt journal application, part of what I wanted to achieve was a really easy way to get to common actions like tool changes. My two biggest gripes with Xournal were always (a) that I had to keep adding an extra page after the one I was working on if I wanted to be able to see the bottom of this one (I fixed this in Quill, and made adding a new page easier, by introducing a `virtual’ last page, which does not appear on any output but gives you scrolling space and, when drawn upon, becomes a regular full-blown page) and (b) that if I wanted to change tools or colors, I had to shoot up to the toolbar to do it. That’s just no fun.

So, my latest foray into Quill has been adding a radial menu to the canvas area, so that you can get a menu for doing stuff in a way optimized for the stylus. The coolest aspect of this was that it let me develop my own Qt widget, of course. The results, though, were pretty good, I thought:

Details follow…

Part of the awesomeness that is Qt is that every QWidget has an associated list of QActions. QWidgets in Qt are the base graphical unit—equivalent to Swing’s Component, for example. QActions are an encapsulated way of representing various actions. They are used equally in menus and toolbar, so, for example, the File>New action and the New toolbar button would be separate representations of the same QAction object. A model, if you will, for actions the user can take, where the toolbar and the menu represent views.

Bearing this in mind, I decided to model the RadialMenu class’s API closely after QMenu’s API. The advantage of this approach is that I had a model for what functions I should have. And, indeed, I implemented most of them. The exceptions were few and involved things that made sense for a QMenu but not for the radial variant, and a few things that I decided to wait on until later if they became necessary. I also added a couple of useful functions specific to RadialMenu (such as centerOn() to center it on a particular point). I did not, however, extend QMenu, as the provided functionality is, I believe, too different. Perhaps in the future the functionality will align more and such an inheritance chain might be merited.

The truly painful part was implementing painting. Because Qt has styles that can style the interface differently, and some of these styles use native painting routines (as with Windows and OS X), I had to model the radial gradient I needed for the menu using the Plastique style as a guide. It won’t look quite as similar to the current style’s widgets in styles other than Plastique, but it’s a good approximation. Also problematic in this context was positioning the icons. I wanted to make RadialMenu flexible, so it would adjust to the number of actions needed correctly. Thus, we have the effect of something that works equally well with the five items seen above, or with three or even six:

Notice that at six we start running into some issues. Basically, the current painting code, when determining the size of the icons, only takes into account the distance between the outer radius and the inner radius—it does not take into account the distance between the right and left button boundaries at their narrowest. The painting code, however, is clearly adjustable.

Finally, RadialMenu takes care of highlighting hovered items and pressed items. Plus, thanks to QAction’s signals, the currently selected item amongst a list of, say, tools, is kept synced with the equivalent entries in the toolbar, even when both are open at the same time.

RadialMenu currently also supports, in the QAction, having a QAction with an associated RadialMenu. QActions by default can be associated with a menu, but I did not want to overload that concept—I wanted a QAction associable with both a QMenu and a RadialMenu at the same time. To this end, QActions have the concept of arbitrary user data set via the excellent QVariant class that can be used to transport essentially arbitrary data. For RadialMenu’s purposes, QAction’s user data is currently expected to be a RadialMenu when a RadialMenu should be associated to it. It is very likely that this will switch to being one of the arbitrary properties QObjects can have set and retrieved via setProperty() and property() with the name `radialMenu’. Note that the submenus aren’t currently popped up yet.

Finally, you may have noticed a small wrench in the middle of some of the screenshots. Currently there are two ways to get a hold of this menu. The first is to press the middle mouse button, which on my stylus at least is mapped to holding the side button down and pressing it. This will bring up the radial menu centered on the mouse position. The second way is that wrench. When the stylus pressure goes to zero, Quill waits for a short timeout and then displays, offset slightly from the location where the stylus was lifted, a small button:

This button, when clicked, will also bring up the radial menu. I’m still not sure about this button and how much it gets in the way. I want to take a wait-and-see attitude on it and see what kind of feedback I get on its usefulness. I think it will be particularly nice when a stylus does not have an alternate button on it and the only way to get the menu will be to click on this one.

Also in the works for the radial menu, in addition to full support for submenus, is support for drag-based navigation of the menus; that is to say, clicking, holding, and then moving through to the final menu item you desire. This basically will make menu item selection a one-press affair, though the release will take a while to come. I also want to implement an alternate form of the pen width spinner more suited to the radial paradigm: basically a widget upon which drawing a circle clockwise will increase the value and counter-clockwise decrease it (this might need to be flipped in RTL locales, as well). The idea is borrowed from InkSeine:http://research.microsoft.com/inkseine/, as referred to me on this bug by Kovid Goyal.

Quill: Qt Tablet Journaling Program

2008-01-03T06:00:00Z

What’s been keeping me up late at night recently? Well, recently, it’s been a side job; however, somewhat less recently, I started a program called Quill. Quill is a Qt equivalent to Xournal, which is a tablet journaling program written in Gtk. But I didn’t just want to make a Qt clone, as that’s neither here nor there. Instead, I used Quill to brush up on my Qt skills, as well as to play with some of the cool technology in Qt 4.3 (like the Graphics View framework and, to a much more restricted extent, scripting (which I haven’t yet been able to figure out very well)). Quill is hanging out on Launchpad at https://launchpad.net/quill. I chose Launchpad in part because I wanted to play with Bazaar, which I have been using since I set up the Launchpad site, and damn it’s hot. I only hope I get more opportunities to use it in the future.

Anyway, a couple of the features I’m playing with Quill. First off, I’m adding buttons to the bottoms of pages to allow inserting pages in the middle relatively easily and with no menu interaction. Likewise with deletion of a page. In addition, there’s always a ``virtual page’’ after the last actual page. Writing on this page will make it a full-fledged page, but it does not appear in any printing or PDF exports. This basically makes it a very easy way to add a page to the end of a document. Finally, if you hold the cursor relatively still once you stop writing, a toolbar pops up with some commonly used tools.

A lot of things need work. Right now I’m working on getting the movement of a selection to work appropriately across saves. Currently, since such a change doesn’t break up strokes, reloading the page will connect the moved lines back to their original strokes, yielding some very strange streaks. I’m working on breaking strokes apart when moves like that happen, so that this works correctly. Also, the way the toolbar appears and when it does and where it does can get annoying. This is definitely an area for improvement.

And there is also, of course, performance. QGraphicsView and QGraphicsScene are fantastic pieces of technology, but their performance has some issues—especially when moving a selection, for example, where paint clipping becomes very apparent. I intend on looking at some of these issues to see if they can be resolved with clever coding, or if I’m just being an idiot to begin with. Additionally, after a random time period, the drawing based on stylus input becomes very very very noticeably slow. I’m still not sure why this is happening, but it’s definitely my fault, as saving the document at this point and then reloading it and continuing works fine.

Declarative Optional Parameters in Ruby

2007-12-17T15:17:00Z

There’s a common pattern in Ruby for setting up keyword parameters—optional parameters with default values that are accepted as a hash at the end of the method call. Ruby’s syntax facilitates this, with end-hashes not requiring curly braces; instead, you can do this:

obj.method :keyword => 'value', :keyword2 => 3

Many people have written about various ways of implementing this in as painless a syntax as possible, but they all involve writing a lot of boilerplate. My intent is to write code that doesn’t require such boilerplate at all, instead providing a declarative way to say `here are the optional parameters, here are their default values’ and have everything taken care of for you. Moreover, the ability to provide introspection into what optional parameters are expected at runtime and the default values thereof is also something I want to try and give.

Specing and Thought Process

Just to clarify, as I write this, I will be BDDing it up, so RSpec code will come first. Moreover, I’m going to write this post in a stream-of-consciousnessesque fashion. This is just something I try to do so that my thought process in getting from point A (here’s an idea!) to point B (here’s how I’ll do it!) to point C (well that’s just dumb, let’s do it this way instead…) to point D (here’s a completed bit of code!) is exposed. It’s an interesting exercise for me, and I hope it’ll be interesting for you as a reader to see this process.

Also, this idea is fairly complete in my mind, and, though its usefulness may be questionable (it’ll induce some interesting overhead), I think it’s another interesting investigation into what you can achieve with Ruby, much like my earlier implementation of the `with’ keyword.

Syntactic Sketch

What we’re trying to get to is a way of declaring optional parameters that is almost as good as declaring them in line. We can’t declare them in line, of course, because that isn’t supported by the syntax. The ideal syntax would be something vaguely like this:

def my_method(arg1, arg2, :arg3 => 5, :arg4 => 'test')

Since we can’t do this directly, we’ll change things up a bit. Let’s envision a situation where we do this instead:

optional(:arg3 => 5, :arg4 => 'test', :arg5 => 'magic')
def my_method(arg1, arg2)
  # ...
end

Some people will likely be uncomfortable given the lack of indication as to where the `optional’ method stops working (if I declare another method after that, does it still apply?). The idea would be that it would only apply for the next method definition; however, let’s provide an alternate syntax to clarify that:

with_optional(:arg3 => 5, :arg4 => 'test', :arg5 => 'magic') do
  def my_method(arg1, arg2)
    # ...
  end
end

Naturally, a set of optional parameters can be shared with several methods with this syntax:

with_optional(:arg3 => 5, :arg4 => 'test', :arg5 => 'magic') do
  def my_method(arg1, arg2)
    # ...
  end

  def my_other_method(arg6, arg7)
    # ...
  end
end

Ok, so now we have a sketch of what we want our syntax to look like. Let’s make it happen!

A Little Bit of Planning

First of all, let’s think about how things will fit together. Clearly, to do this, we’ll need to hook into method creation. Fortunately, Ruby provides the Module#method_added method. This method gets called when a method is added to a module (or a class). It receives, as a parameter, the name of the new method that was defined.

So we can hook into when the method is created; what we can’t do is modify the code in the actual method at runtime (well, we probably could with ParseTree, but let’s not go there). Instead, we’re going to wrap the new method with our own method that will sit in between and intercept calls and take care of optional parameters automagically.

Additionally, we’re going to need at least two methods: optional and with_optional. In fact, we can make these the same method (by checking block_given? or whether a parameter &block is provided, we can just specialize within the method itself). We’ll also need some helpers—one to define the wrapper method, and one or two to set when we’re monitoring added methods and when we aren’t (i.e., we’d enable monitoring when we need to add optional parameters to whatever method(s) is/are defined next, and we’d disable it once we’re done with adding optional parameters).

The details of this breakup will become clearer as we create the actual code. The last thing we’re missing is how to provide the optional parameters to the method once we’re done. In a perfect world, we could maybe inject a local variable into the method at runtime and be happy, but we can’t really do that. We could also use some external instance or global variable, but that’s both dirty and not thread-safe in any way, shape or form. Instead, we’ll go for the still-intrusive but mildly less dirty approach of making the last parameter to the declared method be some sort of placeholder for the options. This modifies our syntax into:

optional(:arg3 => 5, :arg4 => 'test', :arg5 => 'magic')
def my_method(arg1, arg2, options)
  # ...
end

Initial Specs and Code

Before writing any code, we’ll need to come up with some appropriate specs that will detail how we want the code to work. For each behavior (group of specs), we will create a new testing class to use.

Let’s start with the basic version—declaring a set of optional parameters for the next method.

require 'optional_params'

describe OptionalParams, 'when declaring parameters for the next method' do
  before(:all) do
    class TestClassA
      include OptionalParams # include OptionalParams functionality

      optional :birth => Time.now
      def create_person(name, options)
        options
      end
    end
  end

  before(:each) do
    @test = TestClassA.new
  end

  it 'should pass the optional parameters in' do
    @test.create_person('dude', :birth => 'w00t').should == { :birth => 'w00t' }
  end

  it 'should fill in default values when needed' do
    @test.create_person('dude')[:birth].should be_kind_of(Time)
  end
end

This spec will fail initially since the optional_params.rb file doesn’t exist, then because the OptionalParams module doesn’t exist, and then because there’s still no #optional method. Let’s knock these three out in one shot:

module OptionalParams
  def optional(params = {})
  end
end

The next error won’t actually be with the first example—the first example actually runs fine, since options just takes the value of the hash that’s passed in, the way it usually would. The next problem happens in the second example, where we have no default value specified. So it’s time to actually create some code that’ll do what we want it to. Let’s start with the basics:

module OptionalParams
  module ClassMethods
    def optional(params = {})
      @next_optional_params = params
    end

    def optional_params
      @optional_params ||= {}
    end
  end

  def self.included(base)
    singleton = class <<base; self; end
    singleton.class_eval do
      def method_added(meth_name)
        return unless @next_optional_params # return if there's nothing to do

        old_name = "#{meth_name}_without_params" 
        new_name = "#{meth_name}_with_params" 

        # Store the parameters permanently, then clear it out so that we don't
        # do an infinite method_added loop.
        optional_params[meth_name] = @next_optional_params
        @next_optional_params = nil

        # Create the new method, doing whatever optional parameter housekeeping
        # needs to be done.
        define_method(new_name) do |*args|
          opt_params = (Hash === args.last) ? args.pop : {}

          opt_params = self.class.optional_params[meth_name].merge(opt_params)

          self.send old_name, *(args + [opt_params])
        end

        # alias_method_chain -- replace the just-created method, wrapping it
        # with our own optional parameter handling code.
        alias_method old_name, meth_name
        alias_method meth_name, new_name
      end

      include ClassMethods
    end
  end
end

Okay, so let’s look at what’s going on here. First off, we see that the task of the optional method is pretty minimal—it just sets up the next optional parameters instance variable for method_added to use when it’s wrapping a method with optional parameter handling code¹. There’s also another auxiliary method, optional_params, which provides an accessor to an instance variable. This instance variable basically keeps track of the optional parameter lists for various method names².

The real work is done by method_added. method_added starts off by just returning if it has nothing to do (i.e., if there is no next optional parameter that it needs to set up). Otherwise, it sets up the new names for methods—we follow the Rails alias_method_chain convention of renaming the original method with an _without_feature and creating a new method with a _with_feature appended to the end that gets renamed to the original method name.

Once we’ve got the method names, we pull out the next optional parameters and record them permanently (remember, we didn’t know the method name until method_added was called!), then we clear out the next params variable. Note that if we don’t do this, when we define our new method, we’ll try to set up optional params for it, and we’ll go into an infinite loop of method definition-method wrapping.

Then, we go ahead and define the new method. It follows a relatively common pattern of checking if the last parameter is already a hash, and, if it is, using it instead of creating our own. We then merge in our values, and finally we call the original method. We put the hash back into the list of parameters and splat these (effectively splitting the array into separate parameters instead of one array parameter). Finally, we rename the two methods so that everything’s set up the way it should be and our new method wraps the old one.

Here, by the way, the first example comes in useful—had we merged the wrong way (i.e., if we’d merged the passed parameters with the optional parameters rather than the other way around), that example would have failed.

Adding Blocks

Here’re some specs:

describe OptionalParams, 'when declaring optional parameters for multiple methods' do
  before(:all) do
    class TestClassB
      include OptionalParams # include OptionalParams functionality

      with_optional(:birth => Time.now, :weight => 16.5) do # in pounds
        def create_person(name, options)
          options
        end

        def create_baby(name, options)
          options
        end
      end
    end
  end

  before(:each) do
    @test = TestClassB.new
  end

  it 'should fill in the optional parameters for all methods' do
    results = @test.create_person('w00t')
    other_results = @test.create_baby('other_w00t')

    results[:birth].should be_kind_of(other_results[:birth].class)
    results[:weight].should == other_results[:weight]
  end
end

Adding the block version is a little more involved. We still need to clear out the next optional parameters variable in that case, so that we don’t get into the aforementioned infinite loop. However, we also need to keep it around so that if multiple method definitions are done, we can still use the same parameter list. So, we’ll go ahead and fill in some code. First, we’ll add the with_optional method, and then we’ll update method_added to work in a more friendly way.

  def with_optional(params = {}, &block)
    # Store the parameters and raise the keep_params flag, which tells
    # +method_added+ to keep this variable around.
    @next_optional_params = params
    @keep_params = true

    self.class_eval &block

    # Clear the parameters and the keep_params flag, effectively ending the
    # block of methods that will take these parameters.
    @next_optional_params = nil
    @keep_params = false
  end

  # ...
      def method_added(meth_name)
        # ...
        @next_optional_params = optional_params[meth_name] if @keep_params
      end
  # ...
end

Here, we create the with_optional method, which sets up the next params variable appropriately and also raises the keep_params flag. This flag tells method_added that it should keep the parameters around after it runs once. To do this, we add one last line to method_added, which restores next_optional_params if the keep_params flag is raised. Once with_optional’s block is done running, we go ahead and clear both the next params variable and the keep params flag.

Metadata

This gives us the basic structure for the functionality. All that’s left is adding one more bit of useful functionality: metadata. Right now, we declare the optional parameters and such, but the real win of our approach is the ability to get back information about what optional parameters are expected and what their default values are. Thus, we can modify the method method to give us back, inside our Method object, data about the optional parameters. We can also complete the wrapping of the original method by returning a Method object with the original method’s characteristics (specifically, we can use the original method’s arity, rather than the new method, which will always have an arity of -1 (since it uses *args).

First, some specs; under ‘when declaring optional parameters for the next method’:

  it "should keep the original Method object's arity" do
    @test.method(:create_person).arity.should == 2
  end

  it "should provide access to the optional parameters through the Method object" do
    opt_params = @test.method(:create_person).optional_params

    opt_params.keys.length.should == 1
    opt_params[:birth].should be_kind_of(Time)
  end

To achieve this, we’ll have to modify the Method object itself. To achieve this wrapping, we have to store the metadata, which we can do in method_added, and then we have to wrap the method method to return a modified object. In the self.included method, after we do the singleton’s class_eval, we drop this in:

      base.instance_eval do
      define_method(:method_with_metadata) do |meth_name|
        if self.class.optional_params[meth_name]
          params = self.class.optional_params[meth_name]
          initial = method_without_metadata(meth_name)
          orig = method_without_metadata("#{meth_name}_without_params")

          singleton = class <<initial; self; end
          singleton.class_eval do
            define_method(:optional_params) do
              params
            end

            define_method(:arity) do
              orig.arity
            end
          end

          initial
        else
          method_without_metadata(meth_name)
        end
      end

      alias_method :method_without_metadata, :method
      alias_method :method, :method_with_metadata
    end

First, we define the method_with_metadata method. Inside it, we check whether there are optional parameters for the named method (recall that the singleton class stores the list of optional parameters). Then we grab the initial Method object for the specified method, modify it so it returns the regular, un-wrapped method’s arity and the optional parameters. The extensive use of define_method and blocks lets us share local variables like orig and params. If there are no optional parameters, we just pass through directly to the regular method method. Then, we alias the methods appropriately.

This makes the specs for optional pass; let’s add some for with_optional and make sure they pass, too:

  it "should keep the original Method objects' arity" do
    @test.method(:create_person).arity.should == 2
    @test.method(:create_baby).arity.should == 2
  end

  it "should provide access to the optional parameters through the Method objects" do
    opt_params = @test.method(:create_person).optional_params

    opt_params.keys.length.should == 2
    opt_params[:birth].should be_kind_of(Time)
    opt_params[:weight].should == 16.5

    opt_params = @test.method(:create_baby).optional_params

    opt_params.keys.length.should == 2
    opt_params[:birth].should be_kind_of(Time)
    opt_params[:weight].should == 16.5
  end

And there we go. These specs pass, as well.

You can find the two Ruby files here: optional_params_spec.rb, optional_params.rb .

Parting Remarks

Again, this is a bit of a toy. The performance degradation from wrapping methods and such might outweigh the benefits of the cleaner syntax. However, having this to encapsulate the common optional parameter idiom is, I think, pretty cool, and again showcases the power of Ruby’s metaprogramming.

¹ Clever people will point out that this technique will also not be particularly threadsafe. I’m thinking the best way to get around that is to store the next optional parameter list per-thread, using a hash indexed by the thread object.

² As a side note, this means that technically we could use this technique to wrap methods in optional parameter handlers after the fact, as well.

RubyShell Update: A Better Validator? And Path Handling

2007-10-22T20:15:00Z

So it’s been a while since I’ve made a post on RubyShell. I’m still working on it, so no worries, I’ve just got a little less time than usual. Anyways, an update on some things I’m working on right now; namely, a better validator and some improved path handling.

Yay `SexpProcessor`!

So ParseTree comes with a useful little class called SexpProcessor that happily provides a way to visit nodes in the parse tree. Until now, I was using some nastiness in dealing with the sexps directly to figure out when I had calls that were invalid so I could mark them as invalid Ruby. Now, however, we do something different…

Thanks to SexpProcessor, I’ve replaced this less-than-spectacular methodology with a simple SexpProcessor subclass that looks for the first call (fcall, vcall, or call) and tracks what it was so it can be checked later. Currently, checks for overwriting assignments (assignments to variables that will shadow existing methods) are still done manually, through what is essentially a manual SexpProcessor. I’ll be rolling that into the ValidationProcessor (as the new class is called) sometime today. There’s some more promising stuff that can come out of this, as it’s a cleaner solution than what we had before. Fixing corner cases should hopefully be a little easier now.

For example, invalid initial method calls are currently always marked as commands (i.e., as invalid Ruby). This is bad, as sometimes there are invalid method calls that are still clearly Ruby but are passed on to the shell as a command anyway. Here’s one:


RubyShell> math.magic { |test| power }
Error: Command `math.magic' not found.

That should still be processed as Ruby, and instead give a Ruby error. The processor will let us do this more easily, as we can look for, say, an :iter node (for a block) and decide that that will always be valid Ruby, even if it errors out when it runs.

Paths as First-Class Citizens

Daniel Draper mentioned in a comment on the 0.5 release announcement post an interesting possibility:

I’d love to do this:
Have a dir called foo/bar/ that has say 100 directories in it.

Simply do

ls foo/bar.first to see whats in the first one of those

or maybe

ls foo/bar.children.first

Then if you wanted to move all dirs up a level you could do

foo/bar.children.each_with_parent { |c,parent| mv c, parent.parent }
and so on.

My brain pondered this quietly for a few days, and now I’ve taken quite a liking to the idea. I’ve already opened a feature request on Rubyforge to keep track of my progress on this. Basically, i want to see if I can make paths first-class citizens.

Path notation, it bears mentioning, is fairly closely-related to Ruby’s regular expression syntax and/or its division syntax. This introduces some interesting complications if we try and support `directory literals’, so to speak, within Ruby. So instead, I decided to opt for a middle way. Within the context of commands (i.e., code that has been marked as invalid Ruby), I intend on doing some additional parsing and creating actual path objects that can be operated upon like objects. For this purpose, I’ve created a new RubyShell::Parsing module to contain the Parser and CommandParser that will do that extra handling. Alias handling will also be moved into there.

Now, in the context of Ruby code, you’ll still be able to access the same functionality, but you’ll need to do it via a method. Right now, I’m thinking it’ll probably be named path. So you’d do something like path('foo/bar').children.each_with_parent .... It might be shortened to something else, like d (e.g., d('foo/bar').children...). Finally, I’m thinking of whether I’ll allow this kind of manipulation without that method invocation if it’s the first thing on the line. So, for example:


RubyShell> foo/bar.children.each_with_parent { |c,parent| mv c, parent.parent }
RubyShell>  foo/bar.children.each_with_parent { |c,parent| mv c, parent.parent }

The former would run as if it had been a call to path('foo/bar'). The latter wouldn’t, because it isn’t literally the first thing on the line (there were some issues with the leading whitespace being stripped before it was processed earlier for the alias command (which does the same thing—alias turns into alias_cmd if it’s the first thing on the line, for shortening purposes), but I think I’ve knocked those out). We’ll see whether I’ll implement this latter one or not, as I don’t want to make too many exceptions like that.

Anyway, that’s an update of what’s in the works! Output redirection is also in the back of my mind, so no worries there. I just need to sit down and map out how the implementation is going to work.

RubyShell 0.5: Objects, Files, Interpolation, and Aliasing

2007-09-17T02:16:00Z

The latest release of RubyShell is so cool, it’s skipping right over 0.4 and straight to 0.5. If you want to skip the explanation, you can go ahead and download it now.

It bears mentioning also that this release marks the first release of RubyShell on Rubyforge. Thanks to Rubyforge for being awesome :)

RubyShell 0.5 adds some tasty flavors to the table with four new treats. First up are shell objects.

Shell Objects

Shell objects are not my idea :) viraptor waited until the moment that RubyShell was on Rubyforge to unleash this cool little concept. Basically, these are objects that represent handy Ruby interfaces to useful concepts. Along with an initial implementation, viraptor provided two such object classes: ps and if_config. ps is used to get to information about processes, while if_config gives information about network interfaces (just like their command counterparts). Here are some examples of using ps (with some parts abbreviated):


RubyShell> procs.collect { |proc| proc.name } .inspect
=>["init", "migration/0", "ksoftirqd/0", "migration/1", "ksoftirqd/1",
"events/0", "events/1", "khelper", "kthread", "kblockd/0", "kblockd/1", 
"kacpid", "ksuspend_usbd", ...,"artsd"]
RubyShell> ps.named('nspluginviewer').inspect
=>[#<RubyShell::ShellObjects::Ps::Process:0xb78a3b8c @name="nspluginviewer",
@cmdline=["/usr/kde/3.5/bin/nspluginviewer", "-dcopid", "nspluginviewer-27450"], @pid=27521>]
RubyShell> ps.named(/ns/).inspect
=>[#<RubyShell::ShellObjects::Ps::Process:0xb7878edc @name="mdnsd", @cmdline=["/usr/sbin/mdnsd"],
 @pid=6809>, #<RubyShell::ShellObjects::Ps::Process:0xb785ac20 @name="nspluginviewer",
@cmdline=["/usr/kde/3.5/bin/nspluginviewer", "-dcopid", "nspluginviewer-27450"], @pid=27521>]
RubyShell> ps.pids.inspect
=> [1, 2, 3, 4, 5, 6, 7, 8, 9, 88, 89, 90, 223, 226, 228, 240, 261, 262, 263, 861, 1019,
1066, 1068, 1069, 5006, 5090, 5173, 5174, 5180, 5181, 5182, 5183, 5186, 5199,
5201, 5337, 5381, 5443, 5532, 5949, 6060, 6202, 6224, 6225, 6226, 6316, 6633, ..., 29380, 31581]
RubyShell> ps.from_pid(1).inspect
=>#<RubyShell::ShellObjects::Ps::Process:0xb783a948 @cmdline=["init [3]"], @pid=1>
RubyShell> ps.from_name('init').inspect
=>#<RubyShell::ShellObjects::Ps::Process:0xb7834ae8 @name="init", @cmdline=["init [3]"], @pid=1>

You can also kill a process (ps.from_pid(1).kill ‘TERM’, for example) and do a few other things like check if it’s alive or if it’s a kernel process and get the parent process.

A couple of bugs made their way into the 0.5 release, including output that doesn’t call inspect automatically on its results and a ruby indicator that isn’t followed by a space. There’s also a bug in the Process class used in ps that means if it’s printed without an inspect, an error arises. These will all be fixed in RubyShell 0.6 (and, in fact, already are in trunk).

Here is some of what you can do with if_config:


RubyShell> if_config.interfaces.inspect
=>[#<RubyShell::ShellObjects::IfConfig::Base::Interface:0xb780ace8 @name="lo">,
#<RubyShell::ShellObjects::IfConfig::Base::Interface:0xb780ac48 @name="eth0">]
RubyShell> if_config.names.inspect
=>["lo", "eth0"]
RubyShell> if_config['eth0']
=>#<RubyShell::ShellObjects::IfConfig::Base::Interface:0xb77f4c2c @name="eth0">
RubyShell> if_config.eth0.inspect
=>#<RubyShell::ShellObjects::IfConfig::Base::Interface:0xb77f4c2c @name="eth0">
RubyShell> if_config['eth0'].ip
=>192.168.2.39
RubyShell> if_config['eth0'].mask
=>255.255.255.0

There’s the potential for much much more, but for the time being this is about it for if_config.

So these are shell objects. They’re a very promising idea, and I hope some people will contribute more objects. Note, though, that the way shell objects are implemented in 0.5 and currently in trunk is likely to change, I just haven’t thought it through 100% yet.

File Handling

Until 0.5, RubyShell only operated interactively—i.e., you couldn’t just hand it a file and have it read and run that file. That is no longer the case. Now, you can pass files on the command line to the ruby_shell.rb script and it will execute all of them. Currently, it executes them all in the same context; I’m still deciding whether or not I like this better than executing them in separate contexts.

Hand in hand with this new addition is the addition of an initialization file. RubyShell will now try to read a ~/.rbshrc and run whatever initialization code is there before running, either for files or interactively. This file can contain anything that’s valid RubyShell code. Here’re the contents of mine as it stands:


require 'colored'
hostname | host = $pipe.strip
shell.prompt = Proc.new do
  "\#{ENV['USER']}@\#{host}".green + " \#{Dir::pwd.gsub(File.expand_path('~'), '~')} # ".blue
end

alias 'ls', 'ls --color'

Whammo! And therein lie hints as to the two remaining treats that await within…

Interpolation

RubyShell now supports interpolation of RubyShell code. That is to say, it supports executing code and replacing that section of the line with the code’s results. This is what’s usually done in ``s in other shells like bash. As it turns out, RubyShell 0.5 uses #{} for this, just like string interpolation for Ruby.

Sadly, this interferes with the actual string interpolation. As you can see above, you can escape such an interpolation using \#. If that isn’t done in the above section, you’d get an interpolation that would happen once when the RubyShell parser hit it, and never again—thus resulting in a prompt that never updated its values. RubyShell 0.6 will, instead, use the traditional backticks (``) to just dodge this problem entirely.

Command Aliases

Finally, RubyShell 0.5 supports aliasing commands. This is done through a normal Ruby method, alias_cmd. It just takes a command and what it should become. However, the command can also be a regular expression, or anything that will behave nicely under #===. For example:


RubyShell> alias_cmd 'ls', 'ls --color'
RubyShell> alias_cmd /ram/, 'ls'
RubyShell> arama
*results of alsa*

The value of the regular expression thing is one I’m still not 100% sure of, but it seems like it could be used for interesting and cool things. Also, through some hackery in the RubyEvaluator, instead of alias_cmd, you may use alias. The only caveat is that alias has to be the very very first thing in the statement. Otherwise, the replacement won’t take place (so that regular aliasing can be used). Thus:


RubyShell> alias 'ls', 'ls --color'

What’s Next

On the horizon is an implementation of output redirection. Currently, the plan is to make the syntax `word>’, where `word’ corresponds to an actual method somewhere (haven’t decided where yet) that will take the filename as a parameter and could potentially apply transformations to the output as it goes.

RubyShell 0.3: Pipes, Parsing, and CDs

2007-08-16T06:37:00Z

It turns out, RubyShell 0.2 ran code twice when you asked it to run it once. Whoops…

Anyway, once again that’s only one of the several new features (okay, that one was a bug fix, too…) in RubyShell 0.3. If you want to skip the explanation, you can go ahead and download it now or look at the list of fixed/implemented issues.

RubyShell 0.3 brings a couple of brand spankin’ new things to the table. The first is the validation process.

Parsing/Validation

Validation is the process whereby RubyShell validates code as being Ruby. If it isn’t Ruby, then it’s run as a command. Originally, this was done using the defined? keyword, which, while kind of awesome, unfortunately actually ran the code embedded within it, which I had originally thought was not the case. Fear not, however! I have fixed this. RubyShell 0.3 uses ParseTree to make a real attempt to parse the code without any side-effects. In addition, it brings a restructuring of the validation so that validation is handled in its own class, with the evaluation class (the RubyEvaluator class) acting as the Parser class’s conduit to the validator. Multiline Ruby is now handled by looking for syntax errors of a particular kind, which makes it far more flexible than the previous versions, and means RubyShell should be able to handle most multiline Ruby code.

As part of this, the validation includes exceptions for two common shell practices, the trailing / and the trailing ~. For example:

cd ~

This is technically a valid first line of Ruby, with the next line promising to be the other side of the complement operator. However, since it is more common in a shell to be using ~ as `my home directory’ than it is in Ruby to use the complement operator at the end of a line, this is simply considered invalid Ruby and will be run as a command. The same applies for a trailing /, which is commonly used on the end of a directory name, as in shadowfiend/ruby/.

CDs

The CdCommand got a few updates during this release. It now correctly handles tidle expansion (by using the handy File.expand_path method… Kickass method, that!), and running a `cd’ with no parameter is now synonymous to `cd ~’. It’s kind of awesome.

Pipes

Finally, and, for me, most awesomely, this release brings the improved piping syntax to RubyShell. This syntax lets you use $pipe as an implicit receiver for the first message of a line of Ruby code in a piped expression (actually, in any expression). For example, say you wanted to run an `ls’ and then print each line with a trailing `RubyShell rocks!’. In RubyShell pre-0.3, you would do¹:

ls | $pipe.each_line { |ln| puts "#{ln}RubyShell rocks!" }

Not horrendously ugly, but also not very pretty. RubyShell 0.3 still allows this syntax, of course; however, there’s a new one. Now, any calls to methods that start with with_ or and_ are assumed to mean `$pipe.’. The above example in done this way:

ls | with_each_line { |ln| puts "#{ln}RubyShell rocks!" }

There, doesn’t that look better? How about we want to have everything in the current directory, but we want to censor any mentions of `java’:

ls | and_gsub 'java', ''

Shiny, eh?

Other Fixes/Improvements

The presence of the new validator means a lot of validation-related issues were fixed for this release. Frederick Ros also provided a patch to fix RubyShell’s crashing shamelessly when you hit `enter’ with nothing on the line, ‘cause he’s just a fabulous person. Plus, now if a line of code wasn’t evaluated as Ruby and failed to run as a command, an error is printed. Some refinement need still be done—for example, printing an error is unnecessary if the command itself prints anything to stderr or stdout. Not sure how to determine whether that happened, though. It may be that the error will instead be printed if a valid command cannot be found. Again, however, this will be somewhat difficult to determine, since Kernel#system returns false whether that’s the case or the command just didn’t exit happily. Might be able to use the exit code for that.

Le Future!

Or, in Indo-Chinese², `the future!’ While I wanted to do output redirection for this release, I decided to shelve it for now so I could think out and research the syntax a little better. The feature issue tracking it is over here, and I already posted some initial thoughts on how I might do the syntax. Also on the list is better error handling—i.e., it would be nice if RubyShell didn’t die a horrible death when you made a syntax error in the code you typed.

Further out, there’s the idea of improving shell completion, though that’s really dropping further on the list the more I think about it, since having filename completion (which is already present) is really by far the most important thing for a shell. Also on the `indeterminate future’ list is an initialization file (~/.ruby_shellrc or something) and the ability to run actual files of RubyShell code. These two should be relatively simple, and the former depends on the latter. I also need to implement backtick notation (``), but making that basically interpolation of any RubyShell command, including regular Ruby. That’s all I can think of for now, but more ideas are welcome! And I’m sure I’ll come up with a few more of my own, as usual.

Enjoy!

¹ Actually, whether you would really do this is anybody’s guess, since RubyShell pre-0.3 handled blocks in piped expressions a little oddly, and would sometimes explode in a shower of fiery sparks.

² Really, that should have been `Sino-Indian’, if only to use `sino’.

RubyShell 0.2: Now With 100% More Ruby!

2007-08-12T21:44:00Z

It turns out, RubyShell 0.1 couldn’t handle method definitions or local variables because of the way Ruby code was being evaluated. Whoops…

Anyway, that’s only one of the several new features (okay, that one was a bug fix…) in RubyShell 0.2. If you want to skip the explanation, you can go ahead and download it now or look at the list of fixed issues.

RubyShell 0.2 brings a few new features in addition to the sizeable bug fix of working method definitions and local variables.

Multiline Ruby

First in my mind (probably because it was the most work) is the presence of support for multiline Ruby. For example:

RubyShell> def my_method
RubyShell> 'test'
RubyShell> end
=> nil
RubyShell> my_method
=> "test"

Other constructs supported as multiline constructs are if, while, until, and class.

There’s still no real parser for RubyShell. Our Parser class is more of a decider class—deciding whether code will be run as regular Ruby or as a command. However, for this release it has taken on very limited parsing capabilities. Basically, Ruby can be handled multiline for specific constructs (see above) that appear at the start of a line. Basically, if any of these words start the line, then the `multiline level’ is increased by one. Every time an `end’ is seen, that level is decreased by one. When it reaches 0 again (its initial state), the block is considered finished and is fully evaluated as Ruby. A backslash on the end of the line will also increase the multiline level, this time until a line appears without a backslash on its end. Finally, a do anywhere in the current line will also trigger an increase in the multiline level.

Amongst other things, this approach means that commands that have the same name as any of these keywords will be completely unsupported (since seeing them automatically opens a multiline Ruby mode). This is only a small problem, since these constructs aren’t typically used as command names anyway.

Multistatement RubyShell Code

In addition to handling multiline Ruby, this release also supports the semicolon (;) as a statement separator. So, for example:

RubyShell> def my_method; 'test'; end; echo magic
=> nil
magic
RubyShell> my_method
=> "test"

In this context, Ruby code and regular commands can still be freely intermixed. Internally, the parser takes this line apart and actually parses them as individual statements as if the input had come in on separate lines. Currently, splitting on the semicolon is done in a fairly hackish way (a simple split, then a recombination in a loop to handle semicolons inside strings); this would be better handled by a regular expression, but I was unable to come up with a satisfactory one. If any ideas are forthcoming, I would definitely appreciate some help in this regard.

Shell History

Shell history is implemented. In fact, it was a fairly trivial addition (just had to pass an extra parameter to the readline method). However, it’s still missing part of what I wanted it to be able to do, namely the compression of multiline Ruby input into a single line separated by semicolons (and the ability to make that optional).

Customizable Prompt

The funnest new feature in this release is the customizable prompt. The prompt can now be set to either a string or a callable object (usually a proc, but anything that responds to call will do, so you could technically write your own full-blown prompt-generating class if you wanted to). Some examples:

RubyShell> ReadlineMonitor.prompt = 'Man am I Cool# '
=> "Man am I Cool# " 
Man am I Cool# ReadlineMonitor.prompt = lambda { "#{ENV['USER']}@#{Dir::pwd}> " }
=> #<Proc:0xb7b8b890@./ruby_shell/ruby_evaluator.rb:31>
shadowfiend@/home/shadowfiend/ruby_shell_tags/ruby_shell_0.2/lib> cd ..
shadowfiend@/home/shadowfiend/ruby_shell_tags/ruby_shell_0.2>

ANSI color codes should work for this prompt, too; so, combined with the `colored’ gem:

RubyShell> require 'colored'
=> true
RubyShell> ReadlineMonitor.prompt = lambda { "#{ENV['USER'].green}@#{Dir::pwd.blue}> " }
=> #<Proc:0xb7b8b890@./ruby_shell/ruby_evaluator.rb:31>
shadowfiend@/home/shadowfiend/ruby_shell_tags/ruby_shell_0.2/lib> cd ..
shadowfiend@/home/shadowfiend/ruby_shell_tags/ruby_shell_0.2>

(But with colors this time ;))

I’m hoping to make that syntax cleaner in the future. Specifically, I’m hoping “self.prompt = ‘whatever’” will work. The next step would be a simple “prompt = ‘whatever’”, but that one requires a fair bit of voodoo, since it involves catching the assignment to a local variable and making it call a method instead. As such, I’m uncertain whether or not I’ll be implementing it.

On the Horizon

Items on the horizon are listed on the open issues page on Software Without Incident. Output redirection and more readable pipe handling in Ruby is next, with the following syntax being the target piping syntax:

RubyShell> ls | with_each { |file| puts file[0..3] }
ruby
ruby

Downloading

Again, the 0.2 release file is available on the RubyShell files page on Software Without Incident.

A Ruby Shell? Sure! RubyShell!

2007-07-27T02:49:00Z

Once again, I put the Rails tutorial on the back burner (sorry folks :-/) for a coding project. This one wasn’t quite so little as the last one, though. While thoughts went through my mind about writing something similar to JAXB called RAXB, and I created the project on Software Without Incident and all, that’s not what I ended up working on. Instead, my focus the last month or so has been RubyShell: a UNIX shell written in Ruby with Ruby as its primary language. I’m ready now to make an initial release, so here it is: RubyShell 0.1!

Before I go on, I should mention that there has been a previous attempt at a Ruby shell; it was called ruSH and sits on RubyForge. Unfortunately, its last release was on August 28, 2005, four days after its first release. It was announced on ruby-talk and everything, but for some reason it seems to have died. The domain where it was being hosted (off RubyForge) seems to have similarly expired. That said, I only briefly glanced at the ruSH code while writing what I did, and didn’t really use any of what I saw (though I might look there again for inspiration in the future).

Currently, RubyShell can do a few things:

Run external commands. Currently, this is done through Kernel#system most of the time, which may be a bit of a cop-out; however, I believe it allows most of these commands to run on Windows. Recent revisions to the way I’m using Open3 may have dispelled these concerns, though. Using Kernel#system also gives the ability to run a program like vim without a problem, while running it with Open3 causes serious issues.
Display a prompt (I know, it’s basic, but it’s there :)).
Run single-line Ruby (including semicoloned stuff). There’s currently no parser or anything, instead we use the defined? keyword, as detailed in this post on RedHanded. Sadly, as I mentioned, this means only single-line Ruby is currently supported. Whether support for multiline Ruby will come via a parser or via some quiet hackery that will allow common multiline constructs (e.g., def, class, if, etc.) is up for debate at this point.
Connect commands with pipes. This is the exception to the Kernel#system comment above. When output is being piped, external commands are run through Open3 to capture their output. Ruby code in piped commands has access to the output of the last command via the $pipe variable. Alternate syntaxes are in the works; I’ve written up some ideas as part of the RubyShell Implementation Concepts/Ideas document on SWI.

That’s about all I can think of right now. Some things are hacked together in a nasty way; the way a PassthroughCommand with output capturing is monitored for completion comes to mind (currently I’m using Thread.list.length, which is just stupid).

I’ll be trying to establish a RubyForge project for RubyShell, but in the meantime it can be found, as with all my tinkerings, on Software Without Incident. The RubyShell project page is here, and the release files are here. Comments are welcome, help is welcome.

I’ll be off on vacation starting on Saturday for about a week, but I’ll try and keep an eye out to catch comments and such.

Enjoy!

UPDATE: I’ve had a chance to try it on Windows, and it looks like Kernel#system there handles command-line arguments in a weird way. dir /w, my test, didn’t seem to work, while just plain dir did. Regardless, pipes still don’t work there, unsurprisingly. This is because a gets runs at least once when the first command in a list of piped commands isn’t Ruby code. On Windows, this gets blocks the application (go Windows! Not…), thus disallowing useful things like the termination check from running. Even commands like dir will block it, because the gets runs whether the command expects input or not. As far as I know, there’s no way to differentiate between commands that do and don’t expect input.

I suspicion that it might be possible to use Kernel#select to see if the application wants input, but I seem to recall experimenting and seeing that it returned even if input wasn’t expected. Not only that, but it’ll register as ready for reading and writing. Nonetheless, I’ll see if I can’t try and figure something out at some point to alleviate that issue. It would be kind of rockin’ to have piping on Windows (outside of cygwin and such) :)

A Largely Functional Implementation of `with' for Ruby

2007-06-10T22:27:00Z

As a completely random exercise a month or two ago, I sat down and hacked out a very simple implementation of the Javascript with keyword for Ruby. with basically works like this:

with(object)
{
    instanceMethod(); // no need to prefix with `object'
    var test = 5;
    other_instanceMethod(5);
}

So basically it temporarily scopes function calls to a given object. If you call something in there that isn’t an instance method, it’ll then look for a local function with the appropriate name (note that this is contrary to the assertion I originally made in the aforementioned implementation; I’ve since verified that this is the expected behavior).

Initially, I just passed the block on to instance_eval and happily went my way. It was a satisfactory solution, but not an ideal one, because, as pointed out after the snippet, it failed to respect encapsulation. Moreover, if you called something in there that wasn’t an instance method, it would fail. Now, after being spurred on by my brother’s mention of a friend’s solution that involved proxy objects, I’ve got an implementation which does not have those problems and in fact seems to largely behave the same way as the Javascript construct it is based on.

Without too much further ado, here’s the full implementation:

require 'pp'
module With
  def with(obj = nil, &block)
    raise ArgumentError, 'Block expected.' unless block_given?

    # Collect instance methods from the block's original binding so we can pass
    # them on to the proxy object, since otherwise the +instance_eval+ will lose
    # them.
    instance_method_names = eval('methods + private_methods + protected_methods',
                                 block.binding)
    instance_methods = {}
    instance_method_names.each do |meth|
      instance_methods[meth.to_sym] = eval("self.method('#{meth}')", block.binding)
    end

    proxy = WithProxy.new(obj || self, instance_methods)

    # Inject instance variables from the block's original binding into the proxy
    # object, since otherwise the +instance_eval+ will lose them.
    instance_vars = eval('instance_variables', block.binding)
    instance_vars.each do |var|
      proxy.instance_variable_set var, eval(var, block.binding)
    end

    proxy.instance_eval &block
  end

  class WithProxy
    # Kill all instance methods except __send, __id, and +instance_eval+ (which
    # we need in order to use this as a proxy in a with block.
    instance_methods.each do |meth|
      undef_method(meth) unless meth =~ /^__/ || meth == 'instance_eval' ||
        meth == 'instance_variable_set'
    end

    # Initializes a proxy object that will first try proxying methods to the
    # public methods of the passed +obj+ parameter, and, if that doesn't work,
    # try proxying to any of the +additional_methods+. +additional_methods+
    # should be a Hash indexed by the method name as a symbol.
    def initialize(obj, additional_methods)
      @obj = obj
      @additional_methods = additional_methods
    end

    def method_missing(method, *args)
      if method_allowed?(method)
        @obj.send(method, *args)
      elsif @additional_methods.include?(method)
        @additional_methods[method].call(*args)
      else
        raise NoMethodError, "undefined or inaccessible method #{method} for #{@obj}" 
      end
    end

    # Determines if the specified method is allowed to be passed on.
    def method_allowed?(method)
      methods_allowed[method] ||= @obj.class.public_instance_methods.include?(method.to_s)
    end

    def methods_allowed
      @methods_allowed ||= {}
    end
  end
end

There are a few fixes floating around here to previous problems. First and foremost, it should be clear that, instead of using instance_eval on the object that we’ll be dealing with directly, we instead use a proxy object. For that purpose, we have the WithProxy class. This class has all of its instance methods undef’ed except for the __ methods (send and id), instance_eval (for obvious reasons), and instance_variable_set (for reasons we’ll get to in a second). This proxy object lets us make sure to respect encapsulation by only allowing public methods to be callable.

To this end, this implementation of with uses method_missing to catch method calls, and checks whether they can be proxied to the original object. The method_allowed? method checks whether a given method is in the list of a class’s public methods, and memoizes the result so that further calls to that method don’t induce the overhead of searching the list of public methods every time.

The second piece of the puzzle is the @additional_methods instance variable in the proxy object. This is a hash that maps method names (as symbols) to Method objects. It’s used to specify additional methods that are allowed beyond the public instance methods in the original object. If you’ll recall, the behavior of with in Javascript is to look for functions in the enclosing scope when no instance methods are found with the given name. This list of additional methods lets us do that. If a method is not allowed to be called on the given object, then the list of additional methods is searched and, if a matching method is found, that method is called.

Notice also that the list of additional methods is passed in to the WithProxy’s constructor in the with method, and it consists of all public, private, and protected methods in the enclosing scope of the passed block (retrieved using block.binding and eval). These are essentially all the methods that should be available without qualification within that block.

Finally, we also have to do a similar trick with instance variables. We again use eval and block.binding to nab instance variables available in the enclosing scope and inject them into the proxy object. Usually, instance variables are available in the context of a block, since a block is a closure; however, in this case, because we’re using instance_eval, the instance context of the block changes, so that it’s executing in a different object instance than the block was originally in. Thus, we lose both the original instance methods and the original instance variables. This rather hackish fix takes care of that.

So, that said, is it worth using? I dunno. The added complexity hanging around here isn’t necessarily nice to incur in an application. Maybe it’s worth using in a short script for the purposes of saving a few keystrokes, but in general I’d say it’s just an interesting mental exercise in what’s possible in Ruby when you really put your mind to it.

You can also grab the original source file and the related spec.

life.events.each { |event| learn_from(event) } - Home

awesome_fields Gets A FormBuilder, New headerize Plugin

awesome_fields and the LinedBuilder

headerize—Javascript and Stylesheets Where They Belong

magic_fields Moves to GitHub, Renamed to awesome_fields

Qt Radial Menu for Quill

Quill: Qt Tablet Journaling Program

Declarative Optional Parameters in Ruby

Specing and Thought Process

Syntactic Sketch

A Little Bit of Planning

Initial Specs and Code

Adding Blocks

Metadata

Parting Remarks

RubyShell Update: A Better Validator? And Path Handling

Yay SexpProcessor!

Paths as First-Class Citizens

RubyShell 0.5: Objects, Files, Interpolation, and Aliasing

Shell Objects

File Handling

Interpolation

Command Aliases

What’s Next

RubyShell 0.3: Pipes, Parsing, and CDs

Parsing/Validation

CDs

Pipes

Other Fixes/Improvements

Le Future!

RubyShell 0.2: Now With 100% More Ruby!

Multiline Ruby

Multistatement RubyShell Code

Shell History

Customizable Prompt

On the Horizon

Downloading

A Ruby Shell? Sure! RubyShell!

A Largely Functional Implementation of `with' for Ruby

`awesome_fields` and the `LinedBuilder`

`headerize`—Javascript and Stylesheets Where They Belong

Yay `SexpProcessor`!