» Scripting Gedit +

Gedit allows you to write python scripts which interface with its backend (and frontend via pygtk). This is very cool, for reasons obvious to VIM and Emacs users. You can write your own plugins to manipulate the document you are editing in many useful ways.

Well I was using various external tools to run my various scripts for test purposes (the External Tools plugin is itself written in python!). So I'd bind F5 to ruby, F6 to python and F10 to perl. Then, depending on the script I was editing, I'd press the corresponding accel key to run it. But I realized that I could just write a plugin that would run any script according to its bang line. The following is the plugin I came up with (place it in ~/.gnome2/gedit/plugins):

First is the plugin definition file (named script_runner.gedit-plugin):

[Gedit Plugin]
Loader=python
Module=script_runner
IAge=2
Name=Script Runner Plugin
Description=This plugin runs scripts by reading the bang line.
Authors=Jordan Callicoat <MonkeeSage@gmail.com>\nJonathan-Marc Lapointe
Copyright=Copyright © Jordan Callicoat, 2006
Website=http://rightfootin.blogspot.com

Then is the actual python script file (named script_runner.py):

#  Script Runner plugin
#  coding: utf-8
#
#  Copyright © 2006 Jordan Callicoat
#  Copyright © 2008 Jonathan-Marc Lapointe
#  Released under the python license.

import re
import os
import gtk
import gedit
from externaltools.functions import capture_menu_action
from externaltools.ElementTree import Element

class RunScriptPlugin(gedit.Plugin):
  """
  A simple plugin that runs the current document
  in the interpreter specified by the bang line
  """

  def run_document(self, action, window):
    """
    Just Do It!
    """
    # read in the first line of the buffer...
    buff  = window.get_active_view().get_buffer()
    siter = buff.get_iter_at_line_offset(0,0)
    eiter = buff.get_iter_at_line_offset(1,0)
    data  = buff.get_text(siter, eiter)
    # now check if it contains a bang line...
    if data:
      text = re.match(r'^#!(.*)$', data)
      if text: # it does...
        text = text.group(1)
        # extract a label (for use in the output
        # panel) from the bang line...
        label = os.path.basename(text).split(' ', 1)
        if label[0].lower() == 'env':
          label = label[1]
        else:
          label = label[0]
        # build an ElementTree Element to feed capture_menu_action...
        elem         = Element('tool')
        elem.command = text
        elem.name    = label # .title()
        elem.input   = 'document'
        elem.output  = 'output-panel'
        # now run the script!
        capture_menu_action(None, window, elem)

  def activate(self, window):
    """
    Setup stuff
    """
    actions = [
      ('Run', gtk.STOCK_EXECUTE, 'Run Script', 'F5', 'Run Script', self.run_document),
    ]
    # store per window data in the window object
    windowdata = {}
    window.set_data('RunScriptPluginWindowDataKey', windowdata)
    windowdata['action_group'] = gtk.ActionGroup('GeditRunScriptPluginActions')
    windowdata['action_group'].add_actions(actions, window)
    manager = window.get_ui_manager()
    manager.insert_action_group(windowdata['action_group'], -1)
    ui_str = """
    <ui>
      <menubar name="MenuBar">
        <menu name="ToolsMenu" action="Tools">
          <placeholder name="ToolsOps_3">
            <menuitem name="Run" action="Run"/>
            <separator/>
          </placeholder>
        </menu>
      </menubar>
    </ui>
    """
    windowdata['ui_id'] = manager.add_ui_from_string(ui_str)
    window.set_data('RunScriptPluginInfo', windowdata)

  def deactivate(self, window):
    """
    Teardown stuff
    """
    windowdata = window.get_data('RunScriptPluginWindowDataKey')
    manager    = window.get_ui_manager()
    manager.remove_ui(windowdata['ui_id'])
    manager.remove_action_group(windowdata['action_group'])

  def update_ui(self, window):
    """
    UI Callback
    """
    view       = window.get_active_view()
    windowdata = window.get_data('RunScriptPluginWindowDataKey')
    windowdata['action_group'].set_sensitive(bool(view)) # and view.get_editable()))

After you add these files, you need to restart Gedit. Then you go to Preferences -> Plugins tab -> check the Script Runner Plugin option, and you're set. Now pressing F5 will automagically run any script with a bang line. Cool stuff! :)

Note: The External Tools plugin needs to be enabled for this script to work! Updated to work with newer gedit versions, thanks Jonathan.

» Conventional wisdom +

The conventions of a given community of programmers are usually time-tested and often they make life much easier on the programmer. For example, the convention of writing self-documenting code rather than using one or two letter variable names is a very helpful guideline.

But as with many other issues, what is generally beneficial and a good practice for most people in most situations gets promoted to the absolute best thing for everyone always. And anyone who doesn't think so is either a bad programmer or is stupid for departing from the conventional wisdom of the community.

It seems to me that this kind of thinking can lead to stiffling creativity and growth in problem solving. You just can't get your head around approaching a problem in a different way than the common way (i.e., "only right way"); you'll say "why do you want to do it like that, you should do it like this!" (I've seen this happen many times; regarding everything from XHTML to JavaScript to python to ruby).

And what is silly is that many of these conventions are simply matters of personal taste: for example, coding style. But yet people are instructed that it is best to ignore their own prefered coding style and to just follow along with the community, because the way the community likes it is The Right Way®™; and should the person not follow the conventions of the community, then they must be trying to "program in a different language". "Haven't you read PEP 8?!" it is asked in awe and reverence of the almighty style guide.

Or, when something really is a bad idea sometimes; people get all fearful and turn that into every time. "What? You use...eval?" it is intoned in frightened, hoarse half-wispers, lest the devil drag you into hell for mentioning the name of his son. Oh, but instance_eval is fine though(!).

I'm not some kind of rebel iconoclast, standing against everything conventional. I like most of our conventions, actually. I just don't like it when we cling to them so tightly we refuse (or literally cannot) look past them to mabye improve them or find something better; or when we take what amounts to subjective preference and make it the absolute "right way" to do things; or when we get so worried we'll do something wrong we make it harder to do something right. Uh, yeah. That's about the long and short of it, heh.

And of course, I never do any of that; you see, I have this convention that says... ;)

» Spicey curry in your ruby +

I've been learning OCaml for the past few days, and for being a member of the evil functional programming paradigm (invented by people who actually liked doing calculus!), it's not too bad. It has some cool features like pattern matching, subtyping (variants) and some other fun stuff. But over all, it's just too much of a pain to use (speaking personally). If I ever had to do something that absolutely needed speed or type safety, like write a program that drives my car around town for me and does my errands, then I would probably use it over C/C++; but I honestly can't see myself using it that often. But as I said, it does have some nice features. More than a couple times as I worked through the tutorial I was heard to say "nice", "nifty" and I think I even said "funtastic" once (I talk to myself, out lound, often -- the meds aren't working, heh).

One of the great things about OCaml (and most of its functionalistical kindred) is currying (or "partial application"). Currying has something to do with Lambda Calculus and Category Theory and other fields of knowledge that are only useful to people who can count to infinity on their fingers using only self-reference and let bindings. The only thing mortals like us need to know is that it is where you take a method that has multiple arguments, partially apply it (i.e., make a new method with one of the arguments consumed), and return a new method that takes n-1 arguments. So, for example, method m:

def m(a, b, c)
  print a, b, c
end

Could be curried (doesn't actually work in ruby, just showing the idea) like so:

m_bc = m('value a')

And what that does is, in essence, make a new method named m_bc, that looks like:

def m_bc(b, c)
  print 'value a', b, c
end

In other words, m with parameter a consumed. Pretty cool little feature of most functional languages.

Well, python will be getting curry support and some other goodies in 2.5 via the functools module (PEP 309), and you can already find some implementations of curry for python floating around the net, such as:

curry = lambda func, *args, **kw:\
           lambda *p, **n:\
             func(*args + p, **dict(kw.items() + n.items()))

I just think ruby should have some curry, too! Curry chicken and saffron pilaf for ruby, I say! So to that end, here is a (probably buggy -- I just wrote in in about 10 minutes) implementation of curry for ruby.

class Object
  def curry(*args)
    if args.size == 1
      m = self
      a = args.first
    elsif args.size == 2
      m, a = args
    else
      raise(ArgumentError,
        "`curry': wrong number of " +
        "arguments (#{args.size} for 2)",
        caller)
    end
    def _unbnd(m, a)
      lambda { |*b| m.bind(a).call(*b) }
    end
    def _proc(m, a)
      lambda { |*b| m.call(a, *b) }
    end
    if m.is_a?(UnboundMethod)
      fun = _unbnd(m, a)
    elsif not m.is_a?(Proc)
      begin
        m   = method(m)
        fun = _proc(m, a)
      rescue NoMethodError, NameError
        begin
          m   = self.instance_method(m)
          fun = _unbnd(m, a)
        rescue
          fun = lambda {}
        end
      end
    else
      fun = _proc(m, a)
    end
    fun
  end
end

def add(a, b, c=0)
  a + b + c
end

## here's how you use it
add2 = curry(:add, 2)
p add2[4] # => 6
p add2[6] # => 8
add4 = curry(:add, 4)
p add4.call(0) # => 4
add6 = add4.curry(2) # currying other curries
p add6.call(2) # => 8

## you can also curry class/instance methods
## ps. these examples are stupid
arr_rev = Array.curry(:reverse, [1, 2, 3])
p arr_rev.call # => [3, 2, 1]
arr_len = curry(Array.instance_method(:length), [1, 2, 3])
p arr_len.call # => 3
arr_new = Array.curry(:new, [4, 5, 6])
p arr_new.call # => [4, 5, 6]

Anyway, it's not like curry is so amazingly wonderful and let's you do all kinds of things you can't already do, but it's nice for creating little custom Procs out of methods quickly.

In fact, here is one actually useful way to use curry:

debug = curry(:print, '*** DEBUG -> ')
# ... bad stuff
debug["Oh no! You broke something (again)!\n"]
# ... crud, better fix this!
debug["Dude, you totally suck at programming...\n"]

» Code formatting +

I try to be consistant about code formatting. I pick a convention and stick to it. But I like the convention to make sense, as well as being easy to read (easy for me, I could care less if you can read it, heh).

If you've seen any of my code in my posts, you probably noticed that I use parentheses for everything -- method invocation, conditionals / other control structures, built-in methods like require, &c. So I write like this:

require('mickeyds')
if (combo_meal and a_drink)
  supersize('mr pib')
else
  puts('you cheapskate, pay the extra buck')
end
case (eat)
  when (/good/)
    i_m_lovin_it
  else
    get_refund("I'm goin' to taco bell!")
end

But I've been thinking lately that it doesn't really make sense to parenthesize with control structures, unless it is for the explicit purpose of grouping. They aren't methods, so they shouldn't be written like methods. I think I'm going to start writing them without the parens and see how I like it. I'm still not sure about require, and puts and so forth. They are methods, but then they are not regular methods -- they are top-level methods. Mabye I'll try writing them with space rather than parentheses as well.

» ruby reopen +

In ruby we have a nifty reopen method we can use on references to file-handles that are already opened. This is helpful when you need to open a file as read-only, then you want to write to the same file, or you just want to use the same reference to point to a new file-handle. You simply reopen and go on about your merry way. Usually this is used within a File.open block. But with python you have to explicitly close the original file-handle and open a new one. So here is a convenience class-wrapper/method to emulate the ruby functionality:

class file(file):
  def reopen(self, name=None, mode='w', bufsize=None):
    if not name:
      name = self.name
    if not self.closed:
      self.close()
    if bufsize:
      self.__init__(name, mode, bufsize)
    else:
      self.__init__(name, mode)
    return self

fh = file('test.txt', 'rb')
print fh # <open file 'test.txt', mode 'rb' at 0xb7c92814>
fh.reopen(mode='wb')
print fh # <open file 'test.txt', mode 'wb' at 0xb7c92814>
fh.close()

This way we can reopen a different file, or the same file in a different mode, just as with ruby, and the custom method will take care of closing the existing file-handle, opening the new one, and keeping the same reference.

» More on python flatten +

The standard (but for some unknown reason, not built-in) python flatten method looks something like this:

def flatten(l):
  out = []
  for item in l:
    if isinstance(item, (list, tuple)):
      out.extend(flatten(item))
    else:
      out.append(item)
  return out

This obviously runs into recursion errors pretty quickly for highly nested lists; what suprised me is that it can't grok even relatively shallow nesting, e.g., 10 levels deep:

a = []
for i in xrange(10):
  a = [a, i]
a = flatten(a)

Traceback (most recent call last):
  File "test.py", line 13, in ?
    a = flatten(a)
  File "test.py", line 5, in flatten
    out.extend(flatten(l))
  ...
  File "test.py", line 5, in flatten
    out.extend(flatten(l))
RuntimeError: maximum recursion depth exceeded

Someone smarter than I am (viz., Danny Yoo) wrote a better method using an iterator, which looks basically like this:

def iter_flatten(iterable):
  it = iter(iterable)
  for e in it:
    if isinstance(e, (list, tuple)):
      for f in iter_flatten(e):
        yield f
    else:
      yield e

a = []
for i in xrange(300):
  a = [a, i]
a = [i for i in iter_flatten(a)]

As you can see, this works with deeply nested arrays (up to 499 levels on my box). He also wrote a very opaque version using a form of tail recursion (via continuation passing), which is "not meant to be read by humans". That version can handle lists nested as deeply as the system recursion limit (1000 on my box)! That's pretty cool (even though my brain implodes when I try to understand it, heh)!

Just for fun, I decided to see what I could come up with. Here is my offering:

def flatten(l, limit=1000, counter=0):
  for i in xrange(len(l)):
    if (isinstance(l[i], (list, tuple)) and
        counter < limit):
      for a in l.pop(i):
        l.insert(i, a)
        i += 1
      counter += 1
      return flatten(l, limit, counter)
  return l

Nothing fancy. It's about as fast as Yoo's continuation version, but it breaks at 499 levels like the iterator version (but the iterator version is slower and requires the extra list-comprehension syntax if used for assignment). I also added an (optional) limit argument, to specify the maximum number of levels to flatten (ala ruby's Array#flatten). Don't worry about the counter argument, that's just to internally track state across recursions.

Addendum: Wow! I just came across a wonderful version of flatten in a cookbook comment. I saw it a few days ago, but I didn't really think about it, just kind of thought "yeah, another recursive flatten method," but this one whoops all of the others for speed, nesting level support and elegance! This is from Mike C. Fletcher's BasicTypes library. The method looks something like this (I've altered it a bit—see comments below):

def flatten(l, ltypes=(list, tuple)):
    ltype = type(l)
    l = list(l)
    i = 0
    while i < len(l):
        while isinstance(l[i], ltypes):
            if not l[i]:
                l.pop(i)
                i -= 1
                break
            else:
                l[i:i + 1] = l[i]
        i += 1
    return ltype(l)

a = []
for i in xrange(2000):
  a = [a, i]
a = flatten(a)

Freakin' genius!

* Fixed for empty lists/tuples based on Noah's comment
* Fixed again based on Greg's comment
* Fixed yet again based on John Y's comment

Addendum: I got to wondering how Mr. Fletcher's version would stack up against the built-in Array#flatten method in ruby. Granted, his doesn't have a flatten limit, and I think it would be kind of hard to add one, but then I have never really needed that feature. So here is the ruby version with times:

def flatten(l)
    i = 0
    while i < l.size
        while l[i].is_a? Array
            if l[i].empty?
                l.delete_at(i)
                i -= 1
                break
            else
                l[i...i+1] = l[i]
            end
        end
        i += 1
    end
    l
end

And now the times (tested with 1.8.6 final, best out of three). First, Fletcher's version:

p flatten(1500.times.inject { | m, i | m = [m, i] })

# time ruby test.rb
#
# real 0m0.038s
# user 0m0.028s
# sys 0m0.004s

Then the built-in version:

p 1500.times.inject { | m, i | m = [m, i] }.flatten

# time ruby test.rb
#
# real 0m0.027s
# user 0m0.016s
# sys 0m0.004s

Wow! That's pretty neat! A flatten implemented in ruby that is competitive with the C backend, heh! Now I know there are reasons for this, and like I said, Fletcher's version will just smash everything at every level, without regard. But still, it's always cool to find a bit of interpreted code that breaks out a can of Chuck Norris on the interpreter! (ruby is so manly it can almost beat itself up! heh!)

» No builtin flatten in python... +

One view of why there is no built-in flatten in python is that "there are several open semantic questions which are not intuitively answered". But that seems like a cop-out. So what if there are some academic questions about what exactly a flatten method should do? The basic fact of the matter is that people want a method to be able to flatten a multi-dimensional list into a single dimension. Philosophical questions aside, there should be a built-in method to do this. Special cases should be treated as special; let the implementors in those cases create a custom flatten method that will do what they want; edge-cases should never dictate the behavior of the core language. Bottom line is that a flatten method is used often enough to warrent inclusion as a built-in method, even if it doesn't frost the cheerios of every single person who might ever use it.