Don't Try This At Home: Stealing from the stack

I think there are days, when I want to do things I know I shouldn't as a programmer. Do others experience this. Some have said that Smalltalk is like a gun, that "with great power comes great responsibility." Some times, some of the tricks tempt me, and if I know no one's looking (read: I'm not going to be putting this in any production code), I find myself looking around for opportunities to flex a little bit of language super power muscle. Just for the grins. Just because I can.

Messing with thisContext and the stack is one of those things. When I was implementing the _1:_2:_3:_4:_5: message I was talking about the other day.

The proper and boring way to implement it was like this:

_1: one _2: two _3: three _4: four _5: five

     | arguments |
     arguments := Array new: 5.
     arguments at: 1 put: one.
     arguments at: 2 put: two.
     arguments at: 3 put: three.
     arguments at: 4 put: four.
     arguments at: 5 put: five.
     ^(StringParameterSubstitution default)
          originalString: self;
          args: arguments;
          expandedText

But, I didn't want to do that. That was too much typing I think. I wanted to be clever, so I did this instead:

_1: one _2: two _3: three _4: four _5: five

     ^(StringParameterSubstitution default)
          originalString: self;
          args: (thisContext stack copyFrom: 1 to: 5);
          expandedText

There are no references to any of the method arguments. Knowing that the stack is already an array with the arguments already placed in them, exactly what I want, I just grab that, instead of making my own array populated with the method arguments.

Don't do this in production code. It's tricky and evil. But sometimes, it's good to remind yourself, or learn from others, what this great environment really is capable of doing. Who knows, having one be aware of it, there may come a point where playing with thisContext or the stack, may help you solve a real problem, in production code, or not.

A thought about field identifiers

When I was playing with template field substitution last couple of days, I was again reminded that the common Smalltalk substitution syntax (John Brant informs me that it is indeed in more than just VisualWorks and Squeak) is really frustrating to use for any sort of HTML/XML generation. The use of the < and > means you have to constantly escape those characters in your templates if you are generating any kind of output that you actually want to include the alligator brackets.

And a thought occurred to me. As long as I'm not generating HTML or XML, I couldn't think of a better field identifying character to use. There are some others such as { } or [ ] that work visually as well. But for the last 15+ years, we've all been reading more and more and more of the "when Lisp met alligators" syntax. It's been pounded mercilessly into our brains. We've all gotten quite accustomed to parsing, in our heads, the text shows up between brackets, apart from the rest. Because of that, using them as field identifiers, is actually the best thing possible.

It's the meta that's the achilles heel. When you want to use these same field separators to generate other field separators. In other words, I could posit that if HTML/XML had used { } to enclose tags, we'd find ourselves appreciating it in a substitution syntax the most, and hating it most when we tried to use it to generate more of the same.

A Tragedy: When Localization met Interpolation (repost)

This is a repost from my old blog. When I end-of-lifed that Blog, I said might pull some of those over here. Lukas's comments on the previous Syntactic Tartness for Macro Expansion reminded me of this one. At the time I wrote that, I failed to give credit to Steve Dahl, who I had spent about 2 days kicking ideas back and forth with on Skype about this.

Adrian Kuhn commented in a previous post:

"In general, I think that Smalltalk desperately misses String interpolation!"

Yes, I agree. But the devils running amuck in the details. The problem is that I18n translation made it first. There's two ways that I've seen to do translation: 1) statically recompile your program, making a sweep of all (or subset) of the strings in the program and translating them 2) doing translation at runtime, by looking up the string in some database. The first is pretty old school. :)

The challenges with String Interpolation are a few:

Far Reaching Changes

You need to modify the compiler. You might be able to try to have a unary message that evaluates a string, compiling expressions on the fly in the context of the sender. In this case, the tools can do nothing to help you get the code right. They get confused, because you create variables that don't appear to be referenced. You can't do it right in the context of clean closures. So a parser/compiler change is definitely in order. And don't forget the usual rant, it's not enough to modify just the base compiler. You've got to look at it in the context of the debugger's ability to put breakpoints as well as step through code, in the context of the RB parser, which you want for rewrites and formatting, as well as code highlighting maybe. This does not make it undoable. It simply means it's not a quick thing you can whip up over the weekend.

What Is It When?

So lets say you have the expression:

string := '[[customer name]] is [[customer age]] years old'.

If you use it as the argument of a nextPutAll: send, you probably want it to be in expanded form. But, if you want to look it up in an I18n dictionary, then you would rather have it in symbolic form.
You could detect string literals with whatever the expression preamble is ([[ in the examples above) at compile time and build some sort of complex literal that held both blocks as well as a string. But you'd still have to figure out how to resolve the message =. In an assert: form, you'd probably want expanded, but at dictionary look up time for translation you'd want the compressed form.

What Happened Elsewhere

It's been interesting to spend a little time looking at how Ruby and Python do these. This page makes it clear that the Ruby gettext guys also understood that it is hard to have your cake and eat it too when it comes to mixing interpolation and localization. The answer isn't as definitive with Python, but I think I've convinced myself it's a similar story after looking at docs for a little while.

Syntactic Tartness for Macro Expansion

One thing that is very natural for Smalltalk image based programming, is to programmatically assemble source and install it into the very same running program. I've been using the RB Change framework to do just that with something I'm working on lately.

To piece together the appropriate source, working with a template source and string, and then fill in the variables is something that's desirable. VisualWorks and Squeak (maybe some other Smalltalk use this same approach?) have an ability to expand templates strings with macro substitution. The template for a setter method might look something like

'<1s>: anObject
     <1s> := anObject'

To fill out those fields, you send messages like expandMacrosWith:, expandMacrosWith:with:, expandMacrosWith:with:with:, and expandMacrosWithArguments:.

I am no fan of this API. First, it is too verbose. When I'm looking at template substitution, I don't want a bunch of other longish selectors. They dilute the information I'm trying to glean as I piece together the template and what's being substituted.

Secondly, I find it doesn't scale well when evolving the code. It's common that I start with a simple template in the first cut of code. Something like

    'Hello <1s>' expandMacrosWith: aName

But as I refactor and discover more needs, the need to add parameters arises. As long as I only add two more parameters, I can just use the variants with the additional with: keywords.

'<1s> ^self <2s> <3s>'
     expandMacrosWith: aVariableName
     with: aBasicAccessingMethod
     with: sizeof + 1

As soon, as I go to 4 fields though, I have to change gears and use the expandMacrosWithArguments: API and build the sequence myself. It could be argued that it's best to just always start with this version, but it's the very longest of the selectors. If you're using VisualWorks, and don't have the language syntax for array construction (e.g. {statement. statement. statement.}), then it's even funner, because you can use the Array with:with:with:with: expression, but if you need to move to 5 fields, then you've got to change your code all around again.

Over the years, I've tried a couple of different experiments to make this all something I liked a little better. They've used involved interesting binary selectors (e.g. "%") and proxy objects, or at least fun with doesNotUnderstand: messages. I thought I'd try something a little different. I wanted something that correlated well with the numbered fields, but was uber-terse as well.

So I went with the shortest selector that could possibly work: _1:, _1:_2:, _1:_2:_3:, etc. Written in selector shorthand like that, it's pretty ugly. When actually used in code though, it improves some:

'<1s>At: anOffset
     ^self <2s> anOffset * <3p> + <4p>'
     _1: aVariableName
     _2: aBasicAccessingMethod
     _3: aByteSize
     _4: sizeof + 1

and

'inspector<1s>Field
 %<inspectorFields>
 ^Array with: (Tools.Trippy.DerivedAttribute
    label: ''<2s>''
    valueBlock: [self <2s>])'
     _1: upperVariableName
     _2: aVariableName

and

'<1s>
     ^(0 to: <2p>) collect: [:n | self <3s> <4p> + (n * <5p>)]'
     _1: aVariableName
     _2: anInteger - 1
     _3: aBasicAccessingMethod
     _4: sizeof + 1
     _5: aByteSize

I don't dare call this syntactic sugar. The use of the underscores is too ugly to be sugary. It's a sort of bitter sweet thing, thus the "tart" label.

It does solve two problems nicely. It is very terse. You see a minimal amount of "scaffolding" getting the job done, and are free to spend more time looking at the template and the substitutions. One could get that though by using inlined arrays with a shorter selector, something like:

'<1s>
     ^(0 to: <2p>) collect: [:n | self <3s> <4p> + (n * <5p>)]' macro: {
          aVariableName.
          anInteger - 1.
          aBasicAccessingMethod.
          sizeof + 1.
          aByteSize.}

One thing you lose with this though, is the strong association between each substitution and field. In the former example, when I glance at the template and see field 4, and then want to know what's being substituted, I see it instantly. I just find the 4 in the selector below. Without that direct link, I have to parse the array linearly to find it.

I haven't used this _syntax enough to decide if it's usefulness would overcome its ugliness, but it was interesting to play with it and discover the visual readability aspect. I'll likely use it on some more non-production stuff to get a better feel.

I've Inherited an Interesting Dilemma

Consider two browser screenshots. The first is the class side of AlphaBlendedIcons, with the needsComment method selected.


This second is the instance side of NameSpace, the needsComment method selected.


In both cases, if you invoke the menu of the selectors list, and choose the Hierarchy Implementors option, you get the following two results, respectively...



So what's the issue? The issue is the presence of ClassDescription in the Implementors browsers. Should it be there or not?

On the one hand we have the notion of "hierarchy." In RB land, a Hierarchy Browser is a pretty common thing. It is a limited view of classes always going as high as Object. If you are on the instance side, it tops at Object. If you are on the class side, it tops at Object class. From this point, where the term "Hierarchy" refers to a browsing view or scope, it is surprising to see these entries for ClassDescription here when you ask for Hierarchy Implementors. What you expect to see, is only methods that you'd find as you navigated around in the Hierarchy Browser.

Then there is a more pedantic side. It tends to be those who like to think of themselves as "I have ascended into the thirteenth circle of Smalltlak Uber-Knowledge." They know the zen-secret that there's a "crossing of the streams" that occurs at Object class. When a method is not found at Object class, the lookup for the method doesn't stop there, but continues up on the instance side at Class, then ClassDescription, then Behavior, and finally Object (instance). The little up/down arrows we place next to methods, even show this. If you look at the first one, it shows that there is a super implementor of needsComment for AlphaBlendedIcons, that is found farther up the search tree.

These two notions of "hierarchy" when viewed from the class side, are at odds with another. When someone sees that menu option, which kind of "Hieararchy" are they more likely to think about at that moment. Should it be left the way it is, but reword the menu option to be "super/sub implementors?" What would you want it to do?

OSX Lion: First Day Thoughts

I downloaded and installed Lion yesterday. Mostly, I'm still acclimating before drawing judgements and throwing darts. But one thing stands out already.

There's this great story about how Steve Jobs convinced Bill Atkinson that rectangles with rounded corners were a good thing. The idea being that we as humans tend to prefer them and so would naturally prefer them in our UIs. The rest is history. Over the years, more and more of our UI elements get sanded off corners.

There is another pendulum that's been swinging along the Apple UI Design vector that annoys me. Color. Or lack thereof. It's no secret that Apple uses some of its frontline apps to feel out UI evolutions. Back in 2006, I remember discussing the latest iTunes version, and describing what I saw as an attempt on Apple's part to create "Lickable Motiff."

The first thing that keeps hitting me with the subtle changes made to Mail and Finder, are that they both seem to have been through another bleach cycle. Mails top button bar has had all hue rinsed from it. Finder's side bar is nearly colorless. I keep thinking "Those icons are just glyphs from some foreign language. Ignore them." Maybe I should just spend more time in Terminal and use good ol' ls.

I understand some of the influences that drive Apple designers to do this. It's easier to harmonize sets of icons if you use less colors. It helps deal with colorblindness issues. Etc.

On the other hand... I'd like to take the Apple UI designers on a walk around their campus. And point at things. And say "See? Color!" If we get rounded rectangles (problematic as they are to graphics libraries) because they show up in real life, why can't we have some color too?

Who knows, maybe everyone at Apple wears black turtlenecks now, and they all really see less color. Yeah, maybe that's it.

Rope: an evolved Chain

One of the areas my Chain did not come out ahead was in the area of adding truly new keys to a Dictionary (as opposed to just updating what already was stored at a given key) or removing them. Both of these operations potentially involve #become: sends. While VisualWorks #become: is very fast, it's not as fast as some other operations. And not all Smalltalk implementations have a fast #become:. Here's the chart of how well Chain compared to IdentityDictionary for original key add/removal.


While I was on vacation, John Brant pinged me encouraging me to use a #changeClassTo: strategy instead of #become:. To do so, first I cloned the Chain class as a new class called Rope (and the Empty variant as well).

Then we make EmptyRope a subclass of Rope. This is important, because we need an EmptyRope to have the same instance variable layout as a Rope. #changeClassTo: will only change the type or class of an object, if they have the same instance variable layout. We'll just make sure that never actually set or reference the key, value, or nextKnot variables in the EmptyRope subclass.

In the case of adding a new key/value pair, when our #at:put: reaches the end of the list, the EmptyRope subclass can just implement it as

at: aKey put: anObject
   self
      setKey: aKey
      value: anObject
      next: EmptyRope basicNew.
   self changeClassTo: Rope

And the only thing #removeKey: has to do in addition to the original #replace: trick is do a

   previousKnot changeClassToThatOf: self

to get the last link to be an EmptyRope.

Most of the rest of the code stays the same. We can reduce a couple of duplicates since the classes are now in a parent-child relationship, rather than a sibling one.

The at:put:/removeKey: test is the interesting one to rerun and see how things fair. First of all, we compare the new Rope approach to the original Chain one.


That shows some improvement across the board. The real test though, is how does it compare to the original IdentityDictionary?


The good news is, it's actually faster. Not by a huge amount. But it removes it from something you have to worry about.

I've replicated the Rope code up to the Open Repository (package TAG-Ropes). If you know you're using smallish dictionaries (size <= 15) and need to care about performance, then this might be the thing for you.

It would be interesting to see how such an approach fared in GNU Smalltalk or one of the other Smalltalks.