Off The Air

I think it is time to call this blog complete. There will always be exciting things happening in the Lisp world, but I just don't have time to blog about it (as you have seen!) So in a while this blog will disappear. Bye, friends!

Someone should make a mirror because I can't figure out how to download my posts!

Hope against hope

This would be too good to be true! Hopefully this would be the keynote every day!

Evolve, evolve, evolve

Arc has hit version 2 already. This is pretty amazing! It's already one version better than a leading Common Lisp competitor, SBCL, which is languishing at version 1. It will soon catch up to CLisp at version 3.44. It gets tougher after that, with Allegro CL at version 8 and CMUCL at an incredible version 19! But the grand pappy of them all must be mzscheme, which is sitting pretty at version 372. I love arc but it will take a while to catch up to that!

The Uni-can'ts and Uni-can of Uni-code!

Arc was famously released with a defiant attitude towards Unicode. If you think you need Unicode, you don't need Arc, or so the saying goes. This sure caused a ruckus among the idiots who would never use Arc anyway!

Then one guy had the audacity to say this:

"Okay pg, it's time to call you out. Unicode support is not trivial, like you make it out to be, and it's not a waste of time. It's a critical piece of infrastructure for any runtime. You fail."

Then came the knockout punch. Patrick Collison of CROMA fame added full Unicode support to Arc by adding -2 (that's minus two!) lines to Arc! Mr. Graham came out with this followup to the above sass:

"I'm glad this is preserved for posterity, since it did turn out to be trivial and in fact got added with about -2 lines of code a few days after this comment was posted..."

It's no joke. Unicode required removing things from Arc, not adding it! That's how damn elegant the language is! And the support is much simpler than it is in other, less elegant, more gross programming languages. For example, here's the entire definition of "upcase":

(def upcase (x)
  (let upc (fn (c)
             (let n (coerce c 'int)
               (if (or (< 96 n 123) (< 223 n 247) (< 247 n 255))
                   (coerce (- n 32) 'char)
                   c)))
    (case (type x)
      string (map upc x)
      char   (upc x)
      sym    (sym (map upc (coerce x 'string)))
             (err "Can't upcase" x))))

Other languages have to deal with databases, which are inherently slower and bureaucratic. Viaweb used zero databases!

Already mainstream

When was the last time you saw an article about Lisp in The Register? Me neither! Thanks for the linkage!

Freedom is in Sight!

I didn't know if Arc could replace my current dabblings in CL, but now I know it should very soon: Mr. Graham wrote Arc to get away from CL. Where he goes, there I will follow! Unfortunately a lot of my code uses arrays right now, but porting it to lists (or tables keyed with integers) shoudl be easy enough.

Brevity and Arc

Thankfully Paul Graham has addressed the kerfluffle over utf8 characters. It's amazing that people were so eager to jump on some technical shortcoming instead of the real purpose of Arc: to make programs shorter. And what program is shorter than the one you can't write in Arc at all? That is a meta solution that too many are overlooking in their eagerness to be high school bullies towards Mr. Graham!

Arc Brevity

Here is an idiom I am forced to use a lot in Common Lisp:

(incf (gethash table key 0))

The wacky historical names make it hard to tell (incf? come on, ANSI!), but it increments the table entry for key in table. If key is not in table it uses a default value of 0, adds one to it, and stores it back in the table.

Arc does not have this onion. Instead, you see this elegant style used again and again in the arc sources:

(= (table key) (+ 1 (or (table key) 0)))

As you can see, this is much shorter (just line up the code above with this code). Partly because the symbols are very short; aside from the table and key name, the only arc symbols needed are =, +, 1, 0, and or. There is almost a mathematical purity to those symbols. It reminds me of the famous equation: e^pi i - 1 = 0.

Arc debarks

Arc is out. I didn't know if I would be writing these words in my lifetime. I was beginning to think I was stuck writing Scheme (ugh) Common Lisp (double-ugh!) forever. Don't get me wrong, those languages have their places but on the whole they are incredibly gross and messy. And not in the wanton, sleek way Arc is gross and messy.

First of all, Arc is a compiler. This is a boon. There are too many toy Lisp interpreters out there already. It compiles the Arc language to a lesser language. In this case, the lesser language is Scheme, but that's just an implementation detail. I'm sure these Arc will be ported "to the metal" (or is that "to the meta"? ha ha!) soon enough by an enterprising student who doesn't listen when the master says it can't be done, just like John McCarthy and the student who ported Lisp to the metal. That guy.

Another remarkable thing about Arc is what Mr. Graham has been able to strip away. He has gotten down to the essence of the language. Everything else can be added on. Common Lisp is incredibly and unnecessarily complex, so by stripping things off you can get a lot of power and flexibility. For example, you can take Common Lisp. Then take away packages, specialized vectors, CLOS objects (OO, ugh!), structures, complicated lambda lists, complicated streams, generic functions, a tree shaker, delivery options, native code generation, Emacs integration, the potential for a commercial IDE, arglist hints, symbol completion, Unicode support, debuggers, profilers, inspectors, GUI tools, and a lot of other cruft.

What you are left is is the bare essense of coding. Everything written is essential. Nothing is superfluous. There are truly no onions. The specification of Arc (which is the implementation of Arc) can be printed on just a few pages (full duplex); the whole thing is 4609 lines of code!

Let's be clear. Paul Graham is the #1 Lisp luminary right now. When he writes that he is a typical Lisp hacker, it's also 100% true. But there's another factor. He's typical, to the nth degree. When he says Lisp needs to go in a new direction, people who know nothing about Lisp will sit up and listen. It's his kind of leadership that will bring Lisp back into the mainstream.

This blog post is a little long. I'll leave you with the axioms of the new Lisp world:

(define (ac s env)
(cond ((string? s) (string-copy s))  ; to avoid immutable strings
      ((literal? s) s)
      ((eqv? s 'nil) (list 'quote 'nil))
      ((ssyntax? s) (ac (expand-ssyntax s) env))
      ((symbol? s) (ac-var-ref s env))
      ((ssyntax? (xcar s)) (ac (cons (expand-ssyntax (car s)) (cdr s)) env))
      ((eq? (xcar s) 'quote) (list 'quote (ac-niltree (cadr s))))
      ((eq? (xcar s) 'quasiquote) (ac-qq (cadr s) env))
      ((eq? (xcar s) 'if) (ac-if (cdr s) env))
      ((eq? (xcar s) 'fn) (ac-fn (cadr s) (cddr s) env))
      ((eq? (xcar s) 'set) (ac-set (cdr s) env))
      ; this line could be removed without changing semantics
      ((eq? (xcar (xcar s)) 'compose) (ac (decompose (cdar s) (cdr s)) env))
      ((pair? s) (ac-call (car s) (cdr s) env))
      (#t (err "Bad object in expression" s))))

ARC IS RELEASED!!!!!!

I am almost hyperventilating here! You can go download Arc right NOW! I can't wait to feel the elegant language-tsunamis this unleashes as thousands of hackers realize their potential!!! Can't talk...must download!! I promise a full review in the next few days as I bask in the reflected macro, closure, and elegance embodied in ARC!!!!!!!

YAY!!!

Brothers

Here is a blog post that teaches an important lesson: An elephant like CLOS isn't needed to dance on the deftly woven webs of closures.

Arc inspired by...moi??

Paul Graham has deigned to share more tantalizing glimpses into the future sublime juggernaut we call Arc. I was so pleased and surprised to see that he has a construct very much like my own w/file called w/infile! (See the def for codelines)

I can tell that switching from BruceLisp to Arc will be like slipping into an old, familiar, elegant boot!

Hello to comp.lang.lisp

Oh my! It appears that I have been discussed on comp.lang.lisp recently. Sorry I haven't written more lately, but I've been very busy and haven't been able to work on Lispy topics as much as I have wanted to lately.

So what's on my mind? The same themes that have puzzled Lisp enthusiasts for decades!

1. Macros -- I always come back to Macros with Lisp. Along with tail recursion, it's the way that Lisp stands out from other languages.

2. Closures -- another feature unique to Lisp, closures are basically object orientation...done right! No more spaghetti code with so-called classes, objects, etc. You can just get all the functionality you need by deftly weaving closures together. As you may know, spaghetti doesn't weave. It snaps or is just too greasy to hold together. Lisp closures are more like thread. (But not like threads, which are another barbaric construct.)

That's it for now...I'll write more later!

Exploration and Experimentation

One of the nice things about Lisp is the REPL. That stands for Read, Eval, Print, Loop. I like to think of it as Read, Experiment or Explore Print, Loop. That's because when I have questions about how Lisp works, I Experiment and Explore at the REEPL.

Other languages do this too, such as python with its dir() function and ruby with the ruby shell. Perl also has the debugger, which is a great interactive perl shell too.

A big downside of Common Lisp is the size of its specification. Scheme's spec is 50 pages. Lisp's spec is over 1000! There's an old joke: how many common lispers does it take to change a light bulb? One, he just stands on the spec, reaches up, takes out the bulb, puts in a new one, and the light's changed. By that, I mean the spec is pretty big.

But the good thing is that you rarely have to read it. If you want to know how a function works, just experiement! Here's an example. I want to get random numbers. So I just start by typing a bit:

CL-USER> (ran <-- cursor is here

Then I push TAB, and out pops "random"! Lisp is great that way! But how does random work? Easy, let's just experiement a little:

CL-USER> (random 10)
0
CL-USER> (random 10)
5
CL-USER> (random 10)
1

From that short experiment it's easy to see that random is a lisp function that returns a random integer between 0 and 1/2 its argument (in this case 5, which is 10 times 1/2). Why they picked 1/2 I don't know, but you can always multiply by 2 if you want to do something like select a random element from a directory. There are also some web services that will return random numbers. I don't have all the answers, get creative!

What's good about Common Lisp?

My last post has received too many negative comments. Let's brighten up the tone by discussing some of the things Common Lisp got right.

1. Obviously, macros have to get the top billing. Macros are like little compilers. They take your source code, do some transformations, and turn it into unreadable CPU-specific code that is blazing fast. For example, (defmacro my_plus (&rest; args) (apply '+ args)). You don't have that kind of flexibility in other languages (except C with some inline ASM, but Lisp gives you more abstractions to work with (also C suffers from the too-fast problem, but that's a different blog post (oh no, look at the parentheses (can you tell I love lisp? :))))))

2. Pathnames

3. The ability to make your own object system at will. This is partly an advantage of macros but I need to come up with at least a few good points.

4. Elegance & grace of some parts (if you avoid the unelegant parts). For example, the ability to rename defun to define and make it more logical and memorable in the process.

Can you think of more? Please share in a comment...and let's keep it positive, please!

Why Lisp?

Sometimes people ask me why I stick with Lisp if I am critical of it. First of all, I love Lisp. It's Common Lisp that gives me fits sometimes. There are a lot of things that need fixing, and even people who think Common Lisp is ok will give you a long laundry list of problems. For example, there are problems with the type system, the object system (no Visitor pattern built in??), the macro system (dirty unhygienic macros by default??), the number system (slow "big nums" by default?? this breaks every algorithm out there), the variable namespace system (Lisp-2?? This was discredited a long time ago! just read any blog), the type system, the condition system (exceptions are slow...this was proved by C++ in the 80s). What was ANSI thinking? The good thing is that some of these can be fixed by the Lisp community. How about it, community?

So why Lisp? And why Common Lisp? The alternatives are worse.

• Python - this one is easy. Slow, whitespace sensitive.
  
• Perl - hard to read. Good mix of fast and slow.
  
• Ruby - too slow.
• C - hard to read, too fast
• C++ - hard to read

I hear you asking: Why not Java? Java's not too slow, not too fast, and easy to read (especially after reading the sublime The Java Programming Language, 3rd Ed). It isn't whitespace sensitive. It runs on every platform. It has a wealth of design patterns and enterprise tools. However, it lacks two key things: macros and closures. If they added clean and elegant macros and closures to Java, I wouldn't feel bad about using it. At that point, it would be a form of Lisp!

Arc is released!!!

I don't know how I missed it, since I'm probably the biggest Arc fan out there, but Arc has been released! ...after a fashion! Paul Graham has been testing its sublime waters with its first public application, Startup News from YCombinator. (I thought it was going to be a spam-proof web-based email service, but I'll take what I can get!)

The source code for the site is not available, but based on Mr. Graham's past writing I can imagine the elegance. Instead of writing a bunch of code to display pages, it is likely a symphony of Little Language construction. This Little Language (built in Arc) is customized very precisely for the problem at hand. Then all the tasks in the application boil down to actions expressed in the most concise, powerful way possible.

Of course I cannot be sure, but I bet the exclamation point on the end of the Startup News sentence is this tour de force of brevity, concision, and domain specificness:

(mkreddit frontpage-downvotes: #false)

Mr. Graham can confirm or deny in the comments, if he deigns.

Long Time No Post

I am still working on finding more time to blog. I know there are people who really enjoy Lisp at Light Speed. Rest assured that I am still reading a lot and thinking a lot about Lisp, and sometimes I write Lisp too. Here are some links for today:

• What about Lisp?
• SubX

The author of these two pieces gets into a key issue for Lisp: intuitiveness. I wonder if he's aware of Bruce's First Law of Lisp?

Design, or Consideration of Happenstance

Whether, if, upon inspection, the "design" of a thing, may be taken, it is not altogether clear taken as true, whether to trust in the thought of a planner &mdsah;; can it be seen as "true design" or rather a fortunate accident, not being visible to the planner directly? Yet this is how without introspection and -- before reconsideration -- the planner must decide the merit, or congrutiy, of said thing.

There may be some of the lispers who taken with power decide their view is "over the horizon" and yet perfect, without adaptability, and thus must take all others as "contrary, foolish, and without merit" -- yet to do so, premature, invites bickering.

Yet it is an accident mostly, yet they do not see it thus, but the collision of "reality" must show the "design, of a thing" a failure or success, by chance.

Critical mass

Now this is what I'm talking about! People unbiased by knowledge of or experience with Common Lisp are the real movers and shakers in the Lisp community!

Concisification

I think my previous post was a little too long winded. Let's call the following pithy quip "Bruce's First Law of Lisp."

If it does not do exactly what you expect with zero hours consideration or experience, it is a bug in Lisp that should be fixed.

There's no reason why Lisp shouldn't reward your existing intuition and common sense instead of punish it. Fortunately it provides the tools to knock off these rough edges.

Teaching guinea pigs to dance

One annoying thing about Lisp is its community. As a group they are the most resistant to new ideas of any group I know. For example, a lot of my blog posts get a lot of comments from "community experts" pointing out how stupid and shortsighted my solutions are. Maybe they should just try to get over their 50-year history of failure and compromise and just get with the program! Newbies don't want to learn nonsense-words like "cons" and "cdr" and "make-instance". There is no excuse, in 2006, not to use unambiguous, clear names like "make-two-valued-structure", "first-value", "second-value", and "mkobj".

Lisp, the programmable programming language. Ha! What good is being able to program it if people shoot down your ideas all the time? That's why I'm glad things like TwinLisp and newLISP, Paul Graham's sublime ARC, and CROMA are coming around. The people behind them know the key insight: the best way to work with Lisp is to customize it into something completely different.

In summary, never try to teach a guinea pig to dance. They are way too small and scientists have recently discovered that they have a complete lack of rhythm-detecting neurons, perhaps because of their native desert habitat. Which reminds me, I love Brian Herbert's take on the Dune universe.

Macros

I like to browse comp.lang.lisp from time to time. Today that practice was rewarded with a nice macro that I wish I wrote myself: wif. Here is the code:

(defmacro wif (stream filename &rest; body)
   `(with-open-file (,stream ,filename :direction :input
                            :element-type   'character
                            )
                   ,@body))

As you can see, it compresses a very common thing down into three letters. The only problem is the conflict with my existing wif macro, "Wonderful IF".

List accessors

One of the nice things about Lisp is the ability to think of a problem in English, write it out in a Lisp file verbatim, then add parentheses and macros until everything works. For example, I have an application that manages a recipe list, and each recipe is made up of lists of lists of ingredients. The first element of a recipe list is the name of the dish, the second is the cooking time, the third is the original author, etc. How to change the cooking time? "Setf second list new-value" is how I would phrase it in English.

(Aside: I tend to use a lot of lisp forms in english, which makes translation easier. For example, I might ask a coworker "Hungry-p" to determine if he is ready for lunch. I am thinking about switching over to the scheme predicate notation, though.)

What does that look like in lisp? Just add parentheses!

(Setf (second list) new-value)

One step and we're done.

The recipes have grown a lot, though, and now I have more than ten elements. Since even I don't walk around talking about the "nth 15 list" item of a list, I added new accessors to Lisp to make it easier to refer to further items in the list. Here is the list of new accessors:

11TH, 12TH, 13TH, 14TH, 15TH, 16TH, 17TH, 18TH, 19TH, 20TH, 21TH, 22TH, 23TH, 24TH, 25TH, 26TH, 27TH, 28TH, 29TH, 30TH, 31TH, 32TH, 33TH, 34TH, 35TH, 36TH, 37TH, 38TH, 39TH, 40TH, 41TH, 42TH, 43TH, 44TH, 45TH, 46TH, 47TH, 48TH, 49TH, 50TH, 51TH, 52TH, 53TH, 54TH, 55TH, 56TH, 57TH, 58TH, 59TH, 60TH, 61TH, 62TH, 63TH, 64TH, 65TH, 66TH, 67TH, 68TH, 69TH, 70TH, 71TH, 72TH, 73TH, 74TH, 75TH, 76TH, 77TH, 78TH, 79TH, 80TH, 81TH, 82TH, 83TH, 84TH, 85TH, 86TH, 87TH, 88TH, 89TH, 90TH, 91TH, 92TH, 93TH, 94TH, 95TH, 96TH, 97TH, 98TH, 99TH.

If you're interested in the code for these, just let me know...I can send you named-nths.lisp.

Long time, no blog

Sorry to everyone for not writing more lately. I've been very busy at work, and my time for Lisp has suffered as a result. I haven't been completely idle; I have been working a code-walker that rewrites standard Common Lisp into a more elegant dialect. I call this process "Brucifixion." Expect to see some more info about it soon!

Narrowed again

It looks like I am no longer on Planet Lisp. I'm not sure of the exact cause. Bookmark my site to keep up with my writing!

A Wider Audience

Wow! I just discovered my blog is on Planet Lisp! This is truly an honor, and one that I don't take lightly. For those of you reading me for the first time via that venue, I invite you to peruse my blog, which has code examples, ideas, and even improvements for Common Lisp. Lisp is an adventure...won't you share in mine?

Concision is equivalent to powerfulness

Personalization is a key feature of Common Lisp. Eventually, you are not writing Lisp, you are writing a custom language for solving your problem, and then writing in that custom language to solve your problem. I haven't gotten to the problem solving part yet, but I have been making great progress in language customization. (In fact I like to think of myself as the lead coder on the BruceLisp project, which is probably a life-long endeavour!)

There are many long function names in Lisp: MULTIPLE-VALUE-BIND, WITH-OUTPUT-TO-STRING, CHAR, etc. I have written a mkalias macro that lets me redefine certain Lisp functions to shorter, more aesthetically pleasing values. Here are a few examples:

• with-open-file => w/file (similar fixes for other with-macros)
  
• make-array => mkbuf (I always consider my arrays to be buffers for things, so many array functions have aliases with "buf" in the name. the only glitch is the clash between bref as "buffer reference" and my bref macro, "beautiful aref".)
  
• char => chr
  

Now I can write:

(defun filestr (file)
 (w/file (strm file)
   (let-1 buf (mkbuf (strmlen strm) 'chr)
     (dotimes (i (strmlen strm) buf)
        (setf (chr buf i) (readchr strm))))))

See? Much more concise, and all within the limitations of Common Lisp (except for strmlen, which shells out to the "stat" shell command to get the file length).

Scheme -- a new direction?

I have been working mostly with Common Lisp. I have read some books about Scheme, including the first two chapters of Structure and Interpretation of Computer Programs by Abelson and Sussman, but I never really looked at it very closely. That changed today when I came across an article in SysAdmin magazine! Here is a link. From the article:

Scheme is a dialect of Lisp that stresses elegance and simplicity. It is a stripped-down version of Common Lisp. In fact, the language specification is about 50 pages, while the Common Lisp’s specification is about 1,300 pages. Scheme is often used in computer science courses to teach abstraction and programming concepts. Even though it is used primarily as a teaching language, Scheme has numerous applications — the GIMP allows users to write GIMP-loadable modules in Guile.

I had always admired Scheme for its purity and adopted some of its notations in my Common Lisp code (to this day I write lst instead of list), and now it looks like that carefulness may pay off as I embark on porting my Common Lisp utilities and macros to Scheme.

Beauty in motion

One of the "onions" (Paul Graham's wonderful phrase) of Lisp is IF. There are multiple problems:
First, IF only allows one form for both the then-form and else-form. That means you have to introduce ugly progn forms to
group multiple statements. How this got past the ANSI committee
I'll never know. Second, IF works on T and NIL but many sources in
other languages treat 0, NULL, and the empty string as false. I
have fixed both of these problems with two macros of my own. (See?
Lisp may have its flaws, but its true strength is the ability to
correct them without going "outside the language.")

I try to introduce idioms as soon as the urge strikes me and use
them as pervasively as possible, so their use quickly feels more
natural. Therefore I bring you GIF and NIF.

GIF ("Graceful IF") is a macro that can have multiple forms for
each part:

      (gif (some-test)
         (print "The test is true") (print "And this also prints when true")
         (print "The test is false") (print "And this also prints when false\")
   

The key is that the then-forms and else-forms have to be on one
line. Lisp does not have good support for determining if forms are
entered on the same line, but I have a graceful work-around: I
check the length of all the arguments and if they would fit into
80 characters they are considered part of the same clause. This
has worked for a lot of situations but I still need a little
debugging for some stuff, especially when the line runs a little
long.

NIF ("Natural IF") is like IF in other languages. It accepts many
different values as false: NIL, the empty list, 0, the empty
string, a cons cell whose car and cdr are both nil, the empty hash
table, the string "0", the symbol :FALSE, and packages with no
exported symbols. I don't think I need to show you, gentle reader,
any examples; I bet you can think of many ways to plug this into
your active coding already!

I have made a code package for these two utilities called
STANDARD-PRACTICES; soon I'll upload it to cliki so other people
can see the same productivity benefits I have.