Advent of Code 2021: Day 1

Oh oh oh! Finally it’s that time of the year again!

I love Advent of Code and I’ve played it with friends and colleagues for years. Usually, I try to solve it in Ruby, since I’m very familiar with it, and so the effort is only on task solving rather than on learning the language and tooling.

But this year, I thought I might do it both in Ruby and in Raku (The Language Formerly Known As Perl 6), by first solving it in Ruby, and then if I have time trying to solve it again in Raku.

So, here’s a quick view of a couple solutions to the problems in Day 1. Stick to the end for the cool solution past the initial line noise 🙂

SPOILERS AHEAD

The first problem is basically: given a list of values, count the number of times they increase

In Ruby 3, this could look like this

def solve1(array)
  array.                # [1, 2, 5, 4]
    .each_cons(2).      # [(1, 2), (2, 5), (5, 4)]
    .count { _1 < _2 }. # 2
endCode language: PHP (php)

AKA: get a sliding window of pairs from the initial array, then count the occurrences where the first value in the pair is less than the second value in the pair. Notice the fancy anonymous positional block arguments!

You can do more or less the same in Raku, but I’m not aware of a .count method that takes a block§, but well, we can still do this in three steps:

  • build the sliding window
  • select the items matching the condition
  • get the size of the filtered list

Raku does not have each_slice, instead it has a .rotor method which is a generalization of “iterate and group“, you can define in one go both the size of the group and the offset, i.e.

> my @ints = [1, 2, 3, 4, 5, 6, 7, 8, 9]
[1 2 3 4 5 6 7 8 9]
> @ints.rotor(2)  # group by 2 
((1 2) (3 4) (5 6) (7 8))
> @ints.rotor(2 => 1)  # group by 2, and move by one after 
((1 2) (4 5) (7 8))
> @ints.rotor(2 => -1)  <meta charset="utf-8"># group by 2, and move back by one after 
((1 2) (2 3) (3 4) (4 5) (5 6) (6 7) (7 8) (8 9))Code language: PHP (php)

The actual signature of this method is even more flexible, but this should be enough for us.

To filter we can use .grep, which takes a block. We can do this javacript-like, passing a block with a pointed arrow.§

Then we get the size of the list, which is done with prefix '+' . This is an odd perlism, just accept it.

+ @ints.rotor(2 => -1).grep(-> ($a, $b) { $a < $b })Code language: PHP (php)

So, that should work. But at this point it’s just un uglier Ruby! We can do better!

To get there, let’s first see a different way we can build the list of pairs: we can slice the list and merge it with itself, an operation which most languages call zip.

Raku also calls it zip, but it has something cooler than a function: it has a zip metaoperator! §

A metaoperator is basically an operator which takes another operator as argument, and augments it’s behaviour.

The simplest I can think of is the reduce metaoperator. Suppose you want to get the product of all items in a list. You know the operation you want, which is *, and in Ruby, Python, Smalltalk etc.. you would call a method or function called reduce or inject passing a function to it.

In Raku, you apply the prefix reduce metaoperator ([]) to the binary product operator (*):

> [*] [1,2,3]
6Code language: CSS (css)

The Zip metaoperator is the same:, it takes a binary operator and returns a binary operator, which applies its argument to items from both sides zipping them up:

> [1, 2, 3] Z* [10, 20, 30]
(10 40 90)Code language: CSS (css)

See where we’re going? Yep, we can apply the first metaoperator to the result of the second!§

> [Z*] [1, 2, 3], [10, 20, 30]
(10 40 90)Code language: CSS (css)

So, if we want to know “is the item in the left list larger than the corresponding in the right list?” we can just choose the right operator

> [Z>] [1, 2, 3, 1000], [10, 20, 30, 0]
(False False False True)Code language: PHP (php)

This means we can apply it to the slices of the original input to obtain a list of Booleans. Then we just have to count them, and you can do this easily by knowing that in Raku you can add Booleans, and they will be treated as 1 and 0.

Do you know how to add up all items in a list? Yep, reduce metaoperator and binary plus (+)!

So the full solution is

[+] [Z<] @ints, @ints[1..*];Code language: CSS (css)

Or in even shorter form

[+] [Z<] @ints[*,1..*]Code language: CSS (css)

This is at a glance impossible to understand, but it’s actually pretty expressive, and I guess it should get easier with habit, we’ll see 🙂

See you another day, happy hacking!

PS

Problem 2 left as an exercise for the reader. You may want to invest time in learning how to use hyper operators, they are cool!

How to use certbot to get SSL from Let’s Encrypt for EasyWP

I’ve been pretty absent from this blog, so from now I’m going to try to write something at least once a week, on Monday. I may not be able to keep this up, but the first stop to form a habit is to start doing it once.

My requirements for blogging these days are basically: a web UI, I don’t have to maintain deployments, no effort security maintenance, some backups. And frankly the backups are not that important.

Since I already had my domain at Namecheap for a few years, I ended up choosing their EasyWP offering, which is pretty basic but should be fine for now.

While doing this, I took the chance to learn a bit about LetsEncrypt, because it seemed fun.

Let’s Encrypt is a nonprofit Certificate Authority. A CA is basically an entity that can provide a cryptographical proof that you are really you (or rather that some domain like www.riffraff.info is really serving content provided by the owner of www.riffraff.info). For a long time you had to pay for this kind of service, but then Let’s Encrypt appeared and started providing this service for free, with the stated intent of making the internet a safer and better place. Yes, they’re awesome.

The way all this works is by essentially providing some APIs and command line tools that allow you to:

  1. request a certificate for domain.com
  2. get some sort of challenge to prove that you actually own that (e.g. if you really are who you say you are put this string in your DNS records or put this file on your website)
  3. receive a certificate once Let’s Encrypt verifies the challenge is met

The fun part about this is that they will only provide you with short-expiring certificates (2 months), because they want you to automate this. Not automating a renewal will always result in you messing up by forgetting to renew a certificate, like it may happen with anything that you only need to do every few years (checks driving license quickly).

To do that, you can use a wonderful tool called certbot, which has a ton of plugins to be able to interface with many existing services, but does not have direct support for EasyWP/Namecheap.

But fear not! The nice thing about certbot is: you can plug in any kind of script you want, simply providing them as command line options, like this

certbot certonly --manual-auth-hook some-script.sh --manual-cleanup-hook some-other-script.sh -d my.domain.fooCode language: CSS (css)

So, how do we do this for EasyWP? Well, there’s no API for domain management but there is SFTP access to upload custom files. This means we can upload a file and use the HTTP verification method.

When you run certbot, it will talk to the Let’s Encrypt servers and receive a challenge, which it will pass through the environment variable $CERTBOT_TOKEN to the manual-auth-hook and manual-cleanu-hook scripts.

The job of your script is thus to get this value and put it in a correspondingly named file in a specified HTTP accessible directory

# authenticator.sh
#!/bin/bash
set -x -e
# assume there is a .well-known/acme-challenge/ directory already
FULL_PATH=".well-known/acme-challenge/$CERTBOT_TOKEN"
echo $CERTBOT_VALIDATION > $FULL_PATH
echo "put -R ./$FULL_PATH /$FULL_PATH" | sftp $SFTP_USERCode language: PHP (php)

Notice you will need to provide the sftp password on the command line for this to work. You can use expect, lftp or whatever to automate this properly, this is just for example purposes.

Once this runs, certbot will tell the Let’s Encrypt servers to check if the challenge is met, and if it is it will receive a new certificate, which you can upload through the EasyWP admin UI.

But: what about the cleanup hook? Well, once everything is done, certbot will run the hook to allow you to get rid of the leftover files. Once again, you will have $CERTBOT_TOKEN so your cleanup script can look like this

#!/bin/bash
set -x -e
# keep this challenge around, but make it clear we used it
mv ".well-known/acme-challenge/$CERTBOT_TOKEN"  ".well-known/acme-challenge/used.$CERTBOT_TOKEN"Code language: PHP (php)

The full command line would look like this, and notice you can get multiple certs at once

certbot certonly -v --manual --preferred-challenges=http --manual-auth-hook $PWD/authenticator.sh --manual-cleanup-hook $PWD/cleanup.sh -d domain.foo,other.domain.fooCode language: PHP (php)

This is of course more work then necessary as you can just pay a bit more to get an SSL certificate from Namecheap, or you can (I think) proxy everything through Cloudflare, but I had fun playing it with it, and I hope this can be useful to someone else 🙂

Summer readings 2021, part 3

Last summary of my summer readings! Fitting, as the summer is over. I did not read a lot of fiction this summer, but hey, I did read some good stuff!

Hearts of Oak

A short novel by Eddie Robson, which features a weird city made of wood in constant expansion. I would be providing spoilers by adding anything else about the plot, so I will just say that it’s pretty original, and I’ve enjoyed it, after a couple false starts (the beginning is somewhat… dull).

Also, I read the MacMillan paperback edition , which for some reason has the most pleasant paper I’ve experienced in a book, feels like silk. OTOH, the cover has a rubbery feel to it, which is kinda annoying.

7/10: I would like to read more of this universe.

The Flying Sorcerers

An anthology of comic fantasy stories by Peter Haining. I acquired this book, together with The Wizards of Odd, simply because it contained some Terry Pratchett. I sort of hoped the rest of the stories would be good.

On average, they’re not. Neither good, nor fun, and they’re not even about flying sorcerers! Yet, a couple of the stories are quite nice.

5/10: there are probably better anthologies in which to spend your time.

The Wind Through the Keyhole

Ah, yes, a Dark Tower novel. You see, I read the first book of the Dark Tower saga, The Gunslinger, about 30 years ago. Most of it in a single afternoon, while going to the dentist. I loved that strange world, mutants, cowboys, and people singing Hey, Jude. When the Man in Black described the Tower, the Beast§ , the Door, my curiosity reached at the stars.

So I proceeded to read the second book immediately after, and I liked it even more.

And… to read the third book I had to wait years! I found it during a summer vacation, in a small library in a mountain town, and when I got it my father wanted to read it too. He loved it, and so every time a new one came out, I would buy it for him, and read it as soon as he finished it. Fun fact, my father had read The Talisman years before, and I had not.

Me, Sai King, and my dad got to the end of the journey many years later, and while I did not love the last book as much as the earlier ones, it was nice to reach it.

Then The Wind Through the Keyhole came out, again many years later. I didn’t read it for ten years, and I only realize as I write, it’s because my dad had died shortly before it came out.

You see, I never got the chance to buy it for him, so I never read it myself.

As Stephen King says

For longtime readers, this book should be shelved between Wizard and Glass and Wolves of the Calla, which makes it, I suppose, Dark Tower 

and content wise it’s also half way: I loved Wizard and Glass, but didn’t particularly enjoy Wolves of the Calla§. And yet, this book is a much better book than the (chronological) last book, maybe because the author wasn’t so worried about dying before finishing the tale.

Reading this book was like going back to my home town, visiting familiar places, meeting the faces I grew up with. Oh, hey Ronald and Jake!

And the book is good. The plot is good, the structure is good, and for once, the ending is good. Also, as Ronald would have it, it seems I remember the face of my father.

8/10: I really wish there was more of this.

Summer readings 2021, part 2 (Comics)

This summer I finally caught up with a bunch of small comics I had kept on the side, but it would make little sense to discuss all those.

Rather, I’d like to mention a few larger things which had been recommended to me. Spoiler: I liked them all.

Providence

This is a completed 12-issue series written by Alan Moore. It’s part of a larger trilogy (with Neonomicon and The Courtyard) of Lovecraft-inspired stories, and I found it quite entertaining. Basically, Moore plays with the Lovecraft material by having the main character encounter various elements of his legendarium, and tying them up with some of Moore’s favorite obsessions.

I think it’s a good comic, but not fantastic, but if you like either Lovecraft, Moore, or In the Mouth of Madness you will enjoy this one too.

6.5/10: bit more than a passing grade, saved by the ending.

Les Indes fourbes

This is a french comic book (and indeed, the only page on wikipedia for it is in french) but I read the version in italian by Rizzoli Lizard (Nelle Indie Perigliose §).

It’s a picaresque graphic novel inspired by an actual novel from the XVII century, and if you like the genre (I do!) you will love it. The plot is good, the characters entertaining, the writing is great.

As a visual thing: this thing is gorgeous. The art is outstanding, rich in details, cleverly structured, and it makes full use of the medium, as a comic book should. Moreover, the edition is a thing of beauty, as a 33.2 x 24.8 (cm) hardcover it might be a bit hard to fit on a shelve, but I’d leave this thing laying around for the sheer joy of looking at it.

9/10: almost perfect.

The Thrilling Adventures of Lovelace and Babbage

This is an odd book, and by the end I enjoyed it but.. I’m unsatisfied. I went into the book expecting to find either a semiserious account of Lovelace & Babbage’s life, or a made up story using them as character.

What the book actually is: a series of episodes, using a cartoon/steampunk/uchronic version of the titular characters, with no actual plot. They’re entertaining, I really like the drawings and I loved the characters, but I’m left wanting more of them.

The other oddity is that each page is about 30% drawings and 70% footnotes explaining them. And each footnote has references to end-of-chapter notes. This is frustrating§ because you have to choose how to read this

  • read comic -> read footnote -> read endnote (requires you to keep a mental stack)
  • read comic -> read footnote; ignore endnotes (never!)
  • read comic -> read footnote; read all endnotes at the end (you will have forgotten what they’re about by the time you read them)

No choice is satisfactory, and I missed the effortless way you can follow up nested footnotes in, say, Terry Pratchett’s Discworld.

At the end of the book you have some appendixes on various things which I found quite dull, but might be of interest to some people.

So, the book is quite interesting, and if the author decided to actually write a steampunk novel using the characters I would run to buy it; but I got to the end of the book feeling that I had a lot of appetizers, and never got the main meal.

7-/10: could be great, but does not deliver.

Summer readings 2021, part 1 (audiobooks)

It’s been so long since I wrote a blog post, and I’m feeling pretty rusty. But there’s no other way to remember how to do something than practice it, so I’m going to attempt to write a new post every week, until the end of the year.

Anyway, I’m sort of short of arguments, but I was lucky enough to be able to have long holidays this summer, so in the footsteps of more successful writers, I’m going to make a short list of the things I read this summer, in the hope they will be useful to other people. But I read a lot! so I’m going to split it. This is the first instalment, talking about stuff I listened to as audiobooks.

The Box: How the Shipping Container Made the World Smaller and the World Economy Bigger

The humble shipping container might not seem like an interesting topic, but it is! The invention/adoption/perfection of the container changed society in a way that few other things did. For example, a whole category of jobs (dockworkers) has been more or less wiped out. Entire cities that relied on people working on interchange traffic lost their income. Worldwide production chains become possible, which in turn enabled “globalization” and all the good and bad that comes with it.

Plus, the book contains pretty fascinating stories of the people who pioneered this, how they succeeded, how they failed. And of those who resisted or embraced the change. And of course, that marvel of human nature: the standardization committee!

There is something to criticize in the book, and it’s the fact that it is, for all its interesting content, far too rich of details. Honestly, it seems like it could have been 6 or 7 blog posts and still be just as informative.

Surely You’re Joking, Mr. Feynman!

I’ve been recommended this book a million times but always put it off. But I recently found out it’s included in the basic Audible membership I have, so I decided to finally listen to it.

Oh, I am so happy I did it. Feynman is hilarious, the list of anecdotes and odd situations which he experience is mindbogglingly, his insights on human nature are pretty enjoyable.

Also, it’s clear he sort of identifies himself as “anti-intellectual” which kinda resonates with my own education, although I think what he really should identify himself as “anti-pomp” (I think he mentions this at some point). It’s easy to mix up the two.

I cannot recommend this book enough. While reading it tho I couldn’t shake the feeling that, had the book been written today, people would have showed up with pitchforks; Feynman is a womanizer of the extreme kind, and plenty of the adventures (e.g. the one that starts with his regular hanging out at some topless diner (!!)) might not mix well with modern sensitivities.

The New Confessions of an Economic Hit Man

This is another of the free books on Audible. The premise is pretty good: a guy who worked for a big corporation tells his story, explaining how there is a corrupt and ever increasing powerful force of “economic hit men”.

Such people in cooperation with the IMF, World Bank, and USAID, manage to bribe, threaten, and coax politicians and government offices from developing countries so that they get loans, the invest money in projects that promise to have grand returns for the population, but end up being realized by the same foreign corporations, which this way manage to syphon back the money, leaving the countries in deeper debt and without having achieved much. Rich people get richer, poor people get poorer, rinse and repeat.

If the people do not cooperate, then “the jackals” enter the scene, and get rid of the non-complying figures through violent means. If that fails too, there’s always war.

This is not a global conspiracy, but a modus operandi that has been initially used by the US and then by other countries, and effectively constitutes a new form of imperialism.

So far, so good. This, is honestly quite familiar to anyone who read a book or newspaper in the last 50 years, but the idea of reading a first person account is intriguing.

Sadly, the book does not hold up. It’s basically a collection of random stories which are so transparently fake that it’s incredible someone thought to write them down. Like, the guy gets recruited by the NSA be a spy. But the NSA is a code-breaking organization, it does not do that?

Then he meets this woman who explains him of the economic hit man concept Then she obviously tells him “if someone does not play by the rule <throat slash sign>“. Other than obvious corniness, I kinda think this gesture was not popular until much more recent times.

Or, the guy shows up after 9/11 in New York, and obviously meets an old muslim dude who gives him life lessons. He goes to Iran, and also meet people who give him life lessons and tell him how the West is killing their traditional Bedouin culture of leaving in the desert, “a true Persian would not allow that“. But Bedouins are like 1% of the population of Iran?

Anyway, you might say, sure, he’s embellished a story, but there will be some meat in it.

Nope, nothing. He spends an inordinate amount of time describing some situations were the US overturned a government, which you can trivially read on wikipedia, and that in all cases do not show the pattern of the countries being highly indebted, or not even match his own timeline (I.e. they predate the existence of IMF/WB). His first example, the one where he “sells his soul for the first time”, is Indonesia, which still has a debt/GDP ratio of 30-40% (compare: Italy has 130%, Germany 80%) and the poverty rate fell from >60% to 15% in the ’80s, after they started to take on debt (yes, correlation is not causation).

The even more embarrassing thing is that examples of the issue that he wants to talk about would be easy to find! He just doesn’t bother!

It’s a terrible book. But, if you never heard of any of this, it might be interesting, and the last part, basically a “what can I do“, is probably valuable. Caveat emptor.

Matching nested structures with Regexps in Ruby 1.9

NOTE: this was originally published on the older version of this blog, some content may be broken/outdated.

Following a discussion on hacker news I have found myself wondering about regular expressions in ruby 1.9.

In this major version ruby switched its regex engine to oniguruma (and, since a few days ago to a fork of it called Onigmo ).

This engine is widely more powerful than the original ruby one, supporting positive and negative lookbehind, named captures, better support for backreferences and most importantly for our topic, callable backreferences.

The latter means that, among other things, ruby has gained the ability to recognize arbitrarily nested structures.

Since I hadn’t played with this before, I spent some time to craft a Regexp that would match a nested tree like structure, and what better example is there than good ol’ lisp?

This is what I got

# this pattern has many issues, consider it an example
R = %r{
      (?<atom> [-\w]+){0}
      (?<quote> '\g<expr>){0}
      (?<expr>  
        \g<atom> | 
        \g<quote> | 
        \(  
          \g<expr>?
          (\s+  \g<expr>)* 
        \)
      ){0}
      ^\g<expr>$
    }imx

which has a rather nice feel to it, reading bottom up

  • a regex is an expression
  • an expression is
    • an atom or
    • a quotation or
    • an expression (possibly empty) followed by other expressions
  • a quotation is an expression with a single quote in the front
  • an atom is a sequence of word characters or a dash

syntax wise, there are only a couple of things that are unusual: the “grammar rules” are defined like

 (?<foo> some pattern){0}

which is a clever trick (which I first saw in the oniguruma documentation): create a named capturing group, but say that it should match zero times. This way, the definition of the group stays around, but it is not required to actually match anywhere.

The other thing is how we refer to this rules:

\g<foo> 

this simply means: apply the named pattern in this spot, via the awesome “callable backreferences” of oniguruma (also available in PCRE now). Notice that we also get forward declarations, which is rather cool.

Some quick tests show that this works (for an arguable value of “works”) in some simple cases:

t('()')
t('(ciao)')
t('(ciao miao)')
t('(ciao miao)')
t('(ciao (miao))')
t('(ciao (miao bau))')
t("(ciao '(miao bau))")

t('(sum-iter k 1)')
t(<<-SEXPR)
(define sum1  (lambda (k)
    (sum-iter k 0 1)))
SEXPR
t('ok')

t("nope '(miao bau))")
t('(nope')
t(<<-SEXPR)
(define sum1  (lambda (k)
    (sum-iter k 0 1))
SEXPR

t(<<-SEXPR)
(define @  (lambda (k)
    (sum-iter k 0 1))
SEXPR


which correctly returns:

(): ok
(ciao): ok
(ciao miao): ok
(ciao miao): ok
(ciao (miao)): ok
(ciao (miao bau)): ok
(ciao '(miao bau)): ok
(sum-iter k 1): ok
(define sum1  (lambda (k)
    (sum-iter k 0 1)))
: ok
ok: ok
nope '(miao bau)): no
(nope: no
(define sum1  (lambda (k)
    (sum-iter k 0 1))
: no
(define @  (lambda (k)
    (sum-iter k 0 1))
: no

So the pattern correctly matches recursive structures with balanced parenthesis.

Sadly, this approach appears to have one serious downside: it’s impossible to get at the matched elements.

From what I understand, when the regexp engine compiles the pattern, it reserves a certain number of slots for the capturing groups which is independent of the actual string being examinated.

This means that each named group only stores the value of the last match, e.g.

# no reference to "first"
>> R.match('(first last)') 
=>#<MatchData "(first last)" atom:"last" quote:nil expr:"(first last)"> 

This is actually true also for all the other regexes, without named groups or backreferences calls, e.g.

>> /((\w)*)/.match('abc')
=> #<MatchData "abc" 1:"abc" 2:"c">

except that, in my normal life as a regexp user, I’d solve that specific issue by using and getting rid of the capturing group:

>> 'abc'.scan(/\w/)
=> ["a", "b", "c"]
>> 'abc'.scan(/(\w)/)
=> [["a"], ["b"], ["c"]]

but I can’t do it here, as that would mean unrolling the recursion to infinite depth.

I digged for quite a while trying to understand if this was something inherent in the ruby version of oniguruma, or in oniguruma itself, or if I was just dumb.

Turns out, people way smarter than me over at pcre-dev already had a discussion related to this, from which I quote

> Is there a workable approach if the repeating pattern itself has desired
> subpatterns? With the following I find the “b” captured only once:
> /(a(b)c)(\g1)?/ with data abcabc

That’s correct. Nobody has invented a scheme where the contents of capturing subpatterns are expressed as vectors. PCRE returns the value from the last time the parens captured something. So if you had /(a(b|d)c)(\g1)?/ with data abcadc you would get “d”.

Anyway, it seems to me that it should be possible to hook into the regex engine so that when a match happens it is passed to a user defined routine. I seem to recall perl has something like that, and PCRE’s callouts seem to be in the same league.

To be fair, I’d rather use a proper parser than regular expressions in real problems related to nested structures, as regular expressions have a tendency to grow exponentially in complexity. Still, since we can already check if a structure is recursive, it would be cool to be able to get the matched tree structure.

Even better, I’d like regular expressions in ruby to become something akin to pattern matching in perl6, and subsume the issue of basic regular expressions and full blown parsing in one go.

Step, a scala web picoframework

NOTE: this was written on an old version of this blog in 2009, some formatting, links etc.. may have rotted since.

So, after playing with scala on Google App Engine two days ago, I spent a bit of time yesterday (while waiting for food from girlfriend) and today (while waiting plane to go eat at my parents’ easter breakfast, unrelated) writing my hack for a web framework.

I mostly wanted to try scala’s cool abstraction features, so I thought I could try to produce something that had more or less the feeling of immediateness of sinatra.

To get ruby-like blocks in scala you only need to define functions with by-name arguments in this way:

def foo(x:String)(fun: =>Any) 
foo("something") {block returning something to be evaluated later }Code language: Scala (scala)

Of course, what was needed at this point was something that allowed me to match the url and the http method. This is the code to allow that, using a simple Map from path+method to anonymous functions

abstract class Step extends HttpServlet {
  val Map = new HashMap[(String,String),(Any=>String)]() 
  override def service(request:HttpServletRequest ,response: HttpServletResponse) {
        val method=request.getMethod
        val path=request.getRequestURI
        response.getWriter.println(Map(method,path)());
    }
  
  def get(path:String)(fun: =>Any) =    Map.put(("GET",path),x=>fun.toString)
  def post(path:String)(fun: =>Any) =    Map.put(("POST",path),x=>fun.toString)  
}Code language: Scala (scala)

I believe the types could be simplified, as I’d just need a ()=>Any function not Any=>String, but I could not come up with the proper definition yet 🙂

Anyway, this can be used in a pretty simple way, and thanks to Scala’s xml literal you can have code like this:

class ScalaHello extends Step {
  get("/") {
    <form action="/post" method="POST">
      <textarea name="key"/>
      <input type="submit"/>
    </form>
  }

  post("/post") {
    "received a post"
  }

  def hello() = "hello scala world!"
  
  get("/hello) {
    hello
  }

  get("/number") { 1 }
  
}Code language: Scala (scala)

Now, obviously we have the issue of passing arguments. The by-name trick does not allow us to pass parameters and we cannot inject a new variable into the closure scope (I believe), nor evaluate the block in a different scope a-la ruby’s instance_eval.

It is possible in scala to define implicit variables that have dynamic scope, but I could not find example on how that would be possible with byname blocks.

What we can do though, is define a method params to access such variable.

Yet, we are defining all the “get”s inside the same object, so even if we can access the method to access the parameters we just shifted the problem of having that shared object, and what if different threads access it at the same time?

Yet, scala already provides what we need, namely dynamic scope, in the form of the DynamicVariable class, which can be used like

  private val dynamic = new DynamicVariable[SomeType](init)
  dynamic.withValue(aValue) {
    block where dynamic contains aValue
  }Code language: Scala (scala)

Having defined the params method properly we have this code working nicely

class ScalaHello extends Step {
  get("/") {
    <form action="/post" method="POST">
      <textarea name="key"/>
      <input type="submit"/>
    </form>
  }
  post("/post") {
    "hello, this is your input "+params("key")
  }Code language: Scala (scala)

I did not implement the pattern matching in the URL as sinatra does, but that should also be simple.

The code is pretty ugly and having started really reading about scala only few days ago it can be probably improved in many ways. The first things that come to my mind

  • I am shuffling around parameters to keep the function closure, but probably cleaning up the types should avoid this
  • I am using java.util.Map and java arrays, which means even if the code is well typed it can still go wrong
  • I have a cast that could probably be avoided
  • there is no error handling
  • the code still needs to me mapped via the evil web.xml to the / path. It should be possible to avoid this, but I have no clue on how to do it 🙂

Anyway, as the code is 30 lines of which about 9 are “real” code, I am fairly impressed 🙂

The full code

package step
import javax.servlet._;
import javax.servlet.http._;
import scala.util.DynamicVariable
import scala.collection.mutable.HashMap

abstract class Step extends HttpServlet {
  type Params = java.util.Map[String,Array[String]]
  val Map = new HashMap[(String,String),(Any=>String)]()
  val paramsMap = new DynamicVariable[Params](null)

  def params(name:String):String = paramsMap.value.get(name)(0)
  override def service(request:HttpServletRequest ,response: HttpServletResponse) {
      try {
        response.setContentType("text/html")
        paramsMap.withValue(request.getParameterMap.asInstanceOf[Params]) {
          response.getWriter.println(Map(request.getMethod,request.getRequestURI)());
        }
      }
       catch {
         case ex:NoSuchElementException => response.getWriter.println("requesting "+request.getMethod+" "+ request.getRequestURI+" but only have "
                                                   +Map)
       }
    }  
  def get(path:String)(fun: =>Any) =    Map.put(("GET",path),x=>fun.toString)
  def post(path:String)(fun: =>Any) =    Map.put(("POST",path),x=>fun.toString)
}Code language: JavaScript (javascript)

Please post suggestions if you want, but remeber that for heavy lifting it would still better to use a lift ðŸ˜‰

PS this code does not work on google app engine as of now, but I am getting this message

Error: Server Error
The server encountered an error and could not complete your request.

If the problem persists, please report your problem and mention this error message and the query that caused it.

which may mean this is a bug on their side, probably related to some Thread behaviour as I believe DynamicVariable is defined in terms of ThreadLocal. I’ll let you know if I sort it out. UPDATE actually, it seems related to scala.collections.mutable.HashMap not being loaded, even thought the jar is in lib as the others. Mh..

UPDATE 2 well, it seems I had just a misconfigured path, everything works like a charm on Google App Engine too, yay!

Both my languages suck

NOTE: this was written on a different version of this blog many years ago and may contain outdated links and references.

Over at desperately unenterprise there is a nice article about the things the author dislikes about haskell, and an invitation to share your own quirks about your favourite language. Well, I almost equally like ruby and python. so here is a double rant.

Why ruby sucks

First, i can reopen classes but so can the others. It’s all nice, and the usual idea that this will make development impossible is dumb, but it’s true that it can get annoying. Matz once used to say this could be fixed by namespaces in ruby2 but I haven’t heard anything about it in years and I’m not sure it will happen. Meanwhile, I still got the occasional oh, damn why isn’t this working? moments, and it sucks. People may remember that nice chainsaw infanticide logger maneuver.

Modules as namespaces also suck. They rely on a wrong assumption, that everyone else will use modules as namespaces, which of course is not always true, so if you happen to use something poorly designed you may end hupo with their User class conflicting with your. Python got it right, namespaces are implicitly defined by files/directories and you have to explicitly import stuff in the current one.

But not just that, you can also import a single name from a python module. Try that with ruby’s require if you can. You’re either forced to split every single method in it’s own file, like facets does or you impose on the end user namespace pollution.

And it’s not even that namespaces are cheap to define. The usual way is

  • create a foo.rbmain file
  • create a foo/ directory
  • create many foo/xyz.rb files
  • in every single file define the module And it gets much worse with nested namespaces. And yeah I know I can do class Module::ClassName except that then I can’t load that single file separately.

(I also dislike the thing about constants being lexically scoped to be fair, but probably it’s just my disliking of class variables and the fact that I’d like to use constants for that)

What is nice about ruby modules is their usage as mixins, and even though I’d like them a bit different they are cool. But so under used.

The core classes in ruby are so fat that they are almost impossible to subclass. Have you ever tried subclassing String so that you can, for instance have a TextileString where you can do substitution without considering the markup? You’ll never get it right enough, just String#[] has 7 different behaviors depending on th arguments, and String#[]= has 8 of them.

The Comparable mixin rocks, everybody loves Enumerable but go figure how to make an Hash-like or File-like object without subclassing

Why python sucks

The problems with python are, well, what makes it pythonesque. Yeah, I understand why you need to explicitly define self in a function, I just believe it’s dumb and too verbose. But why can’t I have a bit of syntax sugar ? Make .foo be a shortcut for self.foo ? At least that would also enforce the convention!

Python has such a big set of conventions that are not supported by the language. Do not use global variables, but look, there is a useful global keyword, even though you have nested namespaces.

On the other hand, it enforces some ugly conventions, suchs as the avoidance of assignments in conditionals by splitting statements and expressions. That distinction is so annoying that as time passes more and more things become both. Conditions are statements, so here is also the expression version. Functions are statements so here is the crippled lambda expression. Assignment is a statement so here are a handful of setters that avoid using the evil = symbol.

update: sorry, this got published by error, it was still largely incomplete, so I’ll juust add some more comments

A lot of things that I dislike are in the “kernel library” design, and not syntax. len, map and so do not make the language less OO, I know, but they are ugly and polluting the top namespace, why can’t they be made part of proper abstractions? And I hate not being able to use mutable values as key for dictionaries, even java manages to do that, it should not be that hard.

Something that I miss syntax wise instead is the lack of a case statemente, especially of a destructuring/pattern matching one. Python people will usually tell you that you just use a dictionary. Yeah, but why I have to? I can remove else-if from the language too, but it does not mean it makes sense. case/switch statements are actually a place where the language can provide very useful behaviours once someone get past the C/Java versions of it. Take a look at ruby, SML or Scala.

And, pet peeve: why I have to type “:” in definitions? the parser does not need that, and come on, it adds nothing readibility-wise.

ECMAScript 4, the fourth system syndrome

NOTE: this was written in 2007 in an old version of this blog, some links or comments are lost to time.

The second system syndrome is that evil effect that often happens when redesign a small, working system so that it becomes a huge leviathan built by piling new features over new features.

Today I was reading the overview of ECMASCript 4 from ecmascript.org, and I got this very bad feeling that maybe the fourth version of ES, is suffering of an extremely strong case of this problem.

I believe that part of the success of ES in its current incarnation is that it is quite simple. Yeah, it has its shortcomings and oddities, but in general it is simple enough that everyone can learn it in no time.

ES4, on the other hand, seems to have included almost everything that came out from programming languages in the last 50 years, save Prolog and APL.

For example, it has prototypes, interfaces, classes, metaclasses, higher order types and generic functions, which for what I can tell entails all the work ever done on object-orientation (maybe predicate classes are missing, but they may be there too).

The argument list of a function may have named arguments, rest/variadic arguments, optional arguments, optional types including: non-nullable arguments, like arguments (this seems the same as in Eiffel, used to link the type of an object to another), wrap arguments that perform automatic conversion of an object of type X into another of type Y,

Whereas many languages I know provide a concept of Module or Package to handle namespacing issues, ES4 provides packages and namespaces and program units to support name hiding and modularity. Not only this, but you can introduce variables with var to have it in the object scope or with let to have it in a block, so even more namespace separation 🙂

Finally, ES4 incorporates some cool features such as list comprehensions, a kind of destructuring assignment/pattern matching, python-like slicing, triple-quotes and generators (though AFAICT this are only one way generators like in older python releases, where you can yield but you can’t receive values Thanks to Neil Mix for pointing out that they are complete coroutines, and support bidirectional value passing).

And operator overloading, program pragmas, required tail call elimination, a host of interesting keywords suchy as intrinsic to reify operators, dynamic to have classes that allow adding of new fields, static to have class-level functions, final to avoid inheritance and override to use it, internalpublic and so on. And of course, 6 different values of falsity, with NaN""0falsenull and undefined, but not {} and [].

ECMAScript 4 seems cool, I want to use a lot of those things (yay for multi method dispatch!), but maybe it is changing the language a little bit too much .

How many of the current user of ES in one of its incarnations have ever heard of product types? Will they grasp at once the logic behind the subtyping relation between functions, with all that covariant and contravariant stuff? Will they need it at all?

I hope the internet pipes will resist to the number of “metaclass vs meta namespace vs prototype vs superclass” questions on usenet.