BeatDown Metronome

I'm the developer of BeatDown Metronome, an iOS metronome application. I wrote it to teach myself objective-C, and found it useful enough to release in the App Store. It's free, and I enjoy practicing with it.

Web Development

I have a bunch of projects in progress over at my Github page. The site you are reading now is part of my attempt to get better at jQuery, which is why it uses jQuery/javascript for lots of unnecessary things.

Other Interests

Through studying javascript, I've developed a strong interest in functional programming, so far in scheme and racket, although I'm looking forward to learning more about Haskell. I also enjoy coding in Python, improving my C skills, and look forward to continue learning new languages and ways of thinking.

Recent posts on software:

Strange Loop 2014

by paul, on 09.20.2014

Sarah and I just got home from Strange Loop, a conference about emerging languages, data stores, and distributed systems. It's the first conference I've ever personally gone to just out of pure interest. It's held in St. Louis, and I must say it was a great experience. Really well run, not too crowded (even though it sold out). They place a big emphasis on diversity which felt really great for Sarah, and for me.

On friday, we went to the pre-conference party at City Museum, which, no lie and I'm not even exaggerating, is one of the coolest places I've ever been to. We climbed through wire tunnels above planes perched on columns, we rode a ferris wheel on the roof, and slid down a two story slide. The event was great with complimentary food and beer.

The main conference days were thursday and friday. Instead of giving a play by play, I'll highlight my favorite talks. A major challenge at Strange Loop is picking the talks you want to see out of the many awesome concurrent talks happening. This really speaks to the quality of the talks, most of which are now available on youtube. Another fun thing about the conference is that I felt totally comfortable going up to anyone, introducing myself, and then starting a conversation, something I usually struggle with. I think this is because it's a small conference and folks have a common passion for technology.

  • Joe Armstrong gave the first Keynote. One of the creators of Erlang, Joe has been coding since the 1960s, and talked about complexity, the limits of computation, and the wonderful mess we're in.
  • Pete Hunt talked about the principles of REST, from the original Roy Fielding paper, and how they apply to React.js.
  • Great talk on Apache Samza, which is a storm-like system. Martin, the presenter, tied everything back to relational database concepts, which really helped me grok everything. I'm going to take a much closer look at Samza.
  • Consistency without Consensus, by Peter Bourgon. Peter talked about how Soundcloud moved from a "inbox" style architecture (fan out on write), to a "outbox" (fan in on read), and how they used CRDTs to do it. This was especially impactful for me, as I went to a "Papers We Love" meetup on CRDTs, and didn't grok how they could be used in practice. Peter went over exactly how they were implemented using Redis.
  • Nada Amin's keynote on day 2 was about the towers of interpreters, and she did a lot of live coding using black, running on top of scheme.
  • Kyle Kingsbury gave a wonderful talk on Jepsen and Knossos, and talked about his results for RabbitMQ, Consul, and Elasticsearch. Even though Kyle's work, which is top notch, is about highlighting problems in systems, he presented with a tremendous amount of empathy, which I thought was really great: Distributed systems are very difficult to get correct, and there's not a coder alive that hasn't taken a shortcut or made a design decision that had unintended costs or consequences. This talk inspired me to go through cmeik's readings in distributed systems more closely.
  • Rich Hickey's talk on transducers was fantastic. I had read about transducers and quite honestly found the motivation behind them confusing, this talk made everything perfectly clear.
  • The guys from Kitchen Table Coders have gotten a clojure REPL running against Unity 3D, using the clojure CLR. Wow. This was awesome awesome awesome.
  • In one of the most inspiring talks I saw, Julia Evans talked about how you, too, can be a Linux Kernel Hacker. strace, perf, and the Eudalypta challenge. She also wrote the funniest kernel module example ever.
  • Last but not least was Sam Aaron and Carin Meier giving "Our Shared Joy Of Programming". This featured overtone, a music synthesis library I've spoken on before that Sam created, as well as dancing robots. An inspiring, beautiful end to a great conference.

I went to Strange Loop looking for inspiration. I definitely found it. I'm excited to start on some linux kernel hacking, messing around with unity, getting back into the clojure world, and in general having fun with programming.

Categories: Technology, Clojure

Book Review: Secrets of the JavaScript Ninja

by paul, on 05.11.2014

I've done a lot of work in JavaScript, but never really studied the language in a formal way, which means there are often significant gaps in my understanding. To gain a deeper understanding of the power of the language, I decided to read and work through a few books. First on my list was "Secrets of the JavaScript Ninja", by John Resig and Bear Bibeault. The first five chapters of the book are essential reading if you are new to JavaScript, or just want to improve your understanding.

I worked through the book by going through the first five chapters in order, taking notes, and typing in most of the code examples given myself. I learned this habit from the Big Nerd Ranch iOS guide, and I've found that actually typing in the code myself helps me internalize it. I would recommend going through the first chapters on functions, closures and objects in great detail, and then read the rest more casually.

The first chapter gives a high level overview of JavaScript foundational concepts: Functions, Objects, and Closures. The authors emphasize the importance of understanding Closures in order to truly understand the language.

The second chapter introduces considerations around testing and debugging your code, which I found unique and refreshing. By the end of these chapters, you've read an overview of the language, and you've got a basic HTML/JS/CSS setup that lets you try out your code and assert things about the results which you will use throughout the rest of the book.

The third chapter is all about functions. Different ways to declare them, different ways to call them, properties of functions, and how they are treated. The most important section in this chapter for me was understanding the context around functions, namely, "this". Like most developers, I've inadverdently bound many a function to an un-intended context. The most important sentence in the chapter for me is "The Context of a function is determined by how the function is invoked".

The fourth chapter talks Closures. Not all programming languages support Closures, so this chapter is especially important if you've come from a Java background. The nice thing about this chapter are the real world examples of simplifying and improving code using closures. The most important sentence in the chapter for me is "In most languages, scope is derived from the block of code they are in. In JavaScript, scope is derived from the closure that they are in."

The fifth chapter, and the last one I'll talk about in detail, is how Objects work in JavaScript. This is another important chapter to read if you're not familiar with how prototypal inheritance works; as it's a different approach than using classes in Java or C#. This chapter also contains common mistakes, I found this particularly useful as I have made most of these mistakes myself.

The remainder of the book is a collection of other topics: Regular Expressions, Manipulating the DOM, using Timers, and similar. These chapters are fairly self contained, and I found I could skip around a bit and follow my interests.

If you are looking for a refresher in JavaScript, or enjoyed "JavaScript: The Good Parts" and are now looking for a book that has more real world examples, I'd highly recommend "Secrets Of the JavaScript Ninja". It's a great exploration of the JavaScript language.

Categories: Technology

Use the Source: an annotated clojure function

by paul, on 03.04.2014

I was doing a bit of R&D this evening, and I found myself looking at the source of the always useful clojure function "select-keys". One of the best things about clojure is that much of the source is available to you right from the REPL. I've personally learned quite a lot from reading and understanding the clojure source.

As an exercise, I've annotated the "select-keys" function with comments that explain what's going on. It's mostly easy to understand, and this particular function isn't exactly magical, but it's a solid example and I definitely learned a few things in the process of annotating.

defn select-keys
  "Returns a map containing only those entries in map whose key is in keys"
  {:added "1.0"

   ; I looked this up, and it's a no-op
   ; http://stackoverflow.com/questions/7552632/what-does-static-do-in-clojure
   :static true}
  [map keyseq]

    ; set up a point for recursion, initializing the return map to empty,
    ; initialize keys to a seq, note that (-> [1 2] seq seq) is a no-op,
    ; so this is just a good general practice to ensure we've got a seq.
    (loop [ret {} keys (seq keyseq)]

      ; this is more idiomatic than what I would have done, if (not (empty? keys))
      (if keys

        ;I'm not sure what the clojure.lang.RT function call does here.
        ;in the cases I tested at the REPL, it's equivalent to "(find map (first keys))"
        (let [entry (. clojure.lang.RT (find map (first keys)))]

          ; define a recursion point
          (recur
           (if entry

             ; if the entry exists, (conj {} (find {:a 1} :a)) => {:a 1}
             (conj ret entry)
             ; otherwise, recur without modifying the map to return.
             ret)
           ; next returns either nil or a non-empty sequence.
           ; this is necessary when tied with the "(if entry..." syntax,
           ; as only nil is false.
           (next keys)))

        ; make sure to include the metadata passed with the original map
        ; in our return value. another good thing to keep in mind.
        (with-meta ret (meta map)))))

Categories: Technology, Clojure

Seven Databases: Redis

by paul, on 05.09.2013

I've been working my way through the book "Seven Databases in Seven Weeks", in an attempt to get a better understanding of the NoSQL landscape and to start to see use cases for the various stores.

This week I totally cheated and skipped ahead to Redis, because it sounded awesome. Day one lived up to my expectations; it was easy to build from source and install, and the REPL (called redis-cli) makes it easy to interact with, much like a nice SQL interpreter. I wouldn't say the Redis command syntax is particularly intuitive, but I can see myself learning it quickly, and more importantly, the set of commands is quite powerful and also seems simple enough to not get too bogged down in.

The first day runs over storing, mutating and fetching basic data structures: Key/Value pairs, Lists, Hashes, and Sets. Much like SQL databases, Redis supports transactions, and also has some blocking functions on lists that make it useful for coordinating messages. When I read that Redis was sometimes used as a PubSub Queue I was at first confused, but after running through an example on blocking lists, it makes sense. Another nice feature is builtin cache expiries, which obviously make it useful for caching purposes.

I'm looking forward to learning more about Redis, and I wouldn't hesitate to look at it if I were in the market for an application level cache; although I don't take adding caching to an application lightly, as it can introduce a significant layer of complexity.

Categories: Technology

Seven Databases: Riak

by paul, on 05.01.2013

Week 2: Riak

Riak is a key value store that works kind of like S3. It's implemented in Erlang. I had a few problems getting things up and running because the book is a bit out of date, but once I got through the usual ulimit issues, I learned a lot. I then watched a video on a talk that some guys from Yammer gave, and I've included my notes below (taken in real time, so a bit disjointed). Riak is really interesting, particularly some of the more advanced features which are really a superset of what something like S3 gives you. One of my big takeaways here is to read the amazon DynamoDB whitepaper.

Notes from Riak @ Yammer talk

Streamie: The use case is notifications to users: showing things like "likes", "new followers", "replies in thread", etc. This data gets rolled up into a Rollup, a grouping of category and property. For example, a rollup could be "Category" and "Message_id" to see all replies to a thread. Rollups are then processed: notifications are retrieved into "seen" and "unseen", rollup logic is applied: sort, trim, and merge the notifications; they drop results after a certain point in time, so you're not showing a user 2000000 new notifications. This means that they can bound the size and scale accordingly. Based on this data model, they chose Riak. Note that they use twitter's snowflake for ID generation, which is distributed and scalable. So, the data model stored is id, timestamp, category, and properties, where properties is a hash of things like liker_id, followed_by, etc.

    Reasons for choosing Riak
  • Fault-tolerant
  • Low-latency
  • High-throughput
  • great packaging: don't underestimate advantages of getting stuff up and running quickly.
  • great administrative tooling
  • HTTP + JSON fits well into the Yammer architecture
  • Support + Community is both good

So, how does this stuff map to the bucket+key architecture that Riak is based on?Their buckets are: cursors, keyed by user_id + cursor_name. Different cursors might be "web cursors" vs "email cursors". Also stream buckets, one bucket per stream. "email" bucket, keyed by "user_id". One helpful point is to namespace keys, use the hostname as part of the key. They use protocol buffers to store the data in Riak; but a downside is that protocol buffers isn't Riak MapReduce friendly. They don't use Riak Links and a lot of the advanced features of Riak.

    Disadvantages
  • Client API doesn't handle failures particularly well.
  • Writes involve passing around vector clocks so that writes are ordered.
  • Complexity is pushed to the developer, largely because of shortcomings in the client APIs, as the backend is remarkably fault tolerant.

Recommendation: Make consistency a first-class concern. Anticipate failure and use retries correctly (didn't take good notes). They use HAProxy in the middle, 2 app servers, requests are 99.9% <= 35ms, including GC pauses(!). 3 weeks from start of project to staging, 6 weeks to production. They very rarely need to worry about Riak, and it works quite well for them as a result.

    Big Takeaways
  • Data Model has to fit.
  • Deal with failure conditions and consistency issues.

Categories: Technology

7 databases in 7 weeks: chapter one

by paul, on 04.28.2013

I'm currently working my way through seven databases in seven weeks, in an effort to increase my awareness of the different NoSQL options that are out there. I have a lot of experience with traditional relational databases, and am rapidly gaining a lot of experience with elastic search, but haven't explored other technologies that are available to the degree that I should.

The first chapter of the book highlights PostGres, and promises to show you something new even if you are a grizzled old SQL veteran like me. It definitely delivers on this promise; I learned about window functions, which function like group by; except they partition data instead of aggregating it. The material on fuzzy text matching packages was new for me, particularly levenshtein distances and trigram comparisons to determine similarity between two strings. These seem like useful techniques to know about in general, and I plan to explore both further. Also, creating rules in order to update views was a new concept for me, I hadn't seen this before.

This chapter is very well done, and covers a lot of ground; I would wholeheartedly recommend it to anyone wanting a great overview of what an RDBMS can do. Indices, Views, Stored Procedures, Triggers, Query Plans, Optimization, Packages are all touched on. If the rest of the book is this well done, I'm in for a treat. Next up is Riak, an Erlang based datastore that's based off the DynamoDB architecture.

Categories: Technology

understanding lazy-seq

by paul, on 02.08.2013

I've been working on improving my clojure skills by occasionally doing problems on 4clojure, a great resource. I've been trying to make my code more idiomatic by comparing my solutions against others' answers, or sometimes clojure core, to see where I could have written a nicer solution. This has been really helpful for me.

One of the 4clojure problems is to re-implement iterate. After I did this, I viewed the clojure source code to see how it's done:

=> (source iterate)
(defn iterate
  "Returns a lazy sequence of x, (f x), (f (f x)) etc. f must be free of side-effects"
  {:added "1.0"
   :static true}
  [f x] (cons x (lazy-seq (iterate f (f x)))))

To someone who's recently learned clojure, this might look funny. Aren't we supposed to be using loop/recur to avoid the dreaded java.lang.StackOverflowError? why is explicit recursion ok here?

The answer is lazy-seq macro, which, when used correctly, allows us to avoid realizing the the entire sequence at once. The lazy-seq macro essentially allows the runtime to step through our sequence; keeping only the portion in memory that is being asked for. It's our responsibility to code our function in such a way that we don't ask for the whole thing at once. Remember, the java.lang.StackOverflowError occurs because each function call creates a stack frame, and those frames are stacked and can't be garbage collected until the recursion is unwound. Wrapping the recursive call with lazy-seq allows us to code recursively, without fully realizing the entire sequence all at once at runtime. The key to lazy-seq is understanding that only part of your sequence should be calculated in memory at once, and writing your code in such a way that we don't inadvertently hold on to a reference for to the full sequence, by storing a reference to the head of the sequence, for example.

There are some caveats to lazy-seq, which are outlined in any good clojure text. Basically, it boils down to the earlier recipe:

  1. Don't keep a reference to the head of your sequence in a top level variable. The "Joy of Clojure" and "Programming Clojure" both have great examples of this, I don't want to plagiarize. They're both well worth the money.
  2. The lazy-seq macro should go at the top level of the function.
  3. Don't use functions that force a full realization. Ensure that your function body doesn't force evaluation; keep everything lazy. The example from the clojure docs is to use "rest" instead of "next". When in doubt, check the docs.

I hope this post is helpful to others; I write these mostly as a way of solidifying my own understanding of the language, not because I think we need yet another blog post explaining lazy-seq. However, I do hope that this might be useful to other folks getting familiar with the language.

Categories: Technology, Clojure

liberator: a simple repo to get you started

by paul, on 12.29.2012

I've just put liberator-boostrap up on github, my attempt to make a simple project that does some boilerplate setup for the liberator library, which allows us to build RESTful web services using clojure.

This project exists because I started taking a look at liberator, the artist formerly known as compojure-rest. The examples in the project are a good starting point, but it took me a little while to figure out how to get a skeleton project configured, so I've created a simple bootstrap project which I think is close to the simplest possible example of how to set up a clojure project configured with liberator, compojure, and ring.

I hope that someone else can benefit from this as well. I'd love to contribute this back to main project, as a submodule in "examples", or something.

Categories: Technology, Clojure

Kaggle's titanic problem in clojure

by paul, on 11.11.2012

I've been getting more into machine learning recently, and after the Hadoop World conference in October, I've registered on kaggle.com and started their titanic competition. Their initial tutorial uses excel, which I don't own, so I decided to code their Getting started with Excel guide in clojure.

My code, which could surely use some improvement, is on github here. It's basic stuff: mostly summing and aggregation functionality. This is really a very basic example of data manipulation, and doesn't get into machine learning at all. I really look forward to getting my hands dirty updating this code in accordance with the python tutorial they provide, as well.

Categories: Technology, Clojure

Clojure: using the source

by paul, on 11.04.2012

One of the coolest things about Clojure is the readability of the core library. Not only is the source to almost any function available at the REPL, but usually, the code is easy to read and understand. In music, one of the best ways to learn is by recording yourself playing a tune, listening to a master musician play the same tune, and note the differences; there is often much you can learn.

Tonight I was prototyping some data wrangling on CSV files for work, and I wanted to merge the header of a CSV file with each row to form a map. Of course, zipmap in clojure does just this. Since it's sunday night and I'm messing around anyways, why not write my own, probably naive version, and compare it with the canonical version, and see what I can learn?

Here's the implementation I came up with

(defn group-row [header data & {:keys [result] :or {result {}}}]
    (cond
    (empty? data) result
    :else (assoc 
        (group-row (rest header) (rest data) :result result) 
        (first header) (first data))))

basically, I assoc the current first elements of each list with a recursion on the rest of the map. I use an optional argument to collect the results that defaults to an empty map. Not terrible, but let's see how it's done in clojure core:

(source zipmap)
(defn zipmap
  "Returns a map with the keys mapped to the corresponding vals."
  {:added "1.0"
   :static true}
  [keys vals]
    (loop [map {}
           ks (seq keys)
           vs (seq vals)]
      (if (and ks vs)
        (recur (assoc map (first ks) (first vs))
               (next ks)
               (next vs))
        map)))

Ok, I actually got pretty close! The author of this function uses a loop/recur to add the additional "result" argument, I used straight recursion even though I should, and do, know better. I didn't know the difference between "rest" and "next", but stackoverflow has a kinda clear explanation. Looking at the at the function definition, I am not sure why the explicit "seq" statements are there, I will need to think about this a bit. in any case, I wanted to clean up my initial function, to re-factor it to use a recur, I needed to move some nested calls around, but I think this looks better:

(defn group-row [header data & {:keys [result] :or {result {}}}]
    (let [hs (seq header) vs (seq data)]
    (cond
    (empty? vs) result
    :else
        (group-row
            (rest hs) (rest vs)
            :result (assoc result (first hs) (first vs) )
            ))))

It's always nice to write a little code and learn a little something in the process.

Categories: Technology, Clojure

overtone: video examples

by paul, on 09.03.2012

I'm proud to have made a very small contribution to the clojure/overtone community today, I added some sample code for Sam Aaron's Quick Intro to Live Programming with Overtone on Vimeo. You can find the source code in overtone's examples. I hope to find ways to contribute more to open source projects like overtone, even something really small like this is rewarding.

Categories: Technology, Clojure

HTML5 Vs Native? You don't have to choose

by paul, on 08.28.2012

Facebook has just launched a fantastic new iOS app, a huge upgrade in performance and user experience from their previous application. This has re-sparked the debate over native vs portable mobile apps. I think that the reality is more nuanced, and talking about html5 vs. a native app presents a false dichotomy.

Native apps are applications coded using the language, developer tools, and APIs provided by the mobile device's SDK, or software development toolkit. If you want to have a native app for both android and iOS, you would by definition have to write two different pieces of software: Android's SDK is for the Java programming language, whereas iOS is for objective-C. Portable apps, which use html5, css, and javascript, promise a "write once, run anywhere" developer experience. You write your app using standard web technologies, package it up using a framework like senchatouch, and bingo bango bongo, you're in business in the android store as well as the apple app store. A lot has been written on the relative advantages and disadvantages of these approaches, and I will not be re-hashing those arguments here.

The problem with the "native vs portable" debate, is that it implies if you're writing a native app, you won't be using html5. In practice, this doesn't have to be true, and frequently isn't. When I was at GS, my team built GS Research, to showcase equity research to our clients. Equity Research functions quite a lot like a newsroom, and our app is built around a business that publishes timely research, sometimes based around market events, sometimes more general economic forecasting, etc. We wanted our clients to be able to download our app from the app store, and have a first class experience browsing through the research we publish.

When we architected the GS Research app, to allow our clients to read our investment research products, we made a couple of important decisions that allowed us quite a lot of flexibility:
Number one, after some debate, we decided we wanted to do a native app. The usability of the portable frameworks we tried like SenchaTouch weren't quite up to speed. Since our app is iPad only, and would have a lot of visibility, we opted to use objective-C. I would note that none of us, including myself, had ever written a line of objective-C before we started on the project. We used books, I even wrote a small metronome app to prepare. Our attitude was that we are good technologists, and can use whatever technology is required to get the job done. Programming languages are not like spoken languages; they usually have a lot of common ground, and are relatively quick to learn, and as programmers, we love to learn new languages.
Number two, we didn't want to rebuild the whole enchilada if we wanted an android client. Most publishing and news apps have a non-trivial server side component, and for our app, we had roughly three times as many developers working on the server side as we did on the client side. I don't want to trivialize client development; but we had to implement services for search, app activation, personalized content, email alerts, etc.
We needed a clean, platform agnostic way to represent our investment research. Guess what we chose? HTML5 for documents, JSON for reference data, and javascript (jQuery) and CSS for styling. The app uses a UIWebView to display research documents, which were semantically marked up and delivered to our app via HTTP.

In general, the decision to use html5 and JSON as a contract between our web services and client application served us very well, and I would readily architect another web application this way. In fact, my colleague Bryan Irace, who was instrumental in the planning, crafting and implementation of virtually everything I've just described has, as far as I know, done just that at his current gig. My current job is pretty far removed from the mobile space, but my experience and feeling is that architecting your app using a native wrapper, but html5 for transient content, is a very sound, flexible approach.

Categories: Technology

clojure basics: argument destructuring

by paul, on 08.05.2012

Argument de-structuring is a really nice feature of clojure that, as someone new to the language, I often find myself forgetting to take advantage of. I wanted to post a simple example to show how it can simplify code.

Part of my new job involves working with statistics, so I've been going through a statistics refresher course, and writing some very basic clojure code to do things like calculate expected value of a probability distribution, etc. A probability distribution can be modelled as a map, with the key being X, and value being the probability, call it P(x), of X occuring. once you have this map, the Expected Value, or EV, of that distribution, is just the sum of the products of X and P(x), this is very basic stuff you probably learned in high school.

Anyways, it's pretty straightforward to write this in clojure, my initial, naive implementation looked like this:

(defn multiply [[p px]] (* p px))

(defn ev [distribution]
    (sum (map multiply distribution)))
This is fine, but having a separate function for multiply kinda sucks, there has to be a better way. The only bit of "multiply" that provides any value is the argument de-structuring, the stuff inside the [[ ]]. We'd really prefer to use an anonymous function, but with an anonymous function, we're going to end up with variables like %1 and %2. How do we de-structure these? Use let! The below code is nicer to my eye:
(defn ev [distribution]
    (sum (map #(let [[p px] %1] (* p px)) distribution)))

Categories: Technology, Clojure

clojure: type hinting and warn on reflection

by paul, on 05.28.2012

I've been working a bit with the javax.sound.midi libraries, using clojure. I just ran across a problem I hadn't encountered before, so I wanted to share the solution. If you are a clojure veteran, this post won't be interesting, but if you're new to the language like me (or googling the error I got), it might help.

Quick MIDI background. A synthesizer is something you can use to generate sound. A Synthesizer has one or more channels, which you use to generate the sound. For a real world analogy, a Synthesizer is like a piano, and a channel is like your left hand.

I was executing some fairly basic clojure code:

(import 'javax.sound.midi.Synthesizer 'javax.sound.midi.MidiSystem)
(def synth (doto (MidiSystem/getSynthesizer) .open))
(aget (.getChannels synth) 0)
IllegalArgumentException Can't call public method of non-public class: public javax.sound.midi.MidiChannel[] com.sun.media.sound.AbstractPlayer.getChannels()  clojure.lang.Reflector.invokeMatchingMethod (Reflector.java:87)

Hey, what's going on here? According to the javadoc, Synthesizer.getChannels() is fair game!

The problem is caused by the fact that clojure's java interoperability works by reflection, and in this case, it's getting confused as to what type the class is. Since this is a common problem, clojure provides a mechanism, type hinting, to help deal. There is also a flag you can set to be notified of potential errors of this type, warn-on-reflection. So, all I had to do to fix this, was let clojure know what type it should be using to call .getChannels(), using a type hint:

(aget (.getChannels ^Synthesizer synth) 0)
#

voila! also, setting *warn-on-reflection* allows us to see what's happening here:

(set! *warn-on-reflection* true)
true
user=> (aget (.getChannels synth) 0)             
Reflection warning, NO_SOURCE_PATH:7 - reference to field getChannels can't be resolved.

Categories: Technology, Clojure

nyc clojure meetup notes

by paul, on 05.17.2012

On Wednesday, I went to my first NYC Clojure meetup, which took place at Google's 8th ave offices. I got there way too early, and ended up standing around in the lobby with Rich Hickey waiting to get in the building. There were three speakers presenting at the meetup; Rich was there to talk about the new clojure reducers library, David Nolen gave a talk on ClojureScript performance, and and Kovas Boguta talked about the future of REPLs on the web.

David Nolen, one of the key developers on clojure script, started off by showing us some JVM vs V8 spectral norm benchmarks, showing that clojure script is quite fast. He talked a bit about protocols in clojure script, and how they use the "satisfies?" method to determine if an object conforms to a protocol. The clojure script developers have implemented satisfies using bitmasking, because bitwise operations in javascript are extremely fast. I hadn't used clojure script at all before this talk, but am definitely inspired to check it out.

Next up was Kovas Boguta, talking about the future of the REPL. Long story short, Kovas has built an extremely cool web based REPL that works with both clojure and clojure script. He demonstrated the mathematica "REPL", an extremely sophisticated environment that's built from a very simple concept: the symbolic representation of an expression doesn't need to be the same as the display, and in fact separating the two can lead to a considerably more natural programming environment. The closest analogy in the traditional coding world I can think to this concept is interface builders in xcode and visual studio, that show you a graphical representation of a UI element, but it's backed by C#/objective-C code all the same. Kovas mentioned how crucial tag literals were to this kind of implementation, something I haven't really grokked yet and need to explore further.

The final talk of the evening was the author of clojure, Rich Hickey, talking about the new reducers library. Rich noted that it used to be that the easiest way to make your code faster was to wait 18 months and buy yourself some new hardware. This is an old joke, but his point was this is no longer true, we are adding cores now, not clock cycles. The inherent sequential nature of programs, even in functional programming, does us no favors. The idea behind reducers is to decouple a lot of the "how" aspects of higher order functional manipulation: map, filter, fold, functions that are fundamentally using reductions in some way. Rich has blogged about reducers extensively, and does a far better job than me of explaining, so I won't rehash what he talked about. He did come up with a very cool analogy to illustrate how order is often unneeded, using a bag of apples. A lot of times, you just want to filter out all the rotten apples (filter), or take all the stickers off the apples (map), you don't care a thing about order in this case. Yet, map and filter give you order, like it or not. If you're baking a pie, you might not even care about the full bag of apples, just the ones you need, but you're going to get the full bag anyways. Anyway, I am going to do a poor job of rehashing Rich's talk if I go any further, I hope it's posted online so we can re-watch and try to understand a bit more. The endgame of his lecture was a paralellized fold, that was very cool to see.

I'll definitely be going to these meetups in the future, the speakers were all very impressive, and it's really interesting to talk to folks about what they are working on.

Categories: Technology, Clojure

self-help: the clojure edition

by paul, on 04.29.2012

I've been teaching myself clojure recently, and was doing a few problems on the 4clojure site, just to get more familiar solving small problems using clojure. one of the problems is to implement a function that returns the last item in a sequence. this is a pretty basic thing to write, here's my implementation:

(fn [l]
    (loop [l l previous nil]
    (if (empty? l)
        previous
        (recur (rest l) (first l)))))

One of the coolest things about clojure is that there is a lot of documentation available right at your fingertips, at a clojure REPL. There are 3 functions I've found particularly useful so far:

  • find-doc: lets you search doc strings for possible matches. (find-doc last), for example
  • doc: prints a documentation string. (doc last) prints a description of the function
  • source: allows you to view the source of a function. this is one of the coolest features of the language, all the source code is right there, unlike java. (source last) will show you the source code of the "last" function.

In jazz, you learn from the masters by listening to what they played, learning how to play it on your instrument, and using that knowledge to craft your style. Viewing the source for the language itself has got to be a great way to learn something about writing idiomatic clojure. So, with that in mind, I thought it would be interesting to compare my implementation to the "last" implementation in clojure

(source last)
(def 
 ^{:arglists '([coll])
   :doc "Return the last item in coll, in linear time"
   :added "1.0"
   :static true}
 last (fn ^:static last [s]
        (if (next s)
          (recur (next s))
          (first s))))

Interesting, there are several things I can learn from this:

  • My solution stored the previous element, because I didn't realize clojure had "next" available. In the future, I'll check the super-awesome clojure cheat sheet.
  • Seeing the "recur" without a corresponding "loop" threw me, so I read up on recur. Turns out that recur will recur to both loop as well as function, depending on arity (the number of arguments).
  • I wasn't sure what the ^:static metadata item does, stackoverflow to the rescue once again.

All pretty small stuff, but if I keep doing this, I should get much better at writing idiomatic clojure, and more importantly, using the full power of the language.

Categories: Technology, Clojure

Understanding Clojure, continued

by paul, on 04.08.2012

Had last friday off of work, and I've still been banging away at learning clojure, working on some music software that I will post about soon. Until then, I'm including some more notes.

Sequences are extremely powerful in clojure, and the seq API is very deep and has a lot of batteries included. The first thing I realized whilst coding is that it's pretty important to understand when you are dealing with a seq/list/vector, as things like the conj function will behave very differently depending on your collection type. conj will append to the tail of a vector, but to the head of a sequence (this makes sense because seqs are lazy and can easily be infinite). Laziness is an important concept, and it is very similar to generator functions in python, although it feels more baked in to the language than generators do in python. Lazy sequences have a lot of musical possibilities, as it's easy to generate an infinite series of overtones and then take the ones you need.

One thing I got confused about at first is the difference between def and defn. Luckily, stackoverflow set me straight. defn is just a macro that maps to (def name (fn ...)).

Recursion and other Functional stuff

When it comes to recursion, Clojure makes some JVM specific compromises, as it's relatively easy to blow out the stack if you're not careful. using loop+recur helps greatly, as do the aforementioned lazy sequences. Until learning clojure, I did not realize that the JVM does not guarantee tail call optimization.

In terms of other functional programming constructs, Clojure has partial application via the "partial" function, and ability to compose functions via "compose". I haven't messed around with these things too much, and don't yet fully understand the difference between currying and partial.

Categories: Technology, Clojure

Learning Clojure

by paul, on 04.04.2012

Since checking it out via Bruce Tate's book "Seven Languages in Seven Weeks", I've gotten a taste of Clojure, and want to invest more time in it. It's the nicest Lisp I've ever used, has a cool looking music project library called Overtone, and I enjoy functional programming a lot. So, I picked up Stu Halloway's book "Programming Clojure". I'm going to attempt to document what I learn in a series of posts, in the hope that I will remember it a bit better as a result of posting about it.

Chapter One

Chapter one covers the basics of using Clojure. The Read Eval Print Loop (REPL) has a few handy tricks, like references to the last 3 previous evaluated expressions with *1, *2, *3. This is handy, because unlike every unix shell since the 70s, the REPL doesn't have up-arrow history. Some of the handiest stuff I've learned in chapter one are around documentation:

  • find-doc("thing to find") - finds relevant documentation
  • (clojure.repl.source "function name") - shows the source of a function. super cool!
  • (doc thing) - shows docstring. you can document your own docstring by including a string between the function name and arguments.

Note that I've put my implementation of Djikstra's "The Sleeping Barber" problem as a Gist on Github. I'd like to get it reviewed by an experienced clojure programmer at some point, as I'm pretty sure the global defs aren't the greatest idea ever. Most of the program was straightforward to write, as it's a basic producer/consumer problem, although I did get stuck on the consumer side, until I discovered add-watch. Live and learn, I guess.

Chapter Two

I've been on a bit of a clojure bender, so I covered chapter 2 in its entirety today. I'll summarize highlights for me in my notes below.
In Clojure, as in a shell like Bash, you can sometimes get yourself into a situation where you are passing a vector to something that really wants a variable length list of arguments, like

(str (interleave "asdf" "qwer"))
In bash, you would use xargs to get around this, and in Clojure, you use apply.

User defrecord to define classes, Clojure makes creating classes easy. Once instantiated, you can treat them a little like maps. Clojure defines reader macros like ";" for comments, and # for anon functions, but you can't create your own macros, unlike most lisps.

Like python, docstrings are part of the function signature, except they go between the function name and the parameters. they are accessible from the repl via (doc ). Function overloading is really intuitive in Clojure, the defn call simply takes a list of parameter/implementation pairs. This is the kind of thing I love about functional languages, much less code duplication and much more readable.

Function bindings are lexically scoped. Another really nice feature is destructuring, which is the ability to decompose an object passed as an argument into its constituent parts in the function definition itself. So, if your function takes a Car, and all you need is the bumper, you can make that explicit in the function definition. Really very cool feature.

This chapter also covered namespaces and calling java functions. There are a bunch of conventions around calling java things directly which I find slightly non-intuitive, but whatever, being able to have that kind of interoperability is such a huge advantage it doesn't much matter.

    Trivia
  • division operations return a clojure Ratio object
  • 100M gives you a BigDecimal, useful for arbitrary floating point math
  • boolean stuff: nil and false are false, everything else is true (including zero!)

Categories: Technology, Clojure

blog now using SQLAlchemy

by paul, on 07.25.2011

I refactored my blog to use SQLAlchemy as an object relational mapper. I initially implemented my blog just using straight SQL, and planned to switch over to SQLAlchemy as kind of an exercise to remind myself what an ORM really buys me. It's great, I was able to delete lots of messy code and make things much cleaner in general. I've spent a fair amount of time wrestling with ORMs, and it's nice to remind myself how great they can be in comparison to the Bad Old Days.

Categories: Technology