For my Nature of Code final I implemented a version of my thesis that allowed path following in Ruby-Processing along 3D vectors. Boxes would start at a point at the bottom of the 3D map and travel along pre-determined routes. These routes represented the flights of U.N. planes over Sudan. The height of the arcs and start and end points represented the relative length of the flights. the number of boxes travelling represented the relative frequency of the flying involved. Unfortunately, I accidentally overwrote the files I showed in class. But I do have the individual class files I wrote to implement this functionality. These are pretty straightforward ports of Shiffman’s particle system class and path-following classes. Below is a 3d Particle / Emitter after running for an hour or so. (I forgot it on my desktop).

The main code I used was this simple Path-Following implementation, which was pretty easily hooked into my project. Just add some start & endpoints, calculated the endpoints, and shoot those boids off into space. Here’s the Github version of Shiffman’s Path Following – just a one-to-one port into Ruby-Processing. Some more customized stuff follows below.

Spirograph

The main code for the spirograph goes something like below. You can get all of this code on Github, and you’ll need to get it all because you need ToxicLibs, and the attractor & particle class. And PeasyCam. But at least you get an idea of what’s going on.

$LOAD_PATH << 'library/'

require 'library/attractor.rb'
require 'library/particle.rb'

class Playground < Processing::App

  load_libraries 'opengl', 'toxiclibscore', 'peasycam'
  include_package "toxi.geom"
  import 'peasy'

  V3 = Java::toxi.geom

  def setup
    size 800, 600, OPENGL
    new_particle
    configure_camera
  end

  def draw
    background 0
    draw_particle
  end

  def new_particle
    @ac = V3.Vec3D.new(0.0, 0.0, 0.0)
    @ve = V3.Vec3D.new(0.0, 1.0, 0.0)
    @lo = V3.Vec3D.new(50, 50, 0.0)
    # Create new thing with some initial settings
    @p = Particle.new(@ac, @ve, @lo, 10)
    # Create an attractive body
    @a = Attractor.new(V3.Vec3D.new(width/2, height/2, 0.0), 3, 2)
  end

  def draw_particle
    @a.go
    # Calculate a force exerted by "attractor" on "thing"
    f = @a.calcGravForce(@p)
    # Apply that force to the thing
    @p.apply_force(f)
    # Update and render the positions of both objects
    @p.run
  end

  def configure_camera
    @cam = PeasyCam.new(self, 800)
    @cam.set_minimum_distance(-1000)
    @cam.set_maximum_distance(5000)
  end

  def mouse_pressed
    puts @p.points
    puts "===================================================================================================================================="
  end
end

Playground.new :title => "playground"

Particle Arcs

Here’s another script that throws out a particle and records / draws past positions of that particle as it moves through the 3D space. You can get this one on Github too. Here’s the main sketch to give you the gist:

$LOAD_PATH << 'library/'

require 'library/attractor.rb'
require 'library/particle.rb'

class Playground < Processing::App

  load_libraries 'opengl', 'toxiclibscore', 'peasycam'
  include_package "toxi.geom"
  import 'peasy'

  V3 = Java::toxi.geom

  def setup
    size 800, 600, P3D
    start = V3.Vec3D.new(0.0,0.0,0.0)
    finish = V3.Vec3D.new(500,500,0.0)
    @ac = V3.Vec3D.new(0.0,0.0,2.0)
    @ve = V3.Vec3D.new(0.0,1.0,1.0)
    new_particle(start, finish)
    configure_camera
  end

  def draw
    background 0
    draw_particle
  end

  def new_particle(start_loc, end_loc)
    t = start_loc.add(end_loc)
    t.scale_self(0.5)
    t.add_self(0,0,300)
    @lo = start_loc
    # Create new thing with some initial settings
    @p = Particle.new(@ac,@ve,start_loc)
    # Create an attractive body
    @a = Attractor.new(t, 3, 6)
    @a2 = Attractor.new(end_loc, 3, 6)
    @target = false
  end

  def draw_particle
    if(@p.loc.z <= 300 && @target == false)
      @a.go
      f = @a.calcGravForce(@p)
      @p.apply_force(f)
      puts [@p.loc.x, @p.loc.y, @p.loc.z].to_s
      @p.run
    elsif(@p.loc.z >= 0)
      @a2.go
      f2 = @a2.calcGravForce(@p)
      @p.apply_force(f2)
      @target = true
      puts [@p.loc.x, @p.loc.y, @p.loc.z].to_s
      @p.run
    end
    @p.draw_points
  end

  def configure_camera
    @cam = PeasyCam.new(self, 800)
    @cam.set_minimum_distance(-1000)
    @cam.set_maximum_distance(5000)
  end
end

Plotter – Data from Recovery.gov from Peter Esveld on Vimeo.

What is Plotter?

Plotter is an open-source tool for designers to map their data into the real world. Plotter asks the you – the designer – to define that connection between data and visual presentation before making anything. In other words, whatever else you make with Plotter, you’ll be making a map.

The system is designed so that in theory you could map anything you wanted – buildings, animals, or conceptual frameworks.

Plotter asks that whatever other characteristics each individual component of the dataset contains, that it also include a specific timestamp and location vector. The timestamp and location can be of any scale – seconds to centuries, millimeters to light-years – but they need to be embedded within the data. Using the two characteristics of space and time, Plotter then asks the user to provide the imagery to layer data on top of. This is the context, and can take the form of any raster-based photograph, scan, sketch, rendering etc. There are plans to allow Plotter to accept vector imagery such as KML files and 3D models as well.

After appropriate data and map imagery have been assembled, the core of Plotter goes into effect – a 3D engine for rendering, displaying, and exploring data in real time.

Prototyping

Plotter – Early Prototype from Peter Esveld on Vimeo.

This project didn’t set out to be a visualization tool. Originally, Plotter was conceived as a client project. A manager at the U.N. in New York had asked for a sophisticated way to track and view U.N. flights all around the world. The goal was to be able to see individual routes and fuel consumption – and get a really detailed look at the carbon footprint generated by all this air travel on both a local and a global scale. This would have been really simple to accomplish with charts and graphs, but I wanted to build something more sophisticated. The first attempt is what you see in the video above.

So, prototyping started not by creating data visualizations in Processing, but in trying to pick a technology to parse data in Excel Documents and upload that into something usable (A CouchDB database, as it turns out). All the U.N. flight records were in relatively unstructured Excel format – lots of extra junk in there, extra cells and rows, gif images, etc. Here’s an example, freely available online. So, I needed a really flexible way to program something to pull this data out. Where I wouldn’t have to write thousands of lines of code, and could hopefully leverage existing libraries to deal with the primitive nuts and bolts. This is where Clojure fits the bill. I wasn’t really anticipating learning a Lisp language or the basics of functional programming, but that’s what happened. Clojure is built on top of the Java Virtual Machine, which means that you can call methods from any Java library – including JExcel, Java’s Excel document library. Clojure also lets you accomplish tasks with 5 lines of code where Java might require you to write 30. Here’s an example that uses JExcel to open up an Excel doc, look at each sheet, and print each row of that sheet to the console:

(defn parse-it [file]
  (let [wb (Workbook/getWorkbook (File. file))]
    (doseq [sheet (.getSheets wb)]
      (let [num-rows (range 0 (.getRows sheet))]
        (doseq [r num-rows]
          (let [row (.getRow sheet (r))]
          (prn (format-temp row))))))))

[Link to full parsing code here]

Once the Excel docs are parsed, they are pushed up into a CouchDB database. If a database doesn’t exist, one is created on the fly. This is all made possible by the excellent Clutch library for Clojure. CouchDB isn’t super well documented yet, but it’s basically a very fast database that allows versatile indexing and searching. More information can be found here. CouchDB is excellent for data visualization, because it allows for the creation of specific views, or subsets of data, to that you can use to pull in only the data you need for a given application state. For purposes of testing and design, I used a local installation of CouchDBX, which provides a bare-bones GUI so that you can interact with Couch, browse the records, and write views—all inline.

The core of Plotter is built using Processing, a Java library for visual programming. I further clouded the issue by using a more obscure fork of Processing called Ruby-Processing, developed by Jeremy Ashkenas. Ruby-Processing runs on the JVM as well, using JRuby, and is great for building cool stuff very quickly. Plotter is, at its heart, a big, tile-based, scaling interactive map. As such, it owes a heavy debt to Modest Maps, developed by the amazing guys over at Stamen, a design studio in San Francisco. Plotter’s basic functionality comes from Modest Maps, although most of it was completely re-worked to work as a concurrent, thread-safe 3D mapping engine (written in Ruby, not Java).

A typical useful Jruby paradigm might be something like creating a new Java thread pool:

# Thread Pool
@executor = java.util.concurrent.Executors::newFixedThreadPool(13)
# Array to hold queued threads.
@future = Array.new

Later, add a thread to the pool (Loader needs to implement Java’s Callable)

@future.push(@executor.submit(Loader.new(v, @list[v[2]][v])))

Here’s the Loader class

class Loader

  include Processing::Proxy
  include java.util.concurrent.Callable

  def initialize(key, path)
    @key = key
    @path = path
    #puts "loader #{@key}"
  end

  def call
    img = PImage.new
    img = load_image(@path)
    #puts "the key is: #{@key}, the path is #{@path}, the image is #{img}"
    return @key, img
  end
end

Then you could check to see if those threads have finished, and pop the return values (an image and hashtag, in this case) into another array:

@future.each_index do |i|
  if(@future[i].isDone())
    key, val = @future[i].get
    @images[key] = val
    @future.delete_at(i)
  end
end

Look & Feel

Plotter’s look and feel is largely up to the developer. In general, it attempts to keep a very neutral design and let the interaction and data take center stage. Even so, things should look extremely clean and tidy, and very modern. Plotter’s design aesthetic tends toward the work of Edward Tufte, Herbert Bayer, and contemporary Dutch graphic designers. Here are a few sample images to better explain the ideal Plotter mapping aesthetic.

Image from this amazing Flickr set.

In Design frontiers class we were tasked with coming up with applications for Konarka Power Plastics. We were given certain specifications for how solar plastic works – each node needs to be the same surface area, different colors create different power output, etc. Here’s our design – a concealable modular power source for outdoor deployment. It’s vinelike, plant-like aesthetic allows for use in areas where traditional solar panels might not be an option, and it’s designed for adjustment to follow the sun.

Konarka Power Plastic is a photovoltaic material that captures both indoor and outdoor light and converts it into direct current (DC) electrical energy. This energy can be used immediately, stored for later use, or converted to other forms. Power Plastic can be applied to a limitless number of potential applications – from microelectronics to portable power, remote power and building-integrated applications.

Bees

Making Non-Newtonian Fluid from Peter Esveld on Vimeo.

Non-Newtonian Fluid – Act 1 from Peter Esveld on Vimeo.

More videos can be found on my vimeo page.

This works because non-newtonian fluids seem to have both properties of solids and liquids. From the wikipedia entry:

A non-Newtonian fluid is a fluid whose flow properties are not described by a single constant value of viscosity. Many polymer solutions and molten polymers are non-Newtonian fluids, as are many commonly found substances such as ketchup, starch suspensions, paint, blood and shampoo. In a Newtonian fluid, the relation between the shear stress and the strain rate is linear (and if one were to plot this relationship, it would pass through theorigin), the constant of proportionality being the coefficient of viscosity. In a non-Newtonian fluid, the relation between the shear stress and the strain rateis nonlinear, and can even be time-dependent. Therefore a constant coefficient of viscosity cannot be defined.

Why were we playing with this? Mostly because it’s fun, but also because of this video:

This was a very cool assignment – and I’d love to fork this code and adapt it to my own purposes. I hadn’t used bitshifting to separate out RGB values before, and found the process enlightening. Also the path class in the download is useful and interesting.

Aaron Koblin’s work was something I had a passing familiarity with – the House of Cards video is incredible and made quite a splash when it first hit the internet. I also had seen (and loved) the Sheep Market project. His work is really clever, and it’s nice to see some of these things, particularly Mechanical Turk used outside of the purely commercial realm to make art. His projects all seem slightly ahead of the technical curve, and they nicely work around one of my big gripes about generative art: that these projects often seems to have no human element. You can see the hand (or hear the voice, in the case of the Bicycle Built for 2000) of the participants in most of Koblin’s work.

Toxi is another artist that I’ve seen all over the place in the last few years. His rapid-prototyped generative typography for Print Magazine is what first piqued my interest in digital fabrication. His work also appears at PostSpectacular. Toxi makes some amazing stuff, and one of the best things about it is that he open-sources much of the code he uses in his projects.

I think Toxi’s work falls slightly more in the graphic design camp than Koblin’s, although it might debatably be even more beautiful. The projects flow directly from the artist without instead of being visualizations or collaboratively generated (for the most part).

Successfully got this up and running. It imports raw data from a CSV file, parses it, and then displays it in a processing sketch.

This won’t run well in your browser, but take a shot anyhow. If you spot any improvements to the code, please give a shout!

http://hellofriend.info/itp/visualizing_data/themevar/index.html

The four members of Front have developed a method which turns freehand sketches into solid objects by combining two advanced techniques – Motion Capture and Rapid Prototyping. Pen strokes made in the air are recorded with Motion Capture and become 3D digital files; these are then materialised through Rapid Prototyping into real pieces of furniture.

I love this – it seems like drawing without paper. The world becomes the ground.

This project created 2 wireless masks that vocoded whatever performers said into animal sounds. The “deerwolf” produced mammal sounds. The “batdragonfly” produced insect/bird sounds.

The pet sounds project ended up producing quite a few things, depending on how the project is considered. The best byproduct for me is that is the realization that I’ve created a reasonably affordable platform to create wireless “performance devices”. The design, construction, and components used in the masks end up being right around $100. You will need a laptop, but most people who would be building the masks already own that. When you’re done, you’ll have a masks that wirelessly vocodes whatever you say, in any way you like.

The biggest letdown of the semester was my inability to get an Xbee and an accelerometer to work reliably alongside an analog FM transmitter (on the same circuit). As far as I can tell, I was unable to isolate the circuit enough to get reliable information from the accelerometer, and that meant that motion sensing masks were not an option.

Read on for all the tech specs.

The masks were simply painter’s masks fitted with lasercut plexiglass. The screw-on portion of the mask contained the electronics. A coin-cell battery with a switch was attached to the outside. Inside the face portion of the mask was a small microphone.

The electret mic and rechargeable lithium ion coin cells are both available at Sparkfun. The mic works quite well, and is very simple to use. WARNING: the coin cells drain extremely quickly with this circuit, and you need a special charger to charge them (which takes hours). Not the best solution as shown here, but there are probably fixes for this. If you do use the coin cells, be prepared to make some DIY enclosures or buy the sparkfun enclosures (shown), which still require you to solder them to something and provide your own ground. The top of the battery is power(+), and the bottom is negative(-).

Here’s what’s inside both masks. There’s an Arduino Mini, which you program with an FTDI cable, an NS73M FM Radio Breakout board, and that’s about it besides power and the mic. Basically, the Arduino pins 4 + 5 hook up to the i2C pins (clock and data) on the FM transmitter. That’s all you need, besides hooking up power and ground to the appropriate places.

Here’s some code that gets the Transmitter up and running on whatever FM frequency you specify. Put it on the Arduino. Thanks to these guys for figuring out all the I2C protocols here – I wouldn’t have been able to do it on my own.

My stripped-down version: FM Transmitter

Here’s the frankensteined version of the FM transmitter, plus an accelerometer and an Xbee Radio. This sort of works – but the interference in both the FM signal and the values coming from the Accelerometer were too intense to be useful. This circuit might actually work if everything was completely shielded with extremely large capacitors between each power + ground, and strong (22uf) resistors on each power connection. I couldn’t fit all that on this board.

Here’s some Processing code you can use to see if your Xbee/accelerometer setup is working or not: Grapher Sketch

Here’s a setup for your Xbees that should help get an accelerometer going. Remember, once you set the Xbee baud rate to 115200, you’ll have to set your ZTerm (or whatever you’re using) to the same baud rate, or it will look like everything just stopped working.

//PETER’S CONFIG
XBee1 (Sender)
ATID328        pan ID
ATMY1        my address
ATDL2        destination address 2
ATD02        pin 0 in analog mode
ATD12        pin 1 in analog mode
ATD22           pin 2 in analog mode
ATIRA        sample rate 10ms (hex A)
ATIT1        samples before transmit 1
ATBD7        baud 115200
ATWR

XBee2 (Receiver)
ATID328        pan id
ATMY2        my address
ATDL1        dest address 1
ATIU1        I/O output enabled (UART) 1=enabled, 0=off
ATIA1        I/O input from address 1 (MY address)
ATIRA        HEX number of milliseconds (20ms)
ATBD7        baud 115200
ATWR

Finally, for the software I used Stanford’s ChucK to produce all the crazy vocoded audio on the computer. ChucK is basically an object-oriented synthesis program. It has great support for audio analysis however, which makes it ideal for applications like on-the-fly vocoding + triggering samples. You’ll want to download their Mini-Audicle program unless you love Terminal.

For FM receivers, I hooked up some cheapo armband tuners from Best Buy. They fed the audio in on the left + right channels into my laptop, which simply looked at the audio, performed some FFT calculations, and played the appropriate samples / sounds.

Here’s a ChucK shred that I was using: Chuck Sample

Please email me with any questions if you have them.