Systemic wins LIFT award, new version of Systemic (2.17)

I am very happy to announce that Systemic has been awarded the LIFT grant. This means that, together with my colleagues Joel Green and Randi Ludwig, I will be able to work full-time — instead of as a side project — on improving and expanding Systemic, and creating new educational apps like Systemic Live and Super Planet Crash, over the next year!

I am also releasing a new release of Systemic 2 (2.17) which addresses some bugs and improves the documentation for installation on Linux. You can download it now.

Below are some of the changes:

– NEW: ktable function for listing the fit values as a table, suitable for exporting to TeX or HTML
– NEW: Bayesian Information Criterion menu item
– NEW: Quadratic trend term
– CHANGED: Periodograms report the normalized power between 0 < p < 1, where power = 1 is a perfect fit. — This is a bug corrected in 2.172.
– FIXED: bug where the semi-amplitude would be calculated incorrectly for massive bodies. (credit: Trifon Trifonov)
– FIXED: bug in simplex that would crash the application if the minimizer encountered a NaN value.
– FIXED: bug in the GUI that would crash the application in case of excessive text output.
– FIXED: Fixed naming of columns in the matrix returned by kdata().
– FIXED: bug where the radial velocity curve or periodogram would look excessively jagged.
– FIXED: bug in kperiodogram.boot where the function could crash.
– FIXED: bug in kperiodogram.boot where the function would only calculate the ‘full’ periodogram (instead of the periodogram of residuals) for certain inputs.
– FIXED: bug in the GUI periodogram routine, where you could receive an error for certain power spectra.
– FIXED: Kernel plot using plot() respects the chosen xlim.
– FIXED: MCMC would crash in certain situations when set up from the menu.
– FIXED: 1LO-crossval returns the signed sum of logs, instead of the absolute value.
– FIXED: clarified the installation instructions (Readme.txt) for Linux. (credit: Franz Feldtkeller)
– FIXED: you can now choose a path for R that is not /usr/bin/R by selecting Help -> Set path to R…
– FIXED: F-test menu item uses the current kernel instead of the kernel named “k”.
– Various bug-fixes in the plotting routines.

Bugs reported at are unfortunately still open due to lack of time to address them. They will be fixed in 2.18.

Colliding N-body spheres: Particle Mayhem!

When Giants Collide — WGC for short — is one of the “fun” projects I am working on. Once finished, it will be a small in-browser simulator where you can collide giant planets together (with some degree of realism). You can see my progress on my GitHub repo and the series of blog posts under this category.

In this little demo app, you can run an N-body simulation in your browser where you make two spheres made of point masses “collide”. You can tweak various parameters (collision speed,  impact parameter, distance and number of particles) to change the outcome of the simulation.

Underneath it all: TreeSPH.js

The app above is powered by the portion of WGC’s code that computes the gravitational force between a set of point particles (a gravitational N-body system). The gravitational force is computed using the Barnes-Hut tree gravity algorithm, and the coordinates of the points are evolved using a third-order embedded Runge-Kutta algorithm. The code is available in the GitHub repo for WGC.

I am now working on writing the hydrodynamical part (via the SPH algorithm), which will let me simulate the collision between two gaseous spheres. The resulting library will be called treesph.js.

treesph.js will be an open-source JavaScript library able to power small-scale hydrodynamical simulations — either in the browser (through web workers), or within a JavaScript environment (e.g. Node.js). It comes with:

– a library to set up initial equilibrium conditions (e.g. Lane-Emden spheres, or N-body spheres with isotropic velocity dispersion);
– a canvas-based library to plot and animate the simulation snapshots;
– a fast library for operating on vectors and matrices that minimizes allocations and copying, and other math routines.

Performance notes

While playing with the app, you may be wondering (a) why there is a “buffering” stage before you can see the evolution of the system, and (b) why so few particles?

Buffering: it’s all about delayed gratification

The app animates the particle motion at 30 frames per second. This requirement places a hard and fast constraint: if you want to compute the particle motion at each frame request, then the computation must take less than 1/30th of a second, otherwise the webpage will freeze as the JavaScript engine tries to catch up with the accumulated frame requests. For reasonable number of particles (say, 100 or more — see below) and the time steps required by the above app, this requirement is way overshot.

This issue can be ameliorated by running the numerical computation in a separate thread (a web worker), and drawing frames on the main thread as soon as they are computed. This is still not as optimal: while it solves the UI freezing issue, the particle motion will appear very jerky as it will be animated at (typically) less than a frame per second!

In order to solve this issue, I created a small JavaScript library (streamingcontroller.js; available in the same GitHub repo, documentation upcoming). Streamingcontroller.js first estimates the expected wall time — the time in seconds — needed to complete the simulation. Then, within the web worker thread, it “buffers” the simulation snapshots by adding them to a pool of snapshots. Once the buffer is big enough that the simulation can be run in real time without hiccups, the library starts streaming the snapshots back to the main thread where the animation is drawn. In the main thread, a second buffer receives the snapshots; the second buffer is then emptied at 30 frames per second.

More particles, pretty please?

The default setting of the app is to animate 250 particles (125 per sphere). Why so few, when typical number of particles quoted for N-body simulations routinely exceed millions — or even billions! — of particles?

There are three bottlenecks at work. The first is obvious: the code isn’t fully optimized and profiled yet, and I am certain there is room for improvement. I am writing a small math library of common mathematical routines called math.js (also in the same GitHub repo) which will be fully optimized for V8.

The second is also obvious: simulations with lots of particles are usually run at full-speed, on multiple cores, and in the background. These simulations can save their snapshots, to be plotted and animated at the end of the run. An online app (or game) with real-time requirements (or, say, a <1 minute buffering time) doesn’t have this kind of luxury!

The third is the worst hurdle, and it is inherent to the nature of JavaScript: JavaScript is slow. I am not a JavaScript guru by any means, but I do have a good amount of experience writing performant numerical code in a variety of languages (mostly C). While JavaScript is typically fast enough for most tasks on the web, it is slow on personal computers and even slower on mobile platforms for physically-motivated, accurate simulations. In its present form, it is not well-suited to run these kinds of numerical tasks as quickly as the underlying hardware allows. Although JavaScript interpreters have been improving by leaps and bounds, and careful code can exploit some of these optimizations, they are hitting a wall of diminishing returns. Since JavaScript is the only runtime available on browsers, it is the ultimate bottleneck.

You can check out the other demos using code from WGC in this webpage.

AstroBetter: Creating Online Apps for Outreach and Education

I wrote a short article on AstroBetter called “Creating Online Apps for Outreach and Education” about some of the tools and resources I use for my online work.

Chrome developer tools + js2-mode and web-mode in EmacsChrome developer tools + js2-mode and web-mode in Emacs

I’ll be blogging more about this topic in the short term, but here are a few things I’ve been looking at that people might find interesting:

  • Languages that convert to JavaScript: I find the constant context switching between the different programming languages I use for my work very annoying. I’ve used Emscripten for converting my C code into JavaScript to minimize the amount of code rewriting. I’m actively looking at languages that compile to JavaScript — the most famous is probably CoffeeScript, but I’m keeping an eye out for Kotlin. Kotlin is a budding language that primarily runs on the Java Virtual Machine (JVM), but can also be compiled to JS. Since I do use Java and the JVM ecosystem for a substantial amount of my code, it looks very promising. Now, if only there was a kotlin-mode for Emacs I’d be very happy :)
  • Three-dimensional visualizations: three.js looks cool but daunting. I’ve been trying to think of a small project to do using the isometric engine Obelisk.js — I’m just enamoured with things that look retro., a magazine for young students interested in science, published a short interview with me about how I came to be an astronomer, and why I developed Super Planet Crash. Read it here!

An interactive Barnes-Hut tree

[TL,DR: if you’d like to play with a simple Barnes-Hut octree code, scroll down to the little embedded app.]

Gah! It’s been quite a while since my last post. Despite my best intentions, work (and a lot of feedback from Super Planet Crash!) has taken precedence over blogging. I do have a sizable list of interesting topics that I’ve been meaning to write about, however, so over the next few weeks I’ll try to keep to a more steady posting clip.

Super Planet Crash has been a resounding success. I have been absolutely, positively astounded with the great feedback I received. My colleagues and I have been coming up with lots of ideas for improving the educational value of SPC, add new, interesting physics, and addressing some of the complaints. In order to have the ability to dedicate more time to it, over the past few months, we’ve been furiously applying for educational and scientific grants to fund development. Hopefully something will work out — my goal is to make it into a complete suite of edu-tainment applications.

When Giants Collide

I’ve recently started experimenting with a new  visualization that I think will turn out pretty darn cool. Its draft name is When Giants Collide. When  Giants Collide will address a common request from planetary crashers: “Can I see what happens when two giant planets collide”?

A sketch of the interface.

When Giants Collide will be a super-simple JavaScript app (so it will run in your browser) that will simulate the collision of two massive spheres of gas. The simulation will have to model both gravity and the dynamics of the gas: to address this, I’ve been dusting off and reviewing an old Smoothed-Particle Hydrodynamics (SPH) code I worked on for a brief period in graduate school. SPH is a very simple technique for cheaply simulating gas flows with good spatial accuracy, and is somewhat straightforward to code. There are some shortcuts that have to be taken, too — large time steps, low particle counts, and more (e.g., a polytropic equation of state for the gas giants; more on this in future posts). These shortcuts come at the expense of realism, but will enable fast, smooth animation in the browser.

Gravity with the  Barnes-Hut algorithm

Gravity is an essential ingredient of When Giants Collide! Even with very low particle counts (say, N = 1000), a brute force calculation that just sums up the mutual gravitational force between particles won’t do if you want to run the simulation at 60 frames per second. Direct summing is an N^2 operation:

(this is a simple force accumulator written in R).

A better way that involves only a slightly more complicated algorithm is to use the Barnes-Hut algorithm (a short Nature paper with more than 1,000 citations!). The algorithm involves recursively subdividing space into cubes and loading them with particles, such that every cube contains either 0 or 1 particles. This is represented in code with an oct-tree structure.  Once such a tree is constructed, one can calculate the gravitational force on a given particle in the brute-force way for close particles, and in an approximate way for distant particles; whether to use one or the other is determined by walking the tree down from the top. An excellent explanation (with great visuals!) is provided in this article.

The other advantage is that, once the tree has been already built for the gravity calculation, it can be used to identify the nearest neighbors of a given particle through the same tree-walking procedure. The nearest neighbors are needed for the hydrodynamical part of the SPH algorithm (see, e.g., this review article by Stefan Rosswog or this one by Daniel Price).

An interactive tree

Below is an interactive JavaScript applet that subdivides space with the Barnes-Hut algorithm. You can add new points by clicking on the surface, or using the buttons to add new, random ones.

The code for building the Barnes-Hut tree from an array of 3D positions is available at the GitHub repository for When Giants Collide. I will be developing the code in the open, and post periodically about my progress. Hopefully by the end of summer I will have an attractive app running on any modern device and web browser. Any ideas on how to gamify it?

2,000,000 systems played!

The high scores as of April 13, 2014 for all of posterity. Good job, brave folks.!
The high scores as of April 13, 2014 for all of posterity. Good job, brave folks!

This week has been quite the ride. Super Planet Crash has been featured on io9, Huffington Post,, Motherboard, and other online publications, and it “suffered” from repeated surges of traffic from imgur. Not bad for a game hacked together over the weekend! It overjoyed me to receive emails, and pictures!, by people enjoying the game, especially from the younger generation.

More than 2,000,000 games have been played as of today, and hopefully a fraction of those players will want to know more about exoplanets. I would also encourage everyone who enjoys this little free game to donate to science education funds, such as McDonald Observatory’s Science Education Fund. I would be oh so happy to have bragging rights due to planet crashers donating en masse!

I’m slowly trying to work through some of the feature requests. Not all are feasible on a short timescale (science is my full-time job, after all!), but I will strive to at least try to address the lowest-hanging fruit. One pet peeve shared by many was the inability to see the high-scoring games. In trying to address this, I discovered two bugs in the implementation of the high-scores.

The first is that the server relied too much on trusting the high-scores that were sent from the client (i.e. the Javascript running in the web-browser). Although I had tried to mitigate it somewhat, several fake high-scores were submitted. I added some stricter checks that should further help address the problem. The right solution would be to run the system on the server in order to check for any shenanigans. Unfortunately, this is unfeasible, as too many games are being played: it would place an unduly amount of stress on my server.

The second is a bug in the way systems were recorded and sent to the server. Some of the highest-scoring systems attempt to score high on masses, “crowdedness” (how close are the orbits of the bodies to each other) and habitability. They do that by (a) adding a binary companion (the “dwarf star”) and (b) putting a lot of planets in the same orbit within the habitable zone.


Something like this.
Something like this.

The resulting systems are likely highly chaotic, so any small error in recording the state of the system [ref]The state of the system being the current position and velocity of each body.[/ref] will change the outcome very quickly (the so-called “butterfly effect“). Unfortunately, one bug in Super Planet Crash resulted in this exact scenario happening. Any rounding or truncation of the floating point values for the coordinates will also affect the evolution of the system. The most common outcome is that these high-scoring systems will appear to be unstable when replayed. Grrr.

The decision I reached is to clean up the high-score table. The systems should now be recorded the correct way, and everyone will be able to see how the scores were achieved.

I understand this is sad news for the current record holders, so the screenshot at the top of this page will record the brave folks who reached upwards of 300,000,000 points for all posterity. (Just imagine someone unplugged the arcade machine by mistake…)

Next up on my agenda is releasing the game on GitHub. I am cleaning up the last few bits. If you are a programmer, you’ll be able to create pull requests for new features there.

In my next post, I will go into a bit more detail about how I created Super Planet Crash (and so can you!).

Go Crash Some Planets!

Super Planet Crash
A screenshot of Super Planet Crash playing in Safari


If you enjoyed playing Super Planet Crash, please consider donating to the Science Education Fund at McDonald Observatory. Every little bit counts. Go support science!

Update 2: 2,000,000 systems were created!
: Systemic and Super Planet Crash were featured on io9Space.comGlobalNews, Motherboard, Huffington Post, The Verge, and two press releases by UC Santa Cruz and McDonald Observatory. Thank you!

Super Planet Crash is a little game born out of some of my work on the online version of Systemic. It is a digital orrery, integrating the motion of massive bodies forward in time according to Newtonian gravity. It works on any recent web browser and modern tablets.

The main goal of the game is to make a planetary system of your own creation be stable (i.e. no planet is ejected, or collides with another body). This is of course exceedingly easy when your system comprises of a few Earth-mass planets, but dynamical instability can quickly set in when adding a lot of heavier bodies (from giant planets, all the way to stellar companions).

The challenge is then to fit as many massive bodies as possible inside 2 AUs (twice the distance between the Earth and the Sun), teetering close to instability but lasting at least 500 years. Accordingly, the game rewards a daring player with more points (proportionally to the mass of each body added to the system). A few simple rules are listed under the “Help” button.

The game always starts with an Earth-mass planet in a random location, but you can also have fun overloading known planetary systems! Clicking on the “Template” dropdown brings up a list of planetary systems to use as starting templates, including the compact system Kepler-11 and the super-eccentric planet HD80606 (more systems to come). You can even share your creations with your friends by copying the URL in the “Share” box.

The game is open-source, and still under active development. The entire code will be downloadable from GitHub (as soon as I get a bit of work done!).In the near future, I will be adding integration with Systemic Live, a longer list of template planetary systems and smartphone support. In the meantime, have fun crashing planets!


The game was made possible by the wonderful paper.js library, which let me quickly prototype the app despite having little experience in web gaming. The palette draws from the base16 color set.

Many many thanks to my wonderful testers: Rachael Livermore, Mike Pavel, Joel Green, Nathan Goldbaum, Maria Fernanda Duran, Jeffrey SilvermanAngie Wolfgang, and other cool people.

My work is funded by the W. J. McDonald Postdoctoral Fellowship. If you enjoyed the game, please donate to the McDonald Observatory fund to support science education.

AstroTRENDS: No so weaselly after all


Substitute damn every time you’re inclined to write very; your editor will delete it and the writing will be just as it should be. — Mark Twain

Damn right!

In my last post, I showed a plot of the number of abstract that contained weasel words, as tracked by AstroTRENDS:

chart-2I interpreted this trend as a steady change in the style and “audacity” of astronomy papers, and I believed that a possible cause was hedging. (See Writing without conviction? Hedging in science research articles.) Note that I was not making a statement about the quality of the research, but merely about an interesting trend I had not seen mentioned elsewhere.

Perhaps I should have used more weasel words in my post! I acted upon Ben Weiner’s suggestion: use a set of non-weasel words as a control to verify whether the trend was due to an increase in verbosity instead. I track this set of keywords (a mix of adverbs and adjectives that I deemed to be neutral):

Fast OR Slow OR Large OR Small OR Before OR After OR High OR Low OR Many OR Few OR More OR Less OR Inside OR Outside OR Recently OR Just

This is available as “Non-weasel keywords” in the AstroTRENDS drop downs.

And here’s the plot!

The two appear to be tracking each other pretty well. It seems to me to be a strong indication of the correctness of Ben’s guess that verbosity is the main driver here. However, simple keyword search may still not be telling the whole story (e.g. because certain keywords “saturate” as the abstracts get longer, appearing more than once), so a better approach could be to study a small sample of abstracts through the years.


Weasels (green) vs. non-weasels (yellow)
Weasels (green) vs. non-weasels (yellow)

Turns out that there’s a comprehensive ADS API, described on GitHub here, so with a bit of rejiggering I will be able to let AstroTRENDS do free-form queries (via Michael Kurtz.), and do a bit of abstract munging myself.

AstroTRENDS: Weasel words

Credit: Cliff

I added a bunch of new keywords to AstroTRENDS, mostly suggested by friends and people in the community who had read my Facebook post.

A thought I had yesterday is the following: has the astronomical literature become more speculative, and perhaps less committed to audacious claims, in recent times? It is difficult to test this hypothesis  by merely querying ADS for abstract keywords. It would certainly be better served by a natural-language processing analysis of the full text, although this is just my uninformed speculation.

A much simpler way is to search for the so-called “weasel words” (such a funny way of describing them from a non-native speaker POV!). Matthew Might (a CS professor from the University of Utah) has a really interesting article about the different abuses of language that are common among technical writers, and he created some automated tools for detecting them. It’s a great read. (There’s even an emacs minor mode called writegood based on his recommendations, which I will be testing for sure). Although I don’t necessarily agree with a strict adherence to all of his points, there are certainly some great pieces of advice there.

Taking his post as a reference, I added a new “weasel words” pseudo-keyword to AstroTRENDS. The “weasel words” keyword shows the result of an ADS query of refereed abstracts containing the following boolean expression:

Could OR Possibly OR Might OR Maybe OR Perhaps OR Quite OR Fairly OR Various OR Very OR Several OR Exceedingly OR Vastly OR Interestingly OR Surprisingly OR Remarkably OR Clearly OR Significantly OR Substantially OR Relatively OR Completely OR Extremely

We can easily disagree on whether using these words in an abstract constitutes “weaseling”, or has any sort of nefarious purpose (I certainly pepper my writing with more than my fair share of those). It is still an interesting exercise to verify whether usage of those words has increased over time. The following plot shows the fraction of articles containing those words (i.e. number of articles containing the words normalized by the total article count) each year.



Keeping all the caveats above in mind, there is a definite upward, pretty linear-by-eye trend going on. I’m not sure whether it has to do with simple evolution of language and style, less boastful writing, an accident of fate/bug on my part, or some other factor.

This is of course a super-shallow analysis that would require far more insight than what I offered in this post, but it’s still intriguing. I tried to altavista whether this is well-known, but have come empty handed so far. Any ideas?

You can play with the interactive plot itself by clicking this link.

UPDATE: Ben Weiner made a really good point on the Facebook astronomer group.   He suggests that an additional, alternative explanation could simply be that abstracts have become, on average, more verbose with time, which would explain the higher frequency of fluffy adjectives and adverbs. This could be checked with a control set of non-weasel words… which I will definitely try.

How did this post do with writegood-mode? Pretty nicely… but I got a grade of “11” on Hemingway, with about 9 out 24 sentences being hard to read.  Oh well.
Weasel image credit: Cliff

The beauty of grainy photographic plates

I’m working on a small bit for a project, involving cross-fades between old-timey photographic plates of spiral galaxies, and their modern high-resolution counterparts.

Although the level of detail in this image of M33 is astounding, there is a certain beauty to the photographic plate (mouse over the image to fade into the black-and-white plate).


I had to resize and rotate both images to make them more or less coincide for a pleasant cross-fade — until I got frustrated at trying to make every pixel match…

The photographic plate holds a certain romantic value. In my early childhood I owned a lot of old astronomy books (and pilfered quite a lot from my relatives). The black-and-white reproductions on those pages had an indefinable fluid, mysterious quality to them.  If I squinted hard enough, perhaps I could catch some hitherto unknown detail or physical phenomenon? My mindset was definitely more serious and reverential to the gorgeousness of the universe back then, and it’s sometimes sad to think I lost some of that  childhood determination in discovering the mysteries of the universe, which felt so close for the picking:

Vedi, in questi silenzi in cui le cose
s’abbandonano e sembrano vicine
a tradire il loro ultimo segreto,
talora ci si aspetta
di scoprire uno sbaglio di Natura,
il punto morto del mondo, l’anello che non tiene,
il filo da disbrogliare che finalmente ci metta
nel mezzo di una verità.
Lo sguardo fruga d’intorno,
la mente indaga accorda disunisce
nel profumo che dilaga
quando il giorno più languisce.
Sono i silenzi in cui si vede
in ogni ombra umana che si allontana
qualche disturbata Divinità.

One of my favorite poems is apropos. [I limoni, by Eugenio Montale; translation].

The blurriness and undeterminedness of the early observations of galaxies gave rise to the split between the idea of nebulas within the Galaxy versus “island universes”. (Read more about it here and here.)

Image credits: The black-and-white photograph is from this page; the color photograph is from APOD (Adam BlockMt. Lemmon SkyCenterU. Arizona).