Vaex: Interactive visualization and exploration of more than a billion objects

What is Vaex?

  • Veax is a program and python library to visualize and explore large tabular datasets.
  • It mainly renders histograms, density plots and volume rendering plots for visualization in the order of a billion (109) objects in the order of 1 second.
  • Statistics such as mean, sum, count, standard deviation etc, can all be calculated on an N-dimensional grid.
  • For exploration it support selection in 1 and 2d, but it can also analyse the columns (dimensions) to find subspaces which are richer in information than others.
  • Downloads: Standalone version or Python package
  • More: Gallery | Program documentation | Tutorial | API docs | Examples

Main features

  • The vaex graphical interface

    • Visualize a billion (109) rows interactively in a graphics interface in 1d (histogram), 2d (density plot) and 3d (volume rendering)
    • Overplot vectors, for instance mean motions, tensors (for instance mean velocity dispersion tensor)
    • Custom expressions, e.g. log(sqrt(x**2+y**2)), calculated on the fly
    • publish quality output (using matplotlib)
    • Linked views: selecting in 1 view will also select it in different views
    • data formats
    • hdf5: gadget, our own format (in the future: other formats can be supported with a few lines of code)
    • hdf5 from Amuse.
    • fits bintable
    • VOtable over SAMP
    • gadget native format
    • Ranking of subspaces: for 2 and 3 dimensional subspaces, a ranking can be calculated that indicates the relative richness of structure and/or correlation in them.
    • Easily showing a fraction of the data: if the rows are uncorrelated in order (random order), a subset of the data can be shown using a slider (which can make the program more responsive)
    • exporting data: the selected data can be exported for further analysis
    • undo/redo: a mistake in selection or navigation can quickly be undone using undo
  • The vaex library

  • Generate the same plots and more as the graphical interface
  • Integration with IPython notebook

Demo movies

See the Gallery for mor examples.

Getting started

If you want to try out vaex as a graphical tool only, download the binary and read the quickstart.

If you want to use vaex as a library, from your script or IPython notebook, install vaex as a library, and go through the tutorial.