Vaex: Interactive visualization and exploration of more than a billion objects¶
What is Vaex?¶
- Veax is a program and python library to visualize and explore large tabular datasets.
- It mainly renders histograms, density plots and volume rendering plots for visualization in the order of a billion (109) objects in the order of 1 second.
- Statistics such as mean, sum, count, standard deviation etc, can all be calculated on an N-dimensional grid.
- For exploration it support selection in 1 and 2d, but it can also analyse the columns (dimensions) to find subspaces which are richer in information than others.
- Downloads: Standalone version or Python package
- More: Gallery | Program documentation | Tutorial | API docs | Examples
The vaex graphical interface
- Visualize a billion (109) rows interactively in a graphics interface in 1d (histogram), 2d (density plot) and 3d (volume rendering)
- Overplot vectors, for instance mean motions, tensors (for instance mean velocity dispersion tensor)
- Custom expressions, e.g. log(sqrt(x**2+y**2)), calculated on the fly
- publish quality output (using matplotlib)
- Linked views: selecting in 1 view will also select it in different views
- data formats
- hdf5: gadget, our own format (in the future: other formats can be supported with a few lines of code)
- hdf5 from Amuse.
- fits bintable
- VOtable over SAMP
- gadget native format
- Ranking of subspaces: for 2 and 3 dimensional subspaces, a ranking can be calculated that indicates the relative richness of structure and/or correlation in them.
- Easily showing a fraction of the data: if the rows are uncorrelated in order (random order), a subset of the data can be shown using a slider (which can make the program more responsive)
- exporting data: the selected data can be exported for further analysis
- undo/redo: a mistake in selection or navigation can quickly be undone using undo
The vaex library
- Generate the same plots and more as the graphical interface
- Integration with IPython notebook
If you want to use vaex as a library, from your script or IPython notebook, install vaex as a library, and go through the tutorial.
- Download and installing
- Datasets to download
- Helmi & de Zeeuw 2000