Python install

PyKeOps is a Python 3 wrapper around the low-level KeOps library, which is written in C++/CUDA. It provides functions that can be used in any NumPy or PyTorch script.

Requirements

  • Python 3 with packages numpy.

  • A C++ compiler compatible with std=c++14: g++ version >=7 or clang++ version >=8.

  • The Cmake build system, version >= 3.18.

  • The Cuda toolkit, including the nvcc compiler (optional): version >=10.0 is recommended. Make sure that your C++ compiler is compatible with the installed nvcc.

  • PyTorch (optional): version >= 1.5.

On Google Colab

Google provides free virtual machines where KeOps runs out-of-the-box. In a new Colab notebook, typing:

!pip install pykeops[colab] > install.log

should allow you to get a working version of KeOps in less than twenty seconds.

From source using git

The simplest way of installing a specific version of KeOps is to use some advanced pip syntax:

pip install git+https://github.com/getkeops/keops.git@master

Alternatively, you may:

  1. Clone the KeOps repo at a location of your choice (denoted here as /path/to):

git clone --recursive https://github.com/getkeops/keops.git /path/to/libkeops

Note that compiled .so routines will be stored in the folder /path/to/libkeops/pykeops/build: this directory must have write permission.

  1. Manually add the directory /path/to/libkeops (and not /path/to/libkeops/pykeops/) to your python path.

  • This can be done once and for all, by adding the path to to your ~/.bashrc. In a terminal, type:

    echo "export PYTHONPATH=$PYTHONPATH:/path/to/libkeops/" >> ~/.bashrc
    
  • Otherwise, you may add the following line to the beginning of your python scripts:

    import os.path
    import sys
    sys.path.append('/path/to/libkeops')
    
  1. Test your installation, as described in the next section.

Testing your installation

You can use the following test functions to compile and run simple KeOps formulas. If the compilation fails, it returns the full log.

  1. In a python terminal,

import pykeops
pykeops.clean_pykeops()          # just in case old build files are still present
pykeops.test_numpy_bindings()    # perform the compilation

should return:

Compiling libKeOpsnumpyb10acd1892 in /path/to/build_dir/build-libKeOpsnumpyb10acd1892:
   formula: Sum_Reduction(SqNorm2(x - y),1)
   aliases: x = Vi(0,3); y = Vj(1,3);
   dtype  : float64
... Done.

pyKeOps with numpy bindings is working!
  1. If you use PyTorch, the following code:

import pykeops
pykeops.clean_pykeops()          # just in case old build files are still present
pykeops.test_torch_bindings()    # perform the compilation

should return:

Compiling libKeOpstorch2ee7a43993 in /path/to/build_dir/build-libKeOpstorch2ee7a43993:
   formula: Sum_Reduction(SqNorm2(x - y),1)
   aliases: x = Vi(0,3); y = Vj(1,3);
   dtype  : float32
... Done.

pyKeOps with torch bindings is working!

Troubleshooting

Compilation issues

First of all, make sure that you are using a C++ compiler which is compatible with the C++11 revision and/or your nvcc (CUDA) compiler. Otherwise, compilation of formulas may fail in unexpected ways. A table of supported combinations is available at this address. Depending on your system, you can:

  1. Install a compiler system-wide: for instance, on Debian-based Linux distributions, you can install g++ with apt and then use update-alternatives to choose a suitable compiler as default. Don’t forget to pick compatible versions for both gcc and g++.

  2. Install a compiler locally: if you are using a conda environment, you can install a new instance of gcc and g++ by following the documentation of conda.

Cache directory

If you experience problems with compilation, it may be a good idea to flush the build folder that KeOps uses as a cache for already-compiled formulas. To do this, just type:

import pykeops
pykeops.clean_pykeops()

You can change the build folder by using the set_build_folder() function:

import pykeops
print(pykeops.config.bin_folder)  # display default build_folder
pykeops.set_bin_folder("/my/new/location")  # change the build folder
print(pykeops.config.bin_folder)  # display new build_folder

Warning

The build_folder variable must be changed at the beginning of a Python session. That is, before importing any pykeops modules.

Verbosity level

To help debugging, you can activate a verbose compilation mode. This can be done by stting the environment variable PYKEOPS_VERBOSE to 1. In a terminal, type:

export PYKEOPS_VERBOSE=1
python my_script_calling_pykeops.py

Alternatively, you can enable verbose compilation from your python script by setting the flag pykeops.verbose to True after your KeOps imports. In a python shell, type:

import pykeops
pykeops.config.verbose = True

Build type

To force the (re)compilation of KeOps shared objects, you can change the KeOps build type from Release (default) to Debug. This is done by changing the value of the environment variable PYKEOPS_BUILD_TYPE, either in a terminal:

export PYKEOPS_BUILD_TYPE="Debug"
python my_script_calling_pykeops.py

Or directly in your python script, altering the value of the (string) variable pykeops.build_type right after your KeOps imports. In a python shell, simply type:

import pykeops
pykeops.config.build_type = 'Debug'

Warning

Beware! The shared objects generated in debug mode are not optimized for speed and should thus be deleted at the end of your debugging session. In order to do so, please flush your cache directory as described in the previous section.