PyLops
🖥️ Installation¶
Dependencies¶
The PyLops project strives to create a library that is easy to install in any environment and has a very limited number of dependencies. Required dependencies are limited to:
We encourage using the Anaconda Python distribution
or its standalone package manager Conda.
Especially for Intel processors, this ensures a higher performance with no configuration (e.g.,
the linking to Intel MKL library, a highly optimized BLAS library created by Intel).
If you are interested in getting the best code performance, read carefully Advanced installation.
For learning, however, the standard installation is often good enough; in that case, we recommend using uv, a modern Python package manager that is easy to use and has a very fast dependency resolver.
Some operators have additional, optional engines that are usually meant to provide improved performance on CPU or enable GPU acceleration. These rely on third-party libraries, which are added to the list of our optional dependencies and must be installed to be able to use the associated engine. Similarly, some operators are implemented on top of third-party libraries, which are also added to the list of our optional dependencies and must be installed to be able to use the associated operator. In both cases, if the dependency is not installed, the rest of the library will still work. For details more details, see Optional dependencies.
Step-by-step installation for users¶
From Package Manager¶
First install pylops with your package manager of choice.
>> conda install --channel conda-forge pylops
which installs also the required dependencies, if not already present in your environment.
>> uv add pylops
which installs also the required dependencies, if not already present in your environment. Refer to Optional dependencies for alternative uv commands that install some of the optional dependencies as well.
From Source¶
To access the latest source from GitHub:
>> pip install https://github.com/PyLops/pylops.git@dev
>> uv add git+https://github.com/PyLops/pylops.git --branch dev
Docker¶
If you want to try PyLops but do not have Python in your local machine, you can use our Docker image instead.
After installing Docker in your computer, type the following command in your terminal (note that this will take some time the first time you type it as you will download and install the Docker image):
>> docker run -it -v /path/to/local/folder:/home/jupyter/notebook -p 8888:8888 mrava87/pylops:notebook
This will give you an address that you can put in your browser and will open a Jupyter notebook environment with PyLops
and other basic Python libraries installed. Here, /path/to/local/folder is the absolute path of a local folder
on your computer where you will create a notebook (or containing notebooks that you want to continue working on). Note that
anything you do to the notebook(s) will be saved in your local folder.
A larger image with conda a distribution is also available:
>> docker run -it -v /path/to/local/folder:/home/jupyter/notebook -p 8888:8888 mrava87/pylops:conda_notebook
Step-by-step installation for developers¶
Fork PyLops¶
Fork the PyLops repository and clone it by executing the following in your terminal:
>> git clone https://github.com/YOUR-USERNAME/pylops.git
Install dependencies¶
We recommend installing dependencies into a separate environment. For that end, we provide a Makefile with useful commands for setting up the environment.
>> make dev-install_conda # for x86 (Intel or AMD CPUs)
>> make dev-install_conda_arm # for arm (M-series Mac)
This creates and activate an environment called pylops, with
all required and optional dependencies.
>> make dev-install_uv
This creates a virtual environment .venv that can be activated at any time with source .venv/bin/activate (Linux/macOS).
Run tests¶
To ensure that everything has been setup correctly, run tests:
>> make tests
>> make tests_uv
Make sure no tests fail, this guarantees that the installation has been successful.
Add remote (optional)¶
To keep up-to-date on the latest changes while you are developing, you may optionally add the PyLops repository as a remote. Run the following command to add the PyLops repo as a remote named upstream:
>> git remote add upstream https://github.com/PyLops/pylops
From then on, you can pull changes (for example, in the dev branch) with:
>> git pull upstream dev
Install pre-commit hooks¶
To ensure consistency in the coding style of our developers we rely on
pre-commit to perform a series of checks when you are
ready to commit and push some changes. This is accomplished by means of git hooks
that have been configured in the .pre-commit-config.yaml file.
In order to setup such hooks in your local repository, run:
>> pre-commit install
>> uv run pre-commit install
Once this is set up, when committing changes, pre-commit will reject and “fix” your code by running the proper hooks.
At this point, the user must check the changes and then stage them before trying to commit again.
Final steps¶
PyLops does enforce the use of a linter (ruff), which is run both as a pre-commit hook and as a GitHub Action.
The linter can also be run locally with:
>> make lint
>> make lint_uv
In addition, it is highly encouraged to build the docs prior to submitting a Pull Request. Apart from ensuring that docstrings are properly formatted, they can aid in catching bugs during development. Build (or update) the docs with:
>> make doc
>> make doc_uv
or
>> make docupdate
>> make docupdate_uv
Advanced installation¶
In this section we discuss some important details regarding code performance when using PyLops.
To get the most out of PyLops operators in terms of speed you will need to follow these guidelines as much as possible or ensure that the Python libraries used by PyLops are efficiently installed in your system.
BLAS¶
PyLops relies on the NumPy and SciPy, and being able to link these to the most performant BLAS library will ensure optimal performance of PyLops when using only required dependencies.
We strongly encourage using the Anaconda Python distribution as
NumPy and SciPy will, when available, be automatically linked to Intel MKL, the most performant library for basic linear algebra
operations to date.
The PyPI version installed with pip, however, will default to OpenBLAS.
For more information, see NumPy’s section on BLAS.
To check which BLAS NumPy and SciPy were compiled against, run the following commands in a Python interpreter:
import numpy as np
import scipy as sp
print(np.__config__.show())
print(sp.__config__.show())
Intel also provides NumPy and SciPy replacement packages in PyPI, namely intel-numpy and intel-scipy, which link to Intel MKL.
These are an option for an environment without conda that needs Intel MKL without requiring manual compilation.
Warning
intel-numpy and intel-scipy not only link against Intel MKL, but also substitute NumPy and
SciPy FFTs with Intel MKL FFT.
Multithreading¶
It is important to ensure that your environment variable which sets threads is correctly assigned to the maximum number of cores you would like to use in your code. Multiprocessing parallelism in NumPy and SciPy can be controlled in different ways depending on where it comes from.
Environment variable |
Library |
|---|---|
OMP_NUM_THREADS |
|
NUMEXPR_NUM_THREADS |
|
OPENBLAS_NUM_THREADS |
|
MKL_NUM_THREADS |
|
VECLIB_MAXIMUM_THREADS |
For example, try setting one processor to be used with (if using OpenBlas)
>> export OMP_NUM_THREADS=1
>> export NUMEXPR_NUM_THREADS=1
>> export OPENBLAS_NUM_THREADS=1
and run the following code in Python:
import os
import numpy as np
from timeit import timeit
size = 1024
A = np.random.random((size, size)),
B = np.random.random((size, size))
print("Time with %s threads: %f s" \
%(os.environ.get("OMP_NUM_THREADS"),
timeit(lambda: np.matmul(A, B), number=4)))
Subsequently set the environment variables to 2 or any higher number of threads available
in your hardware (multi-threaded), and run the same code.
By looking at both the load on your processors (e.g., using top), and at the
Python print statement you should see a speed-up in the second case.
Alternatively, you could set the OMP_NUM_THREADS variable directly
inside your script using os.environ["OMP_NUM_THREADS"]="2", but ensure that
this is done before loading NumPy.
Note
Always remember to set OMP_NUM_THREADS and other relevant variables
in your environment when using PyLops
Optional dependencies¶
To avoid increasing the number of required dependencies, which may lead to conflicts with other libraries that you have in your system, we have decided to build some of the additional features of PyLops in such a way that if an optional dependency is not present in your Python environment, a safe fallback to one of the required dependencies will be enforced.
Note
If you are a developer, all the optional dependencies below (except GPU) can
be installed automatically by cloning the repository and installing
PyLops via make dev-install_conda (conda) or
make dev-install_uv (uv). GPU-enabled equivalents are
make dev-install_conda_gpu (conda) and
make dev-install_uvcu126 / dev-install_uvcu128 / dev-install_uvcu13 (uv)
When using the Conda package manager, only the required dependencies will be installed when installing PyLops. It is recommended to install the optional dependencies manually before installing PyLops or as part of the creation of the environment via an environment.yml file.
Alternatively, from version v1.4.0 some of the optional dependencies can be
installed as part of the pip installation via (see summary table below for details):
>> pip install pylops[advanced]
>> uv add "pylops[advanced]"
Finally, from version 2.7.0, all of the optional dependencies can be installed as part of
the pip installation via (see summary table below for details):
>> pip install pylops[advanced, stat, deep] # CPU
>> pip install pylops[advanced, stat, gpu-cu12, deep-cu126] # GPU with CUDA 12.6
>> pip install pylops[advanced, stat, gpu-cu12, deep-cu128] # GPU with CUDA 12.6
>> pip install pylops[advanced, stat, gpu-cu13, deep-cu13] # GPU with CUDA 13.0
>> uv add "pylops[advanced, stat, deep]" # CPU
>> uv add "pylops[advanced, stat, gpu-cu12, deep-cu126]" # GPU with CUDA 12.6
>> uv add "pylops[advanced, stat, gpu-cu12, deep-cu128]" # GPU with CUDA 12.6
>> uv add "pylops[advanced, stat, gpu-cu13, deep-cu13]" # GPU with CUDA 13.0
In all cases, dependencies are installed from their PyPI wheels.
A summary table of all optional dependencies, the operators that rely on them (and whether they are required to be able to use the operator(s)), and how to install them as part of the installation process of PyLops provided in the table below.
Dependency |
Operator(s) affected |
Required |
Install with |
|---|---|---|---|
ASTRA |
✅ |
pip install pylops[advanced] / uv add “pylops[advanced]” |
|
dtcwt |
✅ |
pip install pylops[advanced] / uv add “pylops[advanced]” |
|
Devito |
✅ |
pip install pylops[advanced] / uv add “pylops[advanced]” |
|
FFTW |
|
🔴 |
pip install pylops[advanced] / uv add “pylops[advanced]” |
MKL-FFT |
|
🔴 |
N/A (see below for installation instructions) |
Numba |
|
🔴 |
pip install pylops[advanced] / uv add “pylops[advanced]” |
PyMC and PyTensor |
✅ |
pip install pylops[stat] / uv add “pylops[stat]” |
|
PyWavelets |
|
🔴 |
pip install pylops[advanced] / uv add “pylops[advanced]” |
scikit-fmm |
✅ |
pip install pylops[advanced] / uv add “pylops[advanced]” |
|
SPGL1 |
✅ |
pip install pylops[advanced] / uv add “pylops[advanced]” |
|
Sympy |
|
✅ |
pip install pylops[advanced] / uv add “pylops[advanced]” |
Torch |
✅ |
pip install pylops[deep] / uv add “pylops[deep]” (or GPU equivalents) |
|
CuPy |
Almost all operators (see 🎮 GPU / TPU Support for details) |
🔴 |
pip install pylops[gpu-cu12] / uv add “pylops[gpu-cu12]” |
JAX |
|
🔴 |
pip install pylops[deep] / uv add “pylops[deep]” (or GPU equivalents) |
More details about the installation process for the different optional dependencies are described in the following:
ASTRA¶
ASTRA is library used to perform computerized
tomography. It is used in PyLops in the operator pylops.medical.CT2D
To use this library, install it via:
>> conda install --channel astra-toolbox astra-toolbox
>> uv add astra-toolbox
dtcwt¶
dtcwt is a library used to implement the DT-CWT operators.
Install it via:
>> uv add dtcwt
Warning
dtcwt does not support NumPy 2 yet, so make sure you use NumPy 1.x
to be able to use the DTCWT operator.
Devito¶
Devito is a library used to solve PDEs via
the finite-difference method. It is used in PyLops to compute wavefields
pylops.waveeqprocessing.AcousticWave2D
Install it via:
>> uv add devito
FFTW and MKL-FFT¶
Four different “engines” are provided by the pylops.signalprocessing.FFT operator:
engine="numpy" (default), engine="scipy", engine="fftw" and engine="mkl_fft".
Similarly, the pylops.signalprocessing.FFT2D and
the pylops.signalprocessing.FFTND operators come with three “engines”, namely
engine="numpy" (default), engine="scipy", and engine="mkl_fft".
The first two engines are part of the required PyLops dependencies.
The third implements the well-known FFTW
via the Python wrapper pyfftw.FFTW. While this optimized FFT tends to
outperform the other two in many cases, it is not included by default.
To use this library, install it via:
>> conda install --channel conda-forge pyfftw
>> uv add pyfftw
The fourth implements Intel MKL FFT via the Python interface mkl_fft. This provides access to Intel’s oneMKL Fourier Transform routines, enabling efficient FFT computations with performance close to native C/Intel® oneMKL
To use this library, you can install it via:
>> conda install --channel https://software.repos.intel.com/python/conda --channel conda-forge mkl_fft
>> uv add mkl_fft --index-url https://software.repos.intel.com/python/pypi --extra-index-url https://pypi.org/simple
Installing mkl-fft triggers the installation of Intel-optimized versions of NumPy and
SciPy, which redirects numpy.fft and scipy.fft to use MKL FFT routines.
As a result, all FFT operations and computational backends leverage Intel MKL for optimal performance.
Although the library can run without Intel-optimized NumPy and SciPy, maximum performance is achieved when using NumPy and SciPy built with Intel’s Math Kernel Library (MKL) alongside Intel Python.
Note
mkl_fft is not supported on macOS.
Warning
pyFFTW may not work correctly with NumPy + MKL. To avoid issues, it is recommended to build pyFFTW from
source after setting the STATIC_FFTW_DIR environment variable to the absolute path of the static FFTW
libraries.
If the following environment variables are set before installing pyFFTW, compatibility problems with MKL
should not occur:
export STATIC_FFTW_DIR=${PREFIX}/lib(where${PREFIX}is the base of the current Anaconda environment with thefftwpackage installed)export CFLAGS="$CFLAGS -Wl,-Bsymbolic"
Alternatively, you can install pyFFTW directly with conda, since the updated recipe is already available
and works without any manual adjustments.
Numba¶
Although we always strive to write code for forward and adjoint operators that takes advantage of the perks of NumPy and SciPy (e.g., broadcasting, ufunc), in some case we may end up using for loops that may lead to poor performance. In those cases we may decide to implement alternative (optional) back-ends in Numba, a Just-In-Time compiler that translates a subset of Python and NumPy code into fast machine code.
A user can simply switch from the native,
always available implementation to the Numba implementation by simply providing the following
additional input parameter to the operator engine="numba". This is for example the case in the
pylops.signalprocessing.Radon2D.
If interested to use Numba backend, install it via:
>> conda install numba
>> conda install --channel numba icc_rt # optional
>> uv add numba
>> uv add icc_rt # optional
Note that it is also advised to install the additional package icc_rt to use optimised transcendental functions as compiler intrinsics.
However, it is important to note that icc_rt will only be identified by Numba if
LD_LIBRARY_PATH is properly set.
If you are using a virtual environment, you can ensure this with:
>> export LD_LIBRARY_PATH=/path/to/venv/lib/:$LD_LIBRARY_PATH
To ensure that icc_rt is being recognized, run
>> numba -s | grep SVML
__SVML Information__
SVML State, config.USING_SVML : True
SVML Library Loaded : True
llvmlite Using SVML Patched LLVM : True
SVML Operational : True
Numba also offers threading parallelism through a variety of Threading Layers.
You may need to set the environment variable NUMBA_NUM_THREADS define how many threads to use out of the available ones (numba -s | grep "CPU Count").
It can also be checked dynamically with numba.config.NUMBA_DEFAULT_NUM_THREADS.
PyMC and PyTensor¶
PyTensor is used to allow seamless integration between PyLops and PyMC operators. Install both of them with:
>> conda install -c conda-forge pytensor pymc
>> uv add pytensor pymc
Warning
OSX users may experience a CompileError error when using PyTensor. This can be solved by adding
pytensor.config.gcc__cxxflags = "-Wno-c++11-narrowing" after import pytensor.
PyWavelets¶
PyWavelets is used to implement the wavelet operators. Install it via:
>> conda install pywavelets
>> uv add PyWavelets
scikit-fmm¶
scikit-fmm is a library which implements the
fast marching method. It is used in PyLops to compute traveltime tables in the
initialization of pylops.waveeqprocessing.Kirchhoff
when choosing mode="eikonal". As this may not be of interest for many users, this library has not been added
to the mandatory requirements of PyLops. Install it via
>> conda install --channel conda-forge scikit-fmm
>> uv add scikit-fmm
SPGL1¶
SPGL1 is used to solve sparsity-promoting
basis pursuit, basis pursuit denoise, and Lasso problems
in pylops.optimization.sparsity.SPGL1 solver.
Install it via:
>> uv add spgl1
Sympy¶
This library is used to implement the describe method, which transforms
PyLops operators into their mathematical expression.
Install it via:
>> conda install sympy
>> uv add sympy
Torch¶
Torch is used to allow seamless integration between PyLops and PyTorch operators.
Install it via:
>> conda install -c pytorch pytorch
>> uv add torch
Optional Dependencies for GPU¶
PyLops will automatically check if the libraries below are installed and, in that case, use them any time the input vector passed to an operator is of compatible type. Users can, however, disable this option. For more details of GPU-accelerated PyLops read 🎮 GPU / TPU Support.
CuPy¶
CuPy is a library used as a drop-in replacement to NumPy and some parts of SciPy for GPU-accelerated computations. Since many different versions of CuPy exist (based on the CUDA drivers of the GPU), users must install CuPy prior to installing PyLops. To do so, follow their installation instructions.
JAX¶
JAX is another library that can be used as a drop-in replacement to NumPy and some parts of SciPy. It provides seamless support for multiple accelerators (e.g., GPUs, TPUs), Just-In-Time (JIT) compilation via Open XLA, and Automatic Differentiation. Similar to CuPy, since many different versions of JAX exist (based on the CUDA drivers of the GPU), users must install JAX prior to installing PyLops. To do so, follow their installation instructions.