Installing HoloPy for Developers¶
If you are going to hack on holopy, you probably only want to compile the scattering extensions.
For Mac and Linux:
Download or clone the latest version of HoloPy from Git Hub at https://github.com/manoharan-lab/holopy.
Let’s say you downloaded or cloned HoloPy to
/home/me/holopy. Then open a terminal,
/home/me/holopy and run:
python setup.py build_ext --inplace
This puts the extensions inside the source tree, so that you can work
/home/me/holopy. You will need to add
/home/me/holopy to your python_path for python to find the
module when you import it.
Note for Mac users: gfortran may put its library in a place python can’t find it. If you get errors including something like
can't find /usr/local/libgfortran.3.dynlib you can symlink them in from your install. You can do this by running:
sudo ln -s /usr/local/gfortran/lib/libgfortran.3.dynlib /usr/local/lib sudo ln -s /usr/local/gfortran/lib/libquadmath.3.dynlib /usr/local/lib
For Windows: Installation on Windows is still a work in progress, but we have been able to get HoloPy working on Windows 10 with an AMD64 architecture (64-bit) processor.
Install Anaconda with Python 3.6 and make sure it is working.
Install the C compiler. It’s included in Visual Studio 2015 Community. Make sure it is working with a C helloworld.
From now on, make sure any command prompt window invokes the right environment conditions for compiling with VC. To do this, make sure
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.batis added to the system path variable. This batch detects your architecture, then runs another batch that sets the path include the directory with the correct version of the VC compiler.
Install cython and made sure it works.
Install Intel’s Fortran compiler. A good place to start is the trial version of Parallel Studio XE. Make sure it is working with a Fortran helloworld.
Install mingw32-make, which does not come with Anaconda by default.
Download or clone HoloPy from https://github.com/manoharan-lab/holopy.
Open the command prompt included in Intel’s Parallel Studio. Run
holopy/setup.py. It is necessay to use Intel’s Parallel Studio command prompt to avoid compiling errors.
Install the following dependencies that don’t come with Anaconda:
conda install xarray dask netCDF4 bottleneck conda install -c astropy emcee=2.2.1
Open an iPython console where holopy is installed and try
If the above procedure doesn’t work, or you find something else that does, please let us know so that we can improve these instructions.
How HoloPy Stores Data¶
Images in HoloPy are stored in the format of xarray DataArrays. Spatial
information is tracked in the DataArray’s
according to the HoloPy Coordinate System. Additional dimensions are
sometimes specified to account for different z-slices, times, or field
components, for example. Optical parameters like refractive index and
illumination wavelength are stored in the DataArray’s
detector_grid() function simply creates a 2D image composed entirely
of zeros. In contrast, the
detector_points() function creates a DataArray
with a single dimension named ‘point’. Spatial coordinates (in either Cartesian
or spherical form) track this dimension, so that each data value in the array
has its own set of coordinates unrelated to its neighbours. This type of
one-dimensional organization is sometimes used for 2D images as well. Inference
and fitting methods typically use only a subset of points in an image (see
Speeding up Fits with Random Subset Fitting), and so it makes sense for them to keep track of lists of
location coordinates instead of a grid. Furthermore, HoloPy’s scattering
functions accept coordinates in the form of a 3xN array of coordinates. In both
of these cases, the 2D image is flattened into a 1D DataArray like that created
detector_points(). In this case the single dimension is ‘flat’ instead
of ‘point’. HoloPy treats arrays with these two named dimensions identically,
except that the ‘flat’ dimension can be unstacked to restore a 2D image or 3D
HoloPy’s use of DataArrays sometimes assigns smaller DataArrays in
which can lead to problems when saving data to a file. When saving a DataArray
to file, HoloPy converts any DataArrays in
attrs to numpy arrays, and keeps
track of their dimension names separately. HoloPy’s
save_image() writes a
yaml dump of
attrs (along with spacing information) to the
imagedescription field of .tif file metadata.
Bayesian inference of Parameter Values returns a lot of information, which is stored in the form of a
This object stores the model and
EmceeStrategy that were used in the inference calculation as attributes.
An additional attribute named
dataset is an xarray Dataset
that contains both the data used in the inference calculation, as well as the raw output.
The parameter values at each step of the sampling chain and the calculated log-probabilities at each step are stored here under the
Adding a new scattering theory¶
Adding a new scattering theory is relatively straightforward. You just need to define a new scattering theory class and implement one or two methods to compute the raw scattering values:
class YourTheory(ScatteringTheory): def _raw_fields(self, positions, scatterer, medium_wavevec, medium_index, illum_polarization): # Your code here def _raw_scat_matrs(self, scatterer, pos, medium_wavevec, medium_index): # Your code here def _raw_cross_sections(self, scatterer, medium_wavevec, medium_index, illum_polarization): # Your code here
You can get away with just defining one of _raw_scat_matrs or _raw_fields if you just want holograms, fields, or intensities. If you want scattering matrices you will need to implement _raw_scat_matrs, and if you want cross sections, you will need to implement _raw_cross_sections. We seperate out _raw_fields from _raw_scat_matrs because we want to provide a faster fields implementation for mie and multisphere (and you might want to for your theory).
You can look at the Mie theory in HoloPy for an example of calling Fortran functions to compute scattering (C functions will look similar from the python side) or DDA for an an example of calling out to an external command line tool by generating files and reading output files.
Adding a new inference model¶
To perform inference, you need a noise model. You can make a new noise model by inheriting from
NoiseModel. This class has all the machinery to compute likelihoods of observing data given some set of parameters and assuming Gaussian noise.
To implement a new model, you just need to implement one function:
This function receives a dictionary of parameter values and a data shape schema (defined by
detector_grid(), for example) and needs to return simulated data of shape specified. See the
_forward function in
AlphaModel for an example of how to do this.
If you want to use some other noise model, you may need to override _lnlike and define the probablity given your uncertainty. You can reference _lnlike in
HoloPy comes with a suite of tests that ensure everything has been
built correctly and that it’s able to perform all of the calculations
it is designed to do. To run these tests, navigate to the root of the
/home/me/holopy) and run: