.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "examples/processing/filtering_data.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_examples_processing_filtering_data.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_examples_processing_filtering_data.py:


Filtering Data (Averaging Neighboring Patterns etc.)
====================================================

If you have a low number of counts in your data, you may want to filter the data
to remove noise. This can be done using the `filter` function which applies some
function to the entire dataset and returns a filtered dataset of the same shape.

In this example, we will use the `filter` function to apply a Gaussian filter in
only the real space dimensions of the dataset. This is useful for removing noise
as it acts like a spatial bandpass filter for high frequencies (noise) in the dataset.

Because the STEM probe is Gaussian-like, and the convolution of two Gaussian functions
is another Gaussian function, the Gaussian filter is a good choice for filtering and is
equivalent to aquiring the data with a larger probe size equal to
:math:`\sigma_{Filtererd} = \sqrt{\sigma_{Beam}^2 + \sigma_{Filter}^2}`.  The benefit is
that the total aquisition time is much shorter than if the probe size was increased to
reach the same number of counts.  For detectors with low saturation counts, this can be
a significant advantage.

.. GENERATED FROM PYTHON SOURCE LINES 21-45

.. code-block:: Python


    from scipy.ndimage import gaussian_filter
    from dask_image.ndfilters import gaussian_filter as dask_gaussian_filter
    import pyxem as pxm
    import hyperspy.api as hs
    import numpy as np

    s = pxm.data.tilt_boundary_data()

    s_filtered = s.filter(
        gaussian_filter, sigma=1.0, inplace=False
    )  # Gaussian filter with sigma=1.0

    s_filtered2 = s.filter(
        gaussian_filter, sigma=(1.0, 1.0, 0, 0), inplace=False
    )  # Only filter in real space

    hs.plot.plot_images(
        [s.inav[3, 3], s_filtered.inav[3, 3], s_filtered2.inav[3, 3]],
        label=["Original", "GaussFilt(all)", "GaussFilt(real space)"],
        tight_layout=True,
        vmax="99th",
    )




.. image-sg:: /examples/processing/images/sphx_glr_filtering_data_001.png
   :alt: Original, GaussFilt(all), GaussFilt(real space)
   :srcset: /examples/processing/images/sphx_glr_filtering_data_001.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    [<Axes: title={'center': 'Original'}, xlabel='kx axis (px)', ylabel='ky axis (px)'>, <Axes: title={'center': 'GaussFilt(all)'}, xlabel='kx axis (px)', ylabel='ky axis (px)'>, <Axes: title={'center': 'GaussFilt(real space)'}, xlabel='kx axis (px)', ylabel='ky axis (px)'>]



.. GENERATED FROM PYTHON SOURCE LINES 46-48

The `filter` function can also be used with a custom function as long as the function
takes a numpy array as input and returns a numpy array of the same shape.

.. GENERATED FROM PYTHON SOURCE LINES 48-63

.. code-block:: Python



    def custom_filter(array):
        filtered = gaussian_filter(array, sigma=1.0)
        return filtered - np.mean(filtered)


    s_filtered3 = s.filter(custom_filter, inplace=False)  # Custom filter

    hs.plot.plot_images(
        [s.inav[3, 3], s_filtered3.inav[3, 3]],
        label=["Original", "GaussFilt(Custom)"],
        tight_layout=True,
        vmax="99th",
    )



.. image-sg:: /examples/processing/images/sphx_glr_filtering_data_002.png
   :alt: Original, GaussFilt(Custom)
   :srcset: /examples/processing/images/sphx_glr_filtering_data_002.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    [<Axes: title={'center': 'Original'}, xlabel='kx axis (px)', ylabel='ky axis (px)'>, <Axes: title={'center': 'GaussFilt(Custom)'}, xlabel='kx axis (px)', ylabel='ky axis (px)'>]



.. GENERATED FROM PYTHON SOURCE LINES 64-67

For lazy datasets, functions which operate on dask arrays can be used. For example,
the `gaussian_filter` function from `scipy.ndimage` is replaced with the `dask_image`
version which operates on dask arrays.

.. GENERATED FROM PYTHON SOURCE LINES 67-80

.. code-block:: Python


    s = s.as_lazy()  # Convert to lazy dataset
    s_filtered4 = s.filter(
        dask_gaussian_filter, sigma=1.0, inplace=False
    )  # Gaussian filter with sigma=1.0

    hs.plot.plot_images(
        [s_filtered.inav[3, 3], s_filtered4.inav[3, 3]],
        label=["GaussFilt", "GaussFilt(Lazy)"],
        tight_layout=True,
        vmax="99th",
    )




.. image-sg:: /examples/processing/images/sphx_glr_filtering_data_003.png
   :alt: GaussFilt, GaussFilt(Lazy)
   :srcset: /examples/processing/images/sphx_glr_filtering_data_003.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    [<Axes: title={'center': 'GaussFilt'}, xlabel='kx axis (px)', ylabel='ky axis (px)'>, <Axes: title={'center': 'GaussFilt(Lazy)'}, xlabel='kx axis (px)', ylabel='ky axis (px)'>]




.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 1.315 seconds)


.. _sphx_glr_download_examples_processing_filtering_data.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: filtering_data.ipynb <filtering_data.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: filtering_data.py <filtering_data.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: filtering_data.zip <filtering_data.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_