Data set for Noise reduction of coherent diffraction image data using deep learning method

About

This repository provides the simulation data, learning programs, and learning results data introduced in T. Ishikawa et al. "Sub-photon accuracy noise reduction of single shot coherent diffraction pattern with atomic model trained autoencoder", arXiv:2403.11992 (2024).

License

CC BY 4.0. We would appreciate it if you could cite the paper above when you use the data, the documents and the scripts for your scientific works.

T. Ishikawa et al. "Sub-photon accuracy noise reduction of single shot coherent diffraction pattern with atomic model trained autoencoder", arXiv:2403.11992 (2024).

BibTex:

@misc{ishikawa2024subphoton, title={Sub-photon accuracy noise reduction of single shot coherent diffraction pattern with atomic model trained autoencoder}, author={Takuto Ishikawa and Yoko Takeo and Kai Sakurai and Kyota Yoshinaga and Noboru Furuya and Yuichi Inubushi and Kensuke Tono and Yasumasa Joti and Makina Yabashi and Takashi Kimura and Kazuyoshi Yoshimi}, year={2024}, eprint={2403.11992}, archivePrefix={arXiv}, primaryClass={physics.optics} }

Folder structure

-License
-ReadME.txt
-data
  -ref
    -simulation_data
        -triangle.npy
        -ellipse.npy
        -random.npy
    -mask_data
        -w.csv
    -learned_parameter_triangle
        -nn.h5
        -cnn.h5
        -unet.h5
-program
    -padding_simulation_data
        -add_noise.py
    -learning
        -nn.py
        -cnn.py
        -unet.py
    -denoise_using_learned_model
        -denoise_nn.py
        -denoise_cnn.py
        -denoise_unet.py

Contact

takuto-ishikawa__at__issp.u-tokyo.ac.jp k-yoshimi__at__issp.u-tokyo.ac.jp kimura__at__edm.t.u-tokyo.ac.jp

Please replace at with @.

How to use these program

1. Program operating environment python3, numpy, scipy, tensorflow (or tensorflow gpu)

2. Padding the simulation data

   Prepare these data and put the same directry

       -add_noise.py
       -triangle.npy
       -w.csv
   

   Please type the following command: $ python3 add_noise.py

   You can get 2000 noisy-data "x_test.npy" and correct-data "x_test_correct.npy" for validation and 20000 noisy-data "x_train.npy" and correct-data "x_train_correct.npy" for learning. If you want to change the shape or mask data, change the input data in line 17 or line 19, respectively.

3. Learning process

   Prepare these data and put the same directry

       -unet.py
       -x_train.npy
       -x_train_correct.npy
       -x_test.npy
       -x_test_correct.npy
   

   Please type the following command:

   $ python3 unet.py

   You can get the parameter "autoencoder.h5" as the learning result, the value "Train_Loss.dat" "Valid_Loss.dat" of the Loss function of the learning process, and 100 correct-images "true_data.npy" noisy-images "noisy_data.npy" decoded-images "decoded_data.npy" for the simulation data not used for learning. If you want to change the model, change the unet.py to other program. If you want to do transfer learning, change "check=0" to "check=1" in unet.py and prepare the learned parameter "autoencoder_data20_epoch50_batch20.h5".

4. Denoise the experimental data

   Prepare these data and put the same directry

   -denoise_unet.py
   -autoencoder_data20_epoch50_batch20.h5
   -experimental_data.npy
   

   Please type the following command: $ python3 denoise_unet.py

   You can get decoded-data "decoded_data.npy". If you want to change the model, change the denoise_unet.py to other program.

Correspondence with figures in the paper

• "2. Learning process" made the Fig. 5 and Fig. 6

• "2. Learning process for transfer learning" made the Fig. 7 and Fig. 8

• "3. Denoise the experimental data" made the Fig. 10

  We put the parameters obtained by "2. Learning process" in the directory "data/ref/learned_parameter_triangle".