Fourier Ptychography with the OpenFlexure Microscope

General
1

Hi everyone, as a follow-up to Has anyone successfully done Fourier ptychography using the Delta Stage - #10 by rooksoup I thought I’d share my approach of doing Fourier Ptychographic Microscopy with the OpenFlexure microscope. It’s all independent spare time low-budget work that I mostly did at CoMakingSpace Heidelberg. Naturally, this means that I did not have ideal conditions for imaging / handling the optical components and that I didn’t have enough time to dig into all the different components that I used. I’d appreciate comments, especially if I adapted the OpenFlexure software correctly into my workflow and where it could be improved.

Following the recommendation by @rooksoup, I changed my LED matrix mount to the one used in OpenFPM ( GitHub - YoItsLewis/Open-Fourier-Ptychographic-Microscopy: Open Fourier Ptychographic Microscopy (Open FPM) is an open-source, low-cost platform for high-resolution computational microscopy. · GitHub ) with modifications for a commercial XY stage. For the first time now I got good reconstructions from the 9 most central brightfield images. It is very hard to capture any signal under darkfield illumination of just one activated LED with the setup I have, and so I didn’t manage to do FPM with higher illumination angles.

Hardware

  • OpenFlexure microscope v7 beta2, high-resolution microscope
    • LED matrix mount modified from OpenFPM (using FreeCAD) with an adapter to an XY stage from aliexpress
    • Extended optics module to 160mm mechanical tube length and removed the tube lens Assembly Instructions (OpenSCAD)
    • Additional microscope stand to accommodate to the extended optics module (Onshape)
    • Default camera (Raspberry Pi Camera Module v2)
  • Pimoroni Unicorn HAT HD
  • 4X NA 0.1 objective
  • USAF 1951 target

Modified OpenFlexure Microscope

setup
setup1920×1477 291 KB

Software

  • I created an extension to the OpenFlexure Microscope Server for LED selection and running FPM acquisition series
    • Image capturing with use_video_port=False, bayer=True
    • Pause time between acquisitions to prevent camera freeze
    • Take multiple images per LED position to exceed maximum possible exposure time
    • Take background images with no LED active
  • ImageMagick and Fiji for processing the acquired images
  • FPMApp for reconstructions Release FPM app 1.1 · MRogalski96/FPM-app · GitHub
    • I don’t have a matlab license - installed FPMApp with the released installer on windows, then copied over to linux and ran with wine

Workflow

1. Acquisition in the OpenFlexure software

  • No calibrations
  • Analog gain=1, digital gain=1
  • LED centering by comparing the different brightfield/darkfield transition images
  • Set exposure time to match LED PWM frequency (don’t get visible horizontal stripes on images)

2. Image processing

  • Extract channel of interest from Bayer DNG image with rawpy and by pixel subsampling
  • Sum projection of images for each LED position
  • Background subtraction for summed images of each LED position

3. Fourier ptychography reconstruction in FPMApp

  • Select ROI, acquisition order, central LED
  • Enter parameters of the FPM setup

Example reconstruction from 9 brightfield images

reconstruction
reconstruction1830×643 169 KB

FFT of raw image and recovered image

fft_comparison
fft_comparison611×340 39.7 KB

System parameters in FPMApp

  • Position correction with Authors method 2 (fast)
  • 50 epochs
  • 9 images, all brightfield
  • 64mm
  • Synthetic NA: 0.24
  • 620nm wavelength

Limitations

  • Default v2 camera introduces a lot of background noise
  • The selected exposure time probably does not perfectly remove the PWM flicker from the acquired images, a global shutter camera could address this better
  • Cannot process darkfield angles, probably due to a too low quality camera or not having an optics room and vibration isolation table

Remarks / questions

I couldn’t find a definitive source for the following points

  • Exposure time in the OpenFlexure GUI. Is it possible to take pictures at the full ~10s exposure time supported by the v2 camera? Apparently it is limited by the camera FPS, but that should only apply to the use_video_port=True and not the capture pane. Also, when I set an exposure time, it is always modified to a different value close to the one I entered. It is not clear to me, by what rule.
  • With analog gain>1, sometimes the camera image does not update between acquisitions (automated batch acquisition in a custom extension) and I have to add a few hundred ms pause in between acquisitions. Not sure if this is normal behavior or an issue with the microscope software.
4 Likes
2

Thank you for sharing @lennartg