Multi-Exposure Image Fusion Based on Illumination Estimation

The low dynamic range of imaging equipment compared to the dynamic range of real settings is one of the most significant issues with current imaging technology. A camera's dynamic range, or the range between its brightest and darkest pixels, is often far less than the range between brightest and darkest points in a genuine picture. Since portions of the picture will always be underexposed, overexposed, or both, it is difficult to capture an HDR scene with a single image exposure. As may be seen in Fig. 1. Taking many photographs of the same scene at various exposure settings might be a workable solution to this issue because to the greater dynamic range that results from doing so. The succession of images will include some pixels that are appropriately exposed and others that are under or overexposed. One or more of the photos in the series, however, will have correctly exposed pixels (Fig. 1). After capturing the series of images, two methods are available I HDR imaging, and ii) exposure fusion.

As a result of the many ecological services they provide, including carbon dioxide (CO2) sequestration, oxygen generation, and silica cycling diatoms are widely acknowledged to play an important role in ecosystem functioning. Only 0.78 percent of Earth's surface is covered by freshwater, while the oceans take up the remaining 70.2 percent. Diatoms, a kind of microorganism, are essential to the stability of aquatic ecosystems because to their unique biological properties. Their short life cycles and high rates of environmental adaptation make them ideal sentinels for detecting changes in ecosystem health. Diatomite, the fossils found on the characteristics. Since diatom research has so many potential uses, it continues to pique the attention of both fundamental and applied scientists. Though diatoms have been called "a wonderful natural work of art," their study requires a thorough look at various microscopic microstructures that are difficult to see at low magnification. Due to their extreme sensitivity to their surrounding environment, diatoms may be utilized to detect changes in water quality and to get information on ecological state at a level of detail unattainable by more traditional water chemistry methods. Diatom studies have also found use in forensics and the oil industry. All of these uses call for a thorough survey of the sample in question, numbering and labeling all of the different species that are there. Therefore, the specimen is thoroughly analyzed before automatic identification is performed. In this study, we will outline a system for preserving fine detail in images that were captured with many exposures. Today's optical microscopes can take many photos of a specimen and analyze them automatically to determine which ones are best. Diatoms are tiny algae that are categorized according to the pore patterns and contours of their shells. Different types of microscopies, such as bright field and dark field microscopy, may capture different but equally important images of diatoms due to their wide diversity of shapes and sizes.

Wenlong Zhang (2018) This research proposes a novel wavelet-based approach to multi-exposure picture fusion. The luminance inversion is mitigated, and the contrast of input photographs is improved, by factoring in the input photos' brightness as part of the well-exposedness weight. The merged picture is much better. The weight is used for fusing in the wavelet domain. those approximated input image bands that make up the whole. Similarly, the modified contrast When using weight, the input pictures' detail sub-bands are fused to prevent information loss around sharp edges. Additionally, a new detail-enhancement option is shown. The concept of the merged picture was put out. First, by adjusting the input photos such that they are in YUV space, using the saturation value to blend the U and V color difference components; by (2) wavelet-domain-transforming the luminance-component Y and merging the pairs of well-exposedness-related approximation and detail sub-bands adjusted contrast weight, and the body weight; (3) modifying the fused red, green, and blue space t Experiments indicate that the recommended strategy efficiently retains features, increases contrast, and maintains the brightness distribution of input pictures consistent. Fang Xu (2022) Research into multi-exposure image fusion (MEF) is gaining momentum in the fields of an easy way to improve the imaging system's dynamic range without breaking the bank. wide-ranging, useful in a variety of contexts. There has been a lot of development in this area. topic in recent years, because of the progress made in representing visual information on the web. multi-scale analysis and deep learning are examples of such ideas. Currently, this study examines examine the current research landscape for MEF methods. Key technology and related theories components used to construct MEF models are organized and examined. Average MEF by Type Some Techniques are Presented and Summarized. We then provide an analysis of the differences. for representative MEF methods with 9 commonly used goal fusion criterion, based on sequences of still and moving images with many exposures. Finally, the most pressing problems in the field of MEF research are outlined, and a research A long-term pattern in development is postulated. Liu Huang (2021) The authors provide a novel approach to merging photographs with multiple exposures, one that is based on feature assessment with a variable factor. The current multi-exposure fusion approach has been shown to be unsuitable for many types of input images. which are either very light or extremely dark, that the resulting picture quality from fusing them is poor, and that the details are not entirely maintained. In light of this, an accommodating factor is provided. to adjust the brightness of incoming photos. To find out, we utilize a sliding glass to the significance of evaluating exposure, the significance of evaluating changes in texture, and the significance of evaluating changes in color intensity. Finally, a pyramid is used to join the images together seamlessly. There were twenty separate instances of exposure. selected input photographs from a variety of scenarios, subjective and objective evaluations different approaches for fusing images from several exposures are examined and evaluated. According on the findings of the experiments, the suggested method may significantly improve the quality of static sceneries. save more information and create pleasing visual effects. Nail Hayat (2019) Based on the dense SIFT descriptor and the guided filter, this paper suggests a strategy for fusing images from many exposures without producing ghostly results. The results demonstrate that regular cameras may provide high-quality images using the suggested method. and also preventing the ghosting problem. This article uses the dense SIFT descriptor to extract neighborhood features. To do this, we need contrast information from the original photographs. The disparity in hues Histogram median and equalization are used to determine the feature for changing scenarios. filtering. Initial estimates of the weights are calculated using the following three weighting So, the initial weights are sent into the guided filter in an effort to lessenchaotic and erratic. In the end, the pyramid decomposition technique is employed to carry outmerge into one. According to both subjective and objective criteria, the outcomes of the experimentsdemonstrate that the proposed approach is superior to the current best practices. Yuma Kinoshita (2019) Here, we provide a unique method for adjusting brightness in the context of multi-exposure picture fusion. Two novel techniques for scene segmentation using brightness distribution are also offered for the tweak. Specifically, this is done by multi-exposure picture fusion is the process of combining many photographs with varying exposures to create a single composite image. These are meant to outperform all others in terms of informational content and visual attractiveness of the source pictures. On the other hand, existing fusion methods typically result in result in hazy merged pictures if the input photos don't have enough quantification of the amount of danger posed. The quality of the resulting fused pictures may be increased by adjusting the input image brightness, as we demonstrate in this research. The basis of the suggested approach based on this understanding. The suggested strategy enables us to do this even when presented with imperfect data. create professional-grade snapshots. The suggested approach may give images that clearly visual comparison findings that include the whole scenario. Moreover, in respect to image quality in terms of the MEF-structural similarity score, discrete entropy, and tone mapping multi-exposure picture fusion using the suggested measure, and statistical naturalness approach outperforms state-of-the-art fusion algorithms.

The primary concept presented here is to employ picture fusion to take use of the best aspects of many local and global contrast enhancement methods while eliminating their drawbacks. Figure 2 is a summary of the fusion-based contrast enhancement system. In order to create the fused output pictures, image fusion often includes extracting the most relevant regions from the source photos and blending these regions together. Image fusion techniques that use multiple resolutions (MR) are quite popular. Since the HVS is more attuned to changes in local contrast (such as edges), and since MR decompositions facilitate space-scale localization of these changes, this fact motivates the use of MR in image fusion.

A generic MR fusion technique takes many MR representations as input and applies fusion rules to create a single, unified MR representation. Inverse decomposition is used to piece together the merged picture. Fusing the input photos as a weighted blending of the input images is a simple method. To merge the N input photos into one, we simply take an average along each pixel's length and apply weights calculated from the quality measurements.

where Ik represents the kth input picture in the sequence, k represents the kth weight map, Fi, j represents the composite image, and k represents the kth weight map. At each pixel, the values of the N weight maps are normalized such that they add up to one (i, j).

The weighted blending in Equation, however, might result in distracting seams in the merged picture if the weights shift rapidly. In many techniques are given for merging pictures seamlessly. In order to prevent seams, MR-based blending approaches are preferable since they mix picture characteristics rather than intensities. A method (based on MR pyramid decomposition) for fusing several pictures into one is devised in to achieve smooth blending. The authors demonstrate that, unlike the weighted average blending method, MR-based blending does not result in artifacts like fuzzy edges or a double-exposure effect on the blended picture. The pyramidal decomposition strategy presented in and uses a weighting scheme that is independent of the pyramid's actual contents. In order to direct the fusion of contrast-enhanced pictures using a weight map, we use the MR technique provided in. (computed from quality metrics defined for luminance, saturation and contrast of the enhanced images). Any quality metrics that can be calculated per pixel or in a localized area may be used.

For this purpose, we trained an encoder-decoder network using the massive natural dataset MS-COCO (Lin et al., 2014). MS-COCO has almost 70,000 photos from the natural world. For the sake of efficiency, we downscaled all the photos to 256x256 and turned them to grayscale. Even though numerous competing MEF algorithms have been presented, there is currently no standardized MEF benchmark on which to compare them. As a testbed, we implemented Zhang 2021, the most recent multi-exposure image fusion benchmark dataset. This benchmark dataset includes different sceneries and objects in 100 multi-exposure photograph pairings.

With a batch size of 64 and 70 epochs, our network was trained on an NVIDIA GTX 3090 GPU. We implemented a cosine annealing weight decay of 0.0005 and a learning rate of 1e-4 using an ADAM optimizer. We produced 10 randomly sized subregions from a 256 x 256 training picture to make up the set being modified. To create the sequence xseq, we used a patch size of 16 by 16 pixels taken from the converted input picture in Trans Block.

Both subjective and objective measures were used to thoroughly assess our procedure. The observer's opinion on the quality of the fused pictures, taking into account such criteria as sharpness, detail, and contrast, is known as a subjective evaluation. We chose 12 objective assessment indicators across four viewpoints to give a fair and complete comparison with other fusion techniques in the objective evaluation. QMI, QTE, QNICE, PSNR, and FMI are all information-theory-based measures; QA/BF, QP, STD, and QG are feature-based metrics; SSIM and CC are structural-based metrics for images; and CC and QG are human-perception-inspired metrics. Figure 3: Two examples of source image pairs and fusion results from different methods. (a1)-(b1) and (a2)-(b2) are the source image pairs, and (c1)-(n1) and (c2)-(n2) are the fusion results from various methods.

Quantifiable, measurable VIF, metrics. See Section 3 of the Supplemental Materials for a breakdown of the measures. The 100 fused pictures are averaged to determine all objective parameters, and higher values imply greater performance across the board. We compared our approach to 11 other techniques in the MEF area, including both classical approaches (Li, Manjunath, and Mitra 1995; Liu and Wang 2015; Lee, Park, and Cho 2018; Ma and Wang 2015; Ma et al. 2017b,a) and deep learning approaches (Zhang et al. 2020b; Xu et al. Here are several ways to evaluate them: Some examples of conventional approaches include DWT (Li, Manjunath, and Mitra 1995), DSIFT-EF (Liu and Wang 2015), MEFAW (Lee, Park, and Cho 2018), PWA (Ma and Wang 2020b). Subjective Evaluation Indoor and outdoor fusion results from our approach and from the competition are shown in Figure 2. Section 4 of the supplementary materials displays more fusion findings. The luminance maintenance of DSIFT-EF, MEFAW, MEFOpt, SPD-MEF, and MEFNet is poor when fusing the first pair of source pictures shown in Figure 2 (a1) and (b1). Artifacts are introduced by PWA, and the color is off. Even while DWT, Deepfuse, PMGI, IFCNN, and U2Fusion all keep the brightness around the same, their fusion products lack contrast and don't do a good job of showing off the image's finer features. Our approach preserves the highest levels of brightness and contrast while while displaying high-quality details with enhanced clarity. Most approaches fail to maintain the correct brightness when fusing the second pair of source photos in Figure 2 (a2) and (b2). While MEFNet and PMGI manage to keep the brightness substantially higher, they also create artifacts and blurring. Clearly, our approach maintains the right balance of brightness and contrast while storing more information. Objective Evaluation All comparison strategies on the reference dataset are objectively evaluated and shown in Table 1. Nine out of the twelve indicators are best served by our approach, while the other three metrics show very modest deviations from best practice.

In this study, we introduced a novel image fusion technique for combining photos taken at different exposures. The approach relies heavily on an illumination estimate module, which provides an initial assessment of whether a given area is under- or over-exposed. Here, we offer a technique for preserving both under- and overexposed areas of microscope images by the use of multi-exposure picture fusion. In the current method, the weight map function is calculated using a local entropy measure. Using this weight map function, we can create a unified representation of the input data in the form of a modified Laplacian pyramid. Background noise in non-cell areas is reduced by using NMHE as a pre-processing operator, and discrepancies between input multi-exposure pictures are eliminated during the fusion process. Future plans include including a noise amplification component into the suggested approach, conducting experiments to comparing the findings to those of other fusion approaches, and determining an appropriate contrast measure. We'll put the outcomes to the test by combining the data from many different regional and international techniques. Using the many criteria outlined in the article, we want to devote particular emphasis to the development of a quantitative measure for evaluating the performance of contrast-enhancement algorithms. Extensive trials state-of-the-art performance. Both Trans Block and self-supervised reconstruction tasks offer promise for use in additional image fusion challenges and other fields of image processing.

1. Zhang, Wenlong& Liu, Xiaolin& Wang, Wuchao& Zeng, Yujun. (2018). Multi-exposure image fusion based on wavelet transform. International Journal of Advanced Robotic Systems. 15. 172988141876893. 10.1177/1729881418768939. 2. Li, S.; Kang, X.; Fang, L.; Hu, J.; Yin, H. Pixel-level image fusion: A survey of the state of the art. Inf. Fusion 2017, 33, 100–112 3. Nie, T.; Huang, L.; Liu, H.; Xiansheng Li, X. Multi-exposure fusion of gray images under low illumination based on low-rank decomposition. Remote Sens. 2021, 13, 204. 4. Naila Hayat, Muhammad Imran, Ghost-free multi exposure image fusion technique using dense SIFT descriptor and guided filter,Journal of Visual Communication and Image Representation, Volume 62, 2019, Pages 295-308, ISSN 1047-3203, https://doi.org/10.1016/j.jvcir.2019.06.002. 5. Kinoshita, Y.; Kiya, H. Scene segmentation-based luminance adjustment formulti-exposure image fusion. IEEE Trans. Image Process. 2019, 28, 4101–4115 6. Z. Li, J. Zheng, and S. Rahardja, ―Detail-enhanced exposure fusion,‖ IEEE Trans. on Image Processing, vol. 21, no. 11, pp. 4672-4676, 2012. 7. D. Das, and S. Collins, ―Fixed-pattern-noise correction for an integrating wide-dynamicrangecmos image sensor,‖ IEEE Trans. on Electron Devices, vol. 60, no. 1, pp. 314 – 319, November 2012. 8. L. C. Gouveia, M. Waqas, and B. Choubey, ―A reconfigurable CMOS pixel for applying tone-mapping on high dynamic range images,‖ in Proc. of IEEE Instrumentation and Measurement Technology Conference (I2MTC), Montevideo, Uruguay, pp. 1098 - 1101, May 2014. 9. V. R. Garcia-Hansen, M. Cowley, S. S. Smith, and G. Isoardi, ―Testing the accuracy of luminance maps acquired by smart phone cameras,‖ in Proc. of the CIE Centenary Conference on Towards a New Century of 10. M. Song, D. Tao, C. Chen, J. Bu, J. Luo, and C. Zhang, ―Probabilistic exposure fusion,‖ IEEE Trans. Image Process., vol. 21, no. 1, pp. 341-357, January 2012 11. D. D. Busch, ―Mastering digital SLR photography,‖ Third Edition by Cengage Learning PTR, January 2011. 12. J. P. Carrere, S. Place, J. P. Oddou, D. Benoit, and F. Roy, ―Dictionary learning based color demosaicing for plenoptic cameras,‖ in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 455 – 460, June 2014. 13. J. P. Carrere, S. Place, J. P. Oddou, D. Benoit, and F. Roy, ―CMOS image sensor: Process impact on dark current,‖ in Proc. of the IEEE Conference on Reliability Physics Symposium, Waikoloa, HI, pp. 3C.1.1 - 3C.1.6, June 2014. 14. Richards, and Dan, ―Black & white photography,‖ PSA Journal, vol. 77 no. 12, pp. 38– 40, December 2011. 15. Ahmet Akyüz, KeremHadimli, Merve Aydinlilar, and Christian Bloch, ―Style-based tone-mapping for HDR images,‖ in Proc. of the 6th ACM SIGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia, vol. 23, pp. 1-4, November 2013.

Research Scholar, LNCT University