"Success in creating AI would be the biggest event in human history." (Stephen Hawking)
I am a Machine Learning researcher with a primary focus on deep learning techniques for 3D perception, including automating 3D reconstruction, localization and mapping. My research interests revolve around developing innovative solutions using artificial neural networks for 3D applications. Over the course of my career, I have actively contributed to the design and development of various neural solutions. Some have been listed below.
Projects
Planar Gaussian Splatting (WACV 2024)
Planar Gaussian Splatting (PGS) introduces a novel neural rendering approach to learn the 3D geometry and parse the 3D planes of a scene, directly from multiple RGB images. The PGS leverages Gaussian primitives to model the scene and employ a hierarchical Gaussian mixture approach to group them. Similar Gaussians are progressively merged probabilistically in the tree-structured Gaussian mixtures to identify distinct 3D plane instances and form the overall 3D scene geometry.
The PGS achieves state-of-the-art performance in 3D planar reconstruction without requiring either 3D plane labels or depth supervision. In contrast to existing supervised methods that have limited generalizability and struggle under domain shift, PGS maintains its performance across datasets thanks to its neural rendering and scene-specific optimization mechanism.
Neural Mesh Fusion: UNSUPERVISED 3D PLANAR SURFACE UNDERSTANDING (IEEE ICIP 2024)
Neural Mesh Fusion (NMF) is a method for joint optimization of polygon meshes, estimated from multi-view image observations and unsupervised 3D planar-surface parsing of the scene. In contrast to implicit neural representations, NMF directly learns to deform surface triangle mesh and generate an embedding for unsupervised 3D planar segmentation through gradient-based optimization directly on the surface mesh. The conducted experiments show that NMF obtains competitive results compared to state-of-the-art multi-view planar reconstruction, while not requiring any ground-truth 3D or planar supervision.
- Arxiv version (link)
VaLID: Variable-Length Input Diffusion for Novel View Synthesis (WACV 2024)
VaLID is a deep generative AI model. It performs novel-view image synthesis by processing either a single input image or variable-length multi-view images. This highly adaptive property of VaLID enables various applications in 3D reconstruction and novel view synthesis.
- Arxiv version (link)
Deep Learning Frameworks for Weakly-supervised Indoor Wi-Fi Positioning (2020-2021)
Deep learning solutions for the large-scale indoor localization of people and robot agents via Wi-Fi signal with accessing only to a few room/zone level annotations.
- NeurIPS demo, 2021 (link)
- Video on youtube (link)
Corresponding manuscripts:
- Modality-agnostic topology aware localization (link)
Farhad G. Zanjani, Ilia Karmanov, Hanno Achermann, Daniel Dijkman, Simone Merlin, Max Welling, Fatih Porikli.
NeurIPS, 2021.
- WiCluster: Passive Indoor 2D/3D Positioning using WiFi without Precise Labels (link)
Ilia Karmanov, Farhad G. Zanjani, Simone Merlin, Ishaque Kadampot, Daniel Dijkman
IEEE Globecom, 2021.
- Deep Learning Frameworks for Weakly-Supervised Indoor Localization (link)
Farhad G. Zanjani, Ilia Karmanov et al., NeurIPS Competitions and Demonstrations Track, 2021.
Mask-MCNet: Teeth-Instance Segmentation in 3D Point Cloud
(2019)
Mask-MCNet is one of the first deep learning-based research for automating teeth segmentation in 3D point cloud of intra-oral scan (IOS) data. This model is able to process the IOS in its original spatial resolution without down-sampling to maintain the spatial precision. This work has been funded by Promaton
Ltd.
The application of Mask-MCNet holds significant potential in assisting dentists in orthodontics and implantology. By leveraging this AI-enabled technology, dental professionals can benefit from automated teeth instance segmentation, streamlining their workflow and enhancing accuracy in diagnosis and treatment planning.
Corresponding manuscripts:
- Mask-MCNet: Tooth instance segmentation in 3D point clouds of intra-oral scans. Neurocomputing Journal, 2021 (link).
- Mask-MCNet: Instance Segmentation in 3D Point Cloud of Intra-oral Scans; Medical Image Computing and Computer-Assisted Intervention
(MICCAI), 2019.
- Deep learning approach to semantic segmentation in 3D point cloud intra-oral scans of teeth,
Medical Imaging with Deep Learning (MIDL), 2019.
DCGMM: Deep Convolutional GMM for Stained-Color Normalization
(2018)
Deep Convolutional Gaussian Mixture Modeling (DCGMM) presents an advanced approach to address the challenge of stained-color normalization in histopathological slides. The primary objective is to enhance the performance of subsequent automated image analysis methods used for diagnostic purposes.
DCGMM offers a high-fidelity solution for stained-color normalization. This process ensures that the color appearance of histopathological slides is accurately adjusted, enabling more reliable and consistent image analysis results.
Corresponding
manuscripts:
- Deep convolutional gaussian mixture model for stain-color normalization or histopathological images; Medical Image Computing and Computer-Assisted Intervention (MICCAI),
2018.
- Histopathology stain-color normalization using deep generative models,Medical Imaging with Deep Learning (MIDL), 2018.
- Stain normalization of histopathology images using generative adversarial networks; IEEE Int. Symposium on Biomedical Imaging (ISBI), 2018.
Needle detection and localization in 3D Ultrasound Volumes
(2017-2018)
Ultrasound-guided interventions have emerged as a valuable non-invasive medical imaging technique, offering enhanced patient safety and improved health outcomes. However, the precise positioning of needles and transducers during these procedures poses significant challenges. To address this, automated localization of the needle in 3D ultrasound (US) volumes has emerged as a promising solution, overcoming the limitations of 2D approaches and facilitating accurate needle guidance.
In collaboration with Philips Healthcare, we introduced a novel approach utilizing a multiplane convolutional neural network as an efficient and accurate neural architecture for the detection and localization of needles in 3D US volumes. Our developed method demonstrated remarkable performance and efficacy, leading to its recognition with the Best Paper Award
from the Medical Image Computing and Computer Assisted Intervention Society (MICCAI) in 2018.
Corresponding manuscripts:
- Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks;International Journal of Computer Assisted Radiology and Surgery,
2018.
- Improving needle detection in 3D ultrasound using orthogonal-plane convolutional networks;Medical Image Computing and Computer-Assisted Intervention
(MICCAI), 2017.
- Coherent needle detection in ultrasound volumes using 3D conditional random fields;Proc.or SPIE Medical Imaging, 2018.
- Localization of partially visible needles in 3D ultrasound using dilated CNNs; IEEE International Ultrasonics Symposium (IUS), 2019
Microscopic Image Analysis using Convolutional Neural Networks and Graphical Models
(2018)
The detection and segmentation of cancer represent significant challenges in computational pathology. We conducted a research on three core problems in this field, including cancer metastases detection and localization, stain-color normalization, and measuring the impact of pathological image compression on performance of a deep convolutional neural network (CNN).
For cancer metastases detection and localization, we introduced a solution based on recent advancements in deep CNNs and graphical modeling techniques. Through rigorous experimentation, our proposed technique has demonstrated its effectiveness in achieving a notably high performance in pathology metastases detection. In fact, our framework achieved the second-place position in the internationally recognized
CAMELYON17 competition in 2018, which encompassed a large-scale dataset featuring samples from multiple laboratories.
Corresponding manuscripts:
- From detection of individual metastases to classification of lymph node status at the patient level: the CAMELYON17 challenge; IEEE Transaction on Medical Image (IEEE TMI), 2019.
- Impact of JPEG 2000 compression on deep convolutional neural networks for metastatic cancer detection in histopathological images;Journal of Medical Imaging, 2019.
- Cancer detection in histopathology whole-slide images using conditional random fields on deep embedded spaces;Proc.of SPIE Medical Imaging, 2018.
Analysis of Mass Spectrometry Imaging by Artificial Neural Networks
(2018)
Mass Spectrometry Imaging (MSI) is a technique to visualize the spatial distribution of molecules on the surface of tissues by their molecular masses. We employed recurrent neural networks (RNNs) in compared with 1-D ConvNets for classification of high-dimensional mass spectrometry signals for detecting cancer fingerprint in human tissues.
Corresponding manuscripts:
- Cancer detection in mass spectrometry imaging data by recurring neural networks;IEEE Int.Symposium on Biomedical Imaging (ISBI) 2019.
- Cancer detection in mass spectrometry imaging data by dilated convolutional neural networks;Proc.or SPIE Medical Imaging, 2019.
Brain Perfusion-Map Estimation in 4D-CT Scan (2015)
Estimating hemodynamic characteristic of brain parenchyma by using 4D CT data is widely used as clinical diagnostic tools.
This analysis is utilized for acute cerebrovascular diagnosis and is known as a supporting treatment decision for acute ischemic patients.
During my internship at Diagnostic Image Analysis Group (DIAG)
of Radboud University Medical Center in Nijmegen, the Netherlands, I worked on exploring the architectures of convolutional neural networks for accurate estimation of brain perfusion maps in spatiotemporal CT scans.
3D Vision Traffic Video Analysis
(2008-2013)
Real-time detection, tracking, and speed measuring of vehicles in stereo vision high-speed traffic cameras.
Our designed machine learning algorithm using high-speed infrared imaging sensors performs equally well in day and night.
The developed license plate recognition (LPR) module, enables the travel-time estimation across distributed cameras.
Specifications:
- Accurate speed camera (less than 1% error and maximum 2 km/h).
- Vehicle category classification e.g. passenger, truck, etc.
- Accurate license plate recognition (more than 99% accuracy).
Mobile 3D Vision Vehicle Speed Recorder (2008-2013)
Real-time 3D scene analysis on vehicle. The developed system is able to detect, localize, and track multiple vehicles using cameras in real-time and reports their speed in motion. Our designed system utilizing low-power GPUs, accurate GPS sensors, video tracking and ego-motion analysis, estimates the vehicles speeds by less than 5% error rate, up to 250 km\h.
Specifications:
- Real-time multi-target detection, tracking, and speed measurement.
- Extra large field of view: functional distance to target in range of 15-100 meters.
- Online recording, LPR and archiving in the database.
- Auto-validating of the cameras calibration
Granted Patents
- Multi-object positioning using mixture density networks; US Patent 11,696,093, 2023; F.G. Zanjani, A. Behboodi, D.HF. Dijkman, I. Karmanov, S. Merlin, M. Welling, et al.
- Passive positioning with radio frequency sensing; US Patent 11,575,452, 2023; S. Merlin, B. Mohanty, D. HF. Dijkman, F. G. Zanjani, I. Karmanov, et al.
- Passive positioning with radio frequency sensing labels; US Patent US Patent 11,755,886, 2023; S. Merlin, J.M. Lin, B.M. Buesker, R. Liang, D.HF. Dijkman, F.G. Zanjani, et al.
Publications
Journal Articles
- Mask-MCNet: Tooth instance segmentation in 3D point clouds of intra-oral scans; Farhad G. Zanjani, Arash Pourtaherian, Svitlana Zinger, David Anssari Moin, Frank Claessen, Teo Cherici, Sarah Parinussa, Peter HN de With, Neurocomputing, 2021.
- From detection of individual metastases to classification of lymph node status at the patient level: the CAMELYON17 challenge; Bandi, P., Geessink, O., Manson, Q., van Dijk, M., Balkenhol, M., Hermsen, M., Bejnordi, B. E., Lee, B., Paeng, K., Zhong, A., Li, Q., Zanjani, F. G., Zinger, S., Fukuta, K., Komura, D., Ovtcharov, V., Cheng, S., Zeng, S., Thagaard, J., Dahl, A. B. & 16 others, Feb 2019, In : IEEE Transactions on Medical Imaging. 38, 2, p. 550-56 11 p.
- Impact of JPEG 2000 compression on deep convolutional neural networks for metastatic cancer detection in histopathological images; Ghazvinian Zanjani, F., Zinger, S., Piepers, B., Mahmoudpour, S., Schelkens, P. & de With, P., 24 Apr 2019, In : Journal of Medical Imaging. 6, 2, 9 p., 027501.
- Optimize transfer learning for lung diseases in bronchoscopy using a new concept: sequential fine-tuning; Tan, T., Li, Z., Liu, H., Ghazvinian Zanjani, F., Ouyang, Q., Tang, Y., Hu, Z. & Li, Q., 30 Aug 2018, In : IEEE Journal of Translational Engineering in Health and Medicine. 6, 8 p., 1800808.
- Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks; Pourtaherian, A., Ghazvinian Zanjani, F., Zinger, S., Mihajlovic, N., Ng, G. C., Korsten, H. H. M. & de With, P. H. N., 1 Sep 2018, In : International Journal of Computer Assisted Radiology and Surgery. 13, 9, p. 1321-1333 13 p.
- Lesion Segmentation in Ultrasound using Semi-Pixel-Wise Cycle Generative Adversarial Networks; J. Xing, Z. Li, B. Wang, Y. Qi, B. Yu, F. Ghazvinian Zanjani, a. Zheng, R. Duits, T. Tan; Transaction on Computational Biology and Bioinformatics, 2020.
International Conference Proceedings
- Modality-Agnostic Topology Aware Localization; FG Zanjani, I Karmanov, H Ackermann, D Dijkman, S Merlin, M Welling, Thirty-Fifth Conference on Neural Information Processing Systems (NeurIPS), 2021.
- WiCluster: Passive Indoor 2D/3D Positioning using WiFi without Precise Labels; Ilia Karmanov, Farhad G. Zanjani, S Merlin, I Kadampot, D Dijkman; arXiv preprint arXiv:2107.01002, 2021
- Lesion segmentation in ultrasound using semi-pixel-wise cycle generative adversarial nets; J Xing, Z Li, B Wang, Y Qi, B Yu, F. G. Zanjani, A Zheng, R Duits, T Tan; IEEE/ACM transactions on computational biology and bioinformatics, 2020.
- Multi-scale Ensemble of ResNet Variants; J. Putten, F. G. Zanjani; Computer-Aided Analysis of Gastrointestinal Videos, 115-119
- Mask-MCNet: Instance Segmentation in 3D Point Cloud of Intra-oral Scans; Ghazvinian Zanjani, F., Anssari Moin, D., Claessen F., Cherici T., Parinussa S., Pourtaherian A., Zinger S., de With, P.H.N.; Medical Image Comp. and Computer Assisted Intervention (MICCAI), 2019.
- Deep learning approach to semantic segmentation in 3D point cloud intra-oral scans of teeth; Ghazvinian Zanjani, F., Anssari Moin, D., Verheij, B., Claessen, F., Cherici, T., Tan, T. & de With, P. H. N.; International Conference on Medical Imaging with Deep Learning (MIDL), 2019.
- Cancer detection in mass spectrometry imaging data by dilated convolutional neural networks; van Kersbergen, J., Ghazvinian Zanjani, F., Zinger, S., van der Sommen, F., Balluff, B., Vos, D. R. N., Ellis, S. R., Heeren, R. M. A., Lucas, M., Marquering, H. A., Jansen, I., Savci-Heijink, C. D., de Bruin, D. M. & de With, P. H. N.; Medical Imaging: Digital Pathology; SPIE, 8 p. 1095601 (Proceedings of SPIE; vol. 10956), 2019.
- Cancer detection in mass spectrometry imaging data by recurrent neural networks; Ghazvinian Zanjani, F., Panteli, A., Zinger, S., van der Sommen, F., Tan, T., Balluff, B., Vos, D. R. N., Ellis, S. R., Heeren, R. M. A., Lucas, M., Marquering, H. A., Jansen, I., Savci-Heijink, C. D., de Bruin, D. M. & de With, P. H. N.; IEEE 16th International Symposium on Biomedical Imaging (ISBI), 2019.
- Localization of partially visible needles in 3D ultrasound using dilated CNNs; Pourtaherian, A., Mihajlovic, N., Ghazvinian Zanjani, F., Zinger, S., Ng, G. C., Korsten, E. & de With, P.H.N.; IEEE International Ultrasonics Symposium (IUS). 2019.
- Cancer detection in histopathology whole-slide images using conditional random fields on deep embedded spaces; Ghazvinian Zanjani, F., Zinger, S. & de With, P. H. N.; Medical Imaging: Digital Pathology. SPIE, Vol. 10581, 8 p. 105810I. (Proceedings of SPIE; vol. 10581), 2018.
- Coherent needle detection in ultrasound volumes using 3D conditional random fields; Ghazvinian Zanjani, F., Pourtaherian, A., Tang, X., Zinger, S., Mihajlovic, N., Ng, G. C., Korsten, H. H. M. & de With, P. H. N.; Medical Imaging: Image-Guided Procedures, Robotic Interventions, and Modeling; SPIE, Vol. 10576, p. 1-8 105760W. (Proceedings of SPIE; vol. 10576), 2018.
- Deep convolutional gaussian mixture model for stain-color normalization of histopathological images; Ghazvinian Zanjani, F., Zinger, S. & de With, P. H. N.; Medical Image Computing and Computer Assisted Intervention (MICCAI); Springer, p. 274-282 9 p. (Lecture Notes in Computer Science; vol. 11071 LNCS), 2018.
- Histopathology stain-color normalization using deep generative models; Ghazvinian Zanjani, F., Zinger, S., de With, P. H. N., E. Bejnordi, B. & van der Laak, J. A. W. M.; 1st Conference on Medical Imaging with Deep Learning (MIDL). p. 1-11 11 p., 2018.
- Stain normalization of histopathology images using generative adversarial networks; Ghazvinian Zanjani, F., Zinger, S., Bejnordi, B. E., van der Laak, J. A. W. M. & de With, P. H. N.; IEEE 15th International Symposium on Biomedical Imaging (ISBI); Piscataway: IEEE Computer Society, Vol. 2018-April, p. 573-577 5 p., 2018.
- Improving needle detection in 3D ultrasound using orthogonal-plane convolutional networks; Pourtaherian, A., Ghazvinian Zanjani, F., Zinger, S., Mihajlovic, N., Ng, G. C., Korsten, H. H. M. & de With, P. H. N.; Medical Image Computing and Computer-Assisted Intervention (MICCAI); Proceedings, Part II. Dordrecht: Springer, p. 610-618 (LNCS; vol. 10434), 2017.
Contact
mind2machine.info@gmail.com