Early identification of human pathogens is crucial for the effective treatment of bloodstream infections to prevent sepsis. Since pathogens that are present in small numbers are usually difficult to detect directly, we hypothesize that the behavior of the immune cells that are present in large numbers may provide indirect evidence about the causative pathogen of the infection. We previously applied time-lapse microscopy to observe that neutrophils isolated from human whole-blood samples, which had been infected with the human-pathogenic fungus Candida albicans or C. glabrata, indeed exhibited a characteristic morphodynamic behavior. Tracking the neutrophil movement and shape dynamics over time, combined with machine learning approach, the accuracy for the differentiation between the two Candida species was about 75%. In this study, the focus is on improving the classification accuracy of the Candida species using advanced deep learning methods. We implemented (i) gated recurrent unit (GRU) networks and transformer-based networks for video data, and (ii) convolutional neural networks (CNNs) for individual frames of the time-lapse microscopy data. While the GRU and transformer-based approaches yielded promising results with 96% and 100% accuracy, respectively, the classification based on videos proved to be very time-consuming and required several hours. In contrast, the CNN model for individual microscopy frames yielded results within minutes, and, utilizing a majority-vote technique, achieved 100% accuracy both in identifying the pathogen-free blood samples and in distinguishing between the Candida species. The applied CNN demonstrates the potential for automatically differentiating bloodstream Candida infections with high accuracy and efficiency. We further analysed the results of the CNN using explainable artificial intelligence (XAI) techniques to understand the critical features and patterns, thereby shedding light on potential key morphodynamic characteristics of neutrophils in response to different Candida species. This approach could provide new insights into host-pathogen interactions and may facilitate the development of rapid, automated diagnostic tools for differentiating fungal species in blood samples.