Benchmarking Large Language Models for Transcriptomic Data Analysis and Visualization: Multi-Model Workflow Evaluation

Abstract

This study presents a comprehensive benchmarking evaluation of large language models(LLMs) and agentic coding systems for automated analysis and visualization workflows for transcriptomic data, in particular human responses to COVID-19 infection. Multiple models, including ChatGPT 5.5, Claude Sonnet 4.6, Gemini 3.1, and Llama4-Scout, were assessed across a series of scientific prompts designed to test their ability to reproduce workflow components such as dimensionality reduction, clustering, outlier detection, reproducibility, and code generation. The benchmark protocol involved varying prompt specificity and examined both quantitative metrics such as clustering scores and outlier counts as well as qualitative aspects such as code quality and reproducibility. Results indicate that prompt specificity improves code structure and methodological correctness, but quantitative performance depends heavily on preprocessing choices and execution environment. While some models produced strong code and reproducible outputs, alignment with scientific targets like label-based clustering was inconsistent. The findings highlight the promise and limitations of current LLMs for automating scientific data analysis and visualization and suggest directions for future benchmarking and workflow standardization.