The application of AI tools in weather and climate science

Recent years have witnessed significant advancements in the development of Artificial Intelligence (AI) tools, notably Large Language Models (LLMs), with prominent systems including ChatGPT by OpenAI, Gemini by Google, and Copilot by Microsoft. Despite inherent limitations, the diversity of these tools' applications across various fields of life, including scientific research, has expanded significantly.

This study evaluates the utility of various AI tools within the fields of meteorology and climatology, ensuring their applications follow ethical standards in scientific publication. The tools assessed include ChatGPT versions 3.5 and 4.0, Gemini (Google), Copilot (Microsoft), Perplexity, and GPT-based systems such as DataAnalyst, Consensus, ScholarGPT, and Academic Assistant Pro, among others. Each tool was subjected to identical inputs (prompts, data, photographs) and their responses were evaluated on a 0-10 scale for accuracy and relevance. The scoring was based on the percentage of verifiable content in the responses to ensure objectivity. The research spanned from May 2023 to April 2024.

The AI systems were tasked with responding to queries on climate change in Poland, identifying key research papers on humid heat waves, classifying cloud types, creating a climate map from provided data, and comparing two climate maps.

The outcomes varied significantly across tasks. ChatGPT 3.5 demonstrated an answer accuracy of 30-40% (topic: climate change in Poland). The Consensus system excelled in identifying and summarizing key papers on humid heat waves research. ChatGPT 4.0 emerged as the most effective tool for cloud classification, with Copilot also delivering commendable results; however, Gemini (Advanced) struggled with cloud recognition tasks. DataAnalyst proved capable of generating basic climate maps, but with some inaccuracies such as station misplacements. When comparing two climate maps, all systems performed adequately, with the most precise descriptions provided by Bard (Google).