AI literacy and its implications for prompt engineering strategies
Грамотность в области искусственного интеллекта и ее значение для стратегий разработки промптов
2024-04-17
SCID: 54.1/fyuskw9v
Discuss with AI
AI literacygenerative AIhigher educationlarge language modelsprompt engineering
Figures from the paper
Abstract (AI)
Artificial intelligence technologies are rapidly advancing. As part of this development, large language models (LLMs) are increasingly being used when humans interact with systems based on artificial intelligence (AI), posing both new opportunities and challenges. When interacting with LLM-based AI system in a goal-directed manner, prompt engineering has evolved as a skill of formulating precise and well-structured instructions to elicit desired responses or information from the LLM, optimizing the effectiveness of the interaction. However, research on the perspectives of non-experts using LLM-based AI systems through prompt engineering and on how AI literacy affects prompting behavior is lacking. This aspect is particularly important when considering the implications of LLMs in the context of higher education. In this present study, we address this issue, introduce a skill-based approach to prompt engineering, and explicitly consider the role of non-experts' AI literacy (students) in their prompt engineering skills. We also provide qualitative insights into students’ intuitive behaviors towards LLM-based AI systems. The results show that higher-quality prompt engineering skills predict the quality of LLM output, suggesting that prompt engineering is indeed a required skill for the goal-directed use of generative AI tools. In addition, the results show that certain aspects of AI literacy can play a role in higher quality prompt engineering and targeted adaptation of LLMs within education. We, therefore, argue for the integration of AI educational content into current curricula to enable a hybrid intelligent society in which students can effectively use generative AI tools such as ChatGPT.
Key Findings
1
Higher-quality prompt engineering predicts higher-quality LLM outputs, supporting prompting as an essential skill for goal-directed generative AI use.
2
Qualitative findings reveal students’ intuitive behaviors when interacting with LLM-based AI systems.
3
Specific aspects of students’ AI literacy contribute to better prompt engineering and more targeted adaptation of LLMs in education.
4
The authors argue that AI education should be integrated into existing curricula to help students use generative AI effectively in a hybrid intelligent society.
5
The study introduces a skill-based approach to prompt engineering focused on non-expert students using large language models.
Research Object
Non-expert students interacting with LLM-based AI systems via prompt engineering
Research Subject
The relationship between students’ AI literacy, prompt-engineering skills, and the quality and adaptation of LLM outputs in higher education
Publication Details
Publication Date
2024-04-17
Journal
Publisher
ISSN
Cited by
368
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai13
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing2022
Conceptualizing AI literacy: An exploratory review2021
GPT-3: Its Nature, Scope, Limits, and Consequences2020
ChatGPT for Education and Research: Opportunities, Threats, and Strategies2023
A SWOT analysis of ChatGPT: Implications for educational practice and research2023
Artificial intelligence literacy in higher and adult education: A scoping literature review2022
The role of ChatGPT in higher education: Benefits, challenges, and future research directions2023
Impact of AI assistance on student agency2023
Technology, Humanness, and Trust: Rethinking Trust in Technology2015
Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI)2023
War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education2023
Prompting AI Art: An Investigation into the Creative Skill of Prompt Engineering2024
Anthropomorphism and social presence in Human–Virtual service assistant interactions: The role of dialog length and attitudes2022