Autonomous artificial intelligence, scientific research, and human values
Автономный искусственный интеллект, научные исследования и человеческие ценности
2026-02-01
SCID: 54.1/2y493j32
Discuss with AI
AI agentsAI research ethicsAI-augmented scientific researchautonomous artificial intelligencehuman supervision and control
Figures from the paper
Abstract (AI)
During the initial stage of AI's incorporation into scientific research, AI systems have functioned predominantly as tools under direct human supervision and control. However, AI incorporation into scientific research is now entering a stage in which AI Agents perform research tasks with partial autonomy while remaining under human supervision and control. In the not-too-distant future, a third stage may arise when autonomous AI systems conduct their own research and generate knowledge without human supervision or control. While the second and third stages of AI-augmented research may offer substantial benefits for science and society, they also create novel ethical issues, including (1) Conducting immoral research that may harm humans and other forms of life; (2) Increasing rate of biased, erroneous and deceptive research; (3) Confidentiality challenges; (4) Overreliance on AI; (5) Diffusion of responsibility and accountability; (6) Deskilling; (7) Job losses; (8) AI-generated research beyond human comprehension; and (9) Erosion of trust. We suggest specific solutions to minimize the negative consequences of these issues and offer proposals for ensuring that incorporation of AI into scientific research supports human values.
Key Findings
1
AI integration into scientific research is progressing from human-controlled tools toward partially autonomous agents and potentially fully autonomous knowledge-generating systems.
2
Partially and fully autonomous AI research could provide substantial benefits for science and society, but may operate without direct human supervision in later stages.
3
The abstract identifies nine ethical risks, including harmful research, increased bias and deception, confidentiality breaches, overreliance, accountability gaps, deskilling, job losses, incomprehensible outputs, and declining trust.
4
The paper proposes specific measures to reduce these risks and argues that AI incorporation into research should be guided by human values.
Research Object
Autonomous AI systems performing scientific research
Research Subject
The ethical risks, societal consequences, and value-aligned governance solutions associated with increasing AI autonomy in scientific research
Publication Details
Publication Date
2026-02-01
Journal
Publisher
ISSN
Cited by
6
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai7
Highly accurate protein structure prediction with AlphaFold2021
Artificial intelligence: A powerful paradigm for scientific research2021
Self-Driving Laboratories for Chemistry and Materials Science2024
Accountability in artificial intelligence: what it is and how it works2023
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery2024
Accelerating scientific discovery with Co-Scientist2026
How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data2009