What does AI mean for Open Source?

Adam Retter

Abstract

The recent meteoric rise of LLMs (Large Language Models) and associated tools was largely unexpected and surprising to most. The rapid ascent of this technology has caught many software developers unawares, leaving them suddenly somewhat ignorant, and arguably under-skilled.

LLMs, whilst still advancing, have recently demonstrated impressive capabilities in their ability to assist software developers in their day-to-day tasks (e.g., coding new features, and locating and fixing issues). However, the use and adoption of LLMs presents many larger challenges for society as a whole; many of which are not in themselves technical concerns.

This paper examines the current and perceived impact of this technology in the context of Open Source. We identify several social, economic, environmental, political, legal, and technical concerns regarding the use of LLMs in Open Source projects.

We contribute guidance around defining an AI Policy for Open Source projects. We further offer an AI Policy Score Card to assist projects in clearly defining and declaring how they wish to work with AI or not.

Introduction

Research into LMs (Language Models) and NLP (Natural Language Processing) has been a subset of AI (Artificial Intelligence) research since the 1960’s. Developments such as ELIZA demonstrated a natural language chat-like interface between human and machine Wizenbaum66.

Whilst research continued and incremental progress was made, it wasn’t until more than 50 years later, in 2017 that Google published their seminal paper introducing the Transformer Architecture Vaswani17 that created the foundation for modern LLMs. In the last 9 years we have seen exponential growth in the development of new and more powerful LLMs:

2018: GPT-1, and BERT
2019: GPT-2
2020: GPT-3
2022: GPT-3.5, and ChatGPT
2023: GPT-4, Claude 1, Claude 2, Claude 2.1, LLaMA 1, LLaMA 2, Gemini 1, and DeepSeek
2024: Gemini 1.5, GPT-4 Turbo / GPT-4o, Claude 3, Claude 3.5, Claude 4.7, LLaMA 3, DeepSeek v2, DeepSeek v2.5, Mistral, Cohere, and Qwen
2025: GPT-4.5, Claude 4, LLaMA 4, Gemini 2, Gemini 3, DeepSeek v3, DeepSeek v3.1, and DeepSeek v3.2
2026: GPT-5.2, GPT-5.3, GPT-5.4, GPT-5.5, GPT-5.6, Gemini 3.x, Claude 4.x, Claude Opus 5, Grok 4.x, and Qwen 3.x

These new generations of LLMs not only improved the human-machine chat interface, they also added audio voice chat capabilities and autonomous capabilities.

Significantly for Software Developers an "Agentic" approach was developed. This provided: (a) APIs (Application Programming Interfaces) to access huge LLMs running in the Cloud, and (b) Software Agents that whilst operating locally within a Software Developer’s development environment can utilise these APIs to service tasks requested by developers. Such tasks include automated running of various tools and tests, development of new software code, and providing fixes to issues in existing code.

Furthermore, the latest trend is towards orchestration of Agents, whereby you may have a fleet or cluster of LLM powered Software Agents running across multiple systems and utilising different LLMs collaborate on solving a particular task. Remarkably, using Open Source tools such as LangChain Chase22 or n8n Oberhauser19, it is quite possible today to have a team of Software Agents that act in a hierarchy, whereby one or more agents may be supervising or providing quality assurance on the work of one of more other agents.

Whilst the modern capabilities of LLMs were barely imaginable just a few years ago, and may appear extraordinarily useful, their use and operation is not without concern. We will examine such concerns in the context of Open Source in section “Concerns for AI in Open Source”.

Open Source, not to be confused with Free Software Stallman98, as a term is rather broad and arguably overloaded, itself an umbrella that covers several different aspects of both technical and legal domains.

Officially, The OSD (Open Source Definition) Perens98 sets out 10 criteria that a piece of computer software and its associated license must meet in order to qualify as "Open Source". The OSI (Open Source Initiative) recognises over 80 licenses that comply with the OSD; however, in practice there are far more Open Source licenses available, each offering different grants and restrictions on use. One may choose to publish their software under such a license if they wish it to be classed as Open Source

More organically, there are what are known as "Open Source Projects", these are most usually software projects of a specific purpose that are developed by one or more persons and/or organisations. These persons and/or organisations collaborate, often online in public, to produce software that is licensed and published under one of the Open Source licenses.

These Open Source Projects vary wildly in structure and governance. A project may be as simple as an individual creating software for their own pleasure, or perhaps a group of like minded individuals have a shared cause, and have coalesced around an idea to create software to assist anyone in improving some environmental concern. Conversely, at the other end of the spectrum, a project, may involve one or more large corporate/government organisation(s) producing a complex product through the collaboration of large teams of developers, upon which their business model depends.

Ultimately, Open Source can be thought of as an ideology, part of which is encoded into a license for use of the product (e.g., software). An open Source license requires you to allow anyone else to both use your software and modify it, furthermore you cannot restrict the use of your software by any party that you might consider undesirable for any reason. This shared ideology infers not only the free exchange and development of ideas and technology, but from preventing any other entity from restricting this GiovanH21.

Whilst there is nothing in the OSD that requires development of Open Source software to be carried out in the open and/or online, or even for the source code to be published online, the expectation of users since around the mid-2000’s is that Open Source projects will host their source code publicly online, and be developed in the public eye in a real-time fashion Peterson13 AlMarzouq22. Therefore, modern code projects are produced in public via online collaboration through a code repository such as GitHub, Sir Hat, or Code Berg.

It is typical for an Open Source project to have multiple contributors who are geographically separated and perhaps not known to each other outside of their online presence. Differences in expectations, technical ability, language barriers, and differing human cultural and social norms of contributors, can be a strength, but can also contribute friction, making such projects challenging to govern.

One could argue that any project involving multiple contributors is susceptible to such conflicts; however, for private companies with bespoke (closed) software projects and strong governance structures, such issues are easily hidden from public view within the confines of their organisation. As there is now an expectation that all aspects of an Open Source project are carried out and documented in public, interactions that lead to such friction and/or conflict, can often feel amplified or more significant when under the scrutiny of their peers. In an attempt to avoid such issues, many projects have adopted Code’s of Conduct such as Contributor Covenant Ehmke25, The Citizen Code of Conduct Koehler18, or Anticode Webb20; however, such policies are only of use if they are correctly enforced by the administrators and/or moderators of the project itself.

Historically, all of these problems have been very human centric, and do not directly relate to the use of AI. However, it is our position that the use of AI, either by developers, or autonomously via agents, to submit changes to Open Source projects is likely to not only increase friction between humans, but also potentially introduce friction between humans and machines.

It is therefore important that both the human and artificial contributors, current and future, to an Open Source project, all understand the project’s policy with regards to both human and LLM/AI produced contributions. Many Open Source projects already include documentation for humans about how to contribute, but very few have yet decided upon and/or published their AI Policy towards human-machine contributions, and few have documented these in a manner suitable for both humans and machines. To try and solve this issue, our contribution is an "AI Policy Score Card" that may be used by Open Source projects to gauge their AI Policy; see section “An AI Policy Score Card for Open Source Projects”.

Concerns for AI in Open Source

The tragic impacts that LLM/AI can affect through sycophantic traits, including Chatbot Psychosis WIKI01, continue to be documented by psychiatrists Østergaard25 Morrin26 Girgis26, The Human Line Project THLP25, and various news outlets Moore26 Proven26/2.

Open Source projects and code have historically been solely built by humans, with the rise of LLMs assisting Software Developers to generate code, Open Source founders should consider the impact of this upon their projects and contributors, and therefore must define their policy around AI carefully. First and foremost they must remember that Open Source contributors, i.e., software developers, however remote, are also human beings (to date), and as such are susceptible to the same AI biases and sycophantic reinforcement as any other sector of society; perhaps more so, as their use of AI is not just limited to their personal life, but also pervades their chosen career.

The governance and purpose of Open Source projects span a wide spectrum, whilst some projects adhere to a common foundational model of governance ASF26 Burcher14 TLF26, many more are completely ad-hoc and structured according to the founders and/or contributors preferences. Therefore, the concerns affecting the use of AI in Open Source projects whilst many, will undoubtedly vary from project to project.

We present concerns that our research indicates are likely applicable to most Open Source projects. Whilst we recognise that some of these concerns are multi-faceted and cross-cutting, we have loosely grouped them into the following categories: Social, Economic, Environmental, Political, Legal, and Technical.

Social Concerns

For those human founders and/or contributors whom are not financially compensated for the time and resources they invest in Open Source projects, their contributions can largely be considered a social activity. Indeed, contributing to software that is published as Open Source, through the very nature of the OSD means that you are making something available to society as a whole.

The large majority of people contribute to FOSS because of fun (91%), altruism (85%), and kinship (80%). Moreover, when analysing differences in motivations to join and continue, the study found that ideology, own-use, or education-related programs can be an impetus to join FOSS, but individuals continue for intrinsic reasons (fun, altruism, reputation, and kinship).

— opensource.com - 2021-04-05 OS21

Allowing AI generated code into an Open Source project, regardless of a nett-positive or negative outcome, will have both, an immediate impact on the existing human contributors to the project, and also the ability of the project to attract contributors in future.

On the one hand, human contributors may be keen to embrace AI, with the potential to help non/less-technical contributors implement code that that would otherwise be beyond their reach Karim26, or perhaps allow experienced developers to pair with AI to increase their velocity Zheyuan25 Edwards26.

On the other hand, experienced engineers may be disappointed with the code produced by AI LinearB26, or lament having to review large change sets that have been partially or entirely generated by AI (that was not well instructed) and exhibit many issues Song23.

Whether for or against AI, software developers experiencing fear and/or depression triggered by AI is well documented Lițan25 Spirlet26; they may experience the feeling of loss that they will no longer be able to enjoy socially crafting code Orosz26 Lawson26, or the fear that they will be left behind by competitors that have embraced AI where they have not Kwon26.

Additionally, as human beings we subscribe to and/or create philosophies, ideologies, and belief systems. As such, individuals may, understandably and freely, exhibit and justify ethical and/or non-technical reasoning around the extent to which they embrace or reject AI. Around these new ideologies a number of colloquial terms are forming to describe them, such as "AI Vegan" Mahdawi25, and the less strict "AI Vegetarian" Proven26/1.

I have a religious exemption from using all generative “A.I.”

— Catherine Sawers - 2025-09-12 Sawers25

If human contributors within an Open Source project are unhappy with its policy towards AI use, the project may experience a collapse of its social fabric, resulting in a loss of contributions and/or risk being forked. To avoid social conflict and worse, alienation, it would follow that some form of consensus will be required between the founders and/or contributors to the project on, if and how AI may be used in that project. We argue that that consensus, forms the basis of an acceptable use of AI policy for the project, and whilst it may of course evolve over time, it should be clearly communicated to any potential contributors.

Economic Concerns

It remains unclear whether Open Source in and of itself has had a nett-positive or negative affect on reducing the social-economic divide with regards to developing software.

On the one hand, the cost of access to software tools has been reduced through eliminating licensing fees, and self taught developers can study existing code Pscheidt09 LOS24.

On the other hand, producing and contributing to Open Source software requires hardware, a reliable internet connection, education (specialised technical knowledge), and potentially the wealth to allow individuals to contribute without financial compensation; this can make the cost of entry for those from lower socioeconomic backgrounds impossible Blind24.

We argue that the use of AI in Open Source projects may further increase the social-economic and digital divide between those that can afford to contribute and those that cannot.

For SaaS (Software as a Service) based AI services, which appear to be the most commonly used, these are typically presented to the end user as either free, or with a subscription and/or token pricing model. The current startup companies behind these services have huge infrastructure and staff costs that are fuelled by large VC (Venture Capital) investments that have yet to be realised. Whilst these companies are striving for market dominance and determining the product-market fit for their offerings, they are operating their services at a loss and spending vast sums of VC money Sachs24. During this time, the cost to some of using such services may seem reasonable, whilst to others it may already be prohibitive. We cannot yet determine what the true future price to the user of such services will be when these companies have to start making a profit to repay their investors Aggarwal26.

Sadly, we have not been able to find much evidence of discounted SaaS AI services to help those in developing countries or those from disadvantaged socially-economic backgrounds. The true financial cost of using SaaS AI services cannot yet be determined.

For those who wish to avoid subscription services by operating their own AI/LLM models, there are also considerable costs, some of which are non-obvious Ivchenko26.

It would seem to follow that those Open Source projects that are well funded and/or operated by large private/government organisations may be able to afford to pay for AI use in their projects, whereas smaller or unfunded projects likely will not.

We are not yet aware of any Open Source projects that strictly mandate the use of AI by contributors, but there are certainly already many projects that encourage its use. Unfortunately while the use of AI/LLM comes with a financial cost, which itself seems likely to increase, it seems inevitable that this will further increase both the economic-social and technical divides in contributing to Open Source projects.

We believe that those championing the use of AI in Open Source projects should consider if they are excluding potential contributors due to social-economic status, and how they might remove that barrier.

Environmental Concerns

A great amount of Open Source software is started under the banner of OSS4SG (Open Source Software for Social Good) Fang26; however, serious secondary environmental impacts have been raised in relation to the operation of AI/LLMs which would seem to be at conflict with such a purpose.

The operation of AI/LLMs at scale require large compute resources that pose a number of environmental concerns, the main of which can roughly be divided into three categories:

Manufacturing of Equipment – the required hardware: compute nodes, computer network infrastructure, power and cooling systems, and housing. This requires raw materials (incl. rare earth minerals) that are mined, and then electricity and water during their construction AIEC25.
Construction – dedicated physical data centres need to be constructed to house all of the manufactured equipment. This requires the acquisition of suitable land, and development of it. This has been documented to lead to water pollution of the surrounding environment, and reducing nearby property prices Luscombe26.
Operation – the day-by-day operation of the data centres require large quantities of electricity for power, and water for cooling. The by-product is pollution of the local environment from one or more of waste water, heat, noise, and light Robinson26.

Early research into the environmental impacts of LLM’s focused on the environmental costs of training the LLM’s. One such study, found that in 2019 training an LLM required nearly the same carbon footprint as an average human being creates during 60 years of their life Strubell19. Since that time whilst LLM’s have become much more complex and training them requires ever more data, it is the scale at which they have been deployed, and subsequently made available as a service that has changed this landscape.

Whereas previously training the LLM created a larger carbon footprint than its operation, that position has now been reversed. The latest research focuses on the total environmental cost, or which day-to-day use of the models forms a huge part. The UNU-INWEH (United Nations University Institute for Water, Environment, and Health) recently published a report suggesting that by 2030, global AI data centres will consume 945 terawatt-hours of electricity annually. This will require a massive water footprint for cooling and power generation equivalent to the basic annual needs of 1.3 billion people, alongside a land footprint exceeding 14,500 KM2 Aczel26.

Such environmental concerns affect us all, and it would seem likely that the impact of AI on our physical environment may be of particular interest to those operating under the banner of OSS4G, and in all likelihood, many more.

We believe that such concerns may dissuade people from using AI, and by extension individuals and organisations may be put-off contributing to or using Open Source projects that make use of AI.

At present it can be hard to determine if an Open Source project uses AI or not, we believe that such use should be clearly communicated and form part of the projects AI policy, thus allowing users and contributors to make informed decisions.