Can AI Write Secure Smart Contracts? Exploring Large Language Models in Blockchain Programming 🔗 🔒

Published November 14, 2024 By EngiSphere Research Editors

AI Model Interacting with a Digital Smart Contract © AI Illustration

The Main Idea

This research examines the effectiveness of various Large Language Models (LLMs) in generating secure and efficient Solidity smart contracts, finding that while models like GPT-4 and Claude perform well in basic tasks, they struggle with complex coding and security requirements, highlighting the potential but current limitations of AI in smart contract development.

The R&D

In today’s fast-paced digital world, smart contracts are the backbone of decentralized applications, especially on platforms like Ethereum. These immutable, self-executing contracts hold the potential to streamline operations, ensure transparency, and secure transactions without a middleman. But here’s the big question: Can Artificial Intelligence (AI) help write these smart contracts effectively and securely? This research, conducted by Siddhartha Chatterjee and Bina Ramamurthy, dives into how various Large Language Models (LLMs) perform in generating Solidity-based smart contracts.

Let’s dive into their study and discover how AI models like GPT-4 and others perform in the real world of blockchain coding, where security and precision are key! 🔍

Why Smart Contracts and Why Now?

Since the Ethereum blockchain launched in 2015, smart contracts have transformed industries like finance, real estate, and entertainment by enabling decentralized, automated transactions. But writing secure and efficient smart contracts is tricky. Once deployed, they’re immutable, so any bugs or security flaws can lead to costly issues, potentially exposing sensitive data or funds.

This research tackles three major questions:

How accurate are LLMs in creating smart contracts?
What strengths and weaknesses do different LLMs show in this specialized coding task?
How do these AI-generated contracts compare to those created manually?

Experimenting with LLMs: From Simple to Complex Tasks

The study tested multiple models—including GPT-3.5, GPT-4, Cohere, Mistral, Gemini, and Claude—using both simple and complex smart contract tasks. They set up tasks ranging from basic data storage to creating a custom token on Ethereum, gauging each model’s performance across accuracy, efficiency, and code quality.

Types of Prompts

They used two main prompting techniques:

Descriptive Prompting: These are more open-ended and resemble how users typically interact with AI. They give a general description of what the contract should do.
Structured Prompting: These are more detailed and precise, similar to pseudo-code, outlining exactly what each function should achieve.

Key Findings: AI’s Performance on Smart Contract Generation

1. Storage of a Single Variable

All models succeeded in basic tasks, like creating a contract to store a variable on the blockchain.
Interesting find: GPT-3.5 added unnecessary checks for positive values, despite Solidity already restricting this type of variable to positive numbers, which reflects the tendency for AI to add redundant code at times.

2. Locking Ether with Conditions

In a contract that locks and releases Ether based on a timed condition, models like GPT-4 and Claude excelled. However, some models, including GPT-3.5, missed key security elements.
Security warning: Many models overlooked critical conditions, such as ensuring the lock time was in the future, a vulnerability that could allow bad actors to exploit the contract.

3. Custom Token Creation

When tasked with creating an ERC20-based token, only GPT-4 recognized the utility of OpenZeppelin, a widely-used library for secure smart contracts, whereas other models tried to code from scratch, leading to inefficiencies.

Breaking Down the Results: Strengths and Weaknesses

Best Performers: Models like Claude and GPT-4 came out on top, demonstrating high accuracy in basic smart contract creation.
GPT-4 vs. GPT-3.5: The newer GPT-4 outperformed GPT-3.5 across nearly every task, especially in terms of producing concise and functional code.
Security Concerns: Many models, including GPT-4, overlooked security aspects unless explicitly instructed. For example, the prompt might miss a necessary check for time-based conditions, which could introduce vulnerabilities.
Complexity Challenges: The more complex the task, the more issues arose. For instance, no model successfully created an ERC721 token due to the complexity of this type of contract.

Why These Results Matter: AI as a Coding Assistant (for Now)

The study’s results reveal that while LLMs like GPT-4 and Claude are promising for generating simple, secure contracts, they fall short in more complex scenarios. Here’s a quick takeaway:

Great for Basic Contracts: Tasks like simple storage or conditional Ether release were well-handled by most models.
Struggle with Complex Logic: When coding for more complex conditions or token structures, the models frequently produced code with security gaps or inefficiencies.
Inconsistent Security: Security is a top priority for smart contracts, and unfortunately, many AI models overlook it unless clearly specified.

This highlights that while AI can assist in generating code, smart contract developers need to closely review AI-generated contracts for possible security issues. 💡

Future Prospects: Enhancing AI for Smart Contracts 🔮

This study has sparked exciting possibilities for AI-assisted coding in blockchain but also uncovered critical areas needing improvement. Here’s what the future could hold:

Structured Prompting Evolution: Although structured prompts sometimes led to errors, with further AI training, this approach could become essential. Providing clear, code-like prompts could help LLMs navigate complex coding tasks more precisely.
Expanding to Other Blockchains: Ethereum is just one of many blockchain platforms. Testing LLMs on different blockchains like Solana and Aptos could reveal more about their versatility and potential.
Improving Security Awareness in AI: Security is paramount in blockchain, and this study points to a need for more security-focused training in LLMs. Integrating formal verification methods and security analysis tools could ensure that AI-generated contracts are as secure as manually written ones.
Enhancing AI-Aided Debugging: Rather than fully generating smart contracts, AI could assist developers with debugging and auditing, making it a helpful tool for identifying and correcting vulnerabilities.
Human-AI Collaboration in Code: Rather than replacing developers, AI could become a powerful assistant, enhancing developer productivity by automating simpler tasks and freeing them to focus on complex security aspects.

Final Thoughts: AI’s Potential as a Blockchain Developer’s Assistant 🤖✨

AI in smart contract generation is still in its early days, showing promise but also significant limitations. While models like GPT-4 and Claude have demonstrated their capabilities in simple tasks, the complexity of blockchain security highlights areas where human oversight is indispensable. As we move forward, further refinement in prompting strategies, security measures, and cross-platform testing could pave the way for safer, smarter AI-driven contract development.

For now, AI might not fully replace blockchain developers, but it’s well on its way to becoming an essential tool in their coding toolkit. Keep an eye out—this is only the beginning of what AI can bring to the decentralized world!

Concepts to Know

Smart Contracts: Think of these as digital agreements that run automatically on the blockchain when conditions are met. They’re secure, tamper-proof, and remove the need for middlemen. - This concept has been also explained in the article "Revolutionizing Elections with Blockchain: The Future of Secure Voting 🗳️".
Blockchain: A decentralized digital ledger where information is stored across many computers. It's the backbone of cryptocurrencies like Ethereum and Bitcoin, making transactions secure and transparent. - This concept has been also explained in the article "Revolutionizing Elections with Blockchain: The Future of Secure Voting 🗳️".
Solidity: This is the programming language used to write smart contracts on Ethereum. It’s designed specifically for blockchain apps and has unique rules to ensure code can’t be changed once it’s deployed.
Ethereum: A blockchain platform enabling smart contracts and DApps. Ethereum lets you store and move value like Bitcoin but adds the power to execute code directly on the blockchain.
Large Language Models (LLMs): These are advanced AI models trained to understand and generate human-like text. Examples include OpenAI's GPT series and Google’s Bard, which can generate anything from simple text to complex code. - This concept has been also explained in the article "Explaining the Power of AI in 6G Networks: How Large Language Models Can Cut Through Interference 📶🤖".
OpenZeppelin: A toolkit of libraries and standards used for creating secure smart contracts. Developers often rely on OpenZeppelin to ensure their contracts are safe and follow best practices.
ERC20 and ERC721: These are technical standards for creating tokens on Ethereum. ERC20 tokens are simple and widely used, while ERC721 tokens are used for unique assets, like NFTs.
Prompting: In AI, prompting refers to giving models specific instructions or hints to generate the desired output. It can be descriptive (plain language) or structured (like pseudo-code). - This concept has been also explained in the article "🗣️ Speak My Language: Unlocking the Power of Prompts in AI 🔓".
Gas Fees: On Ethereum, gas fees are the transaction costs paid in ETH to process actions, like running a smart contract. These fees vary based on the complexity of the code and network demand.
Formal Verification: This is a method to mathematically prove that a smart contract does exactly what it’s supposed to do, making it super secure. It’s crucial in blockchain because code on the blockchain can’t be changed after it’s deployed!