Meta has just open sourced the latest version of its AI coding assistant, Code Llama 70B. This massive language model contains 70 billion parameters, making it the largest and most powerful version of Code Llama yet.
Code Llama 70B is designed to help developers write and debug code more efficiently. It can generate code based on natural language prompts, autocomplete code in real-time as developers type, and even identify bugs and suggest fixes for existing code.
According to Meta’s AI research team, Code Llama 70B significantly outperforms previous versions of the model. When benchmarked on the HumanEval test suite, it achieved 53% accuracy compared to 48.1% for GPT-3.5 and 67% for GPT-4 in internal OpenAI tests.
This improved performance allows Code Llama 70B to handle more complex programming tasks and longer blocks of code. Developers can feed it more detailed prompts, and it will generate more accurate and robust code in response.
Code Llama 70B is built on top of Meta’s LLaMA natural language model architecture. While LLama was originally developed for general NLP tasks, Meta customized and fine-tuned it specifically for programming languages and code.
The model was trained on over 1 terabyte of publicly available code from GitHub, as well as internal Meta code bases. This huge training set allowed it to learn multiple programming languages’ patterns, conventions and nuances.
In addition to the main 70B parameter Code Llama model, Meta also released two other specialized tools:
- Code Llama Python – Focused on Python only, with knowledge of common libraries and frameworks.
- Code Llama Instruct – Generates code line-by-line based on instructions in plain English.
All of the new Code Llama models are available on Hugging Face and ready to be used in developers’ own projects. They can be accessed via API, imported into code editors like VS Code, or used via command line tools.
Code Llama 70B represents the latest AI breakthrough aimed at boosting developer productivity. Other tech giants like Microsoft, Amazon and Google have also released programming-focused AI tools recently.
Microsoft integrated GitHub Copilot into Visual Studio Code last year, while Amazon launched its CodeWhisperer assistant in April. These leverage GPT-3 and Codex models trained by OpenAI.
However, Meta’s release of Code Llama under an open source license provides full transparency and modifiability. Developers can even fine-tune the model on their own codebases to tailor its knowledge.
The boon for productivity and reducing mundane coding tasks promised by AI has sparked plenty of excitement. But it has also drawn criticism from some developers worried about generative models producing insecure or biased code.
Others argue AI assistants could discourage developers from thoroughly learning skills and concepts needed to craft quality software.
Meta says it will continue expanding the capabilities of Code Llama models in line with responsible AI principles. The aim is to create an assistant that software engineers can trust and productively collaborate with.
The release of Code Llama 70B provides developers early access to try out and shape this emerging technology. With its impressive benchmarks and capabilities, it could well provide a glimpse of the AI-powered programming future to come.
