A new AI programming system for developing computer code can achieve average human-level performance in solving programming contests, researchers report on December 8 in the journal Science.
The development of an AI-assisted coding platform capable of creating coding programs in response to a high-level description of the problem the code needs to solve could significantly impact programmers’ productivity.
Humans will increasingly come up with problems for the AI programming systems to solve
In fact, it could even change the culture of programming by shifting human work to formulating problems for the AI to solve.
To date, humans have been required to code solutions to novel programming problems.
Although some recent neural network models have shown impressive code-generation abilities, they still perform poorly on more complex programming tasks that require critical thinking and problem-solving skills, such as the competitive programming challenges human programmers often take part in.
In the present study, researchers from DeepMind (which is owned by Alphabet) were testing AlphaCode, an AI-assisted coding system that can achieve “approximately human-level performance” when solving problems from the Codeforces platform, which regularly hosts international coding competitions.
Using self-supervised learning and an encoder-decoder transformer architecture, AlphaCode solved previously unseen, natural language problems by iteratively predicting segments of code based on the previous segment and generating millions of potential candidate solutions.
These candidate solutions were then filtered and clustered by validating that they functionally passed simple test cases, resulting in a maximum of 10 possible solutions, all generated without any built-in knowledge about the structure of computer code.
As good as the “median human”
AlphaCode performed roughly at the level of a median human competitor when evaluated using Codeforces’ problems.
It achieved an overall average ranking within the top 54.3% of human participants when limited to 10 submitted solutions per problem, although 66% of solved problems were solved with the first submission. “
“Ultimately, AlphaCode performs remarkably well on previously unseen coding challenges, regardless of the degree to which it ‘truly’ understands the task,” said the study’s co-author J. Zico Kolter.
Other AI news:
- The 10 best AI movies of all time, based on their overall quality, cultural impact, and portrayal of AI.
- AI-enabled image compression could enable higher-resolution displays and holograms.
