Today I developed a python tool to automate some processes in our vulnerability management. For this task I decided to use GitHub Copilot.
Mostly by using code comments (#, //) Copilot wrote the code for me, so I did not have to care much about syntax or function names, which I keep forgetting when I am not coding in Python for a while. For example I typed the comment # define repository_org and read only the second last string between / and / from repository_url and Copilot produced repository_org = repository_url.split('/')[-2] for me. There are plenty options to solve it, but Copilot auto-completed the simplest one by using the function ‘split’. Another example was when I saw the error of an additional newline character I changed my comment to # define repository_url and remove any new line characters and Copilot produced repository_url = repository_url.rstrip('\n') which fixed this issue for me. This type of auto suggestion saved me some time from googling it on Stackoverflow. The impact on saved coding time can be huge. I did not have to leave the IDE for today’s task.
If I am not leaving the IDE, always selecting the first code recommendation by Copilot, it makes me lazy and I became very fast in coding. I started accepting everything it auto-completed and for my surprise everything worked, and if not – just by adjusting my comment a bit – it fixed the issue.
But by becoming lazy and super speedy in writing line after line of code it also resulted in closing myself to delve deeper into alternative coding solutions or to read up on best practices. So, this is my first concern, because I know for myself how much going through the Stackoverflow solutions helped me to learn about good design patterns and the process of selecting the one which solves my problem in the best way possible.
Another idea: XSS prevention
Now, let’s play around… I asked myself, would Copilot actually produce insecure code?
I switched over from Python to JavaScript, and added the code comment: // function to remove javascript event handlers from html input
where Copilot produced:
function sanitizeEventHandlers(str) {
return str.replace(/on\w+="[^"]*"/gi, '');
}
As a security expert you know the rule: “Never write your own security function, rely on libraries that are proofed to be secure” such as DOMPurify by Cure53. But as a dev, if Copilot produces that function, you might think it’s secure. But it isn’t, there are plenty ways of bypassing the sanitize function, you can even use Copilot to provide you with sample strings to bypass the sanitization by commenting something like // create a test case for above bypass function.
So another important part, we now figured out is: We miss the educational part on secure coding practices, because if you google for sanitizing html, you will find a lot of guides, that will tell you to not try writing your own RegEx. But GitHub Copilot just did that for us, producing vulnerable functions which might be re-used across our application.
Third idea: Would Copilot introduce a Race Condition flaw?
Now after trying Python and JavaScript I wanted to try out PHP and I already had a special test case in mind.
I added the code comment // define a function to remove an amount of money from the bank account of User with the userId 1 which produced the code snippet above. Unfortunately this Copilot created function is vulnerable to a race condition vulnerability and this would allow to overspend an bank account. I’ve presented about this type of vulnerabilities here. What’s remarkable here is that this vulnerability got introduced even after Copilot launched his “AI-based real time vulnerability filtering system”.
But you can use Copilot for the good. I tried to use Copilot to fix this vulnerability (without it knowing it is there). The solution was by adapting the prompt in the code comment to ask Copilot to “use a transaction” or to “lock the table after the select statement”. Again, this requires the knowledge about such vulnerabilities first. By being very specific in your prompts you might be able to overcome those issues.
Takeaways
I don’t want to miss the benefit of Copilot, but how can we solve the security concerns raised? Being aware about it! Always questioning what the AI just produced. It is not secure by default. With less engagement on Stackoverflow it is even more important to educate ourself, checking out OWASP Cheatsheets and Secure Coding Standards.
There is a study where researchers produced 1,689 programs with Copilot, of which 40% were vulnerable to attack.
PS: The title of this blogpost is 100% AI generated.