I agree that this could be helpful for finally getting to that natural language programming paradigm that people have been hoping for. But there’s going to have to be something capable of logically implementing the low level code and that has to be more than just a statistical model of how people write code, as trained on a big collection of random repositories. (I’m open to being proved wrong about that!)
The 90% accuracy could just arise from the fact that the tests are against trivial or commonly solved tasks, so that the exact solutions to them exist in the training set. Anything novel will exist outside the training set and outside of the model.
Performing procedural tasks using a statistical model of our language will never be reliable. There’s a reason why we use logical and proscriptive syntax when we want deterministic outcomes.
I see what you mean about Jesse. It’s been a while since I played it, but I remember her being cast as an unwilling and unlikely hero, which can be a hard role to write well. Also, the acting was a bit stilted at first. I think she got better as it went (both the character and the voice acting).
