Can any model actually write current Rust?

Models
1

Screenshot_20240206_125925
Screenshot_20240206_1259251011×547 102 KB

CodeLlama-2-70B doing creative x86_64 assembly.

I love how these models are evolving and can even work with assemby but it seems that Rust is evolving even faster than them. Even CodeLlama-2-70B struggles in my case with this simple challenge:

Please write a sample Rust app to read a JSON file test.json and validate it against a JSON schema file schema.json.

Often it will give deprecated cargo crates or obsolete code which won’t compile or really just code that compiles but won’t do the job. I find this same behavior with Bard, ChatGPT and everything I’ve tried besides GPT-4, which managed to do it in three tries.

Can anybody find an open source model that can generate a clean code snippet to do this in Rust?

2

I can’t imagine that anyone currently keeps a Rust code model up to date due to a lack of data and computing capacity to constantly retrain it.

I think the best approach is to use Bing chat, which looks up sources from the internet and then writes an answer based on the sources ( kind of like a RAG). This is probably the best appraoch at the moment but I don’t want to go too far out on a limb, these are just my thoughts on it :sweat_smile:

3

Cody combines any of several LLMs with sourcegraph and generates pretty good Rust code for me.