CodeLlama-2-70B doing creative x86_64 assembly.
I love how these models are evolving and can even work with assemby but it seems that Rust is evolving even faster than them. Even CodeLlama-2-70B struggles in my case with this simple challenge:
Please write a sample Rust app to read a JSON file test.json and validate it against a JSON schema file schema.json.
Often it will give deprecated cargo crates or obsolete code which won’t compile or really just code that compiles but won’t do the job. I find this same behavior with Bard, ChatGPT and everything I’ve tried besides GPT-4, which managed to do it in three tries.
Can anybody find an open source model that can generate a clean code snippet to do this in Rust?