Hi,
I’m trying to reproduce the section “How do I run it locally?” from this blog post:
Personal Copilot: Train Your Own Coding Assistant (huggingface.co)
When I execute step number 2, I get the following error:
(copilot) --@-- python % pip install -e "."
Obtaining file:///Users/--/GitLab/---copilot/mlc-llm/python
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
Ă— python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [15 lines of output]
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/Users/--/GitLab/---copilot/mlc-llm/python/setup.py", line 47, in <module>
LIB_LIST, __version__ = get_lib_path()
^^^^^^^^^^^^^^
File "/Users/--/GitLab/---copilot/mlc-llm/python/setup.py", line 27, in get_lib_path
libinfo["find_lib_path"]("mlc_llm")[0],
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/--/GitLab/---copilot/mlc-llm/python/./mlc_chat/libinfo.py", line 72, in find_lib_path
raise RuntimeError(message)
RuntimeError: Cannot find libraries: libmlc_llm.dylib
List of candidates:
/Users/--/GitLab/---copilot/mlc-llm/python/mlc_chat/libmlc_llm.dylib
/Users/-/opt/miniconda3/envs/copilot/lib/libmlc_llm.dylib
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
Ă— Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
However, if I do:
cd ..
and execute:
pip install -e "."
from:
/Users/--/GitLab/---copilot/mlc-llm
The install is successful.
Then, Step number 3 (Model compilation) is successful.
But, Step 5, fails. And I get the following error:
/Users/--/opt/miniconda3/envs/copilot/bin/python: Error while finding module specification for 'mlc_chat.rest' (ModuleNotFoundError: No module named 'mlc_chat')
However, if I install:
pip install --pre --force-reinstall mlc-chat-nightly -f https://mlc.ai/wheels
and execute Step 5, then I get the following error:
(copilot) --@-- mlc-llm % python -m mlc_chat.rest --model dist/starcoder-personal-copilot----q4f16_1/params --lib-path dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so
INFO: Started server process [9607]
INFO: Waiting for application startup.
ERROR: Traceback (most recent call last):
File "/Users/--/opt/miniconda3/envs/copilot/lib/python3.11/site-packages/starlette/routing.py", line 677, in lifespan
async with self.lifespan_context(app) as maybe_state:
File "/Users/--/opt/miniconda3/envs/copilot/lib/python3.11/contextlib.py", line 204, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/--/opt/miniconda3/envs/copilot/lib/python3.11/site-packages/mlc_chat/rest.py", line 137, in lifespan
chat_mod = ChatModule(
^^^^^^^^^^^
File "/Users/--/opt/miniconda3/envs/copilot/lib/python3.11/site-packages/mlc_chat/chat_module.py", line 710, in __init__
self._reload(self.model_lib_path, self.model_path, user_chat_config_json_str)
File "/Users/--/opt/miniconda3/envs/copilot/lib/python3.11/site-packages/mlc_chat/chat_module.py", line 922, in _reload
self._reload_func(lib, model_path, app_config_json)
File "tvm/_ffi/_cython/./packed_func.pxi", line 332, in tvm._ffi._cy3.core.PackedFuncBase.__call__
File "tvm/_ffi/_cython/./packed_func.pxi", line 263, in tvm._ffi._cy3.core.FuncCall
File "tvm/_ffi/_cython/./packed_func.pxi", line 252, in tvm._ffi._cy3.core.FuncCall3
File "tvm/_ffi/_cython/./base.pxi", line 182, in tvm._ffi._cy3.core.CHECK_CALL
File "/Users/--/opt/miniconda3/envs/copilot/lib/python3.11/site-packages/tvm/_ffi/base.py", line 481, in raise_last_ffi_error
raise py_err
tvm.error.InternalError: Traceback (most recent call last):
File "/Users/catalyst/Workspace/mlc-ai-package-self-runner/_work/package/package/tvm/src/runtime/dso_library.cc", line 125
InternalError: Check failed: (lib_handle_ != nullptr) is false: Failed to load dynamic shared library dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so dlopen(dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so, 0x0005): tried: 'dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OSdist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so' (no such file), '/Users/--/opt/miniconda3/envs/copilot/bin/../lib/dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so' (no such file), '/usr/lib/dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so' (no such file, not in dyld cache), 'dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/Users/--/GitLab/---copilot/mlc-llm/dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/--/GitLab/---copilot/mlc-llm/dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so' (no such file), '/Users/---/GitLab/---copilot/mlc-llm/dist/starcoder-personal-copilot----q4f16_1/starcoder-personal-copilot----q4f16_1-metal.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))
ERROR: Application startup failed. Exiting.
Maybe @smangrul could help with this?
Any help would be appreciated.
Best.