https://linzeyan.github.io/tags/mlx/Recent content in MLX on RickyHugo -- gohugo.ioenMon, 01 Jun 2026 10:58:17 +0800https://linzeyan.github.io/posts/2026/20260601-local-llm/Mon, 01 Jun 2026 10:58:17 +0800https://linzeyan.github.io/posts/2026/20260601-local-llm/<ul> <li><a href="https://huggingface.co/mlx-community" target="_blank" rel="noopener">https://huggingface.co/mlx-community</a></li> <li><a href="https://github.com/ml-explore/mlx-lm" target="_blank" rel="noopener">https://github.com/ml-explore/mlx-lm</a></li> <li><a href="https://github.com/jundot/omlx" target="_blank" rel="noopener">https://github.com/jundot/omlx</a></li> </ul> <h4 id="install">Install</h4> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span><span style="color:#75715e"># install hf</span> </span></span><span style="display:flex;"><span>uv tool install <span style="color:#e6db74">&#34;huggingface_hub&#34;</span> --with hf-xet --upgrade </span></span><span style="display:flex;"><span>hf auth login </span></span><span style="display:flex;"><span>hf auth whoami </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e"># install mlx_lm</span> </span></span><span style="display:flex;"><span>uv tool install --force <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> --from <span style="color:#e6db74">&#39;git+https://github.com/ml-explore/mlx-lm.git@refs/pull/1192/head&#39;</span> mlx-lm <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> --with <span style="color:#e6db74">&#39;transformers @ git+https://github.com/huggingface/transformers.git&#39;</span> </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e"># install mlx-openai-server</span> </span></span><span style="display:flex;"><span>uv pip install git+https://github.com/cubist38/mlx-openai-server.git --system </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e"># install [litellm](https://github.com/BerriAI/litellm)</span> </span></span><span style="display:flex;"><span>uv tool install <span style="color:#e6db74">&#39;litellm[proxy]&#39;</span> </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e"># install [claw-code](https://github.com/ultraworkers/claw-code)</span> </span></span><span style="display:flex;"><span>cargo install agent-code </span></span></code></pre></div><h4 id="download-model">Download Model</h4> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>HF_HUB_DISABLE_XET<span style="color:#f92672">=</span><span style="color:#ae81ff">1</span> hf download mlx-community/DeepSeek-V4-Flash-4bit <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> --local-dir deepseekV4 --repo-type model </span></span></code></pre></div><h4 id="run-server">Run Server</h4> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>mlx_lm.server --model /Users/ricky/git/mlx/deepseekV4 --port <span style="color:#ae81ff">8080</span> </span></span></code></pre></div><p><strong>or</strong></p>