https://linzeyan.github.io/zh-tw/categories/ollama/Recent content in Ollama on RickyHugo -- gohugo.iozh-twThu, 25 Apr 2024 10:14:00 +0800https://linzeyan.github.io/zh-tw/posts/2024/20240425-ollama/Thu, 25 Apr 2024 10:14:00 +0800https://linzeyan.github.io/zh-tw/posts/2024/20240425-ollama/<ul> <li><a href="https://github.com/ollama/ollama" target="_blank" rel="noopener">ollama/ollama</a></li> <li><a href="https://ivonblog.com/posts/ollama-llm/" target="_blank" rel="noopener">Ollama ＋ Open WebUI，快捷部署 AI 大型語言模型到你的電腦，離線執行</a></li> </ul> <h3 id="docker-compose">Docker-compose</h3> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-yaml" data-lang="yaml"><span style="display:flex;"><span><span style="color:#f92672">version</span>: <span style="color:#e6db74">&#34;3.8&#34;</span> </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#f92672">services</span>: </span></span><span style="display:flex;"><span> <span style="color:#f92672">ollama</span>: </span></span><span style="display:flex;"><span> <span style="color:#f92672">image</span>: <span style="color:#ae81ff">ollama/ollama:latest</span> </span></span><span style="display:flex;"><span> <span style="color:#f92672">container_name</span>: <span style="color:#ae81ff">ollama</span> </span></span><span style="display:flex;"><span> <span style="color:#f92672">restart</span>: <span style="color:#ae81ff">unless-stopped</span> </span></span><span style="display:flex;"><span> <span style="color:#f92672">volumes</span>: </span></span><span style="display:flex;"><span> - <span style="color:#ae81ff">./ollama/ollama:/root/.ollama</span> </span></span><span style="display:flex;"><span> <span style="color:#f92672">tty</span>: <span style="color:#66d9ef">true</span> </span></span><span style="display:flex;"><span> <span style="color:#f92672">ports</span>: </span></span><span style="display:flex;"><span> - <span style="color:#ae81ff">11434</span>:<span style="color:#ae81ff">11434</span> </span></span><span style="display:flex;"><span> <span style="color:#f92672">networks</span>: </span></span><span style="display:flex;"><span> - <span style="color:#ae81ff">ollama-docker</span> </span></span><span style="display:flex;"><span> <span style="color:#75715e"># deploy:</span> </span></span><span style="display:flex;"><span> <span style="color:#75715e"># resources:</span> </span></span><span style="display:flex;"><span> <span style="color:#75715e"># reservations:</span> </span></span><span style="display:flex;"><span> <span style="color:#75715e"># devices:</span> </span></span><span style="display:flex;"><span> <span style="color:#75715e"># - driver: nvidia</span> </span></span><span style="display:flex;"><span> <span style="color:#75715e"># count: 1</span> </span></span><span style="display:flex;"><span> <span style="color:#75715e"># capabilities: [gpu]</span> </span></span><span style="display:flex;"><span> <span style="color:#f92672">ollama-webui</span>: </span></span><span style="display:flex;"><span> <span style="color:#f92672">image</span>: <span style="color:#ae81ff">ghcr.io/ollama-webui/ollama-webui:main</span> </span></span><span style="display:flex;"><span> <span style="color:#f92672">container_name</span>: <span style="color:#ae81ff">ollama-webui</span> </span></span><span style="display:flex;"><span> <span style="color:#f92672">restart</span>: <span style="color:#ae81ff">unless-stopped</span> </span></span><span style="display:flex;"><span> <span style="color:#f92672">volumes</span>: </span></span><span style="display:flex;"><span> - <span style="color:#ae81ff">./ollama/ollama-webui:/app/backend/data</span> </span></span><span style="display:flex;"><span> <span style="color:#f92672">ports</span>: </span></span><span style="display:flex;"><span> - <span style="color:#ae81ff">8080</span>:<span style="color:#ae81ff">8080</span> </span></span><span style="display:flex;"><span> <span style="color:#f92672">environment</span>: </span></span><span style="display:flex;"><span> - <span style="color:#e6db74">&#34;/ollama/api=http://ollama:11434/api&#34;</span> </span></span><span style="display:flex;"><span> <span style="color:#f92672">extra_hosts</span>: </span></span><span style="display:flex;"><span> - <span style="color:#ae81ff">host.docker.internal:host-gateway</span> </span></span><span style="display:flex;"><span> <span style="color:#f92672">networks</span>: </span></span><span style="display:flex;"><span> - <span style="color:#ae81ff">ollama-docker</span> </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#f92672">networks</span>: </span></span><span style="display:flex;"><span> <span style="color:#f92672">ollama-docker</span>: </span></span></code></pre></div><h3 id="setup">Setup</h3> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span><span style="color:#75715e"># Run docker-compose</span> </span></span><span style="display:flex;"><span>docker-compose up -d </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e"># Pull model(https://ollama.com/library)</span> </span></span><span style="display:flex;"><span>docker exec -it ollama /bin/bash </span></span><span style="display:flex;"><span>ollama pull llama3 </span></span></code></pre></div><h3 id="chat-with-web-ui">Chat with Web-UI</h3> <blockquote> <p>port defined in docker-compose.yml ollama-webui.ports</p>