GeneralBots/botserver
2
1
Fork 0
Code Issues Pull requests Projects Releases 323 Packages Wiki Activity Actions

feat(llm): add deepseek reasoning flag and rename GPU layers

Browse source 
Update the LLM server command construction to include a new `--reasoning-format deepseek` argument, enabling explicit selection of the DeepSeek reasoning format. Replace the short `-ngl` flag with the more descriptive `--n-gpu-layers` to improve readability and consistency with other CLI options. This change enhances configurability for models requiring specific reasoning formats and clarifies GPU layer configuration.
This commit is contained in:
Rodrigo Rodriguez (Pragmatismo) 2025-11-07 17:16:58 -03:00
parent 312503ae66
commit 5379e21bfe
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
src/llm/local.rs
View file

@ -204,7 +204,7 @@ pub async fn start_llm_server(
// Build command arguments dynamically // Build command arguments dynamically
let mut args = format!( let mut args = format!(
"-m {} --host 0.0.0.0 --port {} --top_p 0.95 --temp 0.6 --repeat-penalty 1.2 -ngl {}", "-m {} --host 0.0.0.0 --port {} --reasoning-format deepseek --top_p 0.95 --temp 0.6 --repeat-penalty 1.2 --n-gpu-layers {}",
model_path, port, gpu_layers model_path, port, gpu_layers
); );
if !reasoning_format.is_empty() { if !reasoning_format.is_empty() {