Älyverkko CLI application setup

The application is configured using a YAML-formatted configuration file. Below is an example of how the configuration file might look:

mail_directory: "/home/user/AI/mail"
models_directory: "/home/user/AI/models"
default_temperature: 0.7
llama_cpp_dir_path: "/home/user/AI/llama.cpp/"
batch_thread_count: 10
thread_count: 6
prompts_directory: "/home/user/.config/alyverkko-cli/prompts"
models:
  - alias: "default"
    filesystem_path: "WizardLM-2-8x22B.Q5_K_M-00001-of-00005.gguf"
    context_size_tokens: 64000
    end_of_text_marker: null
  - alias: "mistral"
    filesystem_path: "Mistral-Large-Instruct-2407.Q8_0.gguf"
    context_size_tokens: 32768
    end_of_text_marker: null

Here are available parameters:

mail_directory

Directory where AI will look for files that contain problems to solve.

models_directory

Directory where AI models are stored.

• This option is mandatory.

prompts_directory

Directory where prompts are stored.

Example prompts directory content:

default.txt
writer.txt

Prompt name is file name without extension. File extension should be txt.

Example content for writer.txt:

You are best-selling book writer.

default_temperature

Defines the default temperature for AI responses, affecting randomness in the generation process. Lower values make the AI more deterministic and higher values make it more creative or random.

• Default value: 0.7

llama_cpp_dir_path

Specifies the filesystem path to the cloned and compiled llama.cpp directory.

• Example Value: home/user/AI/llama.cpp
• This option is mandatory.

batch_thread_count

Specifies the number of threads to use for input prompt processing. CPU computing power is usually the bottleneck here.

• Default value: 10

thread_count

Sets the number of threads to be used by the AI during response generation. RAM data transfer speed is usually bottleneck here. When RAM bandwidth is saturated, increasing thread count will no longer increase processing speed, but it will still keep CPU cores unnecessarily busy.

• Default value: 6

models

List of available large language models.

alias

Short model alias. Model with alias "default" would be used by default.

filesystem_path

File name of the model as located within models_directory

context_size_tokens

Context size in tokens that model was trained on.

end_of_text_marker

Some models produce certain markers to indicate end of their output. If specified here, Älyverkko CLI can identify and remove them so that they don't leak into conversation. Default value is: null.

Once Älyverkko CLI is installed and properly configured, you can run following command at commandline to see what models are available to it:

alyverkko-cli listmodels

The selftest command performs a series of checks to ensure the system is configured correctly:

alyverkko-cli selftest

It verifies:

• Configuration file integrity.
• Model directory existence.
• The presence of the llama.cpp executable.

1. Open your terminal.

2. Run the command:

   alyverkko-cli mail
   

3. The application will start monitoring the configured mail directory for incoming messages and process them accordingly in endless loop.
4. To terminate Älyverkko CLI, just hit CTRL+c on the keyboard, or close terminal window.

1. Access the application launcher or application menu on your desktop environment.
2. Search for "Älyverkko CLI".
3. Click on the icon to start the application. It will open its own terminal.
4. If you want to stop Älyverkko CLI, just close terminal window.

During Älyverkko CLI installation, installation script will prompt you if you want to install systemd service. If you chose Y, Alyverkko CLI would be immediately started in the background as a system service. Also it will be automatically started on every system reboot.

To view service status, use:

systemctl -l status alyverkko-cli

If you want to stop or disable service, you can do so using systemd facilities:

sudo systemctl stop alyverkko-cli
sudo systemctl disable alyverkko-cli