Author: Ralph Soika

LLM stands for Large Language Model and is a large-scale AI model that has been trained with an extensive amount of text and code. Beside the well known and widespread Chat GPT, today there are many powerful Open Source alternatives available. The advantage of an Open Source LLM is that you can use such a model in your own application within your own environment. There is no dependency on an external service provider that can raise prices, shut down services, or remove models.

But the question that inevitably arises is: Where to start? At least that’s the question I asked myself. After some research I found out that it isn’t such difficulty as it sounds to run a local LLM.

First of all there is a place called Hugging Face providing a kind of market place for all kinds of AI models. After you have registers yourself on the page you can search and download all kinds of different Models. Of course each model is different and addresses different needs and requirements. But the good news is that there is a kind of common open standard to run a LLM called LLaMA CCP. Lamma CCP allows you to run a LLM with minimal setup and state-of-the-art performance on a wide variety of hardware – locally and in the cloud. And of course there is also a Python binding available. And this makes is easy to test a LLM in a Docker container.

Continue reading “How to Run LLMs in a Docker Container”

This is just a short collection of Git commands and tricks which I personally did not always remember.

Create a new Tag

To create and push a new tag:

1.) List current tags

$ git tag

2.) Create a new Tag

$ git tag -a <TAG-VERSION> -m "next release" 

3.) Push tag

By default, the git push command doesn’t transfer tags to remote servers. You will have to explicitly push tags to a shared server after you have created them.

$ git push origin <TAG-VERSION>

Create a Branch

To list all existing branches:

$ git branch

to create a new local branch

$ git branch <branch>

and checkout it with

$ git checkout <branch>

to push the branch to the remote repo

$ git push origin <branch>

Merge a Branch

Merge another branch into the current (e.g. into the master branch)

List all the branches in your local Git repository using the git branch command:

$ git branch

The output shows all branches and marks the current branch with an *.

Ensure you are on the branch you want to merge into. To switch to the master branch:

$ git checkout master

Now you can start merging. Since merging is a type of commit, it also requires a commit message.

$ git merge -m "Your merge commit message" [source_branch]

Check the result in your current file tree.

Finally push your changes:

$ git push origin

How to resolve Merge Conflicts

Sometime you may forget to pull before you start working on something. Later you can not push your commit directly if a colleague has worked on some other artifacts. In this case you can do pull --rebase. This will resolve the conflict in most cases.

$ git pull --rebase

In any case if your pull produces a merge conflict you still will be warned by git.

Git: pull.rebase – Fast-forward

By default, git pull uses a merge strategy. This can cause issues when you try to push your changes while a colleague has already pushed commits to the same branch – even if the changes don’t conflict at all. Your push gets rejected, and you have to manually pull and merge first.

Setting pull.rebase=false changes the behavior of git pull to automatically merge the remote changes into your local repo.

To enable it globally:

$ git config --global pull.rebase false

This applies to all Git tools – whether you use the terminal, VS Code, Eclipse, or any other IDE.

In Debian 12 the default JDK is Java 17. In case you need Java 11 instead you can follow this blog from Linux Shout .

Here is the short version:

1) Edit your sources.list

Edit the file /etc/apt/sources.list and add the unstable packages at the end of the file

deb http://deb.debian.org/debian unstable main non-free contrib

2) Next Update your apt preferences

Edit the file /etc/apt/preferences and add the following entry:

Package: *
Pin: release a=stable
Pin-Priority: 900

Package: *
Pin: release a=unstable
Pin-Priority: 50

This will make our Debian 12 system only choose the stable packages while updating instead of unstable ones.

3) Install JDK 11

Now you can install JDK 11 and switch the java version using the

$ sudo apt update
$ sudo apt install openjdk-11-jdk
$ sudo update-alternatives --config java

The last command allows you to switch between JDK 17 and JDK 11.

Flameshot is a nice screen capture tool allowing you to mark a screenshot with lines and text and save the screenshot or copy it into the clipboard.

I uses this tool since years. But on Debian 12 it seems not to work. At least it does not open on my installation.

The trick is to start the program form a terminal window with the option gui

$ flameshot gui

Very large companies often tend to hire very large IT consulting firms to solve their IT problems. Often with moderate success. You can observe this all over the world when you read about failed big IT projects.

So why is this happening? Large IT consulting firms are perfect at telling you that they are the only ones able to solve your problem quickly and reliably. Typically, they also have a real expert who recognizes the problem very well and has an idea of a working solution. You should memorize this person’s face well, because you may never see this expert again. And he probably won’t be the one to solve your problem.

But what exactly is happening here? Why should an international big IT consulting company not be the right expert for your IT problem? To understand this, you simply think about the business model of such companies. It consists of selling you IT services at a fixed daily or hourly rate. This means that the more hours you buy, the better it is for the consulting firm. In order to further maximize profits, these companies are often looking for teams consisting of young enthusiastic university graduates. Typically in a completely different place in the world, where much lower wages are paid. It may be that you are lucky and you find an expert able to solve your problem. But it can also happen that he is confronted with this problem for the first time in his life.

So, now you have an international IT team managed by a large consulting firm and you may continue to have your original IT problem for a very long time. How can you solve this situation in a better way? It’s basically quite simple. Just take a look at the open source community. For every problem in IT, there is someone who deals with it. And if he or she makes the problem public and also publishes his proposed solution, then that’s your IT expert! If the problem is big enough, it may happen that the open source project grows a little. This means that there are now more than one experts able to solve your problem. You will save yourself a lot of money and time if you try to solve your IT problems in this way in the future.

So the biggest problem for you is actually only to find these IT experts in the open source community. However, you should not hire a large IT consulting company to search for such experts. If you’re wondering why – just read this article again from the beginning.