< up >
2024-03-24

Chemnitzer Linuxtage 2024

I attended the Chemnitzer Linuxtage 2024 and it was great fun¹:

• date: 16.03.+17.03.
• 6 lecture halls
• lectures (german only
• regular ticket price: 12€
• workshop price: 5€

Disclaimer as within my last conference: I’m paraphrasing (italic) the speaker based on my notes and memories as best as I can.

Content

• Analoge Fotos mit freier Software digitalisieren
• Wie funktioniert ChatGPT?
• RocketLang - Mein eigener kleiner BER
• Der Compiler: Eine Einführung für Anfänger
• Mini-LLM selbst gemacht
• (Fast) alles was man an der Uni nicht lernt
• J Workskop
• Sticker

Analoge Fotos mit freier Software digitalisieren

The kernel developer Heiko Stübner held a talk (video) about digitalizing analog photos using free software. Since I’m a hobby photographer (see various self-made featured images), this was a great opportunity to get more in-depth knowledge.

• photo grains are not evenly distributed and differ in size, in conclusion digital pixels and analog grains don’t have an 1:1 mapping, resolution comparisons are hard
• digitalize negatives:
  • scan with a flatbed scanner with “Durchlichteinheit”² or DSLR
  • import into DarkTable
  • use negadoctor³
  • correct exposure
  • correct size/scale/rotation
• use shadow/sunlight parts of the image for proper white-balancing

I’ve asked what is the best setup to start with: The best analog camera is the on you can get on ebay.[…] If you already have a digital camera, stick with the brand to re-use the equipment.[…]. Start with negative films Fomapan or Kodak Gold.

Wie funktioniert ChatGPT?

Michael Christen’s lecture (video) was a historic journey through 120yrs. of artificial intelligence break-throughs.

• 1906 Markov Chains
• 1948 Claude Shanon introduces entropy within information theory and that it isn’t evenly distributed over an information
• 1957 Rosenblatt’s Perceptron Model
• 1980 Searle’s Chinese room
• 1986 Backpropagation and Autoencoder
• 1995 RNN - Recurren Neural Networks having loops within a network to enable memories (LSTM - Long Short Term Memory)
• 1998 CNN - Convolutional Neural Networks
• 2005 Embeddings encode natural language into tokens into vectors which can be fed into a network
• 2014 Attention Mechanism
• 2017 Transformer Architecture Attention is all you need
• 2018 GPT-1
• 2022 ChatGPT
• 2023 Llama by Meta
• 2023 Stanford Alpaca

Learnings:

• abstraction: reduce the count of nodes within a hidden layer forces the net to learn concepts to overcome this artificial barrier
• open_llm_leaderboard is a AI ranking site
• Mixtral 8x7B and OpenChat are better than ChatGPT
• models can be mixed by pairwise averaging the weights of two networks

RocketLang - Mein eigener kleiner BER

Robert (former colleague, now at Mozilla) gave a talk (video) about his implementation of MonkeyLang called RocketLang based on the book Writing An Interpreter In Go.

It was an insightful journey through one of those never-ending hobby-projects. It was interesting to see how Robert took language decisions and more interestingly reverted/fixed former decisions. He encouraged people to feel ok in abandoning personal projects for a while, come back later and to make more useless projects. I can totally relate to that. Take this blog for example: I write every now and then, correct old blog posts or correct nothing at all. Who is judging my posts anyway apart from myself? This is fun and it should stay fun :)

Shameless plug: I’ve introduced itterable integer in RocketLang.

Der Compiler: Eine Einführung für Anfänger

This compressed roller coaster ride (video) through my compiler module back then at the HTW by Benjamin Stürz reminded me of the good old times :‘)

Mini-LLM selbst gemacht

The learning about language models within this lecture (video) was insane. Martin Neitzel showed how a minimal LLM (Large Language Model) works using the array-language J and markov chains. It’s basically “autocomplete on steroids”.

Minimal approach using one follow-up character:

1. define an input text, e.g. Chemnitzer Linuxtage
2. for each unique character, make a list about each follow-up character, e.g. n -> [i,u]
3. end of the input is connected to the beginning, so e -> [m,c]
4. define an input char and grab a random character from its map, e.g. f('n') = random([i,u]) = i

This works similar to autocomplete but rather than suggesting every follow-up character, it chooses a random one.

Extended approach with n followup characters and m output length:

1. use larger input, e.g. scrape CLT24 website
2. for each unique character, make a list about each n follow up string, e.g. with n=2, e -> [mn, ch]
3. define an input string, append a random followup and repeat until m has been reached

Outputs with different parameters:

• n=1,m=300: looks gibberish
• n=4,m=300: german words are noticeable
• n=10,m=300: text is clear readable, although meaning and grammar might be off

It was very interesting to see how a simple markov-chain can be used to generate readable text. We¹ were so excited about the lecture and the J language, that we registered for the J Workshop.

(Fast) alles was man an der Uni nicht lernt

The lecture (video) from the SUSE Software Engineer Dan Čermák was about contributing to- and maintaining open-source projects. He described his experiences with open-source communities and their dynamics: Technology changes, [but] you always have to work with people.

J Workskop

• Paper
• J Homepage
• Primer
• Playground
• Vocabulary

We¹ had three awesome hours of learning the fundamentals of the array-language J from the inventor Kenneth E. Iversion.

• everything is an n-dimensional array
• primitives over standard library: there are tons of tokens consisting of multiple special chars that have different meaning depending if they’re monadic (only left argument) or dyadic (both arguments):
  • x <.: round x down
  • x <. y: take minimum of x and y
  • x *: return the sign of x
  • x * y: multiply x and y
• Iverson doesn’t want to invent/coin new terms but re-use already existing:
  • token, function: verb
  • data: noun

Sticker