serhii.net

In the middle of the desert you can say anything you want

03 Dec 2023

Masterarbeit eval task LMentry-lite-UA

Context: 220120-1959 taskwarrior renaming work tasks from previous work

Just tested this: DAMN!

2023-12-03-174516_813x723_scrot.png 2023-12-03-175001_773x692_scrot.png

User
Can you, in English, name one word for each of these tasks:
1. Rhymes with "chair"
2. Is a number larger than eleven
3. Has two letters "a"
4. Ends with the letter "k"
5. In the sentence "the cat had four paws and a good mood" is BEFORE the word "paws"

Also:
6. A sentence that starts with the word "dogs"
7. A sentence that ends with the word "beaver"
8. A sentence that uses the word "metal" twice

https://chat.openai.com/share/3fdfaf05-5c13-44eb-b73f-d66f33b73c59

lmentry/data/all_words_from_category.json at main · aviaefrat/lmentry

Not all of it needs code and regexes! lmentry/data/bigger_number.json at main · aviaefrat/lmentry

I can really do a small lite-lite subset containing only tasks that are evaluatable as a dataset.

LMentry-micro-UA

// minimal, micro, pico

Plan:

  • go methodically through all of those task, divide them into regex and not regex, clone the code translate the prompts generate the dataset

Decision on [[231010-1003 Masterarbeit Tagebuch#LMentry-micro-UA]]: doing a smaller version works!

Nel mezzo del deserto posso dire tutto quello che voglio.