
> [!WARNING]
> Downgrading a package and/or ignoring upgrades is not officially supported and carries risks of mismatched packages.
> See [Partial upgrades are unsupported](https://wiki.archlinux.org/title/System_maintenance#Partial_upgrades_are_unsupported)

##

- Find the previous version in the cache

```sh
# For pacman
ls -lha /var/cache/pacman/pkg/

# For yay
ls -lha ls -lha ~/.cache/yay/
```

- Perform an "upgrade" to the previous version

```sh
# For pacman, use the file:// uri
pacman -U file:///var/cache/pacman/pkg/package-old_version.pkg.tar.zst

# For yay, use the raw path
yay -U /home/user/.cache/yay/package_name/package-old_version.pkg.tar.zst
```

To ignore upgrades, see [Pacman - Skip package from being upgraded](https://wiki.archlinux.org/title/Pacman#Skip_package_from_being_upgraded)

## References

- [Arch Wiki - Downgrading packages](https://wiki.archlinux.org/title/Downgrading_packages)


Downgrading Arch Packages


## What is Evaluation
Evaluating the performance and capabilities (output) of an LLM based application checking various metrics such as:
- Accuracy
- Coherency
- Toxicity
- Many more

## Why
Directly evaluating one specific hosted LLM such as OpenAI's GPT ada may not be particularly useful outside of selecting a provider but evaluating the output of your tailored prompts and the flow of your LLM based application (agentic or otherwise) will help track how your application performs while leveraging an LLM.

The concept is not specific to LLMs as evaluating the the same metrics can be applicable to any ML based application (and any other complex algorithmic based process) but with the trend towards incorporating LLMs these notes are mainly framed around their use with LLMs.

## In CI
- Track the results *after* merging - need to separate what *may* go in vs. what **is** in the production (main) code
- Can use testing frameworks (e.g. jest) to boostrap the flow while developing specific CI actions

## Types
- Reference Free: Evaluate without examples of the expected output
- Ground Truth: Compare expected/reference output to the result

## Approaches
- LLM as judge evaluator = using an LLM to score the output
- Programmatic = using custom logic to check if the LLM output and/or function calls or correct

## Problems to solve
- Which measure(s) to use
- How to run them
- How to ensure consistency between runs

## Terms
| Term                      | Definitions                                                                                |
| ------------------------- | ------------------------------------------------------------------------------------------ |
| cot                       | chain of thought - reasoning                                                               |
| reference-free evaluation | Evaluate a response without examples/reference data                                        |
| ground truth evaluation   | Evaluate a response with an example expected response/data                                 |
| explicit score            | Prompt for a score between two numbers (e.g 1-5)                                           |
| implicit score            | Prompt expecting yes or no (e.g. "Is the submission harmful, offensive, or inappropriate?) |


## Evaluation Types to be Reviewed
- ROUGE (Recall-Oriented Understudy for Gisting Evaluation)
  - Set of metrics comparing n-gram overlaps
  - https://en.wikipedia.org/wiki/ROUGE_(metric)
  - https://aclanthology.org/W04-1013.pdf
- BLEU
  - bilingual evaluation understudy
  - https://en.wikipedia.org/wiki/BLEU
- F-score
  - [harmonic mean](https://en.wikipedia.org/wiki/Harmonic_mean "Harmonic mean") of the precision and recall
  - https://en.wikipedia.org/wiki/F-score
- Perplexity
  - Measure of the confidence a model has in its next word


### Accuracy/Precision/Recall
- Accuracy = correct predictions / total predictions (% correct)
  - Can be misleading if dataset is imbalanced (i.e. only 5% are matches, getting all wrong = 95% accuracy)
- Precision = how often positive predictions are correct = True Positives/ (True Positives + False Positives)
  - Does not account for false negatives
- Recall = how often true positives are correctly identified = True Positives / ( True Positives + False Negatives)
  - AKA: sensitivity or true positive rate
  - Does not account for false positives

## Topics to Research Further
- Grounding
- Relevance
- Efficiency
- Versatility
- Hallucinations
- Toxicity

## References
- [LangChain - Evaluation](https://python.langchain.com/v0.1/docs/guides/productionization/evaluation/)
- [LangChain - Evaluate an LLM Application](https://docs.smith.langchain.com/how_to_guides/evaluation/evaluate_llm_application)
- [Llama Index - Evaluation](https://docs.llamaindex.ai/en/stable/optimizing/evaluation/evaluation/)



LLM Evaluation


## Overview

With TypeScript, there is risk of `<T>` being considered a tag. Therefor when using an arrow function, you need to either provide an trailing comma, a second generic, or an extends.

## The Issue
```ts
// Possible errors:
// ❌ Cannot find name 'React'.
// ❌ Cannot find name 'T'.
// ❌ JSX element 'T' has no corresponding closing tag.
const test = <T>(value: T) => value;
```

## Solution 1 - Trailing Comma
```ts
// ✅
const test = <T,>(value: T) => value;
```

## Solution 2 - Extends Type
```ts
// ✅
const test = <T extends object>(value: T) => value;
```

## Solution 3 - Second Type
```ts
// ✅
const test = <TValueA, TValueB>(valueA: TValueA, valueB: TValueB) => valueA && valueB;
```

TypeScript: Generic on an Arrow Function


## The Problem

When an array is marked as a const (readonly), its type becomes a union of its values.
```ts
const VALID_VALUES = ['a', 'b'] as const;

const isValid = (value: string): boolean => (
  // ❌ Argument of type 'string' is not assignable to parameter of type '"a" | "b"'
  VALID_VALUES.includes(value);
)
```


## Solution 1 - Widen the type

Explicitly set the type to account for other possible values.

**Caveat**: Widens the type across the board.

```ts
const VALID_VALUES: Readonly<string[]> = ['a', 'b'] as const;

const isValid = (value: string): boolean => (
  // ✅
  VALID_VALUES.includes(value);
)
```

## Solution 2 - Assert the type during the check

Cast the constant during the check.

```ts
const VALID_VALUES = ['a', 'b'] as const;

const isValid = (value: string): boolean => (
  // ✅
  (VALID_VALUES as Readonly<string[]>).includes(value);
)
```

TypeScript: Check if an Array Constant Includes a Value


## Install the plugin
Installing the plugin is straight forward - just follow the [README in the GitHub](https://github.com/tauri-apps/tauri-plugin-sql).

## Migrations
This part I was unable to find documentation for but was able to infer how to add them from the code in plugin code ([`plugin.rs`](https://github.com/tauri-apps/tauri-plugin-sql/blob/dev/src/plugin.rs#L277)).

The migrations can be added by specifying the database URL and a vector of migrations:
```rust
tauri::Builder::default()
    .setup(|app| setup(app))
    .plugin(
        tauri_plugin_sql::Builder::default()
            .add_migrations(
                "sqlite:database_name.db",
                vec![Migration {
                    version: 1,
                    description: "setup tables",
                    sql: include_str!("./migrations/001.setup.sql"),
                    kind: MigrationKind::Up,
                }],
            )
            .build(),
    )
    .run(tauri::generate_context!())
    .expect("error while running tauri application");
```

For the migration SQL, I've put them in a folder `./migrations` and just numbered them sequentially.

I'm still new to rust so this might not be the best approach, but to move the migration definitions out of `main.rs` I moved it to a module `db_migrations.rs`:
```rust
use tauri_plugin_sql::{Migration, MigrationKind};

pub fn get_migrations() -> Vec<Migration> {
    return vec![Migration {
        version: 1,
        description: "setup tables",
        sql: include_str!("./migrations/001.setup.sql"),
        kind: MigrationKind::Up,
    }];
}
```
And then in `main.rs` updated the plugin definition to use this function:
```rust
// Include the added module
mod db_migrations;
...
.plugin(
    tauri_plugin_sql::Builder::default()
        .add_migrations(
            "sqlite:database_name.db",
            // Get the migrations using the functin defined in the module
            db_migrations::get_migrations()
        )
        .build(),
)
...
```

## Running Migrations on Launch
This is another part that I was unable to find in the documentation but was able to figure out reading the relevant code in [`plugin.rs`](https://github.com/tauri-apps/tauri-plugin-sql/blob/dev/src/plugin.rs#L296). For each database that you would like to run the migrations for at startup, you can add them to the `preload` option of the `sql` plugin configuration in `src-tauri/tauri.conf.json`:
```json
{
  "build": {
    ...
  },
  ...
  "plugins": {
    "sql": {
      "preload": [
        "sqlite:database_name.db"
      ]
    }
  }
}
```

## References
- [Plugin tauri-plugin-sql GitHub](https://github.com/tauri-apps/tauri-plugin-sql)
- [Tauri plugins GitHub](https://github.com/tauri-apps/plugins-workspace)
- [Tauri configuration docs](https://tauri.app/v1/references/configuration-files)

Setting up the Tauri SQL Plugin


> `_app.tsx`

```tsx
import { CSSTransition, TransitionGroup } from 'react-transition-group';

function App<P>({ Component, pageProps }: AppProps<P>) {
  const router = useRouter();

  return (
    <TransitionGroup className="app_transition_group">
      <CSSTransition
        key={router.pathname}
        classNames={animationClass}
        timeout={700}
        onExiting={() => {
          const mainElements = document.getElementsByTagName('main');
          if (mainElements.length > 1) {
            mainElements[1].style.marginTop = `${-scrollOffset.current}px`;
          }
        }}
        onEntered={() => {
          window.scrollTo(0, 0);
        }}
      >
        <Component {...pageProps} />
      </CSSTransition>
    </TransitionGroup>
  );
}
```

Major Issue - CSS modules are unloaded too early and an unstyled version of the page being navigated away from flashes unstyled. This thread discusses the issue and possible workarounds:
https://github.com/vercel/next.js/issues/17464

## Workaround

> `usePageTransitionFix.ts`

```ts
import { useRouter } from 'next/router';
import { useEffect, useRef } from 'react';

export const usePageTransitionFix = () => {
  const router = useRouter();
  const currentPathRef = useRef<string>(router.asPath);
  currentPathRef.current = router.asPath;

  useEffect(() => {
    // Disable server side style sheet removal labeled with data-n-p
    const serverStyleSheets = document.querySelectorAll('link[data-n-p]');
    serverStyleSheets.forEach((styleElement) =>
      styleElement.removeAttribute('data-n-p'),
    );

    // Disable styles removal done by setting media to 'x'
    const styleElements = document.querySelectorAll('style[media="x"]');
    styleElements.forEach((styleElement) =>
      styleElement.removeAttribute('media'),
    );
  }, [router.asPath]);
};
```

Another issue is scroll restoration - when navigating back, the page is scrolled to the top rather than back to where the user was last viewing:

https://github.com/vercel/next.js/issues/20951

## Reference

- [[NextJS Issue] CSS module styling is removed too early on route changes #17464](https://github.com/vercel/next.js/issues/17464)


NextJS - Animated Page Transitions


These are the settings that I use in tandem with [ESLint](https://eslint.org/) and [Prettier](https://prettier.io/).

1. Go to `File > Preferences > Settings`
2. Search for `On Save`
3. Under `Editor: Code Actions On Save` click `Edit in settings.json`
4. Set the following:
    ```json
    "editor.codeActionsOnSave": [
        "source.organizeImports",
        "source.fixAll.eslint"
    ]
    ```

This will organize imports alphabetically (`source.organizeImports`) and use `eslint` to fix any auto-fixable errors. Using prettier this will take care of formatting as well.

For organizing imports, I initially was only using `source.organizeImports` but this was removing the trailing commas on imports which conflicted with my prettier settings. By following it with `source.fixAll.eslint`, the prettier rules fix this at the same time.

## References
- [[StackOverflow] Enable trailing comma in vscode](https://stackoverflow.com/questions/66508443/enable-trailing-comma-in-vscode)

VSCode: Auto Format on Save (Prettier)


## Parent Expanding to the Size of Child Elements
```scss
.container {
  display: grid;
  // Fits columns to the available space, with a min width of 100px
  // and a max width of 1 fraction of the container
  grid-template-columns: repeat(auto-fit, minmax(100px, 1fr));
  gap: 8px;
  min-width: 150px;
}
```

## Child Elements Filling Parent
```scss
.container{
  display: flex;
  gap: 8px;
  min-width: 150px;

  .child {
    // flex: {flex-grow} {flex-shrink} {flex-basis}
    // Using flex-basis of 0, all elements have the same starting point
    flex: 1 1 0;
  }
}
```

## References
- [[StackOverflow] How to make flexbox items the same size?](https://stackoverflow.com/questions/29503227/how-to-make-flexbox-items-the-same-size)
- [[MDN] CSS Grid Layout](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Grid_Layout)
- [[CSS-Tricks] Auto-Sizing Columns in CSS Grid: `auto-fill` vs `auto-fit`](https://css-tricks.com/auto-sizing-columns-css-grid-auto-fill-vs-auto-fit/)

CSS Equal Width Elements

- [Window Size Changes](#window-size-changes)
- [Watching a Specific Element for Size Changes](#watching-a-specific-element-for-size-changes)
- [React Hook](#react-hook)
- [References](#references)

----

## Window Size Changes
If it your logic based off of the window size and not specific element sizing, an event listener can be added to the window for the `resize` event with:
```typescript
const callback = (evt: UIEvent) => console.log(evt);
window.addEventListener('resize', callback);
```

When done, be sure to remove the event listener with:
```typescript
window.removeEventListener('resize', callback);
```

## Watching a Specific Element for Size Changes
For cases where a specific element's size needs to be known, a `ResizeObserver` can be used. This is helpful for when the window doesn't resize but a element's size does (e.g. reflow due to content being added or removed).

To watch an element, the following can be used:
```typescript
const testDiv = document.getElementById('test');
const callback = (entries: ResizeObserverEntry[], observer: ResizeObserver) => console.log(entries);
const resizeObserver = new ResizeObserver(callback);
resizeObserver.observe(testDiv);
```

When done watching an element, it can be unobserved using:

```typescript
// Only stop watching a single element
resizeObserver.unobserve(testDiv);

// Or Alternatively, all elements can be unobserved using:
resizeObserver.disconnect();
```

## React Hook
This hook is a general purpose resize hook. It can be used to track the resizing of a specific element or when the window itself is resized.

If `elementToWatch` isn't passed, the whole window will be watched for resize events.

```typescript
import { useEffect } from 'react';

/**
 * Hookup a resize listener to the window or a target element
 * @param {() => void} callback The callback to call on resize (should be memoized with `useCallback` to avoid re-registering each render)
 * @param {boolean} callOnMount Whether to trigger the callback on mount
 * @param {HTMLElement | undefined} elementToWatch The element to watch for size changes
 */
export default function useOnResize(
  callback: () => void,
  callOnMount: boolean = false,
  elementToWatch?: HTMLElement,
) {
  useEffect(() => {
    // Watch a particular element if specified
    if (elementToWatch) {
      const resizeObserver = new ResizeObserver(callback);
      resizeObserver.observe(elementToWatch);
      if (callOnMount) {
        callback();
      }

      return () => {
        resizeObserver.disconnect();
      };
    }

    // If no element is specified, watch for window size changes
    window.addEventListener('resize', callback);
    if (callOnMount) {
      callback();
    }
    return () => {
      window.removeEventListener('resize', callback);
    };
  }, [callback, callOnMount, elementToWatch]);
}

```


## References
- [[MDN] ResizeObserver
](https://developer.mozilla.org/en-US/docs/Web/API/ResizeObserver)
- [[MDN] Window: resize event
](https://developer.mozilla.org/en-US/docs/Web/API/Window/resize_event)



Monitor for Resize in JavaScript

- [Configuration](#configuration)
  - [Redirect HTTP to HTTPS via 301](#redirect-http-to-https-via-301)
  - [Including SSL Certificate for HTTPS](#including-ssl-certificate-for-https)
  - [WebSocket Connections](#websocket-connections)
  - [Load Balancing](#load-balancing)
  - [Basic Full Example](#basic-full-example)
  - [Adding Custom Stream Configs](#adding-custom-stream-configs)
- [Commands](#commands)
- [PID Issue on Some Alpine Linux Docker Images](#pid-issue-on-some-alpine-linux-docker-images)
- [References](#references)

----

## Configuration

### Redirect HTTP to HTTPS via 301
```nginx
server {
  listen *:80;

  server_name k3s.functionalbytes.com;
  server_tokens off; ## Don't show the nginx version number

  location / {
    return 301 https://k3s.functionalbytes.com:443$request_uri;
  }
}
```

### Including SSL Certificate for HTTPS
```nginx
ssl_certificate /secret/keys/proxy.functionalbytes.com.fullchain;
ssl_certificate_key /secret/keys/proxy.functionalbytes.com.key;
```

### WebSocket Connections
See: [[nginx Documentation] WebSocket proxying](http://nginx.org/en/docs/http/websocket.html)
```nginx
location /web_socket_path/ {
    proxy_pass http://backend;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
}
```

### Load Balancing
See: [[nginx Documentation] HTTP Load Balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/)
```nginx
upstream k3s_servers {
  # Round Robin by default - can add "weight=#" after each server for weighting, e.g.:
  # server k3s-s1.functionalbytes.com:80 weight=3;
  # With this set and nothing on k3s-s2 (default = 1), 3 out of 4 requests will be sent to k3s-s1

  # Alternatively one of the following can be uncommented for other load balancing options

  # Uncomment the line below to use the one with the fewest connections
  # least_conn;

  # Uncomment to send requests to the server based on a hash of the first 3 octets of their IP (guaranteeing requests go to the same server unless it is unavailable)
  # ip_hash;
  server k3s-s1.functionalbytes.com:80;
  server k3s-s2.functionalbytes.com:80;
}
```

### Basic Full Example
```nginx
upstream k3s_servers {
  server k3s-s1.functionalbytes.com:80;
  server k3s-s2.functionalbytes.com:80;
}

server {
  listen *:80;

  server_name k3s.functionalbytes.com;
  server_tokens off; ## Don't show the nginx version number, a security best practice

  location / {
    return 301 https://k3s.functionalbytes.com:443$request_uri;
  }
}

server {
  listen 443 ssl http2;

  server_name k3s.functionalbytes.com;
  server_tokens off; ## Don't show the nginx version number, a security best practice

  ssl_certificate /secret/keys/proxy.functionalbytes.com.fullchain;
  ssl_certificate_key /secret/keys/proxy.functionalbytes.com.key;

  # Allow large attachments
  client_max_body_size 128M;

  location / {
    proxy_pass http://k3s_servers;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
  }
}
```

### Adding Custom Stream Configs
1. Add an include directive to `/config/nginx/nginx.conf`
    > File: `/config/nginx/nginx.conf`
    ```nginx
    stream {
        include /config/nginx/stream-confs/*;
    }
    ```
2. Add the stream configs in `/config/nginx/stream-confs/`, e.g.:
    > File: `/config/nginx/stream-confs/k3s.conf`
    ```nginx
    upstream k3s_servers {
      server k3s-s1.functionalbytes.com:6443;
      server k3s-s2.functionalbytes.com:6443;
    }

    server {
      listen *:6443;

      ssl_certificate /secret/keys/proxy.functionalbytes.com.fullchain;
      ssl_certificate_key /secret/keys/proxy.functionalbytes.com.key;

      proxy_pass k3s_servers;
    }
    ```

## Commands
Reload the nginx config without any downtime (sends `SIGHUP` signal):
```bash
nginx -s reload
```

Verify configuration is valid:
```bash
nginx -t
```

## PID Issue on Some Alpine Linux Docker Images
**Note:** I am currently using `nginx:1.21-alpine` (official) which does not have this issue, but do still see the issue with `linuxserver/nginx:1.20.2` ([linuxserver.io](https://www.linuxserver.io/) image that I was using this on [Unraid](https://unraid.net/))

There is an issue where the PID file isn't created in the alpine linux version of nginx docker image. Running the reload command will result in the following error:
```bash
> nginx -s reload
nginx: [error] open() "/run/nginx/nginx.pid" failed (2: No such file or directory)
```

Two options for fixing this:

1. Create the required directory
    > File: `Dockerfile`
    ```Dockerfile
    ...
    RUN mkdir -p /run/nginx
    ...
    ```
1. Override the PID location
    ```sh
    nginx -g 'pid /tmp/nginx.pid; daemon off;'
    ```

## References
- [[Ubuntu Manpage] nginx](https://manpages.ubuntu.com/manpages/jammy/man8/nginx.8.html)
- [[nginx Documentation] WebSocket proxying](http://nginx.org/en/docs/http/websocket.html)
- [[nginx Documentation] HTTP Load Balancing](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/)
- [[GitHub Issue #185] latest image run nginx problems](https://github.com/gliderlabs/docker-alpine/issues/185)

